Both the phrases "Monotonous, isn't it?" and "Hmmm, could be!" would be used by Bugs Bunny in numerous future cartoon shorts. They were first made popular by radio comic Jerry Colonna.
Although the U. S. had yet to enter WW2 (and would, a few months after this short was released), the world had already seen the cancellation of the 1940 summer and winter Olympiads due to the war overseas. The next cycle of games would not be held again until 1948.
Madison Round Garden is, of course, a send up of NYC's famous Madison Square Garden.
"The Six Days of New York" was a former six-day cycling event, held in New York City's Madison Square Garden's velodrome. Between 1899 and 1961, a total of 73 races were run, sometimes three per year. Only the Six Days of Berlin and the Six Days of Ghent had more runnings. Australian Alfred Goullet and Italian Franco Giorgetti hold the record with eight wins each.
Sloppy Joe's was actually a popular dive bar in Key West, Fl. Hemingway drank there. It's now a popular tourist destination but still has a somewhat authentic atmosphere.