The opera the Penguin and Vreeland attend is "Pagliacci", about a clownish figure who finds out he was being deceived. This mirrors the episode's basic plot.
The Penguin - newly freed from prison - catches a bus. In the background of this scene sits a thin white girl with short black hair. This is Hopey, one of the main characters from the cult comic book "Love and Rockets". The man who drew this scene as a storyboard, Ronnie Del Carmen, is a devoted fan of the comic and put her in as a shout-out to fellow fans.
The Penguin squawks out a grating accompaniment of Pagliacci. His voice actor Paul Williams is actually an acclaimed singer/songwriter with notably smooth, melodious vocals (as can be heard in the Penguin's speaking voice).
The artwork The Penguin is admiring at the top of the episode is the authentic John James Audubon he coveted in The Mechanic (1993).