The scene towards the end of the film in which Emmet Ray is drunk in a New York bar was filmed in "Chumleys," a bar on Bedford Street in Greenwich Village.
Woody Allen pays a clever homage to Federico Fellini's La Strada. The characters of Emmet (Penn) and Hattie (Morton) correspond to Zampano (Quinn) and Gelsomina (Masina) including the final scene with Emmet's/Zampano's breakdown and repentance.
The guitar that 'Sean Penn' plays in the movie is a Selmer Maccaferri of about 1932 (though it seems likely that it's a reproduction of the rare instrument and not an original) This is the same kind of guitar played and made famous by Django Reinhardt.
In the scene where Emmet Ray sees Django Reinhardt for the first time at the gas station and faints, right behind the trees is the Nyack High School Football team. They were asked to be quiet during the filming of the scene and had a stop their practice a number of times.
Woody Allen had planned an initial version of this movie (originally titled "The Jazz Baby") as his first film for United Artists. When UA insisted upon a comedy, Allen shelved the idea for nearly three decades.