"Remember When" was the career directorial debut of Phil Abraham, a longtime Sopranos cinematographer since the first season. Abraham initially started as a camera operator for the series.
The Katz's Deli (famous from 'When Harry Met Sally') is used for this episode. Tony, while being away to "lay low," calls Hesh for a loan. Hesh, back in Manhattan, is enjoying a sandwich in Katz's Deli when talking to Tony.
Lin Manuel Miranda of "Hamilton" fame makes his television debut in a bit part as a bellhop in this episode.
Ken Leung, who plays Carter Chong, was in 2 movies previously referenced in the series. "Kundun," which Christopher yelled to "Marty Scorsese" coming out of his limo in Season 1, and "Saw" which Christopher mentioned a few times in reference/comparison to his own movie "Cleaver."
The show Tony sees Paulie watching is the TV sitcom Three's Company.