Lily's story about why she doesn't want to name her son Rob is a reference to the Pilot when she announced one of her kindergarteners got to second base with her.
When Marshall mentions the name Jeremy as a potential baby name, the song "Jeremy" by Pearl Jam plays during the flashback. The video for that song is parodied in the flashback when red paint is splashed on Lily.
Alyson Hannigan's character comes up with a list of baby names, one of which is Tara, the same name as her character Willow's girlfriend on Buffy the Vampire Slayer (1997).
This episode features Laura Bell Bundy as 'Becky.' Laura Bell Bundy was in the original Jumanji as 'young Sarah' (portrayed by Bonnie Hunt). You can hear the 'Jumanji' drums throughout this episode.