This episode's title takes its name from the piece that Anna Draper's young piano student is practicing when Don enters Anna's house: "In the Hall of the Mountain King," one of the pieces of music that Edvard Grieg originally wrote for Henrik Ibsen's 1876 play Peer Gynt.
Peggy's Popsicle design campaign is an allusion to many Catholic images of the Virgin Mary, complete with the orange circle of the Popsicle logo serving as a halo for the mother. It follows up on Peggy's earlier comment in the episode that the Catholic Church knows how to sell things.
The trivia item below may give away important plot points.
Although the showrunners have stated unequivocally that what Greg does to Joan in Don's office is rape, it is strongly implied that Joan may not have understood it as such because during the time this episode is set (1962) and for about 15 years afterward, marital and "date" rape were not legally or societally recognized concepts in the US. Until 1976, there were laws in all 50 states exempting spouses from being prosecutable for rape; even now (as of 2010), only 17 states (and the District of Columbia) make no distinction between marital rape and other kinds in their laws; all other states have mitigated penalties for marital rape.