Beckett refers to their first suspect, a monk, as a "felonious monk". This is a reference to jazz musician Thelonius Monk.
When Beckett and Castle go back to the monastery, Castle puts his hand in the mouth of the secret door. As he does this, he screams in pain, as if something happened to his hand, before telling a shocked Beckett that he was "just kidding." This is a reference to a scene in the 1954 film Roman Holiday (1953) when Joe Bradley, played by Gregory Peck, surprises Princess Anne, played by Audrey Hepburn, in a similar way. Addition: this also occurred in the second National Treasure when Cage stuck his and between two rocks to open a secret door and screamed in fake pain to scare his mom, dad and ex. Most likely this is the reference considering the entire episode takes after, in part, National Treasure, which was referenced earlier in the episode.