Whilst Caligula (Jay Robinson) is talking to Claudius and Messalina about the death of Marcellus and Diana, he draws a dagger and plays with Claudius by threatening to stab him with it. When he puts it back in the scabbard, the scabbard and dagger inadvertently turn upside down sticking out at an odd angle and irritating Jay Robinson, who tries twice and finally rights the scabbard at his waist and goes on.
Near the end of the movie when Peter goes to the house of Messalina to see Demetrius, Messalina throws wine on Peter. The wine stains Peter's under garment on his shoulder and drips on his draped outer toga on his chest. When the camera cuts to another angle, Peter now has a smaller wine spot only on his shoulder with no dripping stains on the chest of the outer toga. When the camera switches back, Peter again has both stains (shoulder and chest) prominently along the front of his outfit.
Caligula was emperor of Rome from 37 to 41 CE. The apostle Peter is known to have been incarcerated by Herod Agrippa, who reigned from 40 to 44 CE. When he came to Rome is unknown, but this must have been later, only after there were followers of the new religion there (after 50 CE). So, Caligula and Peter were never in Rome at the same time.
Caligula is depicted in this movie and its prequel "The Robe" as persecuting Christians. However, he reigned from 37 to 41 (he's killed near the end of this movie), whilst Christianity was still a nascent religion with most of its followers in the eastern Mediterranean. The first mention of Christians from the perspective of the Roman government, according to the Roman historian Suetonius, wasn't until the reign of his successor Claudius (reigned 41-54). The first major incidents of persecution of Christians did not occur until the reign of Nero (reigned 54-68).
A common misconception was that half the gladiators in the arena would be killed. In actuality, there were many times that the losing gladiator's life would be spared. This was partially due to him, or her, giving a good battle. The gladiators were also very valuable to their owners. Emperor Claudius, who followed Caligula, was known for not showing mercy, even when it was a good fight.
When Claudius and Messalina were interviewing Demetrius on his Christian beliefs, the parrot on the perch next to Messalina's couch was a sulfur crested cockatoo, indigenous to Australia. Australia was unknown to the western world in the first century AD. Therefore, such birds would never have left their native habitat, much less be captive pets in Ancient Rome.