The dress Julie wears at the beginning of playing the records is not the same dress she is wearing when she is playing the last record. This is because she changed into a different dress as well as coat while she is packing to leave and left the record playing.
After the courtroom scene, the stairs don't squeak as he ascends to the second floor with the child. There has been no indication the stairs have been fixed at this point.
When they first put the 5-week-old baby to bed, the baby is lying face down. Later, when the baby wakes up screaming, she is miraculously lying on her back.
When Roger asks the other music store clerk to play some sheet music, she walks off camera, but does not stop the record being played, so she can play the sheet music he selected. The record player keeps on playing and the other clerk just disappears.
The record shown playing is a bat wing Victor that was produced prior to 1925, making it historically incorrect.