When the Berkmans are speaking to Walt's teacher about his plagiarism of the song "Hey You" by Pink Floyd during his performance during the talent show, there is a poster behind the teacher promoting reading featuring the WWE wrestler Hurricane (Shane Helms) who made his WWE debut in 2001. He was only 12 in 1986.
In the middle of the movie, Walt is waiting on a subway platform. A train goes by, and you can glimpse an American flag on the side of the train. Flags decals weren't put on New York subway trains until after 9/11/2001.
When Walt is learning 'Hey You' from the record and from the book, the book has the song in standard music notation and in tablature, but tablature was just starting in the guitar magazines of the mid-1980s and the style of the book is also from a much later date.
Bernie's car has a Statue of Liberty license plate, which didn't start appearing until 1986. But his car's registration says 1986 on it, which means it was issued in 1984, so it should have the older blue-on-yellow license plate.
The ambulance that takes Bernard away is painted FDNY red. EMS was not merged into the fire department until the '90s, and prior to that ambulances were painted in the EMS colors: orange, blue and white.
Bernard misrepresents the end of Breathless: he says that Michel says "You're a bitch!" to Patricia at the end; actually, Michel says "It's disgusting," then Patricia asks Detective Vital what Michel said, and Vital misreports it as "You're disgusting." It's unlikely that someone with such a deep interest in art house film would get it so wrong.
The Squid and the Whale display in the museum is in reality not brightly lit. It is a very dark display meant to simulate the inky depths of the ocean. (It probably would not show up on film as it really exists.)