The calendar in the background at the mission (when Edith Keeler talks to Kirk about a place to stay) shows a 30-day month, with Wednesday the 14th in red, indicating a holiday. However, none of the 30-day months in 1930 started on a Thursday, and the only month in the United States with an observance that is always on the 14th is June (Flag Day). In 1967 (this episode was filmed in February of that year), Flag Day fell on a Wednesday, therefore it is likely that this is a 1967 calendar.
Spock picks a combination lock to obtain tools. This type of single-dial combination padlock was first marketed by Master Lock in 1935, five years after the events of the episode, and the early models had a metal dial, with the plastic dial depicted on the show coming at least two decades later. The particular lock Spock picks appears to be a much later copy of the Master design, possibly from the late 1950s, but much more likely to be from the early to mid 1960s, which was when the show was filmed.
As Edith and Kirk are walking hand in hand outside the mission, Kirk says "Oh", but it is clear to see that Kirk is not talking. Edith then says to Kirk that they have to hurry to see the Clark Gable movie. Kirk responds with "What?". Again, it is easy to see that Kirk is not talking.
During the early bridge scene, shots filmed from the front of the bridge clearly show a red uniformed female standing next to the center seat. When the shot switches to the view from the rear, she disappears, even though that section of the bridge is still in shot, and reappears when the view returns to the view from the front. In addition, Kirk looks different in the rear view shot, as if this section was filmed with a stand in for William Shatner.
The Guardian of Forever first speaks in response to Kirk's questioning of its origin, by responding "A question." Moments earlier, Kirk had asked the same question in slightly different wording, with no response, though this question was directed to Mr. Spock.
When Kirk and Spock are running from the police, you can tell that they simply run around the block back to the exact same alley. The two barrels, one rusty and one yellow, are still in the exact same spot.
When McCoy arrives in 1930 NY, he believes he is in a fabricated environment and marvels at the detail of reconstruction. As he leans on a column less than 6 inches in diameter he remarks on the detail, "...even down to the cement columns." implying the free-standing column he is leaning on is made of cement. Clearly, the size and texture of the column would make it metal. A free-standing street column also would suggest a utility pole, none of which would be made of cement.
The goof items below may give away important plot points.
In a scene often deleted for syndication, a bum called Rodent steals McCoy's phaser and accidentally kills himself with it. In the wide shots you can see that the actor playing Rodent is missing part of the ring finger on his right hand, but the ring finger in the close up of the phaser being activated is undamaged.
While seven people beam up and the transporter platform has only six pads, there are several transporter stations on the Enterprise and the extra crewman could have easily been sent to a second platform.