The battle of Gettysburg ended with Lee's retreat on July 4th, 1863 and the city of Vicksburg surrendered that same day. Lincoln gave his address at the dedication of the Gettysburg Cemetery on November 19th, 1863, more than 4 months after Vicksburg surrendered.
When John and Matt Geyser meet, Matt (a Confederate or "CS" soldier) is wearing a belt with a US buckle - upside down. It's not too farfetched that a resource-poor southerner could have used captured gear, but wearing it wrong would have screwed up all the fittings to hold his equipment.
(at around 26 mins) While Captain Steele is leaving the funeral, there is a close-up of a dead union soldier. He is breathing and his eyes and mouth move.