When Nat and Ritchie start out on their trip Nat makes Ritchie pull over quickly to get a hair cut. In the background you can see a toll booth. Later in the film you can see them swerve off the freeway with the same toll booth in the background.
While in Altoona, PA, Nat and Ritchie talk about how this is where Ritchie was convicted, and then went on to do time in Rahway. Rahway is a prison in Jersey; presumably a person convicted of a crime in PA would go to a prison in that state, not NJ.