When Stuffy gets on the horse with the wagon, he is wearing a coat and dark trousers. As jockey he wears white pants and jockey shirt. While he could have lost the coat easy enough, there was no opportunity for him to change pants.
When the horses head to the track, the announcer notes only eleven horses are headed to the track and that the added horse, Hi Hat, is nowhere to be seen. However, when the board was shown with the horses listed, twelve horses were initially listed for the race with two scratched and then Hi Hat added, so if Hi Hat wasn't heading to the track, there should have only been ten horses.