When Mary asks Tom if he will travel to America, she states they can get him "masses of introductions - starting with Grandmama and Uncle Harold". Tom had already met Ms. Levingston in Series 3, Episode 1.
The dowager tells the butler to go to the kitchen to fetch young Pegg and to have him remove his boots before entering the drawing room. The butler is gone exactly 6 seconds and returns with a bootless Pegg. To go all the way to the kitchen and back, and wait for the boots to be removed should have taken at least a minute.
Mary says that at the end of the night that she would go down before the band left to talk to Jack Ross about Robert paying them. They state it was midnight and as stated in another episode "Downtown is hardly a metropolis" so there was no way that they would leave after paying and Mary could have done it the next morning.