Lord George is sick of keeping Teddy and gives him an ultimatum. Either he marries Madge Cartwright, the heiress to a soap dynasty and a potentially wealthy woman, or he will have to go and work in the family's rubber factory. Teddy, who prefers servant girls to his own class, would rather be dating Madge's maid Rose than Madge herself but George calls his bluff and sends him to the factory. Alf, meanwhile, has found a load of money under Lady Lavender's bed and aims to get his hands on it.
Teddy's sexual harassment of some of the factory girls leads to a strike. Alf persuades Lord George to address the strikers but his efforts are feeble and Alf, knowing that his employer has to be home by evening to entertain the prime minister, conducts his own negotiations - leading to a pay rise and Teddy's dismissal. Lord George asks Ivy and Henry to take the money from under Lady Lavender's bed but she has hidden it somewhere else.
Lord George wants to put Lady Lavender's money in the bank but when she hears an organ-grinder in the street below, she throws it out of the window. Staff and family are told they will be grounded until it is all recovered and the money is all returned - apparently. Alf has taken some for himself and hidden it in one of a set of three vases the Meldrums have given the Bishop for his charity auction. Alf is desperate to recover it.
Teddy resigns himself to his forced engagement to Madge Cartwright whilst the girls start to suspect their father's dalliance with Agatha. Alf pretends to woo Mrs. Lipton so she will lend him fifteen pounds to bid for the vases at the charity auction. He successfully buys them but, due to other bidders, has to pay more than he expected and ends up out of pocket.
Ralph confronts Lord George with his suspicions of Agatha's infidelity but Alf puts him off the scent by claiming that his employer is impotent due to a war wound. George meanwhile believes Agatha is seeing somebody else. James uses Ivy as an excuse to get out of a compromising situation with Poppy but, seeing that he has hurt her, takes her to the pictures - as a friend. Alf stalls Mrs. Lipton by claiming that his wife refuses to divorce him.
James' rough diamond of a father arrives in the middle of the night, on the run from an armed gang for whom he refused to shoot a gun and the police. Despite their differences Alf will not see James disgraced and, as the other staff members discover about the fugitive, he arranges to send him on a banana boat to South America next evening. Helped by the socially-aware Cissy the staff manage to get their passenger on board in time, despite the presence of Lady Lavender, who tags along in the belief that she is eloping with Captain Dalby.
James kisses Poppy, who this time does not protest but Mrs. Lipton's cooking is under fire, following the discovery of an earwig, so outside caterers are hired for a banquet to entertain the impoverished ex-king and queen of Dalmatia, its central attraction being a sugar swan - whose head falls off. Noel Coward also shows up but prefers to spend the evening with Lady Lavender drinking champagne in her room.