Joe and Jerry, a saxophonist and bassist, respectively, are two working-class musicians, working in a Chicago speakeasy in February, 1929. Though they have steady work, they still owe money to many of their friends. However, Joe, the optimist, isn't worried since the gig they have seems to be stable. Joe does flirt briefly with the idea of taking all the money he and Jerry have and betting it on a greyhound race but the plan is soon discarded.
Fortune changes a few minutes later when a police officer, Mulligan, working on a hot tip from a mob informant, Toothpick Charlie, raids the place for illegal liquor sales. The speakeasy itself belongs to Chicago's most notorious mob boss, Spats Colombo. Joe and Jerry barely escape the raid. Jobless, they try to figure out a plan to earn money; Joe suggests they hock their overcoats and bet the money on a long shot at the dog racing track. Joe's plan fails miserably and the guys are more broke than ever during a bitterly cold Chicago winter.
Joe and Jerry go to the offices of their talent agents, whom have no work for them. They go to the last one, Sig Poliakoff's, where Joe talks to the receptionist, Nellie, whom he has been dating. She tells him that Poliakoff has openings for a sax player and bass player in a band that will be traveling to Florida. In Sig's office, Sig is on the phone frantically trying to find replacement musicians for Sweet Sue, the band's leader, and her assistant, Beinstock. Sue's band is all-female and she has a strict "NO MEN" policy; Sue's frustrated because one of her players got pregnant and another ran off to get engaged. Jerry and Joe, not knowing Sue is looking for women, burst into Sig's office and ask for the gig. Poliakoff informs them that they're the wrong gender, but he does have a gig in Urbana, Illinois, for one night. Joe and Jerry accept and con Nellie into loaning them her car to drive to the gig.
Joe and Jerry go to the garage where Nellie's car awaits. A group of shady-looking men are playing cards in the corner, one of them is Toothpick Charlie. While the mechanic fills the car with gas, a large limo rolls into the garage and several gang members, armed with shotguns and Thompson machine guns get out. Joe and Jerry hide behind Nellie's car while everyone else is lined up against the wall. The gangsters who burst into the place work for Spats Colombo, who steps from the limo. He has come to Charlie's garage seeking revenge for the speakeasy being busted. He gives the command and his men slaughter everyone against the wall (a reference to the infamous St. Valentine's Day Massacre perpetrated by Al Capone). The gas nozzle in Nellie's car suddenly pops out when the tank is full and clatters on the ground, drawing the attention of Spats' men, who see Joe and Jerry. Spats orders them to be killed too, while at the same time, Charlie, still alive, tries to reach a nearby phone. Spats grabs a Thompson and kills Charlie, temporarily forgetting Joe and Jerry, who escape. One of Spats' goons shoots at them but only succeeds in hitting Jerry's bass. Joe and Jerry, now on the run, call Poliakoff's office, planning to fool their agent into thinking they're the women musicians needed for Sweet Sue's band.
Joe and Jerry, dressed as women, arrive at the train station. They use false names; Joe becomes Josephine and Jerry becomes Daphne (he'd originally agreed to be Geraldine, but tells Joe he never liked the name). They meet Sue and Beinstock and fool them effectively enough to be hired. They also spot the band's singer and ukulele player, the attractive and blond Sugar Kowalczyk. Both men are instantly attracted to her, especially Joe.
During a band practice on the train Sugar drops a flask of bourbon; alcohol is strictly forbidden by Sue. Beinstock reminds Sugar that he warned her not to hide liquor, however, Jerry tells them it's his flask and covers for Sugar. Later that night, Sugar sneaks into Jerry's berth to thank Daphne for covering for her. Jerry suggests that they both share a drink of whiskey, which Jerry steals from Joe's suitcase in the berth below his. The other women in the band quickly discover Jerry and Sugar and assume they're having a party. Joe and Sugar go to the ladies' room to prepare some ice and during their conversation, Sugar tells Joe that she's had bad luck with romance and men and that she has a soft-spot for saxophone players. Joe is floored but keeps his composure and they share a drink together. In his berth, Jerry finds that the other girls are becoming a bit too physical and, afraid they'll find out his true identity, pulls the emergency cord, stopping the train. All the girls spill out of his berth and the party grinds to halt, Sweet Sue flustered by the whole ordeal.
The train arrives in Florida and the band is taken to their hotel. Joe leaves Jerry to carry all their luggage and as Jerry climbs the steps to the entrance, he loses a shoe. A rich man, Osgood Fielding III places it back on her feet and proceeds with improper advances toward Jerry's alter ego. Jerry fights the man off after he gropes him in the elevator. In their room, Jerry tells Joe they should leave the band and go further into hiding. Joe tells him they should stay in disguise since Colombo's gang would never look for them in an all-woman band. Joe is also attracted to Sugar, though he doesn't sight that as a reason for staying, and has his own plans to woo her. Sugar shows up and invites the two to the beach; Jerry joins her but Joe declines. After they leave, Joe takes out a suitcase he'd stolen from Beinstock (along with the man's glasses) and dresses up in a fashionable sailor's outfit. Joe goes down to the beach and sits in a chair, reading the Wall Street Journal. He attracts the attention of Sugar and presents himself as an heir to the Shell Oil Corporation. Sugar begins to flirt with him, however, he remains aloof, telling her he's waiting for a signal from his yacht offshore. Jerry happens by and instantly recognizes Joe. He convinces Sugar to go back to his and Joe's hotel room to expose him as an impostor. Joe beats them back there and they find him in the tub, covered with bubbles and posing as Josephine. Sugar tells Josephine that she's probably met a millionaire and she leaves. When she does, Jerry launches into a tirade about faking an identity on the beach and that Joe is trying to take advantage of Sugar. Joe responds by rising out of the tub, still in his Shell Oil Jr. outfit, and plops his wet wig down on Jerry's head. Jerry receives a ship-to-shore call from Osgood: Osgood wants to invite Daphne to have dinner with him aboard his yacht. Joe takes charge of the invitation and tells Jerry to persuade Osgood to take him to a dinner and dancing club instead of the yacht. Joe will go to the yacht as Shell Oil Jr. with Sugar.
That night, while Joe & Jerry play with the band at dinner, Jerry receives a giant bouquet of flowers from Osgood. When the gig ends, Joe rushes back to his room and assumes the disguise, while Jerry and Osgood go to the restaurant. Joe arrives at the dock just before Sugar and takes her to the yacht on Osgood's boat. While the two have drinks and eat, Joe again acts aloof towards Sugar, trying to persuade her to kiss him. At first he acts as though he has a psychological block that prevents him from enjoying their romantic evening, but Sugar eventually turns him on. On the shore, Osgood and Jerry dance the tango all night.
Sugar and Joe return to the mainland, apparently in love. He bids farewell to Sugar and climbs up to his room where Jerry is lying on one of the beds. Jerry tells Joe he's engaged; when Joe asks "who's the lucky girl?" Jerry says he is himself because Osgood proposed to Daphne. The two have a brief debate where Jerry reveals his plan to marry Osgood and tell him the truth right after the ceremony. He plans to extort a large settlement out of Osgood and live on the alimony checks he believes he'll receive. Joe convinces Jerry that he's committing fraud and will be caught. Jerry shows Joe the pricey diamond bracelet Osgood gave to him as an engagement gift, saying he'll return it. Joe suggests they keep it, perhaps thinking they can hock it for cash.
In the hotel lobby, Spats Colombo and his goons arrive for a convention of "Friends of Italian Opera", which is actually a meeting of organized crime gangsters. The organization and meeting are being led by Little Bonaparte, the most powerful gangster there. Bonaparte already has a rivalry with Spats, which has been exacerbated by Spats' murder of Toothpick Charlie, who was a good friend of Bonaparte. Jerry and Joe, in the lobby, spot Spats and his crew and immediately get into the elevator to return to their room. Just as the doors are about to close, Spats and his men enter the elevator; Jerry and Joe's disguises work on them and they make it to their room.
They pack hurriedly and Joe wants to take care of one last detail: Sugar. He calls her and once again uses his Shell Oil Jr. voice, telling her that he has to leave suddenly. His parents have told him to marry a woman who is the daughter of another millionaire with a large empire. As a final gesture, Joe leaves a bouquet of flowers outside Sugar's room with Osgood's diamond bracelet (the gift to Jerry) hidden inside. Sugar is devastated but accepts the gift.
Jerry and Joe climb out the window to avoid running into the gangsters again. However, their path takes them right past Spats' balcony and they're spotted. Spats grabs the bass Jerry left behind and sees the bullet holes from when the two escaped in Chicago. Spats and his men chase them through the hotel but lose them. At one point, Jerry and Joe disguise themselves as a bellhop and a man in a wheelchair and duck into a banquet room, the same room all the gangsters will be eating dinner in. While they hide under the huge table, a pair of shoes with spats on them slides under the table. The two sit still and wait.
Little Bonaparte begins the meeting with a lengthy criticism of Spats himself, admonishing him for the assassination on Valentine's Day. Bonaparte seems to forgive Spats' indiscretion and lightens the mood by announcing they will celebrate Spats' birthday. Spats points out that his birthday isn't for a few months but Bonaparte insists they still have a large cake for Spats. After the cake is brought in, the entire room sings to Spats and a gangster pops up out of the cake and shoots Spats and his crew with a Thompson. Jerry and Joe burst out from under the table and run out of the room. Just as Little Bonaparte orders his men to catch them, Chicago cop Mulligan walks in and demands to know what happened. Bonaparte avoids the question and Mulligan promises to start a federal investigation.
Jerry and Joe retreat to their room, fixing their disguises once again. They overhear a gangster saying that they've got all the standard escape routes covered. Joe realizes that they can escape on Osgood's yacht and tells Jerry to call the millionaire and accept his marriage proposal. Before they leave, Joe wanders into the dining and dancing hall of the hotel and sees Sugar singing "I'm Through With Love". Moved, Joe approaches her and kisses her. Sugar realizes who Josephine really is and leaves the band, following Joe and Jerry to Osgood's speedboat. They all board it and head for the yacht. Joe tries to tells Sugar that he's a cad who took advantage of her and that he's a sax player who will only treat her badly. Sugar doesn't care and kisses him anyway. Jerry begins to tell Osgood that he's equally as treacherous, that he smokes and can't have children, however Osgood doesn't care. Jerry, frustrated, finally pulls off his wig and tells Osgood he's actually a man. Osgood replies, "Well, nobody's perfect."