C. C. Baxter (Jack Lemmon) is a lonely office drone for an insurance company in New York City. Four different company managers take turns commandeering Baxter's apartment, which is located on West 67th Street on the Upper West Side, for their various extramarital liaisons. Unhappy with the situation, but unwilling to challenge them directly, he juggles their conflicting demands while hoping to catch the eye of fetching elevator operator Fran Kubelik (Shirley MacLaine). Meanwhile the neighbors in the apartment building assume Baxter is a "good time Charlie" who brings home a different drunken woman every night. Baxter accepts their criticism rather than reveal the truth.
The four managers write glowing reports about Baxter a little too glowing, so personnel director Mr. Sheldrake (Fred MacMurray) suspects something illicit behind the praise. Sheldrake lets Baxter's promotion go unchallenged on condition that he be allowed to use the apartment as well, starting that night. Sheldrake gives Baxter two tickets to The Music Man to ensure his absence. Delighted about his promotion, Baxter asks Kubelik to meet him at the theatre. She agrees and it is revealed to the audience that she is Sheldrake's girlfriend, intending to break off their affair that night but is instead charmed by Sheldrake to the apartment. Baxter is disappointed at being stood up, but is willing to forgive Kubelik.
At an office party on Christmas Eve, Baxter discovers the relationship between Sheldrake and Kubelik, though he conceals this realization, while Kubelik learns from Sheldrake's secretary that she is merely the latest female employee to be his mistress, the secretary herself having filled that role several years earlier. At the apartment, Kubelik confronts Sheldrake with this information and while he maintains that he genuinely loves her, he leaves to return to his family. Meanwhile, a depressed Baxter picks up a woman in a local bar and, upon returning the apartment, is astounded to find Kubelik in his bed, fully clothed and overdosed on Baxter's sleeping pills.
Baxter sends his bar pickup home and enlists the help of his neighbour, a physician, in reviving Kubelik without notifying the authorities. The doctor makes various assumptions about Kubelik and Baxter, which Baxter concedes without revealing Sheldrake's involvement. Baxter later telephones Sheldrake and informs him of the situation, and while Sheldrake professes gratitude for Baxter's quiet handling of the matter, he avoids any further involvement. Kubelik recuperates in Baxter's apartment under his care for two days, during which he tries to entertain and distract her from any possible suicidal afterthoughts, talking her into playing numerous hands of gin rummy, though she is largely uninterested.
Baxter and Kubelik's absence from work is noted and commented on, with Baxter's former "customers" assuming that Baxter and Kubelik were having an affair. Kubelik's taxi-driver brother-in-law comes looking for her and two of the customers cheerfully direct him to Baxter's apartment, partly out of spite since he has been denying them access since his arrangement with Sheldrake. The brother-in-law also assumes the worst of Baxter and punches him several times.
Sheldrake, angered at his secretary for sharing the truth with Kubelik, fires her. She retaliates by telling his wife about his infidelities, leading to the breakup of the marriage. Sheldrake moves into a room at his athletic club and continues to string Kubelik along while he enjoys his newfound bachelorhood. Baxter finally takes a stand when Sheldrake demands the apartment for another liaison with Kubelik on New Year's Eve, which results in Baxter quitting the firm. When Kubelik hears of this from Sheldrake, she realizes that Baxter is the man who truly loves her and abandons him, running to the apartment. Baxter, in the midst of packing to move out, is bewildered by her appearance and her insistence on resuming their earlier game of gin rummy. When he declares his love for her, her reply is the now-famous final line of the movie: "Shut up and deal."