John Shaft, a private detective living in New York City, is informed that some gangsters are looking for him. Police Lt. Vic Androzzi meets Shaft and unsuccessfully tries to get information from him on the two gangsters. After Androzzi leaves, Shaft spots one of the men waiting for him in his office building. He commandeers the first gangster, forcing him into his office where the second gangster is waiting. After a quick fight, Shaft dodges one of them who goes out the window, while the other surrenders and reveals to him that Bumpy Jonas, the leader of a Harlem-based organized crime family, wanted Shaft brought uptown to Harlem for a meeting.
At the police station, Shaft lies to Lt. Androzzi and the detective assigned to the second gangster's death at Shaft's office, by saying that his friend was in an "accident". He is allowed to return to the streets for 48 hours before the police decide to question him again. Shaft arranges a meeting with Bumpy, the leader of these gangsters, in his office. It turns out Bumpy's daughter has been kidnapped, and Shaft is asked to ensure her safe return.
After tracking down Ben Buford as Bumpy suggested, a big shoot out ensues; Shaft is told by Vic after the shooting that Shaft himself, and not Ben, was the target, and that tensions brewing between the uptown hoods belonging to Bumpy Jonas and the downtown Mafiosi have culminated in a couple of murders. But the perception is black against white to the general public, with the possibility of an escalation into full-blown race war on the streets of the city. He also shows Shaft some pictures of two of the Mafia men who just arrived in New York. Vic begs Shaft to explain what's going on, although Vic already knew Bumpy was looking for Shaft.
Streetwise, Shaft surmises that mobsters are watching his pad from a local bar. Shaft pretends to be a barkeep and calls the police to have the mobsters arrested. Shaft later goes to the police station to set a meeting to find where Bumpy's daughter is being held captive.
Vic tells Shaft that the room that he was in at the station house was bugged and he is supposed to bring him in for questioning, but instead leaves. Ben and Shaft go to the apartment where Marcy Jonas is being held to make sure she's alive. Once there, a gunfight ensues during which two hoods get killed and Shaft takes a bullet in the shoulder.
Shaft goes home and receives medical attention from a doctor working underground with him. Shaft tells Ben to round up his men and meet him at the hotel where Marcy has been taken, to prepare to get her back. He also calls Bumpy to tell him his daughter is fine and he is going to need some taxicabs to meet him at the hotel for the getaway.
Shaft's plan resembles a military commando-style operation. That evening, Ben's men all dress as hotel workers to avoid arousing suspicion. Shaft and one of Ben's men go to the roof and prepare to enter the room where Marcy is being held captive. Shaft's plan is to cause a distraction with an explosive thrown through the window of Marcy's room while Ben and his men come down the hall and deal with the Mafia men as they leave their rooms.
The rescue plan is successful. Marcy is spirited out of the hotel into one of the waiting taxicabs. As the others get away in the remaining cabs, Shaft walks to a phone booth to call Vic. Shaft informs Vic as a result of the rescue there will be a huge mess to fix between the uptown crew and the mob in the near future. Vic says to close it for him, meaning he wants Shaft to fix the trouble. Shaft replies, "You're gonna have to close it yourself shitty" then hangs up the phone as he walks away laughing.