Two divers are photographing the wreck of the Orca, Quint's boat from the first film, and are suddenly attacked and killed by a large great white shark, but not before one of the divers' camera gets a photo of the shark's eye. The shark later prowls the coastal waters of Amity Island, killing a female water skier. The female driver of the speedboat tries to defend herself by first throwing a gasoline tank at the shark (accidentally spilling some on herself) and then igniting the fuel with a flare gun. The fire ignites the gas tank and the speedboat explodes, killing the driver and leaving the shark heavily scarred on the right side of his head.
In addition to these incidents, a dead killer whale is beached at a nearby lighthouse with large wounds all over its body, which Police Chief Martin Brody (Roy Scheider) suggests were caused by a great white shark. Once again, Mayor Vaughn (Murray Hamilton) doesn't share Brody's belief that the town has another shark problem and warns him not to do something hasty. Later, Brody spots a section of a ruined speedboat bobbing in the surf just off the beach. When he goes to retrieve it, he encounters the burnt remains of the female speedboat driver.
Brody angrily grounds his son Mike (Mark Gruner) because of his reluctance to find a summer job, preferring to go sailing every day, and gives him a job at the beach. The following day, while Brody is in an observation tower, he sees a large shadow produced by a school of bluefish, which he mistakes for a shark. In his haste, Brody orders everyone out of the water and fires his gun, causing a panic. Later that evening, he receives the photo of the shark's eye, taken by one of the attacked divers. Brody shows it to Vaughn and his Townsmen, but they refuse to accept the evidence put in front of them. Len Peterson (Joseph Mascolo) (who has built a new resort in Amity to attract people) and the town council fire Brody for the beach incident, with Mayor Vaughn being the only one to vote against dismissal, and promote Deputy Hendricks (Jeffrey Kramer) to Brody's position.
The next morning, Mike sneaks out and goes sailing with his friends, but has to take his young brother Sean (Mark Gilpin) along to stop him telling his parents about Mike's trip. Later, they go past a group of divers led by Tom Andrews (Barry Coe). Tom encounters the shark minutes after entering the water to catch lobsters and escapes, but suffers an embolism due to rushing to the surface too fast. Tina (Ann Dusenberry) and Eddie (Gary Dubin) later encounter the shark when he smashes into their sailboat, devouring Eddie and leaving Tina terrified and alone.
Brody and his wife Ellen (Lorraine Gary) find the panicked diver being put into an ambulance, and Brody suspects that something must have scared him to make him come up so fast. Hendricks informs Brody that Mike has gone out sailing to the lighthouse with his friends, so Brody insists on taking the police launch to rescue them, with Ellen and Hendricks both joining him. They find Tina's boat, with Tina hiding in the hull, who confirms Brody's suspicions about the shark in the area. Hendricks and Ellen take Tina ashore in a passing boat, while Brody continues to search for the teenagers using the police launch.
All seems well with the other teenagers, until the shark appears, smashing into one of their sail boats, causing panic and their boats to collide with each other. Mike is knocked unconscious and is pulled out of the water just as the shark appears; two friends take him back to the shore for help. The rest of the teens remain floating on the wreckage of tangled boats, drifting out toward the open sea. A Harbor Patrol marine helicopter arrives and a line is rigged to tow the boats to shore, but before the pilot can tow them, the shark attacks the chopper causing it to crash into the water. Sean also falls into the water, but is quickly saved by Marge (Martha Swatek). As Marge tries to get back into the boat, her hands slip on the wet hull, and she falls back into the water. The shark approaches and devours Marge. Back at mainland, Tina is sent to the hospital, and Ellen berates Peterson for getting Brody fired and denying the shark's presence.
Brody encounters Mike, who informs his father about the situation. Sean and his friends are drifting on the wreckage toward Cable Junction, a small rocky island housing an electrical relay station (with the open sea beyond it). Brody quickly finds the teenagers, but the shark attacks again, which causes Brody to run his boat aground on the rocks. Brody tries to tie a rope line, but snags an underwater power cable instead. Most of the teenagers are tossed into the water during the shark's next attack, and they swim to safety on Cable Junction, but the shark injures Lucy. Using an inflatable raft, Brody gets the shark's attention by pounding the power line with an oar, and gets the shark to bite the power cable. The plan succeeds, the shark gets electrocuted and sinks to the bottom of the sea, dead. Brody collects Sean and Jackie and paddles over to Cable Junction, to await rescue with the other teenagers.