As a boy, Colin Sullivan (Damon) is introduced to crime by Irish-American mobster Frank Costello (Nicholson) in the Irish neighborhood of South Boston. Costello grooms him to become a mole inside the Massachusetts State Police. Sullivan becomes an officer and is accepted into the Special Investigations Unit, which focuses on organized crime.
Billy Costigan (DiCaprio), whose family has considerable ties to organized crime, applies to join the police force. He is grilled and harassed by Captain Queenan (Sheen) and Staff Sergeant Dignam (Wahlberg). They pressure him to disqualify himself as a police officer because he's too smart to be a cop and due to his family's ties to criminals. When Costigan stands up to their hazing, they offer him a job as a police officer on one condition: that he quit the academy, go to jail on a trumped up assault charge, and infiltrate Costello's organization. Only Queenan and Dignam will know that Costigan is working for the police.
As both men infiltrate their respective organizations, Sullivan begins a romance with psychiatrist Madolyn Madden (Farmiga). Costigan sees her as required by the terms of his probation and also develops a relationship with her. After Costello escapes a sting operation, both moles become aware of each other's existence, though not their identities. Sullivan is told to find the "rat" and asks Costello for personal information about members of his crew so he can determine who the informer is.
Costigan follows Costello into a cinema, where Costello gives Sullivan an envelope containing the personal information he asked for on his crew members. Costigan chases Sullivan through Chinatown but neither man learns the other's identity. Sullivan has Costello's men trail Queenan to a meeting with Costigan where Costello's men kill Queenan. As they leave, Costigan joins them, pretending he has just arrived to join them in the assassination. Later, Costello's henchman, Fitzgibbons (O'Hara), reveals that Delahunt (Rolston), a crew member, was an undercover cop. Dignam is forced to step down from the police force as a result.
Using Queenan's phone, Sullivan reaches Costigan, and fails to persuade him to quit his work as a mole. Sullivan learns from Queenan's diary that Costello was an informant for the Federal Bureau of Investigation. He worries that his identity as a mole for Costello may be revealed. With Costigan's help, Costello is traced by the police to a cocaine pick-up, where a gunfight erupts between his crew and police, during which most of Costello's crew is killed. Sullivan confronts the wounded Costello, who admits he is an occasional FBI informant. Sullivan shoots him multiple times. Sullivan is applauded the next day for having killed Costello by everyone on the force.
In good faith, Costigan comes to see him, seeking to get his civilian identity restored and to collect his back pay. He tells Sullivan he intends to resume his civilian life. Sullivan leaves to look up Costigan's employee record when Costigan notices the envelope from Costello on Sullivan's desk. Costigan finally realizes Sullivan is Costello's mole. Returning to his desk, Sullivan realizes that Costigan has figured out his true identity, so he erases Costigan's employee records from the police computer system.
Madolyn tells Sullivan she's pregnant, but does not reveal who the father is. A few days later she receives a package in the mail from Costigan addressed to Sullivan. She opens it to find it contains a CD of Costello's recorded conversations with Sullivan. Sullivan walks in as she is listening and tries unsuccessfully to assuage her suspicions. He contacts Costigan, who reveals that Costello recorded every conversation he had with Sullivan. Costello left the recordings with his attorney, who has given them to Costigan. Costigan says he wants his civilian identity back or he will implicate Sullivan. They agree to meet on the roof of the building where Queenan died.
When they meet, Costigan catches Sullivan off-guard and handcuffs him. As Costigan had secretly arranged, Officer Brown (Anderson) appears on the roof as well. Shocked to see Sullivan in handcuffs and held at gunpoint by Costigan, Brown draws his gun on Costigan. Costigan explains his actions by revealing that Sullivan is the mole. Costigan asks Brown why Dignam did not accompany him, but Brown doesn't answer. Costigan leads Sullivan to the elevator.
When the elevator reaches the ground floor and the doors open, Officer Barrigan (Dale) shoots him in the head, and when Brown arrives, Barrigan kills him too. Barrigan reveals to Sullivan that Costello had more than one mole in the police and that Costello was going to give both of them up to the FBI. When Barrigan momentarily turns, Sullivan shoots him in the head. At police headquarters, Sullivan concocts a story to protect himself, identifying Barrigan as the mole and recommends Costigan for the Medal of Merit.
At Costigan's funeral, Sullivan and Madolyn stand by the grave. Sullivan attempts to talk to her, but she ignores him. As Sullivan enters his apartment, he is met by Dignam, who shoots and kills him.