Set in 1937 Los Angeles, a private investigator named Jake "J.J." Gittes (Nicholson) is hired to spy on Hollis Mulwray, the chief engineer for the city's water department. The woman hiring Gittes claims to be Evelyn Mulwray, Hollis' wife. Mr. Mulwray spends most of his time investigating dry riverbeds and water runoff outlets. Mr. Mulwray also has a heated argument with an elderly man on the street. Gittes finally catches Mulwray during an outing with a young blonde and photographs the pair, which becomes a scandal in the press. After the story is published, Gittes learns that the woman who hired him was not the real Evelyn Mulwray. Evelyn later visits Jake's office and threatens to sue him for defamation. (Her image of Gittes is not helped when he tells a dirty joke that she overhears.)
Clues suggest a scandal in the city government: despite a serious drought and an expensive proposal to build a new dam, the Water and Power department is dumping fresh water into the ocean at night. The dam project is opposed by Mulwray himself, who cites a potential disaster because of weak geological formations in the rock where the dam is to be constructed -- Mulwray had previously supported the building of another dam that had failed due to similar geological conditions. While he addresses a public hearing on the project, that Gittes himself attends, he is ridiculed by several farmers in attendance, who need the reservoir and water the dam will create.
Jake goes to Mulwray's mansion to speak to him but is only able to talk to Evelyn. While he waits for her, Mulwray's Japanese gardener works in one of the small decorative pools. He casually says "bad for the glass" in broken English, a comment that Jake dismisses. Jake notices a shiny object in the pool and tries to retrieve it, stopping when Evelyn appears. She tells him that Hollis usually takes afternoon walks at a reservoir and that she should look for him there.
Gittes looks for Hollis at the reservoir but finds the police there instead, hauling his body out of the water & investigating Mulwray's death from drowning. The lead investigator is Lt. Lou Escobar, a man Jake worked with while he was a cop himself. When the police speak to Mrs. Mulwray about the death, they assume she hired Gittes, which Gittes corroborates. She thanks him and hires him to investigate what happened to her husband. Later, at the county coroner's department, Gittes talks to the coroner himself and finds out that Mulwray lungs were filled with salt water, despite his being found in a freshwater reservoir. Gittes is also told that a local drunk was found dead in a dry riverbed that Mulwray was inspecting, having also drowned.
Later that night, while sneaking into the reservoir's secured area, Gittes is confronted by water department security; a large man named Claude Mulvihill and a short thug (a cameo by Polanski), who slashes Jake's nose for being a "very nosy fella." Gittes, forced to wear a large & ridiculous bandage on his face, receives a call from Ida Sessions, the woman who impersonated Mrs. Mulwray, who admits she was hired to trick Gittes. She refuses to come to Gittes' office to speak about the incident but suggests that Gittes look at that day's obituary column. At the water department, Gittes notices photographs of the elderly man Mulwray quarreled with a few days before his death, identifying him as Noah Cross (Huston). Cross, who is Evelyn Mulwray's father, used to own the water department as Mulwray's business partner. Cross ended his association with the department when the partners sold it to the city, a long-standing desire of Hollis'.
Cross invites Jake to lunch at his home and hires Jake to find the blond girl Hollis had been seeing, saying that she might know what happened to him. Gittes goes to the Hall of Records and looks in a large plat book, finding that new names of people who have purchased large tracts of land. When the attendant in the room refuses to let Jake borrow the book, Jake surreptitiously tears the column out of the book and pockets it. Acting on a further hint from Sessions, Gittes begins to unravel an intricate scandal involving LA's fresh water supply. Gittes first travels to an orange farm to talk with the owner about how his land is being irrigated. As he drives around he is shot at by the farmer and a few of his farmhands and crashes his car into a tree. Jake is dragged from his car, beaten and searched. The farmer explains that the Department of Water has been harassing him by sending agents to run him off his land and poison the water in his wells. While he tries to show the farmer documentation of his investigation, the farmhands claim that Mulwray is responsible for harassment of late and attack Jake. When Jake tries to fight back, he's knocked unconscious. He wakes up to find that the farmer and his wife have called Evelyn, who has come to the farm.
While they drive back to LA, Jake explains the scandal to her: her father and his partners have been forcing farmers in the rural areas surrounding the city off of their land so they can buy it cheap, after which a newly-built (and controversial) dam and water system would start redirecting much of L.A.'s water supply to that land, dramatically increasing its real estate potential & value.
Since Cross wants no record of such transactions, he has partnered with a retirement home community, using the identities of the eldest residents within (one of whom is mentioned in the obituary column): they would legally, but unknowingly, own the land. Jake, having matched one of the obituary names to one of the names in the list he stole from the plat book, has Evelyn drive him to the retirement home and pose as a married couple trying to find a place for Jake's father to live. The host tells them they can tour the facility. They come across an activities board with the names of the people from the plat book. Jake talks to a group of women working on a quilt. One of the pieces of fabric they've sewn into the quilt bears the emblem of the Albacore Club, the yacht club owned by Noah Cross. Jake is confronted by the host who has figured out Jake's ruse. The man takes him out to the lobby where Mulvihill is waiting. Jake tells Evelyn to bring the car around and then severely beats Mulvihill and barely escapes when the short thug who slashed his nose shows up.
Back at Evelyn's house, Gittes and Evelyn share a romantic interlude. As they lie on the bed afterward, Evelyn asks Jake about his past as a cop. He tells her he worked in Chinatown and was responsible for a woman "being hurt", possibly killed because of his actions. The phone rings and Evelyn has a cryptic conversation with someone, then informs Jake that she has to leave for a little while. She gravely asks him to trust her.
Gittes takes Hollis' car and follows Evelyn to a middle-class house and sees Mulwray's girlfriend crying. Evelyn claims the young woman is her sister, who was crying because she had just learned about Hollis' death in the newspaper. Later that night, Jake receives a call at home from a detective named Loach, Escobar's partner, telling him to meet him at a specific address. When he gets there he finds that Ida Sessions is murdered and Escobar and Loach are waiting for him. When Jake asks how they knew to call him Escobar shows Jake his phone number written near the phone. Escobar also points out that he knows the coroner's report proves that salt water was found in Mulwray's lungs even though the body was found in a freshwater reservoir, a fact that Jake had discovered earlier but withheld. He demands that Jake turn over any incriminating photos that may reveal Mulwray's murderer's identity. Escobar's chief suspect is Evelyn herself.
Under pressure from Escobar threatening to revoke his PI's license, Jake returns to Evelyn's mansion looking for her. Evelyn's Japanese gardener is working in the backyard and drops a minor comment about "salt water being bad for the grass". Jake has the man fish out the shiny object he'd noticed in the pool before: it's a pair of eyeglasses.
Gittes confronts Evelyn at the small house where she'd been keeping the young girl. Evelyn reveals that the blond girl, Katherine, is both her sister and her daughter, born from an incestuous relationship she had with her father years before. Gittes asks Evelyn if her father raped her and she shakes her head no. It remains unclear whether the act was consensual or not. It is apparent also that Evelyn resents her father for taking advantage of her in a relationship considered unnatural. Gittes then chooses to help Evelyn escape. Evelyn also states that the eyeglasses Jake found in her back yard pond could not have been her husband's because they are bifocals. Gittes arranges for the two women to flee to Mexico on a fishing boat owned by another of Jake's clients and instructs Evelyn to meet him at her butler's address in Chinatown. Evelyn leaves, and Cross arrives with Mulvihill under the pretext that Gittes has found the girl; however, Gittes confronts Cross with the accusation of murder and the glasses. Cross had Mulwray drowned in the saltwater pond at his own house and lost his own glasses in the pond during the act. Jake asks Cross about the water scandal; Cross tells him that he plans to create a community in the desert with an abundant fresh water supply. The real estate revenues from the sales of the land will generate many millions of dollars for him. Cross considers the plan a way of buying the future, essentially insuring that his family will reap the benefits from such a deal for many years. When Jake pointedly asks Cross about the relationship with his daughter, Cross confidently says "Most people never have to face the fact that, at the right time and the right place, they're capable of anything". Cross then orders Mulvihill to seize the glasses, the only physical evidence Jake has and forces Gittes to take him to the girl. When Gittes arrives at Evelyn's hiding place in Chinatown, the police are already there and arrest Gittes on conspiracy and withholding evidence. Jake vainly tries to explain Cross' plan to Escobar, who won't listen.
Evelyn appears with her daughter, trying to drive away in her car. When Cross approaches Katherine, demanding custody of her, Evelyn pushes him back, shoots him in the arm with a small pistol and starts her car. As Evelyn is driving away, the police open fire and Evelyn is shot and killed. Cross clutches Katherine, taking her out of the car, as a devastated Gittes is comforted by his associates, who urge him to walk away: "Forget it, Jake. It's Chinatown."
The plot is based in part on real events that formed the California Water Wars, in which William Mulholland acted on behalf of Los Angeles interests to secure water rights in the Owens Valley.