The movie begins with an old man named Paul Edgecomb (Dabbs Greer) in a retirement center. He takes two pieces of dry toast from an orderly, who mentions Paul's habit for taking walks outside the ground. The orderly is worried about Paul, but allows him to continue with his daily routine.
Paul and several other residents are watching TV when an old movie with Fred Astaire dancing to the song "Heaven" is on. Paul sees it and walks away, followed by his friend Elaine (Eve Brent). Elaine realizes that the movie has awakened some powerful memories for Paul, and asks about it. Paul tells Elaine his story: that he was a prison guard during the Depression, in charge of Death Row, informally called "The Green Mile," because of its green tile floor. Paul's most powerful memory of this time took place in 1935....
The story then flashes back to the 1930's at the State Prison, where a young Paul (now played by Tom Hanks) is suffering from a urinary infection. Some of the other guards- Brutus "Brutal" Howell (David Morse), Dean Stanton (Barry Pepper) and Harry Terwilliger (Jeffery DuMunn) bring in a new inmate. His name is John Coffey (Michael Clarke Duncan) "like the drink, only not spelled the same." He is a gigantic muscular man, but when Paul talks to John they find that he has the mindset of a small child- very meek and apparently scared of the dark.
When John is brought in, another guard named Percy Wetmore (Doug Hutchison) is sent off Death Row to attend to work elsewhere. Percy is not happy about this, and in frustration he lashes out at another inmate named Eduard Delecroix (Michael Jeter), breaking Del's fingers. Paul is given a copy of John Coffey's records and finds that he was sent to Death Row after being convicted for the murder (and implied rape) of two small girls. John Coffey does not mention his crime, only stating that he "tried to take it back, but it was too late."
Later on, Paul is outside when he is met by Warden Hal Moores (James Cromwell). Hal gives Paul the execution papers for an inmate named Bitterbuck, and has a conversation with him about the young guard named Percy. It's revealed that Percy is the nephew of the governor's wife, and his powerful political connections are what got him hired- and keep him in the job, because Percy is apparently "stupid and mean" according to the other guards. Paul finds out that Percy has put in to be an administrator at a mental hospital, which would mean better pay and better hours. Paul theorizes that Percy wants to witness an execution up close before moving onto a new job. Warden Moores also mentions that his wife, Melinda, is not well she suffers from bad headaches and must have an X-Ray in order to find the source of the problem. That night, Paul meets with his wife Jan (Bonnie Hunt) and discusses the problem with Hal's wife.
Next day, Brutus spots a mouse in the cell block. They watch it run into a small room in the corner, which turns out to be a padded room for dangerous inmates but is currently used for storage. The guards check everything in the room but do not find the mouse. A few hours later, Percy spots the mouse and goes into a fury trying to kill it. Paul berates Percy for scaring the inmates in his pursuit of the mouse. Percy doesn't care, thinking the inmates are contemptible. Paul feels differently, believing that under enough strain the inmates would "snap" and cause serious problems. Brutus grabs Percy, but Percy threatens to use his connections to get the others fired if they hurt him.
We then see Paul and the others doing a rehearsal for the next execution, with the prison's elderly janitor, Toot-Toot (Harry Dean Stanton) helping them. Paul instructs Percy to watch and learn while the others prep the electric chair. That night, the execution of inmate Arlen Bitterbuck is carried out. Afterward, Paul confronts Percy about his new job opportunity. Percy reveals that he wants to "be out front" (meaning placed in charge of an inmate's execution) before he leaves.
Next day, the inmate named Del has found the mouse again, named it "Mr. Jingles" and is trying to tame it. The mouse is able to fetch a spool of thread as a trick. The other guards allow Del to keep Mr. Jingles as a pet.
Paul meets with Warden Hal again, getting word of a new inmate coming in, a man named William Wharton who killed three people in a holdup. Hal is almost in tears; the doctors have told him that his wife Melinda has a tumor the size of a lemon in her brain, virtually inoperable and eventually fatal. That night, Paul suffers from his urinary infection even more; he is almost in constant pain.
Paul intends to see the doctor next day after the new inmate is brought in. Percy and Harry go to retrieve Wharton from a hospital, where he is in an apparent trance presumably from medication. As soon as Wharton gets inside, he springs to life, attacking the guards and kneeing Paul in the groin. Dean is nearly strangled before anyone can get Wharton under control. When the others go to report what has happened, John Coffey asks to speak with Paul. When Paul approaches John's cell, John grabs Paul and puts his hand over Paul groin. John holds on for several seconds, until the lights flare brightly. John then lets go, coughing and gasping until he releases a cloud of gnat-like spores from his mouth. Paul asks what happened, but John can only say that "I helped." Later when Paul visits the washroom, he feels no pain at all. John Coffey's act has healed his infection.
The next morning, Paul goes into town to see John Coffey's public defender, Burt Hammersmith (Gary Sinise) who preceded over John's trial. Burt is absolutely convinced of Coffey's guilt. Back at the prison, Paul presents John with a loaf of cornbread baked by his wife, as a thanks for Coffey's "help". Coffey shares the cornbread with Del & Mr. Jingles, but does not give any to Wharton. This enrages Wharton, who takes his fury out on the guards, urinating on Harry. The guards use a fire hose to catch Wharton off guard, then wrap him up in a straitjacket and send him to the padded room. When Wharton spits on them later, he is sent to the room again.
The rehearsal for Eduard Delecroix's execution takes place the next day. Paul has decided to put Percy in charge, in the hope that he will finally leave the prison right afterward. When Percy walks by the cells later, he is grabbed by Wharton. Percy wets himself in terror, and threatens the men to never mention this. Paul states that "what happens on the Mile, stays on the Mile." They will not say anything about what happened. Del, however, delights in Percy being humiliated.
Later on, Mr. Jingles runs across the room between cells. Percy walks up and stomps on the mouse, coldly uncaring about what he has done leaving Del screaming in shock. John Coffey asks for the mouse, so Paul picks it up and hands it to John. The other guards watch in shock, awe, and possibly horror as light shines from John's hands. John coughs, releases another cloud of spores, and Mr. Jingles runs across the room- good as new.
Percy, seeing that the mouse is uninjured, is furious- thinking the guards have set out to make a fool out of him. Paul confronts Percy and gives him an ultimatum- Percy will transfer out immediately after Delecroix's execution, or the others will go public about Percy's record of mistreatment of the prisoners and his behavior on the Mile. Percy agrees.
Just before he "walks the Mile" to the electric chair, Del gives Mr. Jingles to Paul knowing that he will be taken care of. When Paul points out that he cannot have a mouse sitting on his shoulder during an execution, John Coffey volunteers to take care of Mr. Jingles.
Percy sets everything up for Del's execution, with one small exception- he does not properly soak the sponge required for proper electric conduction, wanting to punish Del one more time. As a result, the execution is excruciating for Del and the entire horrified audience- he rolls in pain, screaming and even catches fire before finally dying.
The guards confront Percy, but Paul tells them that Percy isn't worth fighting over and that he will still honor their agreement to leave.
Paul and his wife go to visit Hal and Melinda the next day. Hal reveals that Melinda is rapidly falling apart, she is losing her memory and experiencing severe behavior changes including uncontrollable cursing. Paul invites the other guards (minus Percy) to dinner later and discusses John Coffey's acts of healing both him and Mr. Jingles. Paul states that he wants to sneak John Coffey out to try and heal Melinda. The others are very skeptical, pointing out that Coffey is a convicted murderer, and it would be disastrous if they are found or if he escapes. Paul puts forth his belief that Coffey is innocent; Paul "does not see God putting a gift like that in the hands of a man who would kill a child."
The next day, they carry out the plan- Paul drugs Wharton so he will not see them leaving, then the others gag Percy and put him in the padded room as supposed "retribution" for Eduard Delecroix. They open up John Coffey's cell, and he is excited at the prospect of going for a ride outside and also seems to already know what they want him to do . John agrees to try and help Melinda. Wharton grabs Coffey as they head out, and John is apparently horrified by what he sees when touching Wharton.
They arrive at Hal's home, and Hal threatens them with a shotgun. Paul talks him down while Coffey goes upstairs to meet Melinda. John gets very close to Melinda's face and something comes out of her mouth and into his, making the light in the room shine intensely. John breaks the connection with her, falling down coughing. Melinda sits up, looking much healthier and having no memory of anything that happened before her X-Ray. Hal collapses, weeping at his wife's restoration. John continues to cough, unable to release the "spores" like before. Melinda gives Coffey a pendant with the mark of St. Christopher-the healer- as a present.
John returns to the prison, still very ill from the encounter. Percy is released, apparently keeping silent but the others still fear that he might talk. John grabs Percy, releasing the spores directly into Percy's mouth. Percy, in a daze, walks over to Wharton's cell and empties his revolver into Wharton's chest. The others seize Percy, who leans back and coughs up the remaining black spores. Upon examination, Percy appears to be catatonic. He is eventually sent to a mental hospital (ironically the same place where he was supposed to be an administrator) for presumably the rest of his life.
Coffey repeats that Wharton and Percy were "bad men," and places his hand on Paul. Paul sees that Wharton was responsible for the murder John Coffey was convicted for. Now that he knows Coffey is innocent, Paul is unsure how to proceed. He talks to his wife that night and he suggests talking to John about it. He even asks Coffey if they should just "let him go." Coffey does not want to escape; he reveals that in addition to healing he can also feel the pain of all others around him and does not wish to continue with such pain in the world. Paul offers John a last request; Coffey states that he has always wanted to see a "flicker show" (a motion picture). They bring in a movie projector with the film "Top Hat," the same movie that the elderly Paul was watching at the beginning of the movie, which is what triggered Paul's memories particularly when Fred Astaire is dancing to "Heaven" and John watches in awe saying "they like angels!"
That night, John Coffey is put to death as the guards watch on in tears. The elderly Paul's voice cuts in and states that he left The Green Mile soon after, unable to carry on after seeing John Coffey die. He and some of the others transferred to a youth corrections' facility.
Elaine admits that Paul's tale is "quite a story," and does not apparently believe it. She also points out that Paul mentioned his son being grown up in 1935, which means he should be much older than he appears.
Paul takes Elaine on a walk, and they come to a cabin in the woods. There is a mouse sleeping in a small box; Elaine is shocked to meet Mr. Jingles- Paul found the mouse again after Coffey's execution and has kept him ever since. Paul states that he is now 108 years old, and that he believes John Coffey "infected him [and Mr. Jingles] with life." Paul feels that this is his punishment for killing a genuine miracle of God- he must stay alive and watch everyone he cares about, including friends like Elaine, grow old and die before his own death.
Later, Paul is seen at Elaine's funeral, quietly wondering just how much longer he has to go. "We each owe a death," he states, "There are no exceptions. But oh God, sometimes The Green Mile seems so long."