Glenn Holland, not a morning person by anyone's standards, is woken up by his wife Iris early one bright September morning in 1964. Glenn has taken a job as a music teacher at the newly renamed John F. Kennedy High School. He intends his job to be a sabbatical from being a touring musician, during which he hopes to have more "free time" to compose. However, he soon finds that his job as a teacher is more time-consuming than he first thought.
As he arrives at the school for the first time, he meets Vice Principal Wolters, who comments on his Corvair, the model of car the Ralph Nader wrote a book about. Inside the building, he meets Principal Helen Jacobs. Having got off to an awkward start with both of them, he goes to the music room and meets his students for the first time. The students are dull, apathetic, and mostly terrible musicians. At lunchtime, he meets the football coach, Bill Meister, and strikes up a friendship with him. At the end of his stressful first day, Glenn and Iris talk about their future. If everything goes according to plan, between his paychecks and what she made with her photography, he should be able to quit in four years and go back to his music, including composing.
Glenn notices one dedicated but inept clarinet player, Gertrude Lang, and starts working with her individually. He continues attempting to teach the class about music and continues working on his music at home as time passes. Grading papers gradually replaces working on his own music during his home time, much to his chagrin. After several months, Glenn grows exasperated when it seems that none of his students have learned anything from his classes. Gertrude, despite diligent practice, does not improve her clarinet-playing. Glenn's exasperation is further compounded when the Principal Jacobs chastises him for not focusing properly on his students. She has noticed that he is even happier to leave at the end of each day than most of the students. Later, Glenn expounds his frustration to Iris, who then informs him that she's pregnant. Glenn is dumbstruck, and his muteness upsets Iris. To comfort her, he tells her a story about how he discovered John Coltrane (his favorite musician) records as a teenager, the point being he could get used to this turn of affairs.
After some soul-searching, Glenn decides to try some unconventional methods of teaching music appreciation, including the use of 'Rock and Roll' to interest students, demonstrating to them the similarities between Bach's "Minuet in G" and rock-and-roll in the form of the Toys' "Lovers Concerto". For the first time, the students are interested in the class, and Glenn appears much happier as he relates this to Iris as they assemble a crib. Their apartment is getting more and more crowded, and Glenn suggests that they get a house. Iris is overjoyed, even though it means using their savings and Glenn sacrificing his summer vacation, which he intended to use to work on his composing, in order to make extra money teaching Driver's Ed. Glenn does right by his family but he knows he can forget about getting out of the teaching gig for the foreseeable future.
Continuing his new, unorthodox teaching methods, he finally gets Gertrude, who was on the verge of giving up, to have a breakthrough and become a more skilled clarinet player. She rediscovers her joy of playing, and the now-competent band go on to play at the 1965 graduation. Summer vacation begins, and Glenn follows through on his plan to teach Driver's Ed, having a series of near-death experiences at the hands of new drivers. Glenn and Iris move into their new house. Soon, we see the Driver's Ed car once again, except this time it is Glenn himself driving like a maniac, breaking every traffic law - so that he could get to the hospital to see his newborn son, Coltrane ("Cole") Holland.
Glenn's unorthodox teaching methods do not go unnoticed by Principal Jacobs, or Vice-Principal Wolters. They, along with the conservative School Board and the parents of the community, are hostile to rock-and-roll. Glenn is able to convince the principal that he believes strongly that teaching the students about all music, including rock-and-roll, will help them appreciate it all the more. The principal and vice principal also hand him a new assignment, to get a marching band together for the football team. Glenn is at a loss with this concept, until his Bill Meister agrees to help, in exchange for Glenn putting one of his football players, Louis Russ, in the band to allow him to earn an extra curricular activity credit, which he needs in order to stay eligible for the sports teams. Louis knows absolutely nothing about music, but takes up drums. He has trouble keeping time and always finds himself out of place with the rest of the band.
Later, Glenn and Bill are chatting while playing chess. Bill, a bachelor, wants to know about Glenn's stories of debauchery as a traveling musician, but Glenn doesn't want to talk about the past, as he is a different person in a different time. Glenn instead tells Bill he is pessimistic about Louis Russ. Bill encourages him to keep trying. Much as he worked with Gertrude earlier, Glenn starts working one-on-one with Louis, helping to get a feel for the tempo of music. After some hard work, Louis gets it, and later, he marches with the band in the local parade, much to the delight of his family.
Immediately behind the Kennedy band in the parade is a fire engine, and its deafeningly loud horn catches everyone by surprise. Iris looks into Cole's stroller to check on him, but the noise hasn't awakened Cole - the boy is deaf. The revelation drives a wedge between Glenn and his son, as it seems that his son cannot understand what he does. A more somber Glenn teaches his students about Beethoven, the deaf composer.
Time passes, and we see a montage of events from the late '60s, as Glenn picks away as his composition a little at a time and watches Iris work with Cole. We stop again in the early '70s, with Glenn still directing his high school band. Cole is old enough to enter school. Because of her mounting frustration with her inability to communicate with Cole, she insists on sending him to a special school for the deaf, whatever the cost. The three of them visit the school. Glenn winces at the cost, but they enroll Cole and set about to learn sign language themselves, though Iris puts more effort into it than Glenn.
Apathetic students still go through Glenn's classes, and one of them, named Stadler, is stoned. Glenn is chewing him out when Glenn receives bad news. He tells Stadler to meet him on Saturday. On that day, they appear at a funeral. Louis Russ has come home from Vietnam after being killed in action. Coach Meister is there, and he and Glenn mourn. At the end of that academic year, Bill reveals that he finally has a steady girl-friend, and Principal Jacobs retires from the high school, praising Glenn for what he has done.
We see another montage of events, this time in the 1970s. Glenn continues teaching Driver's Ed in the summer. We see the class of 1980 being welcomed back, suggesting that it is now September 1979. Glenn and Bill Meister team up to help the Drama Department, when it is rumored that funding may be pulled. Glenn and Bill tell Wolters, now the principal, have an idea to be certain the school will make money rather than lose it; it will be a musical revue of Gershwin classics. During auditions for the musical revue, Glenn becomes entranced and interested in a talented young singer named Rowena Morgan. At home, the teenage Cole comes home and tells Glenn about the science fair, which Glenn missed. Iris is fluent at sign language now, but Glenn is still only fair. Iris reproaches him for spending so much time with the school projects and the students while neglecting his own son. Glenn is frustrated, realizing that his own musical composing has been on the back burner for 15 years now.
Rowena visits Glenn at a diner, where he has gotten into the habit of going to get out of the house and have someplace quiet to work. Unknown to Iris, Glenn writes a small piece that he titles "Rowena's Theme," and takes an interest when Rowena states that she wants to leave town and go to New York to sing professionally. Glenn's life at home is still strained. Iris agrees to come to the school play on Saturday, because she had a meeting with Cole's teachers on opening night (Friday).
The school revue arrives at last, and is a big hit, playing to a packed house. In the audience we see Coach Meister wearing a ring (he married the woman we saw earlier), and Sarah, the drama teacher, shows Principal Wolters something on a new invention, a handheld calculator (presumably, showing him how much gate money made it into the school coffers, as Wolters looks impressed). After the revue, Rowena comes to see Glenn in the auditorium, and she tells him she intends to pursue her dream of singing by going to New York the very next night, after the second and last performance of the revue. Glenn is taken aback. Rowena hints that she'd like Glenn to come with her. Glenn goes home and looks at his photo album, looking at pictures of his family and pictures of his old life as a traveling musician, now half a lifetime in the past. He is tempted to leave everything behind and go with Rowena to restore his old life as a musician. However, he realizes he is no longer the same person as he was then. He visits Rowena at the bus stop and sees her off, giving her the names of someone in New York who will help her find lodging. Glenn watches her depart, and goes home, content in his love for Iris.
The timeline then shifts to late 1980, when John Lennon is killed. Glenn goes home and finds Cole working on Glenn's old Corvair. When Cole asks what is wrong, Glenn tries to explain, but then gives up, feeling that his son wouldn't understand John Lennon or his music. This infuriates Cole who (through Iris), explains that he does care about Glenn and knows about John Lennon, but that Glenn does not seem to be at all interested in communicating with him. Cole berates his father for putting so much effort in to teaching his students and very little towards him, calling him an asshole in sign language as he stalks off. Glenn then makes an effort, and even provides a concert at the high school, which also features lights and other items to enhance the show for deaf members of the school where Cole attends. Glenn, having become somewhat more proficient in sign language, even does an interpretation of Lennon's song 'Beautiful Boy,' dedicated to Cole. Later, Glenn discovers Cole listening to records by sitting on the speakers and feeling the vibrations through his body, and they can start healing the rift between them, even as Glenn's composition continues to gather dust.
Time passes. It's now 1995. Glenn goes to see Principal Wolters, who announces that Art, Music and Drama have been cut from the school curriculum, and Glenn would be out of a job shortly. Glenn, who has become a cynical old man, tells Wolters that to cut the fine arts would lead to a generation of students who would be proficient at reading and writing and math (maybe) but would have nothing to read or write about. Wolters offers to write Glenn a reference, but Glenn, who is now 60 years old, fully recognizes the futility of the gesture. His working days are over and he knows it. Then he looks up at the picture on the wall of the long-departed former Principal Jacobs. He says Jacobs would have fought the budget cuts, and he will too. Glenn pleads to the school board to reconsider, but they refuse.
At home, Iris reads a letter from the now-grown Cole. He has become a teacher himself, and was considering an offer from a university for the deaf in Washington, D.C. He also has taken Glenn's old car, the Corvair that we saw at the beginning and that Cole was working on in his teens, and jokingly writes that he will never give it back. Despondent, Glenn walks through the school on his last day, and he talks to Coach Bill, whose job as football coach is safe, though he can't be far from retirement himself. Glenn figures that he will bring in some money teaching piano lessons on the side, but he's unprepared to be forced into early retirement.
On Glenn's final day at the school, Cole shows up driving the Corvair. School's out for him, too. Glenn is surprised when Iris and Cole lead him to the school auditorium, where they have organized a surprise going-away celebration for him. He sees many of his former students in the audience, including Stadler, the pothead from years before. Arriving next is Gertrude Lang, the clarinetist who Glenn helped in the 60s, who has since become the state's governor. Gertrude thanks Glenn for his dedication, and Glenn is very moved. He is moved to tears when she gives him a baton and asks him to conduct his own composition, which she had got hold of. The curtains open and a band, filled with more of Glenn's former students, is assembled and ready to play. Governor Lang picks up her clarinet and takes her place among them, and they play, for the first time, the musical Opus that Glenn had been picking away at for three decades.