"Holocaust" is the story of two Berlin families from 1935-1945. Dr. Josef Weiss is a well-to-do Jewish physician. His wife Berta is an accomplished pianist. They have three children; Karl, Rudi and Anna. Erik Dorf is a young lawyer who can't find a job until his wife Marta persuades him to join the SS.
The movie opens with the wedding celebration of Karl Weiss and his Christian bride, Inga Helms. Her parents and brother Hans are not happy that she is marrying a Jew. Neither is Heinz Muller, a family friend who is in love 'with Inga. Berta feels that the working-class Inga is not good enough for Karl. Josef's speech about the families now being joined does little to ease the strain. While many Jews are leaving Germany, the Weiss family decides to stay.
Marta Dorf suffers from a heart murmur and Erik brings her to Dr. Weiss for treatment. Josef discovers that he once knew Erik's father, the baker. He prescribes rest and a healthy diet. Once again Marta urges Erik to speak to her uncle about joining the SS. This time he agrees and is hired. When he puts on his uniform for the first time, his small son Peter runs away in fear.
Three years later, Erik again visits Dr. Weiss; this time in his official capacity. He warns Josef to stop treating Christians or face the consequences. When Josef protests, Erik asks why he and his family have not already left Germany as life will not get any easier for them. As he departs, he tells Josef not to come to him if he gets in trouble.
Berta still does not want to leave and in any case it is too late. Most countries will no longer admit Jewish refugees. Then Karl is arrested and sent to a concentration camp. Against Erik's orders, Josef visits him at home and begs him to help free Karl. Marta coldly tells Erik to send him away. Shortly after this incident, Josef is deported to Poland, the country of his birth.
Berta wants to accompany her husband but he reminds her that she must take care of Rudi and Anna. After an emotional farewell at the train station, Berta prepares to move with the children and her elderly parents into Inga's flat. The Germans have taken over Josef's medical practice and the adjoining flat. Before leaving, Berta and Anna play one final piece together on their Bechstein piano. Rudi goes to fetch his grandparents only to discover that they have committed suicide together.
In Warsaw, Josef is taken in by his brother Moses, a pharmacist. They have had little contact over the years. Josef gets a job in the hospital and is made a member of the Jewish Council. Gradually Warsaw becomes a ghetto and the people are starving. Smugglers, some as young as twelve, bring food and news from the outside whenever possible.
Inga moves next door into her parents' flat to give Berta, Anna, and Rudi more room. Berta is still optimistic, writing letters to Josef that he will probably never receive. She tutors a rebellious Anna, reminding her that she can still have a good life. But Rudi has no intention of waiting for the Gestapo to come for him. He leaves Berlin on foot in the middle of the night. Inga sees him before he leaves and gives him some money.
A grieving Anna decides to run away too. Inga tries to stop her, reminding her that Rudi is older and can take care of himself. But Anna rips the gold star from her coat and runs out into the night. She is accosted by several drunken soldiers. They take turns raping her. Eventually she finds her way back to the flat but over time, deteriorates mentally.
Erik and Marta are living a good life. He has been promoted and they enjoy many perks. At a party, he runs into his uncle Kurt Dorf, an engineer who has a contract with the Nazis to build roads. Kurt spends Christmas with the family and Marta receives a Bechstein grand piano. Erik says it was sitting unused in a medical clinic formerly owned by a Jewish doctor. He is evasive when Kurt asks what happened to the doctor. Inside the piano are pictures of Dr. Weiss and his family. Marta is startled as she recognizes the doctor and Erik orders their daughter Laura to burn the pictures.
Several months after the attack, Anna is in a catatonic state. Berta and Inga try to obtain medical help for her. Inga takes her to a doctor who, though a party member, was a good friend of Josef's. He recommends that Anna be sent to Hadamar Sanatorium where she will be given therapy. Inga has misgivings but realizes that Anna cannot go on the way she is now. But when Anna arrives at Hadamar, she and other new patients are taken to a shed and murdered by carbon monoxide poisoning. Later, Berta receives a form letter stating that her daughter died of pneumonia. She is overcome by grief and guilt for sending Anna away.
In Prague, Rudi meets Helena. She owns a luggage shop and saves him from being questioned by soldiers. When he learns she is alone in the world, he invites her to accompany him on his travels. While attempting to steal loaves of bread, they are captured by the Germans and put on a truck. But an explosion enables them to escape and run for cover. They find Inga's brother Hans wounded. Rudi learns that Hans was recently home on leave and asks for news of his family. When Hans reveals Anna's death, Rudi accuses him of murdering her. Frightened, Hans offers to get them passes so they can travel safely. Helena urges Rudi to trust Hans, but when soldiers discover them, Hans denounces them as Jews. Once more Helena and Rudi are prisoners. Later they are able to escape. Hans is killed during another explosion.
Inga has discovered where Karl is being held. She travels to the camp and discovers that Heinz Muller, her parent's friend, is in charge. He offers to pass Karl a letter from her if she has sex with him. Disgusted but desperate to communicate with her husband, Inga agrees. This arrangement continues for some time but eventually Muller taunts Karl with it.
Berta is sent to Poland to join her husband. She tells him of Anna's death, leaving out the part where their daughter was raped. He is told only that Anna died at home of pneumonia. Even though they don't know where Rudi is and Karl is a prisoner, Berta still feels the family will one day reunite. She begins teaching school in the ghetto, but conditions are growing worse each day.
Erik has made enemies and a rumor circulates that he has Jewish ancestors. Although he denies it, he is subjected to an investigation. He and Marta rarely see each other now. Their son Peter is a member of Hitler Youth. Kurt is aware of what is happening to the Jews and his disgust with Erik grows. Watching executions and gassings is affecting Erik's mental health. At home, Marta finds copies of letters detailing these actions. Far from being upset, she reminds Erik that he was only following orders. Their relationship deteriorates to the point that he is no longer able to make love to her.
The Warsaw ghetto is rapidly emptying. Every day Josef must select a certain number of Jews to be sent to a resettlement camp. When he learns that they are really being gassed, he tries to save as many as he can by opening a railway station clinic. He pulls various people off the transport by saying they have symptoms of typhus. This only works for a short time. He and Berta are placed on the transport and sent to Auschwitz. Moses Weiss stays behind but is among those who were executed after the Warsaw uprising.
Karl has been transferred to Theresienstadt and put to work in the artist studio. He and his fellow artists make drawings of starving children and dead bodies, indicating what really happens in the concentration camps. These drawings are kept hidden from the guards. But then one of the artists swaps several drawings for cigarettes. The studio is destroyed and Karl, together with two other men, is beaten and tortured for information. Erik Dorf, now a Major, questions Karl. He reminds Karl that they are both Berliners and even asks if they have met in the past, as Karl reminds him of someone. But Karl refuses to tell where the rest of the drawings are hidden. His hands are broken so that he can never draw again. The other two men die of their injuries.
Inga can no longer bear life without her husband. She persuades Muller to turn her in on false charges so she can join Karl at Theresienstadt. When she arrives, Karl doesn't want anything to do with her because of her affair with Muller. Inga assures him that she hated every minute of it but it was the only way she could get letters to him. They are reconciled and soon Inga is pregnant. Karl begs her to have an abortion rather than let their child be born in such a cursed place. But the female artists promises Karl that they will take care of Inga and she will be fine. Before the child is born, Karl is sent to Auschwitz.
Rudi and Helena join a group of partisans and a rabbi among them performs a marriage ceremony. During an attack, Helena is shot and dies instantly. Rudi is captured and sent to Sobibor. There is a plan to overpower the guards and steal dynamite and guns. Rudi participates and is able to escape. The war is nearly over and he decides to try and find his family.
Josef and Berta are separated upon arrival at Auschwitz. He is put on a road crew with a former patient, Max Lowy. But he isn't physically able to keep up and the engineer in charge, Kurt Dorf, allows him to rest. Erik has received reports that his uncle is taking Jews from the camp to work on the roads, which is forbidden. He orders that it be stopped at once as the Jewish prisoners are marked for special handling. In contempt, Kurt says he knows what that is.
Using forged passes, Josef and Max go to the women's barracks to visit their wives. Berta has brought her music case and pictures of her children. Josef is concerned that it makes her too sad to see the pictures, but Berta says it gives her strength. When the men return the next day, the room is empty. The occupants were gassed the previous day. Soon afterward, Josef and Max are marched from the road crew to the gas chambers.
Karl dies the night before Auschwitz is liberated. Somehow he has managed to make one final drawing, which is eventually sent to Inga.
Erik goes into hiding but is captured by the Americans. During his interrogation, he repeats that he was merely a soldier following orders. When he is left alone for a few minutes, he swallows a cyanide pill and dies instantly. Kurt visits Marta and the children, who insist that Erik was a hero. When Kurt says that Erik was a mass murderer, he is ordered to leave and never return. Sadly he bids them goodbye. After he is gone, Marta and the children light a candle for Erik.
A celebration is in progress when Rudi arrives at Theresienstadt . He meets Inga and learns the fate of his family. Inga proudly shows off her baby son, whom she has named Josef. She and Rudi look at the drawings, which Inga and others had buried to keep safe. Karl's last drawing is among them. Inga plans to return to Berlin but only temporarily. She invites Rudi to come along but he refuses. He notices a soccer game in progress and joins in, giving the boys some tips. A man approaches him and offers him a job taking the boys, who are orphans, to Palestine. Rudi accepts and his new life begins.