The film opens with Colonel Sam Trautman (Richard Crenna) returning to Thailand to once again enlist the help of Vietnam veteran John J. Rambo (Sylvester Stallone) whom has settled in the country since the events of First Blood Part 2. After witnessing Rambo's victory in a stick fighting match in Bangkok, Trautman visits the construction site of the Buddhist temple Rambo is helping to build and asks Rambo to join him on a mission to Afghanistan. The mission is meant to supply weapons, including FIM-92 Stinger missiles, to Afghan rebels, the Mujahideen, who are fighting the Soviets in the Soviet-Afghan War. Despite showing him photos of civilians suffering under Soviet military intervention, Rambo refuses and Trautman chooses to go on his own.
While in Afghanistan, Trautman's troops are ambushed by Soviet troops while passing through the mountains at night. Trautman is imprisoned in a Soviet base and coerced for information by Colonel Zaysen (Marc de Jonge) and his henchman Kourov (Randy Raney).
Rambo learns of the incident from embassy field officer Robert Griggs (Kurtwood Smith) and convinces Griggs to take him through an unofficial operation, despite Griggs' warning that the U.S. government will deny any knowledge of his actions if killed or caught. Rambo immediately flies to Peshawar, Pakistan where he meets up with Mousa Ghanin (Sasson Gabai), a weapons supplier who agrees to take him to Khost, a village deep in the Afghan desert, after Rambo threatens him, close to the Soviet base where Trautman is kept enslaved.
The Mujahideen in the village, led by the village's main leader Masoud (Spiros Focas), are already hesitant to help Rambo in the first place. When their village is attacked by Soviet helicopters after one of Mousa's shop assistants has informed the Soviets of Rambo's presence, they agree to help him. Rambo shows his fighting skills when he mans a 12.7 mm heavy machine gun and uses it to shoot down one Soviet helicopter. However, the village is mostly devastated.
Aided only by Mousa and a young boy named Hamid (Doudi Shoua), Rambo makes his way to the Soviet base and starts his plan to free Trautman. The first attempt is unsuccessful and results not only in Hamid getting shot in the leg, but also in Rambo himself getting wood fragments in the side. After escaping from the base, Rambo tends to Hamid's wounds and sends him and Mousa away to safety, before cauterising his own wound with gunpowder and an open flame.
The next day, Rambo returns to the base, after scaling a cliff on the fort's perimeter to avoid detection, just in time to rescue Trautman from being tortured with a flamethrower. After rescuing several other prisoners, Rambo steals a Mil Mi-24 helicopter and escapes from the base. However, the helicopter is damaged as it departs and soon crashes, forcing Rambo and Trautman to continue on foot.
After a confrontation in a cave, where Rambo and Trautman eliminate several Soviet Spetsnaz commandos, including Kourov, they are confronted by an entire army of Soviet tanks, headed by Zaysen. Zaysen warns them to drop their weapons and comply to his orders saying that they cannot escape and he wants to take them back alive. To which Trautman says to Rambo "what do you say, John?" and Rambo replies, loading his gun, "Fuck 'em". Just as they are about to be overwhelmed by the might of the Soviet Army, the Mujahideen warriors, together with Mousa and Hamid, ride onto the battlefield by the hundreds in a cavalry charge, overwhelming the Soviets. In the ensuing battle, in which both Trautman and John are wounded, Rambo somehow manages to kill Zaysen by driving a tank into the Soviet colonel's helicopter, which he first tried to damage with a Molotov cocktail, handed to him by a man on a horse. Somehow, Rambo survives the explosion and gets out of the tank.
At the end of the battle Rambo and Trautman say goodbye to their Mujahideen friends and leave Afghanistan to go home. The movie ends with two quotes: "This film is dedicated to the gallant people of Afghanistan." and "I am like a bullet, filled with lead and made to kill"
However, this was a not the original quote in the movie. Prior to the American war in Afghanistan, the ending quote of the movie read "This film is dedicated to the brave Mujahideen fighters of Afghanistan". This was then edited to read "people of Afghanistan" during the post 9/11 era.