The story opens with "Eye of the Tiger" during the flashback of Rocky's rematch against Clubber Lang, where Rocky defeated Lang with a KO in the third round to regain his title. The picture then fades to Apollo Creed who is seen presenting his favor to Rocky shortly after the Lang fight for helping him train. Meanwhile, Ivan Drago (Dolph Lundgren), a highly intimidating 7 foot 1, 261 pound Soviet boxer, arrives in America with his wife Ludmilla (Brigitte Nielsen), an Olympic gold medal swimmer, his manager, Nicolai Koloff (Michael Pataki), and a team of trainers headed by grizzled Russian coach Igor Rimsky (George Rogan), and the Cuban Manuel Vega (James "Cannonball" Green) to challenge the best American fighters. His manager takes great pride in showing off the hi-tech equipment which aids in improving Drago's performance. As a demonstration, Drago throws punches at a machine that measures the hit's strength. Motivated by patriotism and an innate desire to prove himself, Apollo is desperate to step back into the ring in an exhibition bout against Drago. A press conference is held to publicise the bout. It begins on affable terms but the mood quickly changes when Apollo is accused of being a "has been" by Drago's manager, who suggests Creed shouldn't even be in the same ring with Drago. Livid with such a statement, Apollo tells Drago they'll "finish this in the ring" before abruptly leaving the conference. Drago appears completely unfazed by this.

Rocky has reservations, but comes round to supporting his friend by helping to train him for the fight. Apollo sets the match between himself and Drago in Las Vegas. With Rocky in his corner, Apollo flamboyantly makes an even bigger show than when he first fought Rocky - including fireworks and a patriotic theme complete with a performance by James Brown, dancing girls and Apollo himself dancing around. Starting the fight in his trademark manner, Drago manages to catch him off-guard quickly and batters Apollo with a series of devastating punches. Apollo is in dire straits as the first round is over. The commentators and audience are visibly shaken by what they've seen, all except Ludmilla, who - smoking a cigerette - appears to be relishing it. Rocky and his trainer Duke are pleading with him to stop the fight. Apollo refuses to do so, and tells Rocky not to stop the fight no matter what. The second round starts just as the first ended. Rocky attempts to throw in the towel but despite Duke's begging, Rocky honors Apollo's wish. It turns out to be a tragic decision, as Drago finally hits a devastating hook that perishes the former champion. Drago displays no sense of remorse, commenting: if he dies .... he dies.

Devastated by the death of Apollo Creed, Rocky comes to the decision he must avenge his death by agreeing to fight Drago in his home country, on Christmas Day. Adrian pleads with him not to as she feels he can't possibly win. Supported by Apollo's manager Duke and his brother-in-law Paulie, he flies to the Soviet Union to train. Unlike Drago, who is attached to electrodes, is constantly monitored by computers, and works out with ultra hi-tech equipment (as well as being injected with what are implied to be anabolic steroids), Rocky trains by carrying logs, chopping down trees, lifting huge weights on the ends of chains, running through the deep snow, and pulling a sleigh containing Duke and Paulie. When Adrian shows up unexpectedly to give him her unconditional support, Rocky's training becomes more focused and energised than ever before. After intense preparation for both fighters, the two men finally meet in the ring.

Much like Apollo did in the previous fight, the Soviets introduce Drago with an elaborate, patriotic ceremony that puts the attending audience squarely on the side of Drago, leaving Rocky to be fiercely booed. In contrast to his fight with Apollo, Drago immediately goes on the offensive, repeatedly pounding Rocky, knocking him 15 feet across the ring on one occasion, and casually shrugging off his punches. After a pulverizing first round, with Drago easily winning, Rocky comes back toward the end of the second and lands a shot that cuts Drago just below his eye. While Drago is visibly shaken by the reminder of his own mortality, Balboa is fired up, and defiantly stands up to Drago, pummeling him until, after the bell rings numerous times, Balboa is visibly pulled off his opponent. Drago punches Rocky in revenge, and Balboa throws Drago to the ground, their managers splitting up the fight. While Duke and Paulie cheer Rocky for his heroism, they remind him that Drago is not a machine, but a man. Meanwhile, Drago's own corner reprimand him for being "weak" in comparison to the "small American." Drago replies in his native tongue: "He's (Balboa) not human. He's like a piece of iron".

At this point, the fight becomes a fierce battle of wills between the two boxers. Drago's confidence drops round after round due to Rocky's seemingly limitless endurance, allowing Rocky to get in under his guard and pound him relentlessly. By the fourteenth round, the crowd has been won over by Rocky's determination and is cheering him on. Koloff, fearing retribution from the Soviet Premier, goes over to Drago and berates his performance, telling him to win. Drago's response is to pick up Koloff by the throat, and proclaim that he only fights for himself.

In the 15th and final round of the fight, Rocky and Drago trade punch after punch. Eventually, Balboa manages to overcome Drago knocking him out, to the shock of Soviet premier (who strongly resembles the then Soviet leader Mikhail Gorbachev) and his aides who have no option but to applaud Rocky for fear of backlash. Following his victory, Rocky gives an impassioned speech to the crowd, acknowledging their initial and mutual disdain for each other, how they've come to respect and admire each other during the fight and how everybody can "change". It has been speculated this speech was a precursor of glasnost.