Rocky II begins with the 14th round of the Bicentennial fight from the first movie. The fight lasts the entire 15 rounds with each fighter suffering many devastating injuries. After the fight, Rocky calls out for Adrian, who runs down to the ring. As Creed is announced the winner by split decision, Adrian and Rocky embrace, and Rocky tells her that he loves her.

After their championship bout, Rocky and Apollo are taken by ambulance to Pennsylvania Hospital in Center City (a WCAU-TV Channel 10 news truck follows behind the ambulance), where they meet once again in the hospital lobby with a mob of reporters. Apollo Creed, who retained the championship on points, is furious at Rocky, the first man to go the distance with him, and starts clamoring for another chance to knock Rocky out. Rocky goes to Apollo's room that night and asks "Did you give me your best?" to which Apollo answers "Yeah". Rocky declines talk of a rematch and tells Apollo that he is quitting boxing. Rocky's decision to retire is strengthened when doctors tell him he needs surgery for a detached retina.

Following his hospital stay, Rocky proposes to Adrian in front of the tiger exhibit at the Philadelphia Zoo, and they later on get married. Rocky begins to enjoy the profits from his fight with Apollo. He buys his first brand new car which is a 1979 Pontiac Trans Am (even though it's still supposedly 1976), goes on a shopping spree (purchasing a black leather jacket with a tiger in the back, a fur coat for Adrian, and some watches, to name a few), and buys a two-story house. He also convinces his previous employer, Gazzo, to give Paulie his old collecting job. Rocky then opts to earn money by appearing in television commercials. His new career, however, is quickly derailed by his sub-par reading skills, which leads to a clash with the director of his first commercial.

Unable to find white-collar work due to his lack of education beyond the 8th grade, and after being laid off (because of cutbacks) from the meat packing plant where Paulie previously worked, he gets a job "helping out" at Mickey's gym. Adrian, now pregnant, returns to her old job at the pet store. He also hands over his new Trans Am to Paulie, who agrees to take over the payments.

Meanwhile, Apollo Creed has been receiving hate mail about his split-decision win over Rocky and is filled with anger and frustration: the letters alternately accuse him of deliberately carrying "that chump" (Rocky) the distance or state outright that he should have lost and has no business calling himself the champ. Determined to prove himself and win back the respect of his fans, Apollo elects to try and pressure Rocky into another fight, despite protests from his trainer, Tony "Duke" Evers, to find a new contender to fight against (when asked by Apollo what Tony is afraid of [regarding Rocky], Tony replies "He's all wrong for us. I saw you beat that man like I ain't never seen no man get beat before. And the man kept coming after you!"). Rocky himself begins to have second thoughts about the fight, but because of the pressure applied by Apollo, Rocky decides to accept the challenge for a rematch against Apollo on Thanksgiving 1976 and begins training.

Mickey, initially unwilling to train Rocky due to his bad eye, decides he wants Rocky to learn to fight right-handed in order to better protect the eye, and to give him a surprise advantage by switching back to left-handed with the intention of throwing off Apollo's timing and confusing him. He also focuses on improving Rocky's speed by using an old training method from his own boxing days: he forces Rocky to chase a chicken around a vacant lot, stating, "If you can catch this thing you can catch greased lightning."

However, Adrian's opposition saps his resolve and his training suffers. When Paulie confronts Adrian about her opinion, her angry reaction creates a serious medical condition that breaks her water prematurely. During the birth of their son, Adrian falls into a coma, and as a result of the premature birth, their son is placed into an incubator. Rocky breaks away from training to spend every waking moment either at her side or in the hospital chapel with Mickey, not even seeing his newborn son until she recovers. When Adrian emerges from the coma, she tells Rocky to "win" and he restarts training in earnest with Mickey for the rematch with Apollo. His condition improves dramatically with increased strength and speed.

As with the first fight, Rocky's training includes his famous brisk run throughout the city of Philadelphia. This time, however, he is greeted with the cheers and encouragement of onlookers as he runs through the streets of South Philadelphia, having become a local celebrity following his narrow loss to Apollo. Along the way, children and pre-teens begin to follow him, their numbers increasing dramatically block by block. By the time he reaches the Benjamin Franklin Parkway in Center City, hundreds of young people have joined in. Ultimately, they follow Rocky to the top of the steps of the Philadelphia Museum of Art, enthusiastically chanting his name as he lifts his arms in a victory pose.

Rocky and Apollo once again fight an intense battle in front of a standing-room-only crowd at the Philadelphia Spectrum. Unlike the first match, Apollo gets off to a fast start, flooring Rocky twice in the first two rounds and building a huge lead on the scorecards as the fight goes on. But throughout the fight, Rocky pounds away at Apollo on and off, turning the fight into a brutal battle. At the start of the fifteenth round, Apollo's trainer begs him not to try to go for a knockout, as he can win simply by avoiding Rocky. Apollo, desperately wanting to prove he can knock out Rocky and very frustrated that he keeps coming back, disregards the advice. Mickey tells Rocky to switch back to boxing left-handed now, and although Rocky initially refuses, he switches back, landing several crashing lefts. Rocky hits Apollo with a final punch that knocks him down, but from sheer exhaustion, Rocky also goes down to the canvas. The referee starts the final ten-count as the fighters struggle to their feet, in a heart-pounding sequence.

As he finally realized all but too late that the last fight was not a fluke and with no more cards left to play, a beaten and exhausted Apollo slumps to a corner. Rocky grabs the top rope and picks himself up at the count of 9, and is declared the WBA's new Heavyweight Champion of the World. In his post-match speech, he thanks Apollo for another hard-fought battle, Mickey for training him, and God. Rocky states "Except for my kid being born, this is the greatest night in the history of my life, and I just want to say one thing to my wife who's home: "Yo, Adrian! I did it!"