The film picks up almost directly after the end of The Karate Kid; John Kreese (Martin Kove), furious over his star pupil Johnny Lawrence's (William Zabka) second place finish in the All Valley Karate Tournament, viciously berates and humiliates Johnny in the parking lot, nearly killing him by putting him in a headlock. Miyagi rescues Johnny, passively immobilizes Kreese, then comically tweaks Kreese's nose instead of dealing him a fatal blow. Horrified by Kreese's behavior, Johnny and his friends quit the Cobra Kai dojo.
Six months later, Daniel is upset that his girlfriend Ali has left him for a football player from UCLA. To make matters worse, he learns that he and his mother are moving to Fresno for the summer. Miyagi surprises Daniel by telling him that he has made arrangements with Mrs. Larusso to have Daniel live with him for summer. Miyagi then receives a letter telling him his father is dying. He intends to return to Okinawa alone, but Daniel decides to accompany him. When Daniel asks Miyagi why he had left Okinawa in the first place, Miyagi answers that he loved a woman named Yukie, who was arranged to be married to Sato, son of the richest man in town, and Miyagi's best friend. Sato and Miyagi had studied karate together under Miyagi's father, in defiance of what was then the strict one-to-one father-to-son tradition of karate. One day, Miyagi had announced before the whole town that he wanted to marry Yukie. Sato had been insulted and had challenged Miyagi to a fight to the death. Rather than fight his best friend, Miyagi left Okinawa.
When they arrive in Okinawa, Miyagi and Daniel are greeted by a young man, Chozen Toguchi (Yuji Okumoto), who is Sato's (Daniel Kamekona) nephew. Sato has not forgotten his feud with Miyagi and demands to fight Miyagi. Again, Miyagi refuses, so Sato calls him a coward.
Miyagi and Daniel are welcomed to Tome village by Yukie (Nobu McCarthy) and her niece Kumiko (Tamlyn Tomita), to whom Daniel is attracted. Sato has become a rich industrialist whose supertrawlers have destroyed the local fish population, impoverishing the other villagers, who have turned to small farming to survive. The villagers must rent their property from Sato, who owns the entire village. Yukie reveals that because she loved Miyagi, she never married Sato.
Despite Miyagi's father's dying wish for his son and student to make peace with each other, Sato insists on fighting Miyagi, though, after his sensei's passing, he gives Miyagi three days to mourn. Miyagi shows Daniel that the secret to his family's karate lies in a handheld drum that beats itself when twisted back and forth. This "drum technique" represents the block-and-defense that Daniel begins to practice. Later, Yukie and Miyagi perform the tea ceremony together, which, Kumiko explains to Daniel, is a sign that they are renewing their love.
Daniel inadvertently reveals that the grocery business of Chozen and his cronies has been defrauding the villagers with rigged weights. The outraged farmers demand compensation. Chozen accuses Daniel of insulting his honor and being a coward. He and Daniel have a series of confrontations. Chozen attempts to humiliate Daniel by demanding that he chop through six blocks of ice, a seemingly impossible feat. Mr. Miyagi takes Chozen up on his bet at a dollar amount which Chozen cannot cover, but which Sato agrees to cover for Chozen. Daniel fulfills the challenge successfully, and Sato honours the terms of the wager despite his grudge.
Meanwhile, Daniel and Kumiko grow closer. She brings him to an old castle on the seacoast that Sato is allowing to deteriorate and be plundered.
The feud between Daniel and Chozen comes to a head when Sato, at the conclusion of the three-day mourning period, shows up to fight Miyagi. Because Miyagi is not present, Chozen and his cronies destroy the Miyagi family dojo and much of the garden, then Chozen attacks Daniel. When Miyagi arrives, Chozen, Taro, and Toshio attack him, but Miyagi defeats them easily. Realizing that he has put Daniel in danger, Miyagi plans to return home.
Before they can leave Okinawa, however, Sato shows up with earth-mover machines and threatens to destroy and redevelop the village if Miyagi continues to refuse to fight. Miyagi gives in on the condition that no matter who wins, Sato must sign the titles to the villagers's homes back over to them. Sato agrees. On the day the fight is to take place, Daniel and Kumiko, like Yukie and Miyagi, perform the ancient tea ceremony, ending with a kiss. Meanwhile, a typhoon strikes the village. The villagers take cover at a storm shelter, but Sato is still at his family's dojo. When the dojo is leveled by the storm, trapping Sato inside, Miyagi and Daniel rush to rescue him. Sato while being trapped believes Miyagi will win unfairly but instead Miyagi breaks a support beam that had pinned Sato down, freeing him.
After the three return to safety, Daniel goes out to rescue a child trapped in the bell tower. Sato orders Chozen to help Daniel, but Chozen refuses most likely due to his grudge. Just as Miyagi goes out to help Daniel, Sato insists in helping him instead as gratitude for saving his life. After helping the child, Sato then disowns Chozen for refusing to cooperate. Chozen runs off in anger.
The next morning, the villagers set about rebuilding the village, and Sato returns with the bulldozers to get rid of debris and repair storm damage. Sato hands over the titles to the villagers' homes and asks Miyagi for forgiveness. Miyagi accepts his old friend's apology. Daniel asks Sato if the village may hold their upcoming O-bon festival on the castle grounds. Sato agrees and has one condition: that Daniel join in the celebration.
At the O-bon festival, Kumiko is performing a traditional dance when Chozen interrupts, taking her hostage at knifepoint. Sato tries to dissuade Chozen to stop this and admitting he was wrong for all the wrongdoings and his hatred towards Miyagi. But refuses now that he disowned him he sees no point in listening to him. Chozen now thirsty for revenge and restoration of his honour, he threatens to kill Kumiko if Daniel does not fight him to the death. Daniel agrees with Miyagi telling him that its not a tournament fight but its a real serious fight. Just when Daniel is on the verge of defeat, Miyagi beats his hand drum. As Chozen closes in for the kill, Daniel utilizes the drum technique to deflect Chozen's attacks and land counter-attacks. Daniel grabs the vanquished Chozen by his hair and cocks his hand back for the fatal blow, demanding, "Live or die, man!" When Chozen responds with "die," Daniel fakes out Chozen before tweaking his nose and dropping him to the ground. Daniel embraces Kumiko, while Miyagi looks on proudly.