Police say more than a dozen heavily-armed men and women stormed into a school just as students and teachers entered the building on the first day of classes in the new school year.
There was some shooting when the militants seized the building and reports of at least two deaths.
Local authorities say the attackers are wearing suicide bomb belts and have threatened to blow up the school if Russian troops attempt to storm it. They say they are holding talks with the militants, who are demanding that rebels imprisoned in the neighboring region of Ingushetia be freed.
North Ossetia is located next to the breakaway region of Chechnya, where Moscow has been trying to crush a struggle for full independence from Russia for most of the past decade. Officials say it is not clear if the militants are from Chechnya.
The attack follows a suicide bombing Tuesday in Moscow and the crash of two airliners last week that killed 90 people, incidents which authorities believe were carried out by Chechen suicide bombers.
President Putin cut short his holiday and returned to Moscow to meet with cabinet and security officials.
Mr. Putin has long refused to negotiate with the Chechens and insists the conflict in Chechnya is solely an issue of terrorism.
Defense Minister Sergei Ivanov says Russia will continue the fight to defeat the rebels. He tells reporters, this is a war without a front line, adding Russia is not the only country which faces this. He says, it is no accident that the bombing near the Moscow metro on Tuesday occurred the same day that two buses were blown up in Israel, referring to the latest suicide bombings in the Middle East.
Chechen militants have seized hostages before, most notably at a Moscow theater in October 2002 when 800 people were held for three days before troops used a knock-out gas and stormed the building. About 130 of the hostages died in the incident.
While denouncing terrorist actions, many world leaders have also urged the Russian president to seek some kind of political solution to the Chechen conflict.