The Indonesian government and separatist rebels from Aceh Province failed to salvage a peace pact during two days of talks.
A mediator confirmed late Sunday that two days of hastily convened talks were unsuccessful.
Negotiators for the Free Aceh Movement refused to give up their demands for independence, despite the government's threat of a military offensive if the talks ended in failure.
On Saturday, Indonesia gave the rebels 24 hours to agree to give up 60 percent of their weapons within a month and immediately agree to accept special autonomy, not independence.
The Tokyo talks nearly ended before they began. Five Aceh independence delegates due to attend were temporarily detained Friday as they were preparing to leave for Japan.
The two days of talks were described as tense. On the first day, negotiators from Indonesia's government and the rebels did not meet face-to-face, relaying their demands only through a mediator.
Indonesia appears determined to end any independence bid for the resource-rich province. It has boosted the number of soldiers and police there to more than 45,000 in recent weeks. Aircraft and nearly two dozen warships are also at the ready for what many observers now predict could be a major assault.
Aceh rebel official Sofyan Ibrahim Tiba says that if Indonesia goes to war, it will have to kill nearly three-quarters of Aceh's four million people because, he says, that is how many desire independence.
Thousands of Acehnese, mostly civilians, have died in recent years in fighting between the military and rebels of the Free Aceh Movement. The province has been wracked by decades of separatist violence.
A peace deal signed in December raised hopes that the conflict would end soon, but that agreement was plagued by alleged disarmament violations and clashing interpretations of autonomy.
The Tokyo talks were called in an effort to sort out these differences. The government's chief negotiator said Jakarta was open to the possibility of further talks. The leader of the rebel delegation said the rebels were ready for war.
Fresh clashes were reported even as negotiations in Japan proceeded. The Indonesian military says seven rebels were killed Saturday in a clash with troops caused by the hoisting of an independence movement flag.