General-grade representatives from the two sides met in Singapore in a bid to resolve "mutual misunderstandings" over a dispute regarding Seoul's use of military radar for humanitarian search operations last month, according to Seoul's defense ministry.
It marked the first face-to-face contact between officials from the two nations over the sensitive case.
Tokyo has accused a South Korean warship of locking fire-control radar on its maritime patrol aircraft on Dec. 20.
Seoul has rejected the claim, saying the ship was on a mission to rescue a North Korean ship that was drifting in the international waters of the East Sea.
"The two sides were apart (throughout the Singapore meeting)," a ministry official said, suggesting that no agreement was reached.
They plan to consult each other in order to schedule additional talks, according to a defense source.
The neighboring countries opened the first round of talks at the South Korean Embassy in Singapore in the morning. They had another round at the Japanese Embassy in the afternoon.
South Korea's delegation was led by V. Adm. Boo Suk-jong of the chief director of military support at Seoul's Joint Chiefs of Staff and Lee Won-ik, director general of international policy at the defense ministry.
In the Japanese delegation were Lt. Gen. Atsushi Hikita, the director of operations at Tokyo's Joint Staff Office, and Takeshi Ishikawa, a senior defense ministry official.
On Dec. 27, the two sides held a video working-level conference on the issue only to confirm their differences over the incident.
Just a day later, Tokyo released a video clip to back up its claim over which Seoul expressed "deep concerns and regrets."
Later, Seoul hit back with its own video footage, demanding Japan's apology for conducting a "threateningly" low-altitude flight toward the South Korean warship focusing on the humanitarian mission.
Seoul has argued that its warship only used a search radar system and did not send out any radar beams to the Japanese plane.
It demanded a "smoking gun" -- Tokyo's analysis of radar frequency data to verify if the warship used a tracking radar against the plane.
The latest spat added to tensions in the bilateral relationship long strained by historical and territorial feuds.
The two countries have recently sparred over Seoul's top court rulings earlier this year about South Koreans forced into hard labor by Japanese firms during World War II. (Yonhap)