The Korea Herald

피터빈트

AI near-aces college entry-level Korean exam

In this year's Suneung, AI answered all but one question correctly in the Korean section, ranking in the top 4 percent of all test-takers

By Yoon Min-sik

Published : Dec. 17, 2024 - 14:06

    • Link copied

An image provided by Markr AI An image provided by Markr AI

An artificial intelligence program has gotten a near-perfect score on the Korean language test of the state-run college entrance exam, according to the company behind the software.

Korean startup company Markr AI said the o1-Preview model, developed by OpenAI, got all but one question right in the Korean language section of this year's Suneung, held in Nov. 14 and was used for 2025 college admissions. This gave the AI a score of 97 out of possible 100 points, which corresponds to a "grade 1," awarded to the top 4 percent of all test-takers.

Question No. 8 was the only question that the AI model had gotten wrong, which 81.5 percent of actual test-takers had gotten wrong. The developers explained that o1-Preview had failed to catch the subtle nuances in the text.

OpenAI o1 is a generative pre-trained transformer (GPT) program developed by US-based AI research organization OpenAI. The "o1-preview" and the cheaper o1-mini models were released in September of this year for the o1 series, which the company noted will be the first of a series of "reasoning models."

Markr AI has been testing the problem-solving capacities of the AI programs by having them take Suneung Korean exams for the past 10 years.

It also had other AI programs take the same exam, with o1-Mini model finishing second by getting 78 points. This would have qualified it for a grade 2, which would still put it in the top 40 percent group.

The text used for Question No. 8 of the Korean language exam for this year's Suneung. The text used for Question No. 8 of the Korean language exam for this year's Suneung.
Question No. 8 of the Korean language exam for this year's Suneung Question No. 8 of the Korean language exam for this year's Suneung