The YandexGPT 2 language model solved several variants of the Unified State Exam in Literature. The neural network approached the average score in this subject among Russian schoolchildren (64 at the end of 2022), Yandex said in a statement.
The average YandexGPT 2 score was 55 points. In this way, the neural network has exceeded the minimum threshold required for admission to a university: 40 points.
It should be noted that this is the first time in Russia that a generative neural network passed the unified state exam, completing all its tasks, including creative ones.
Yandex added that passing the Unified State Exam in Literature is a difficult task for a neural network, since knowledge is tested from several sides at once: there are tasks to assess erudition, writing style and creative abilities.
“The MMLU is considered the standard test for assessing scholarship on linguistic models in the world: it includes questions of varying degrees of complexity from 57 areas of knowledge. This way, you can objectively compare various neural networks, but you cannot evaluate the quality of their answers to open or creative questions. That is why we offered YandexGPT 2 to pass the Unified State Exam in Literature,” explained Alexey Gusakov, technical director of Yandex Search.
For the experiment, Yandex specialists received literature versions of the Unified State Exam, which were used to conduct training exams in 2021-2023, and made sure that the YandexGPT database used for training did not contain answers.
Yandex introduced YandexGPT 2 in early September. This is an improved version of the YandexGPT neural network: it is capable of solving more different types of problems and the quality of its answers has also been improved.
Author:
Anastasia Marina
Source: RB
I am a professional journalist and content creator with extensive experience writing for news websites. I currently work as an author at Gadget Onus, where I specialize in covering hot news topics. My written pieces have been published on some of the biggest media outlets around the world, including The Guardian and BBC News.
