A Russian-language platform LLM Arena for evaluating generative neural networks has appeared in Russia

The first independent Russian-language platform for assessing the quality of large language models based on fundamental tasks has appeared in Russia – Master of Laws ArenaThe platform was created by Roman Kutsev in collaboration with neural network experts and former developers of TrainingData.ru.

The rating shows how generative neural networks cope with real user tasks.

Here you can at first test neural networks and evaluate the quality of their responses in accordance with the request:

► For comparison, the supplier provided two random models (he does not know which ones)

► A person writes any request, compares the answers of the models and chooses the one that he considers the best

► Based on the assessments of the definition of the rating of generative neural networks in English

At the moment, the user platform is available for testing 21 of the most popular generative neural networks, including ChatGPT, LLaMa, YandexGPT, GigaChat Saiga). The list is regularly updated: new models can be added by their developers.

❝

Our goal is to create an objective, open and up-to-date rating of language models in Russian. Even though more and more standards are appearing in the world, and government officials are comparing models, it is very difficult to protest in Russia in the field of LLM in the native language on the side of competitors.

The same LMSYS chatbot arena does not provide access to any Russian neural network. That is why we came up with the idea of ​​creating our own platform so that users could compare Russian and foreign generative neural networks themselves and draw their own conclusions.

— Roman Kutsev, founder of LLM Arena, graduate of the Moscow State University Computational Mathematics and Cybernetics, former CTO TrainingData.ru

In the future, it will be possible to compare neural network responses to multimodal tasks. For example, to evaluate how well the model understands what is depicted in the picture, or how well the image is generated on request.

LLM Arena is created under an open license and operates on the principle of one of the most popular ratings LMSYS Chatbot Arena.