A team of researchers from Stanford University and the University of California at Berkeley found that the accuracy of responses from the paid version of ChatGPT has plummeted.
The scientists found that while in March of this year ChatGPT-4 solved mathematical problems with an accuracy of 97.6%, in June the figure had dropped significantly to 2.4%.
At the same time, the correctness of the answers from the previous version of GPT-3.5 is only growing. If in March the accuracy of mathematical calculations was 7.4%, in June it increased to 86.8%.
In addition to mathematical calculations, the scientists analyzed the dynamics of the accuracy of AI models in the field of code generation, visual thinking, and in terms of answering sensitive or dangerous questions.
GPT-4 indicators have changed when answering dangerous questions, in June the bot provided fewer answers than in March. At the same time, both versions in June stopped explaining why they could not answer this or that question. In March, users received a detailed explanation.
In addition, the number of errors in the generation of program code has increased. If in March GPT-4 performed 50% of the 50 assigned tasks, then in June it was only able to complete 10%. A similar trend is observed in the GPT-3.5 version.
One possible explanation is that the June releases were constantly adding additional non-code text to their builds.
In July, OpenAI, which developed ChatGPT, provided API access to the latest version of the GPT-4 language model. The company developed the version itself in March.
Author:
Natalia Gormaleva
Source: RB
I am a professional journalist and content creator with extensive experience writing for news websites. I currently work as an author at Gadget Onus, where I specialize in covering hot news topics. My written pieces have been published on some of the biggest media outlets around the world, including The Guardian and BBC News.
