A group of Apple artificial intelligence scientists have published a study claiming that modern AI is incapable of thinking.
To prove their point, they created the GSM-Symbolic reference standard, which measures the “thinking ability” of various large language models (LLMs). According to the study, any changes, even minor ones, in the wording of queries can lead to differences in reasons, which increases confidence in the models.
Scientists pay special attention to mathematical tests. They added contextual information to the requests that the person might understand, but which should not influence the decision. AI is tested differently every time.
One of the examples given in the article describes a simple mathematical problem about how many kiwis a person collected in a few days. When the condition included Kiwi community data that was not relevant to the solution, models o1 from OpenAI (used in ChatGPT) and Llama from Meta* changed the answers, which should not be done.
We found no evidence of forms of reasoning in the language models. Their best control of the template selection mechanism is so fragile that changing the names can change the results by about 10%.
Apple
The scientists concluded that the models do not use real logic to solve problems, but instead try to recognize patterns that were added during training. [Appleinsider]
*recognized as extremist and banned in Russia
Source: Iphones RU
I am a professional journalist and content creator with extensive experience writing for news websites. I currently work as an author at Gadget Onus, where I specialize in covering hot news topics. My written pieces have been published on some of the biggest media outlets around the world, including The Guardian and BBC News.
