ChatGPT Doesn’t Do Maths As Well As A Human, Says Study

According to research by scientists at Brigham Young University (BYU) in the United States, ChatGPT’s AI is not that efficient when it comes to accounting. In math tests with hundreds of participants, ChatGPT scored only 47.4% overall, compared to an average of 76.7% in humans.

The study, published in the Numbers Journal of Accounting Education, recruited 327 participants from 186 educational institutions in 14 countries, in addition to BYU’s undergraduate students. In total, more than 27,000 answers were given to questions related to accounting, covering subjects such as accounting information, auditing, financial accounting, among other areas.

In addition to recording a lower overall average than study participants, ChatGPT scored higher on only 11.3% of the questions. In general, AI had the hardest time with questions about financial, financial and managerial considerations.

ChatGPT also had trouble answering short-answer questions, but got 68.7% of true/false questions and 59.5% of multiple-choice questions correct. According to the research team, the AI ​​doesn’t always realize that it’s doing mathematical calculations and starts offering errors and/or meaningless answers.

ChatGPT and math calculations

Anyway, the researchers believe the AI ​​chatbot could be a great option to improve the way people and institutions teach and learn. However, it is still necessary to work on the subject more and wait for the advancement of artificial intelligence.

“When this technology first came out, everyone was worried that students might use it to cheat. But opportunities to cheat have always existed. That’s why we’re trying to focus on what we can do with this technology now, what we couldn’t do before to improve the teaching process for us, the faculty, and the learning process for students. “Testing this was eye-opening,” said David Wood, lead author of the study and BYU accounting professor.