Last year, OpenAI caused a stir when it claimed that GPT-4 outperformed 90% of legal interns on the bar exam. However, new research shows that this claim is exaggerated.
A study published in the journal Artificial Intelligence and Law shows that the GPT-4 model actually failed to reach the top 10% of exam results.
He performed only well compared to re-takers, a generally less successful group.
Moreover, GPT-4 significantly underperformed on the essay portion of the exam, ranking only in the 15th percentile among top performers. This is especially important because the articles are closest to the challenges lawyers face.
We note that this study highlights the need for caution in the use of AI in legal practice, given the tendency of AI to create “hallucinations” (fictional facts or connections).
Source: Ferra
I am a professional journalist and content creator with extensive experience writing for news websites. I currently work as an author at Gadget Onus, where I specialize in covering hot news topics. My written pieces have been published on some of the biggest media outlets around the world, including The Guardian and BBC News.
