AI hallucinations pose a real danger to science

Hallucinations from generative artificial intelligence (AI) pose a major threat to science, according to a study from the University of Oxford in the United Kingdom. According to the article, users’ gullibility towards the information provided by models could be a big problem in the future.

“People who use LLMs often anthropomorphize [atribuem características humanas] “We rely on technology as a human-like source of information,” said professor Brent Mittelstadt, co-author of the paper.

What are hallucinations?

The phrase “hallucination” refers to the moment when AIs present false or completely misleading information. This most often occurs when very deep information is requested from a large database, but it can also occur in more common scenarios.

The main reason for this confusion may be the way these chatbots present themselves: coherent, accessible text that resembles human speech. Additionally, models tend to trust the information provided even if it is not accurate.

Artificial intelligence presents information safely

“As a result, users can easily be convinced that answers are correct, even if they are not factual or present a biased or partial version of the truth,” the professor said.

But this combination of factors is a threat to scientific dissemination and education. In addition to being disseminators of publicly available misinformation, the scientific community is also under pressure to use productive AIs as content translators.

According to Oxford professors who participated in the research, language models are important aids in workflow, especially in research, but it is important that users use the tool responsibly and are aware of its flaws.