Why hallucinations occur:
-
Compression and data inconsistency: Hallucinations in LLM often result from data compression and inconsistencies in teaching materials.
-
Limited or conflicting information: Hallucinations can occur when training data contains limited, outdated, or contradictory information about a particular query.
Various strategies can be used to reduce the effects of hallucinations:
-
Query engineering: Making well-structured queries that encourage step-by-step thinking and fact-based responses can help reduce hallucinations.
-
Using external source Checking the answers can increase their reliability.
Hallucinations in LLM can be broken down into logical fallacies, juggling of facts, and data-driven bias.
Practical approaches to reducing the level of hallucinations highlighted by Sergei Savvov are:
-
Chain of Thought (CoT): Add “Think step by step” option to tooltips and request JSON responses.
-
Self-consistency with CoT (CoT-SC): Asking the model to give more than one answer and choose the best one.
-
Thought Tree (ToT): Encourage step-by-step reflection and self-evaluation of the model’s reasoning process.
-
Marked Context Hints: creating question sets, generating context clues, and including links to sources and facts in answers.
-
self correction: to encourage the model to recheck and improve its answers.
-
Multiple agents: Engaging multiple LLM examples to propose and discuss answers.
Each method has its own advantages and disadvantages, and the choice depends on the specific use case and dataset. More research is needed to improve these approaches and increase the reliability of LLM responses.
Source: Ferra
I am a professional journalist and content creator with extensive experience writing for news websites. I currently work as an author at Gadget Onus, where I specialize in covering hot news topics. My written pieces have been published on some of the biggest media outlets around the world, including The Guardian and BBC News.
