The term “hallucination” is used as a negative term within the field of Large Language Models (LLMs). This is because it’s used to describe instances when an LLM verifies its information with sources that don’t exist. There have been several high-profile cases where people have used ChatGPT or similar models to complete work for them, only to discover that significant portions of what they’ve written are false (the most well known of these being the lawyers who used ChatGPT, only for it to end up fabricating nearly their entire case). In these cases, the LLM has been observed to ‘hallucinate,’ fabricating information with confidence, which can leave the user under the impression that these fabrications are true.
However, there’s a problem with using the term ‘hallucination’ to describe this behavior. It doesn’t accurately capture what’s happening when LLMs ‘create’ sources, because technically, LLMs are always engaged in a form of hallucination. To delve into this issue, we need to briefly explore how LLMs work.
Firstly, LLMs are fed vast amounts of information, which they divide into tokens. The final output is generated by selecting the most likely token for each position in the sequence, based on the model’s learned probabilities. This output then needs to be decoded to produce human-readable text, typically involving the removal of extra spaces, joining tokens together, and handling special characters. Finally, post-processing is often used to make the generated text more coherent, including grammar checking and the addition of contextually relevant information.
Throughout this process, it becomes clear that LLMs are not independent, intelligent machines that replicate human thinking processes, as one might imagine when we refer to them as ‘hallucinating.’ Instead, they are highly advanced machine learning algorithms, capable of processing enormous amounts of data, that have been programmed to provide coherent responses based on input prompts and trained probabilities.
So, what word should we use to define that fuzzy line between success and failure for an LLM? We propose that a far more fitting term is ‘fabrication’. Fabrication has a nice double meaning. In the context of industrial manufacturing it means to create. But in the context of relaying facts it has a negative connotation - it means to lie. For this reason, the word is perfect because context determines if the fabrication is positive or negative. If you’re interested in creating, LLM is great. If you’re interested in faithfully relating facts, it isn’t fantastic.
The moment an LLM fails is when you expect it to stick to facts but instead you get a convincing fiction. LLMs have been trained to source information, but are unable to differentiate fact from fiction. It will always respond to your input with a reasonable sounding answer. The problem is that this information might not be accurate and the LLM doesn’t know or care. Even worse, because the LLM has no concept of fact and fiction, it may present reasonable looking citations of sources for these inaccuracies.
We feel fabrication is also preferable as it avoids the other issue with the term ‘hallucination’ in that it implies some form of consciousness or entity capable of truly perceiving reality as we do. LLMs don’t have a ‘waking’ state from which they can then hallucinate. As previously determined, they’re simply highly advanced machine learning algorithms that process vast volumes of data and have been trained to provide a human-readable, user-friendly output. As much as they might mimic an actual mind capable of independent thought and free will, that simply isn’t true.
Therefore, there’s nothing present in an LLM that is capable of hallucination, at least not how we define human hallucination. If one were to argue that they do hallucinate, then the logical conclusion would be that they’re in a constant state of hallucination, as they don’t perceive or understand reality in any way that we currently comprehend. Whether or not, and to what extent, LLMs do understand reality could be the topic of an entire book, so we won’t get into it here.
In any case, we’re really just delving into semantics here. This is a topic we’ve discussed during meetings, and we thought it would be interesting to share our thoughts on the matter. Regardless of how we define this issue, whether as ‘hallucination,’ ‘fabrication’ or some other term, it’s something that every person and company using LLMs must be aware of. It’s important to work with experts - like us - who can wrap the LLM in code, helping to guide it to truthful outputs. It’s also imperative that the outputs are checked for accuracy by humans, giving further peace of mind.
Implementing these two measures is the most effective way to avoid LLM ‘fabrication’. Effective prompt engineering can have an impact on increasing the likelihood of the LLM providing a factual response, but it’s never guaranteed. It’s best to remember that in the world of LLMs, diligence and collaboration are our greatest allies in the pursuit of accurate and reliable outputs.
