AI hallucination
What is AI hallucination?
An AI hallucination is when AI states something false in a confident voice, as if it were fact: an invented price, offer or policy.
LLMs are built to write sentences that sound plausible, not to know whether they are true. Missing data, they tend to fill the gap with something that sounds right: a promotion that does not exist, a policy never issued.
AI hallucination in a shop
In 2024, an Air Canada chatbot misstated its fare discount policy, and a Canadian tribunal made the airline compensate the customer.
How does AI hallucination work?
- 1An LLM is built to write sentences that sound plausible, not to know whether they are true.
- 2When a question falls outside what it was given to read, it still answers with something that sounds right.
- 3Outdated or contradictory documents make it pick the wrong information.
- 4Instructions that only say "be friendly" without "say you don't know when you don't" make it guess more.
What a shop or café can use it for
- Knowing where chatbots tend to invent things so you block them first: prices, offers, policies, delivery dates.
- Writing chatbot instructions: answer only from the documents, otherwise hand over to a person.
- Telling staff to check figures in AI-written posts before publishing.
What to watch out for
- It cannot be switched off, only reduced: always keep a person approving where money is involved.
- The more confident the answer sounds, the more customers believe it, even when wrong.
- When a chatbot has told a customer something wrong, correct it at once and log it so it does not repeat.
Quick questions
No. It can be reduced with correct documents, guardrails and human approval, which is why the shop remains responsible for what its chatbot says.
It depends on the law and the situation, but the 2024 Air Canada case shows a business can hardly blame "the bot". Correct it with the customer, log it, and block the mistake with guardrails.
It chooses words by how plausible they are, not by how certain the facts are. A confident tone is a writing style, not a sign of truth.
Sources
The dated facts on this page come from the original sources below.
- BC Civil Resolution Tribunal (CanLII)
Moffatt v. Air Canada, 2024 BCCRT 149
Keep learning beyond the glossary
Read next
- Data and accuracyRAGRAG (retrieval-augmented generation) is letting AI find and read your documents before it answers, instead of answering from memory.
- Risks and safetyGuardrailsGuardrails are hard rules around AI so it does not say or do things outside its remit, such as giving a bigger discount than allowed.
- Agents and automationHuman in the loopHuman in the loop is a design where a person reviews and approves before AI does something important: the AI drafts, the person decides.