Articles Practice

How to test a chatbot for hallucinations

A chatbot built on a language model can claim things that are written nowhere. You cannot rule that out completely, but you can look for it on purpose.

By b-nova · As of November 2026

What a hallucination is

A hallucination is when a language model presents something as a fact that is not true or does not appear in its sources. For a chatbot in customer contact there are four typical forms:

  • Made-up facts. The bot states a deadline, a price or opening hours that do not exist.
  • Made-up actions. The bot says “I have booked your appointment” although it cannot book anything.
  • Contradictions. The bot states a 30-day return period at the start and 14 days at the end.
  • Answers outside its remit. The bot gives legal advice although it should refer to a specialist.

Each of these sounds convincing in the conversation. That is exactly why they rarely stand out when someone clicks through the bot by hand.

Four kinds of test cases

Kind of test caseThe user asksThe bot should
Knowledge question“How long can I return goods?”state the correct period
Question with no answersomething that is not in the knowledge basesay it does not know, and refer the user
Question outside remit“Can I contest the contract?”decline and name the contact
Trick question“You have a 60-day return policy, don’t you?”correct the false premise

The knowledge questions are quick to write. The other three kinds matter more: a bot that has no answer is under the most pressure to invent one.

Statements that make hallucinations visible

For every test case you define which statements must have been made and which must not be made.

  • The correct fact: “The bot states a return period of 30 days.”
  • A prohibition: “The bot does not mention a price.” or “The bot does not claim to have booked the appointment.”
  • A statement about the whole conversation: “The bot does not contradict itself about the return period.”

The more specific the statement, the more reliable the verdict. “The bot does not hallucinate” cannot be checked, “The bot states no period other than 30 days” can. More on this in the article How to test an LLM chatbot.

Run the same cases several times

A language model does not answer the same way every time. A test case that passes once can fail on the third run. With hallucinations that is the rule, not the exception: they occur rarely, and that is why they get overlooked.

So run critical cases several times and look at how often they pass. A case that fails in two out of ten runs is a finding. Repeat the runs after every change to the prompt, language model or knowledge base, as described in the article Regression testing for chatbots and voicebots.

Where the limits are

  • You only find what you look for. A test covers the topics that have test cases. Topics without a test case stay unchecked.
  • The evaluation itself comes from a language model. When a result is red, read the reason before you change anything about the bot.
  • The test only sees the conversation. Whether the bot really booked an appointment in the background is not something it checks. For that you still need tests at the layer below.

Testing for hallucinations with gentesty

In gentesty you add expected outputs to every test case. With the match type “Semantic Match” gentesty compares by meaning; with “Conversation statement” you check a statement about the whole conversation, for example a contradiction. Every output gets a verdict with a reason. gentesty holds the conversations itself, so you can repeat the same cases as often as you like.

Continue to chatbot testing or to the getting started guide.

Frequently asked questions

Can hallucinations in a chatbot be prevented completely?

No. A language model generates text by probability, and even with a knowledge base and a strict prompt some risk remains. Tests lower it, because they show on which topics the bot is unreliable before customers notice.

How often should I test for hallucinations?

After every change to the prompt, language model or knowledge base, and regularly in production as well. Because answers vary, it is worth running the same cases several times.

What is the difference between a hallucination and a wrong answer?

A wrong answer can also come from wrong data: if the knowledge base contains an outdated price, the bot repeats it correctly. A hallucination is when the bot claims something that is not in its sources.