Day 23

Why do LLMs hallucinate?

But before that...
What is a hallucination?

Here's what I've understood so far.

1. Hallucination is when an LLM generates information that sounds correct but isn't actually true.

It might invent a fact, a source, or even a quote.

2. Why does this happen?

LLMs are trained to predict the next token based on patterns they've learned.

3. So what happens when the model doesn't know the answer?

It doesn't always stop.

Sometimes it predicts the most likely continuation, even if it's wrong.

That's what we call a hallucination.

Day 23 of 1% Better.