Not a bug

Hallucination is the model working as designed.

A model is trained to produce likely-sounding text, not verified truth. A confident, plausible, wrong answer scores well on that objective. Hallucination is a feature of the goal, not a malfunction.