Why does AI get things wrong so confidently?

Because a fluent answer and a verified answer are two different things.

Generative AI predicts what text is likely to come next. That can produce useful explanations, but it can also produce a false claim, a made-up citation or details that contradict themselves. The U.S. National Institute of Standards and Technology calls this kind of output confabulation.

So don’t ask only, “Does this sound right?” Ask for the exact source, open it, and check whether the cited passage actually supports the claim. For numbers, dates and instructions that matter, verify against the original record. If the answer cannot point to evidence, treat it as a draft—not a fact.

The 20-second check: pick the most important factual sentence, find its primary source, and compare the wording. A confident tone is not evidence.

The check above is practical editorial advice, not a guarantee that every error will be caught.

Source: NIST, Generative AI Profile, §2.2.