When AI Gets It Wrong Addressing AI Hallucinations and Bias
When AI Gets It Wrong Addressing AI Hallucinations and Bias
This guide argues that generative AI failure is not one problem but at least two structurally different ones: hallucinations and bias. It is strongest when it shows that both are built into the way these systems are trained and used, rather than being rare glitches that disappear with a version upgrade.
Bias Is Not Abstract
On the bias side, the source grounds the problem in concrete cases. It points back to the Gender Shades result, where commercial gender-classification systems performed far worse on darker-skinned women than on lighter-skinned men. It then shows the generative version of the same problem through Stable Diffusion, whose outputs amplified racial and gender stereotypes at scale.
The raw keeps pressing on the real-world consequences. Biased image generation is not just embarrassing. If similar systems feed policing tools, professional recommendations, grading, or hiring judgments, they can intensify existing harm while wearing what the source calls a veneer of objectivity.
Hallucination Means Plausible Fabrication
On the hallucination side, the guide focuses on fabricated content that looks authoritative. It highlights legal-research evaluations where both general-purpose and specialized tools invented cases or citations, then uses Mata v. Avianca as the clearest public example: a lawyer submitted authorities generated by ChatGPT that did not exist. The particularly damaging detail is that the chatbot did not only invent the citations — it stipulated that they were available in major legal databases, giving false confirmation to a check that should have caught the error.
That case matters because it makes the mechanism visible. The model did not simply fail to retrieve one fact. It produced a polished false answer, complete with invented references, confident presentation, and fabricated database availability. The raw's broader point is that this is now a cross-domain problem, not something confined to one bad anecdote.
Why These Failures Persist
The guide gives three reasons the problem persists. Training data already contains falsehoods and social distortions. Generative models are optimized for plausible continuation rather than truth-checking. And even a cleaner corpus would not fully solve the problem because the act of generation can still combine patterns into inaccurate or skewed outputs.
Responsible Use Means Changing How You Trust The Tool
The practical response is not "never use AI." It is to change the trust model. The source recommends critical review, cross-checking with stronger sources, and using retrieval-grounded systems when available. It adds two practical levers worth naming precisely. The first is Chain-of-Thought prompting: structuring the prompt to ask the model to explain its reasoning step by step, which can surface logical gaps or unsupported claims before they get buried in fluent prose. The second is temperature control: setting temperature to 0–0.3 for tasks where factual consistency matters, and reserving the 0.7–1.0 range for open-ended creative or brainstorming tasks where variety is the point.
Worth coming back to: this source reframes responsible AI use as a calibration problem. The point is not whether AI is powerful. It is whether the user understands what kinds of mistakes a fluent system can hide.
Sources
raw/When AI Gets It Wrong Addressing AI Hallucinations and Bias (ingested).md