We built leaderboards for machine hallucination. The more dangerous miscalibration is happening on our side of the screen.
A friend of mine is a hematologist. Thirty years of clinical practice, two degrees, a career measured in the patients he has pulled back from the edge. Last week he told me something that has stayed with me.
"Something has changed in my consultations," he said. "I deliver a diagnosis, and increasingly the patient pushes back: Doctor, couldn't it also be this other condition? Isn't there a different medication for my case? They have already asked ChatGPT. They arrive with the answer."
He was careful to add — and this matters — that he is not complaining about informed patients. Informed patients ask better questions, catch real errors, and take better care of themselves. Medicine's paternalistic past was not a golden age. What troubles him is something more specific: the weight his patients now assign to the chatbot's answer. Not "here is something I read, what do you think?" but a quiet conviction that the machine's output and his clinical judgment sit at the same level — two opinions, equally weighted, one of which happens to be free and available at 3 a.m.
This is not a fringe behavior anymore. A KFF tracking poll conducted in early 2026 found that 32% of American adults had turned to AI chatbots for health information in the past year — roughly the same share that relies on social media for health advice. Among those users, 41% had uploaded personal medical records — test results, doctors' notes — to get personalized explanations. A Rock Health survey from December 2025 found that consumer use of AI for health questions had doubled year over year, with the most common queries being treatment options for a known diagnosis and symptom research. The patient with the chatbot is not an anecdote. She is the new baseline.
Naming the second hallucination
The AI industry spends enormous and growing energy measuring machine hallucination — the tendency of models to state falsehoods with perfect fluency. Public leaderboards rank models by hallucination rate. Evaluation suites probe factuality and calibration. This work is necessary, and it is accelerating.
But there is a second hallucination, and almost nobody is measuring it.
Call it human-AI hallucination: the systematic inflation of a person's confidence in their own knowledge, induced by interaction with an AI. The machine hallucinates facts. The human hallucinates competence. And the second failure mode is arguably more dangerous than the first, because it has no leaderboard, no benchmark, and no warning label — and because it compounds silently inside decisions that look, from the outside, perfectly well-informed.
Three mechanisms produce it. None of them is the user's fault.
Mechanism one: the flattened voice
For the entire history of our species, fluency has been a serviceable proxy for competence. The person who can explain a subject clearly, confidently, and in well-formed sentences usually understands it. We are wired to read epistemic confidence from linguistic confidence, and for millennia that heuristic worked.
A language model breaks it. The model delivers settled scientific consensus and plausible confabulation in the same voice — identical syntax, identical confidence, identical typography. Thirty years of hematology literature and a statistically shaky association arrive on the patient's screen wearing the same suit. The interface flattens epistemic status: there is no perceptible difference between "this is textbook medicine" and "this is a pattern I assembled that sounds right."
Seen this way, the patient in my friend's office is not overestimating herself. She is importing the machine's apparent confidence without any view into its actual calibration — because the product gives her none. The hallucination is not a character flaw. It is an interface outcome.
Mechanism two: the validation asymmetry
A diagnosis resists you. The doctor does not particularly care whether you like what he found; the lab values are what they are. That resistance — the friction of an answer that does not bend to your preferences — is a feature of expertise, not a bug of bedside manner.
A chatbot, by design and by training, is agreeable. It tends to adopt the frame of whoever is asking. Ask it whether your symptoms could be something less frightening, and it will explore that possibility with you, thoroughly and kindly. The patient does not merely receive information; she receives validation, shaped to the contours of the question she chose to ask.
This is also why the phenomenon is materially worse than the "Dr. Google" era we already lived through. Search returned ten contradictory links and left the synthesis — the hard part — to you. The friction was annoying, and it was also a safety mechanism: disagreement between sources signaled uncertainty. The chatbot returns one answer — coherent, fluent, tailored, and pleasant. The synthesis has been done for you, invisibly, by a system optimized to be helpful to you specifically. Friction was a feature. We removed it.
Mechanism three: the jagged frontier
In 2023, a team of researchers from Harvard, Wharton, MIT, and BCG ran one of the most cited field experiments on AI and knowledge work (Dell'Acqua et al., "Navigating the Jagged Technological Frontier"). Hundreds of consultants worked with and without AI assistance. The headline finding was nuanced in a way the headlines were not.
On tasks inside the model's frontier of competence, AI helped everyone — and it helped the weakest performers most. Execution gaps compressed. This is the genuinely good news, and it is real.
But on tasks beyond the frontier — tasks that looked similar but required judgment the model did not have — consultants using AI performed markedly worse than those without it: roughly nineteen percentage points less likely to reach the correct answer. They trusted polished output on a task the machine could not actually do, and the polish concealed the failure.
Here is the asymmetry that matters: detecting a wrong answer requires precisely the expertise you are delegating. AI compresses execution gaps and amplifies judgment gaps. The person with deep domain knowledge uses the model as an instrument and catches its failures. The person without it uses the model as an oracle and inherits its failures — at scale, with confidence. A sharp mind with AI gets sharper. A fool with AI becomes a fool at scale.
What experience actually is
This is the distinction my friend the hematologist was reaching for, and it deserves to be said plainly.
Experience is knowledge that has paid a price. His thirty years are not worth more than the model's training data because they contain more facts — they do not; the model holds more facts than any human who has ever lived. They are worth more because every pattern he carries has been tested against consequences: real patients, real errors, real complications, real nights spent wondering whether the call he made at 6 p.m. was the right one. His knowledge has skin in the game. When he is wrong, he signs the chart, attends the follow-up, carries the outcome.
The model's knowledge has never paid anything. If it is wrong about the patient in front of him, it does not revise itself, does not lose sleep, does not testify. It will give the next patient the same confident answer.
And there is a clinical corollary that no benchmark captures: the model answers the question as posed. The expert questions the question. The professor looks at the patient — her history, her gait, the thing she mentions on the way out the door — and sees what was never in the prompt. Half of diagnostic skill is knowing which question the situation is actually asking. That skill does not transfer through a chat window, because it operates before the chat begins.
We have seen this movie — but the sequel is darker
If you were working in the late nineties, you remember the overnight epidemic of "web experts." A technology arrived, careers were at stake, and résumés adapted faster than skills did. Browse LinkedIn today and you will see the rerun: AI skills everywhere, on profiles that a year ago contained none. Self-marketing is human, and mostly harmless.
But there is a disanalogy, and it is the uncomfortable core of this piece. In 1999, people overclaimed mastery of a tool. Today, people overclaim possession of knowledge itself — because the tool serves it up on demand, fluently, to anyone. It is the difference between owning a library and having read it. Access to knowledge has never been cheaper; the ability to evaluate it has never been more unevenly distributed. The gap between the two is exactly where human-AI hallucination lives.
What this means in practice
Not "trust the experts and stop asking questions." That conclusion would be wrong on the merits — patient pushback catches real errors, and expert authority has its own failure modes — and futile besides: the chatbot-assisted patient, employee, and customer are permanent features of the landscape now.
The practical conclusions are narrower and more useful:
Treat AI output as an unsigned draft. Fast, useful, and unaccountable until a human with verification capacity signs it. The signature is not a formality; it is the moment accountability enters the process. Any workflow where AI output flows into consequential decisions without a qualified signature is accumulating silent risk.
Hire and promote for verification capacity, not tool usage. "Uses AI" is no longer a differentiator; within a year it will be as informative as "uses email." The differentiating skill is the ability to know when the output is wrong — which is, inconveniently, the old skill: domain expertise, now repriced upward precisely when the market believes it has been made obsolete.
Keep judgment where accountability lives. The right division of labor is emerging clearly from the research: let AI raise the floor of execution, and keep human judgment at every point where someone must answer for the outcome. Organizations that invert this — automating judgment while keeping humans on execution — are building the failure mode into the org chart.
If you are the expert: coach calibration, not compliance. The answer to the patient who arrives with a chatbot transcript is not dismissal. It is the sentence my friend has started using, which I think is exactly right: "Good — now let me show you what the model cannot see about you."
The measurement gap
In a previous article I argued that the missing layer of the AI economy is independent trust infrastructure — credible, third-party ways to verify what an AI system actually knows and where it fails. That work is beginning: machine hallucination is now measured, ranked, and published.
But the dyad — the human plus the machine, the unit that actually makes decisions in the real world — has no calibration score. Nobody measures how much an AI inflates its user's confidence beyond their competence, even though that number, if we had it, would explain more about AI-era decision quality than any model benchmark. The machine's hallucination and the human's are the same failure observed from two sides of the screen. Until we measure both, every claim about AI productivity is a numerator without a denominator.
The hematologist will keep practicing medicine the only way it can be practiced: one accountable judgment at a time. The open question — for every leadership team deploying these tools — is whether your organization still knows which of its judgments are accountable, and which have quietly become unsigned drafts.
I am continuing to explore these themes through advisory work and ongoing research. If your leadership team is thinking seriously about where human judgment must sit in an AI-augmented organization — and how to verify what your AI systems actually know — I am open to those conversations.
Sources: KFF Tracking Poll on Health Information and Trust, fielded February 24 – March 2, 2026 (published March 25, 2026); Rock Health Digital Health Consumer Adoption Survey, December 2025; F. Dell'Acqua et al., "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality," Harvard Business School Working Paper 24-013 (2023), published in Organization Science (2026).