Ask people building AI for healthcare what keeps them up at night and you'll hear about model accuracy, regulatory approval, integration with ancient hospital IT. All real. But the thing that should keep all of us up at night is simpler and harder: trust. Without it, nothing in medicine works. Not the drug, not the vaccine, not the algorithm. It is the load-bearing wall of the entire enterprise, and right now it's cracking.
The erosion of trust in science and medicine isn't entirely new — it's probably been with us longer than we noticed — but it is more severe and more amplified today than it has ever been. And here's the property of trust that technologists habitually get wrong: <strong>trust is not bestowed; it is earned, constantly.</strong> There is no moment where you've "achieved" trust and get to move on to the roadmap's next line item. It's earned in every interaction, and it can be spent much faster than it accrues. Worse, earning it isn't even sufficient anymore — there are active forces in the world fostering mistrust of science and medicine, and anyone building in this space inherits the job of counteracting them, whether they signed up for it or not.
Three words that survive scrutiny
Frameworks for "responsible AI in health" have proliferated — codes of conduct, principle lists, commitment matrices. Strip away the committee language and I think the ones that matter reduce to three words, and all three are really about the same thing.
<strong>Responsible.</strong> A clear-eyed view of what we're actually optimizing: the best possible health for all people — not engagement, not billing efficiency captured as a side effect, not a demo that impresses a board. Health is in the equation, or the tool doesn't belong in the clinic.
<strong>Accountable.</strong> When the system errs — and every system errs — a human institution owns the consequence, visibly. Nothing corrodes trust faster than a mistake with no one standing behind it. (This is, not incidentally, why "the model made the decision" can never be an acceptable sentence in medicine.)
<strong>Consulted.</strong> Nobody builds this alone. The people affected — clinicians, patients, the communities the tool will serve — are in the room from design onward, not surveyed afterward.
Responsible, accountable, consulted. Each one is a mechanism for earning trust; together they're close to a definition of it.
The fundamental unit
Engineers like to identify the fundamental unit of a system — the atom everything else is built from. Here is medicine's, and I hold this view strongly: <strong>the fundamental unit of healthcare is a clinician and a patient, sitting together in a room.</strong> Every technology we build either serves that unit or damages it. There is no neutral.
This reframes what AI in medicine is <em>for</em>. Suppose an AI system could synthesize everything known about a patient — history, conditions, circumstances — alongside everything the clinical evidence recommends, and lay it out clearly. Enormously valuable! And yet the most important step happens after the printout: the clinician and the patient look at it <em>together</em>, discuss what the patient actually wants and needs, and decide — sometimes in line with the recommendation, sometimes not, because guidelines are built for populations and patients are individuals. It doesn't matter what the best science in the world says in the abstract; what matters is a clinician explaining it to a particular human being, and that human being deciding what they want for their own life. These are life-and-death decisions. The conversation is not a UI inefficiency to be optimized away. The conversation is the product.
I find a certain humility test clarifying here: none of us, however smart, wants to do our own medical care. Not because we couldn't read the literature, but because being a patient is a human condition, not an information problem. We need another human in the loop to know we're getting what we need. Any AI strategy that forgets this is optimizing the wrong system.
What this means for builders
If trust is the infrastructure, some engineering priorities reorder themselves.
<em>Design for the busy clinician, or don't bother.</em> A clinician with thirty patients in an afternoon will not adopt anything that slows them down — and shouldn't. The two motivations that reliably move clinicians are doing better for their patients and delivering care more effectively and efficiently. Tools that let clinicians do more of what they trained to do earn adoption; tools that add clicks earn quiet sabotage. And the burnout the industry promises AI will fix? A good share of it comes from the fact that clinicians will drive themselves into the ground taking care of patients. Respect that motivation; build for it.
<em>Co-develop or fail.</em> Clinicians won't trust — and shouldn't trust — tools they had no hand in shaping. Participation isn't a rollout strategy; it's a design input. The teams getting this right treat the clinical staff, nurses, administrators, all of it, as co-designers from the first whiteboard.
<em>Expect the trust bar to be higher than the accuracy bar.</em> A tool can be statistically excellent and still rightly rejected because its errors are opaque, its accountability is vague, or its designers never sat in the clinic where it lands. That's not irrationality. That's the immune system of a high-stakes profession doing its job.
The technology arriving in medicine right now is genuinely powerful — powerful enough to reshape how care is delivered. Which is exactly why the constraint that matters isn't computational. Medicine ran on trust for two thousand years before it had statistics, and it will not run without it no matter how good the models get. Earn it constantly, spend it never, and build everything — everything — in service of two people in a room, deciding together.
