Will Medical AI Do More Harm Than Good?
Expert weighs promise and peril at a hinge moment for technology and healthcare.
If online behavior is any gauge, artificial intelligence (AI) is already reshaping healthcare in the US. One in three Americans consults AI for medical information and advice, according to a March 2026 survey by the Kaiser Family Foundation, with the young and the poor leading the way. But even as the use of AI continues to grow, so does concern about the technology. In 2025, the annual Bentley-Gallup Business in Society Survey found nearly 80 percent of respondents either somewhat or extremely concerned about the use of AI for medical recommendations and advice.
As a member of the Artificial Intelligence Advisory Group of the College of Family Physicians of Canada and an advisor on the Canadian Medical Association’s AI policy, Sian Tsuei, PhD ’23, helps establish national guidelines for the use of AI in primary care, family practice, and other clinical settings. In his new book, Medical AI: Promises and Perils at a Critical Moment, Tsuei explores the technology’s potential to transform healthcare—and disrupt it. He sat down recently with Harvard Griffin GSAS Communications to discuss his research and why he worries that the power and potential of AI will overwhelm the guiding principle of medicine: first, do no harm.
Your research focuses on the primary care setting, not least because the healthcare system in Canada emphasizes it so strongly. What have you found about the national guidelines in this area and how they play out with clinicians?
I reviewed all AI guidelines applicable to primary healthcare in Canada. Looking at 2021 to 2025, the guidelines address all the core principles highlighted by the World Health Organization. They essentially advise clinicians: “Choose AI carefully. Make sure the software adheres to high standards of privacy and data security. When deploying it, you have to use AI to support your clinical judgment. And make sure you solicit meaningful consent, set up appropriate local data governance, ensure privacy and data security, and give feedback for AI optimization.”
However, regarding the first step—selecting AI properly—clinicians often lack the expertise to evaluate these tools and rely on government vetting. For example, a recent Ontario Auditor General report revealed that a quarter of the AI scribe companies cleared for healthcare use did not submit threat, risk, and privacy impact assessments. Following government clearance alone is therefore problematic.
I recently presented to a group of physicians at a conference and asked how many were using an AI scribe; about 50 percent raised their hands. I then reviewed 40 or 50 standard privacy and security questions. Audience members were looking down, visibly concerned. Many want to be on the frontier of technology, but they do not fully understand the harms and risks. In Canada, government oversight of rapid AI developments remains an enormous challenge.
How quickly do you see the AI rollout happening in healthcare?
I think it can roll out in a few different ways. One is that we could stop where we are now or even revert a little. There is significant political backlash now. I think around 70 percent of Americans in the most recent survey I saw said they do not trust the pace of AI development because it’s way too fast. There are even AI software engineers saying we have to slow this down. If that political momentum takes hold, we may never see artificial general intelligence or superintelligence. Or the rollout could happen slowly, where AI emerges as a teammate that's overseen by humans, and over time, we end up with a pilot model.
But governments and societies face enormous economic and political pressures to do something about healthcare costs, quality, and access. Maybe those get too great. Big Tech comes in and offers a solution. If that happens within a policy window, boom, it goes forth, and it goes forth fast. I mean, you may think AI in healthcare is morally repugnant, but then your kid can’t see a doctor for three months. And then, AI is good enough. My concern is that if it goes too quickly, I think it creates a shock that our society is unprepared for.
I recently presented to a group of physicians at a conference and asked how many were using an AI scribe; about 50 percent raised their hands. I then reviewed 40 or 50 standard privacy and security questions. Audience members were looking down, visibly concerned. Many want to be on the frontier of technology, but they do not fully understand the harms and risks.
Is that your biggest concern—that this technology is rolling out too quickly?
What I worry about right now is that most of the literature I've seen shows that AI devices that can face patients directly or guide clinical decision-making often have not gone through randomized controlled trials, even though they could meaningfully influence people's care choices or clinical decisions. If AI tools with poor-quality evidence get approved, we might end up harming a lot of people.
At the same time, there are patients who cannot see doctors right now. Isn’t doing something better than nothing? Isn’t releasing a potentially helpful AI tool better than the status quo? That’s one of the key problems I've been faced with: the problems of commission versus problems of omission. If we commit to AI, we have a set of problems. But if we delay AI rollout until the AI suppliers jump through sophisticated hoops, we also have a set of problems. The current paradigm in the medical space is that if there are new drugs that come out and don’t yet have all the evidence—sure, they can save a lot of lives, but we have to make sure we don't harm anybody first. First, do no harm, and that's our guiding principle. I lean toward taking the problem of omission. We'll miss saving some people, but we really want to avoid killing somebody.
Before we talk more about your concerns, let’s have the good news. What are some ways AI is changing medicine that make you most hopeful and excited?
I think a lot of times the administrative part of clinical work is what really kills clinicians, or really makes it tough and draining. And we've seen that AI has that capability to help take notes, to synthesize materials, and to even create documents for filling out forms, etc. I think that is pretty helpful and the harms are relatively low. I think the frameworks around privacy and consent are starting to come into place a little bit more, and people are starting to get a little bit more used to it. And it's more tractable, I think, for human clinicians to stay meaningfully involved in the loop and not lose sight of what is going on and not lose their clinical decision-making or their understanding of the situation. I think that's a promising approach.
AI has shown a lot of promise with imaging. There are now randomized controlled trials showing that efficiency can meaningfully improve with AI without dropping quality for breast cancer screening. In terms of general medicine, Google created its Articulate Medical Intelligence Explorer (AMIE), an autonomous AI that can meaningfully take history better than primary care physicians. A randomized controlled trial published recently in Nature showed that clinicians who used AI to support their primary care visits improved their quality of care. Recently, AMIE autonomously rendered the first episode of care for 100 patients without requiring primary care physicians to intercede. And we're seeing some very interesting rollouts in Utah now, where they're using AI to automate prescription renewal.
Then there's psychotherapy. In 2024, Michael Heinz of Dartmouth and colleagues showed that a randomized controlled trial of generative AI did pretty well at psychiatric relief compared to human therapists, without jeopardizing rapport. The last part is surgery. In July 2026, University of California San Diego researchers used humanoid robots to help perform an operation. Down the line, we may be looking at embodied AI doing autonomous robotic surgery.
All these advances are pretty exciting. We could be looking at strong support for a lot of the clinical tasks that we think humans have to do now independently, and potentially even replacement of human clinicians in the near future. And if you do it right and have meaningful oversight, I think it can significantly improve the efficiency, quality, equity, and accessibility of healthcare.
Could AI really replace doctors?
The technical skills are getting more and more powerful, and there's evidence to show that AI models already outperform a lot of doctors in some tasks. There's some evidence that when doctors are using AI and the AI is really smart, doctors do not meaningfully improve their performance with AI. In fact, when AI is prompting doctors to improve their quality of work, doctors sometimes actually end up not correcting their actions, or introducing new errors. So, this line of evidence points to the idea that we should really cut doctors out. And then there are the norms of the current healthcare system, where despite increasing health expenditures, people are dissatisfied, finding care inaccessible, inefficient, and poor quality. That’s why places like Utah are giving AI a shot. And England’s National Health Service has actually automated physiotherapy as well.
The counterargument is that AI can be smarter than humans, but there's also some evidence to say the way humans reason is fundamentally different from AI. So if that is the case, and we're making different mistakes, then we should really work together, and there's evidence to say if you bring doctors and AI together as a team, they achieve better outcomes.
Beyond the outcomes, there's also the thought that human experience cannot be meaningfully replaced. I mentioned that Google AI created a chatbot that can do a good job taking history, and it can maintain empathy better than primary care providers. But there's a thought that empathy is not just whether something or somebody can mirror how you're feeling back to you; it’s also about that other something or somebody experiencing or mirroring that feeling, and being willing to do something about it.
[Advances in medical AI] are pretty exciting. We could be looking at strong support for a lot of the clinical tasks that we think humans have to do now independently, and potentially even replacement of human clinicians in the near future. And if you do it right and have meaningful oversight, I think it can significantly improve the efficiency, quality, equity, and accessibility of healthcare.
You've talked about potential gains in efficiency through the use of AI. Whether it’s administrative support, clinical medicine, or even minimizing medical errors, could the technology be a solution to healthcare costs that are ballooning as global populations age?
It comes down to how it actually ends up being operationalized. For example, right now, doctors and AI companies are lobbying for AI billing codes to be set up. If that happens in a way where you can automate the same volume at the same cost as human doctors, then you're not necessarily saving money. And right now, I think doctors are saying, “AI is not to replace us, so we'll add on an additional fee code to complement our work.” So, if doctors are billing X dollars, and AI just adds on Y dollars, this could end up inflating healthcare costs.
Finally, in the title of your book, you call this a “critical moment” for medical AI. Weighing both the promise and the perils, are you more hopeful or worried about the future?
A lot of people feel there’s something sacred about healthcare. There’s a study from Harvard Business School that shows many people feel a sense of moral repugnance if some jobs are automated. So, in Utah, the medical board pushed back on the AI automation and said, “This is unacceptable. You're skirting the edges.” The American Medical Association has also issued statements about this. Here in Canada, the guidelines I'm reviewing right now for the primary care setting emphasize the centrality of clinical decision-making for humans. So, I think it's going to take a long time for people to feel comfortable seeing doctors that are AI.
Tech capital and tech enthusiasm are so powerful. And I think they overwhelm people's concern for humanity, for safety. So, I’m concerned we will see AI automation overpower those other forces in healthcare. . . . But that's also why I wrote the book. If people don't want that future, they can do something about it. There's still time.
That said, I think tech capital and tech enthusiasm are so powerful. And I think they overwhelm people's concern for humanity, for safety. So, I’m concerned we will see AI automation overpower those other forces in healthcare. In fact, that is what we're starting to see. I imagine that there was resistance against automated prescriptions in Utah, but the government rolled it out anyway. And, of course, patients are already using AI, bypassing doctors to ask for a first opinion. I think that’s only going to ramp up, which makes me really concerned. But that's also why I wrote the book. If people don't want that future, they can do something about it. There's still time.
