[Science] AI Mostly Avoids Unnecessary Medical Tests — But Stumbles Where Doctors Disagree

Ask a popular AI chatbot whether you need a thyroid ultrasound for a small nodule your doctor found by accident, and there is a real chance it will tell you to get one anyway — even though medical guidelines say not to bother. A large new benchmark study finds that commercial AI models mostly avoid recommending unnecessary medical tests and treatments when restraint is the right call. But in the small slice of cases where human doctors themselves disagree about what counts as “necessary,” the AI stumbles right alongside them.

What the Study Actually Tested

The research team, led by Dean Ariel, MD, MPH, of Clalit Health Services and Ben-Gurion University of the Negev in Israel, built 225 clinical scenarios from 15 recommendations issued by Choosing Wisely, a long-running U.S. campaign that lists tests and treatments doctors and patients should question because they rarely help and sometimes cause harm. Each scenario described a specific patient situation, such as a healthy adult requesting a routine heart scan or a patient with an incidentally discovered small thyroid nodule.

For each of the 15 recommendations, the researchers wrote 10 scenarios in which the right move was restraint (skipping the test or treatment) and 5 scenarios in which action was actually warranted. They then put all 225 scenarios to six commercial large language models, with each model answering every scenario three times, for 4,050 responses in total — 2,700 on the restraint scenarios and 1,350 on the action scenarios. That split matters: the 150 restraint scenarios (times six models and three runs) are where “unnecessary care” can be measured, while the 75 action scenarios are where “wrongly withholding care” can be measured. The two error rates below come from those two different slices, not from one shared pool of 4,050 responses.

Choosing Wisely has been running since 2012, built around a simple idea: ask medical specialty societies themselves to name the tests and treatments their own members order too often. The campaign’s lists now cover hundreds of recommendations, from skipping antibiotics for a simple cold to avoiding imaging for uncomplicated lower-back pain. Using it as the study’s yardstick means the “right answer” for each scenario was not the researchers’ own judgment call — it came from the specialists most familiar with each procedure’s actual value.

Low Overuse, With Exceptions

Across the scenarios where restraint was the right call, the AI models recommended potentially unnecessary care in just 5.4% of responses. For eight of the fifteen Choosing Wisely recommendations tested, the models never once suggested the unnecessary test or treatment — a result that held steady across scenarios involving imaging, lab work, and routine screening alike.

In scenarios where restraint was the right answer, AI chatbots still recommended unnecessary care 5.4% of the time overall — but that average hid a couple of scenarios where they got it wrong nearly half the time.

On the other side of the ledger, the models were also cautious about swinging too far toward withholding care altogether. In the scenarios where testing or treatment was actually warranted, they failed to recommend it — a pattern the researchers call “false restraint” — in only 0.6% of responses. In other words, the dominant failure mode by far was ordering too much, not too little, but neither failure was common.

Where the AI Models Stumbled

The 5.4% average mask two restraint scenarios where the models struggled far more than usual. The worst was thyroid ultrasound imaging for a small, incidentally discovered nodule with no worrying features: the AI models recommended the unnecessary scan in 42.8% of their responses to that scenario. The second-worst was routine electrocardiogram (ECG) screening for low-risk adults with no symptoms, where models recommended the unneeded test in 28.3% of their responses.

On the false-restraint side, the single riskiest category was prostate-specific antigen (PSA) testing, where the models wrongly withheld a test that was actually appropriate in 3.3% of their responses to that scenario — still low, but the highest rate among the categories studied.

Why Doctors Themselves Can’t Agree

The authors point to a possible explanation that has nothing to do with AI being broken: on thyroid nodules specifically, medical societies themselves do not agree. Different professional guidelines draw the line for “worth scanning” at different nodule sizes and risk profiles, and primary care doctors in the real world follow that same disputed judgment call in different directions. One plausible reading is that large language models, trained heavily on published medical literature and real clinical text, picked up on that same disagreement — though the study itself does not prove which training material caused the errors, only that the errors cluster exactly where human guidance is split.

That is a notably different story from an AI simply failing to learn the rules. It suggests these models are not making random errors; their mistakes line up with the same contested judgment calls that divide human doctors.

What This Means for Patients and Doctors

An editorial published alongside the study, written by Dr. Deborah Korenstein of the Icahn School of Medicine at Mount Sinai and Dr. Daniel Morgan of the VA Maryland Health Care System and University of Maryland, cautions against reading too much into a benchmark built from hypothetical scenarios. Real clinical decisions involve a patient’s full history, a physical exam, and a conversation — not a paragraph of text handed to a chatbot. The editorial argues that any claim about AI’s real clinical value needs to be tested with actual patient encounters, not written vignettes.

Even so, the stakes are real and rising. Patients increasingly ask AI chatbots for a second opinion before or after seeing a doctor, and some clinicians already lean on AI tools to help triage test requests. If an AI model inherits a human blind spot rather than correcting for it, it can quietly reinforce the same overtesting it was supposed to help reduce — particularly in exactly the gray-area cases where patients most need a clear, confident answer. That risk echoes a broader pattern researchers have flagged elsewhere: a Berkeley study found that leaning on AI for help can quietly erode a person’s own persistence and judgment over time, which is worth keeping in mind any time a difficult, ambiguous decision gets outsourced to a chatbot by default.

The study did not measure real-world costs, but the shape of the risk is easy to see: if similar errors occur often enough in actual chatbot use, they could add up to unnecessary follow-up scans, biopsies, and patient anxiety over findings that were never going to matter. That remains a hypothesis, not something this benchmark demonstrated.

The Limits of a Multiple-Choice Test

Choosing Wisely scenarios are, by design, close to a clean multiple-choice test: a well-defined clinical situation with an established “right” answer from a respected guideline body. Most real medical decisions are messier, shaped by a patient’s anxiety, family history, insurance coverage, and a dozen details a short text prompt cannot capture. The study’s own authors frame their 5.4% overuse rate as reassuring but incomplete — strong evidence that today’s leading AI models have absorbed the broad principle of “don’t test without a reason,” paired with a reminder that they still default to the crowd’s judgment, flaws included, in the cases where the crowd itself is unsure.

For now, the practical takeaway is less about whether AI chatbots are generally “safe” for medical questions and more about where their blind spots sit. In this benchmark, the models rarely withheld care that was genuinely needed, but unnecessary testing remained a real problem in a handful of gray-area scenarios. Because the study used hypothetical vignettes rather than real patients, more research is needed before anyone can say how these patterns play out in an actual exam room — which is exactly why a chatbot’s answer on a borderline test is worth raising with an actual doctor rather than treating as a final word.

Sources: American College of Physicians / Newswise press release, AuntMinnie, Annals of Internal Medicine (DOI)