Xsciens

WorldMAP NOW

WorldMAP NOW

Tag: Human-AI Interaction

  • [Science] AI Mostly Avoids Unnecessary Medical Tests — But Stumbles Where Doctors Disagree

    [Science] AI Mostly Avoids Unnecessary Medical Tests — But Stumbles Where Doctors Disagree

    Ask a popular AI chatbot whether you need a thyroid ultrasound for a small nodule your doctor found by accident, and there is a real chance it will tell you to get one anyway — even though medical guidelines say not to bother. A large new benchmark study finds that commercial AI models mostly avoid recommending unnecessary medical tests and treatments when restraint is the right call. But in the small slice of cases where human doctors themselves disagree about what counts as “necessary,” the AI stumbles right alongside them.

    原論文情報 / Original Paper
    Title: Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios
    Authors: Dean Ariel, MD, MPH, et al.
    Journal: Annals of Internal Medicine
    DOI: 10.7326/ANNALS-26-01667

    What the Study Actually Tested

    The research team, led by Dean Ariel, MD, MPH, of Clalit Health Services and Ben-Gurion University of the Negev in Israel, built 225 clinical scenarios from 15 recommendations issued by Choosing Wisely, a long-running U.S. campaign that lists tests and treatments doctors and patients should question because they rarely help and sometimes cause harm. Each scenario described a specific patient situation, such as a healthy adult requesting a routine heart scan or a patient with an incidentally discovered small thyroid nodule.

    For each of the 15 recommendations, the researchers wrote 10 scenarios in which the right move was restraint (skipping the test or treatment) and 5 scenarios in which action was actually warranted. They then put all 225 scenarios to six commercial large language models, with each model answering every scenario three times, for 4,050 responses in total — 2,700 on the restraint scenarios and 1,350 on the action scenarios. That split matters: the 150 restraint scenarios (times six models and three runs) are where “unnecessary care” can be measured, while the 75 action scenarios are where “wrongly withholding care” can be measured. The two error rates below come from those two different slices, not from one shared pool of 4,050 responses.

    Choosing Wisely has been running since 2012, built around a simple idea: ask medical specialty societies themselves to name the tests and treatments their own members order too often. The campaign’s lists now cover hundreds of recommendations, from skipping antibiotics for a simple cold to avoiding imaging for uncomplicated lower-back pain. Using it as the study’s yardstick means the “right answer” for each scenario was not the researchers’ own judgment call — it came from the specialists most familiar with each procedure’s actual value.

    Low Overuse, With Exceptions

    Across the scenarios where restraint was the right call, the AI models recommended potentially unnecessary care in just 5.4% of responses. For eight of the fifteen Choosing Wisely recommendations tested, the models never once suggested the unnecessary test or treatment — a result that held steady across scenarios involving imaging, lab work, and routine screening alike.

    In scenarios where restraint was the right answer, AI chatbots still recommended unnecessary care 5.4% of the time overall — but that average hid a couple of scenarios where they got it wrong nearly half the time.

    On the other side of the ledger, the models were also cautious about swinging too far toward withholding care altogether. In the scenarios where testing or treatment was actually warranted, they failed to recommend it — a pattern the researchers call “false restraint” — in only 0.6% of responses. In other words, the dominant failure mode by far was ordering too much, not too little, but neither failure was common.

    Where the AI Models Stumbled

    The 5.4% average mask two restraint scenarios where the models struggled far more than usual. The worst was thyroid ultrasound imaging for a small, incidentally discovered nodule with no worrying features: the AI models recommended the unnecessary scan in 42.8% of their responses to that scenario. The second-worst was routine electrocardiogram (ECG) screening for low-risk adults with no symptoms, where models recommended the unneeded test in 28.3% of their responses.

    On the false-restraint side, the single riskiest category was prostate-specific antigen (PSA) testing, where the models wrongly withheld a test that was actually appropriate in 3.3% of their responses to that scenario — still low, but the highest rate among the categories studied.

    Chart showing AI chatbots recommended unnecessary care in 5.4% of responses overall, 42.8% for thyroid ultrasound, 28.3% for routine ECG, and 3.3% false restraint for PSA testing
    Source: Data from Ariel et al., Annals of Internal Medicine (2026)

    Why Doctors Themselves Can’t Agree

    The authors point to a possible explanation that has nothing to do with AI being broken: on thyroid nodules specifically, medical societies themselves do not agree. Different professional guidelines draw the line for “worth scanning” at different nodule sizes and risk profiles, and primary care doctors in the real world follow that same disputed judgment call in different directions. One plausible reading is that large language models, trained heavily on published medical literature and real clinical text, picked up on that same disagreement — though the study itself does not prove which training material caused the errors, only that the errors cluster exactly where human guidance is split.

    That is a notably different story from an AI simply failing to learn the rules. It suggests these models are not making random errors; their mistakes line up with the same contested judgment calls that divide human doctors.

    What This Means for Patients and Doctors

    An editorial published alongside the study, written by Dr. Deborah Korenstein of the Icahn School of Medicine at Mount Sinai and Dr. Daniel Morgan of the VA Maryland Health Care System and University of Maryland, cautions against reading too much into a benchmark built from hypothetical scenarios. Real clinical decisions involve a patient’s full history, a physical exam, and a conversation — not a paragraph of text handed to a chatbot. The editorial argues that any claim about AI’s real clinical value needs to be tested with actual patient encounters, not written vignettes.

    Even so, the stakes are real and rising. Patients increasingly ask AI chatbots for a second opinion before or after seeing a doctor, and some clinicians already lean on AI tools to help triage test requests. If an AI model inherits a human blind spot rather than correcting for it, it can quietly reinforce the same overtesting it was supposed to help reduce — particularly in exactly the gray-area cases where patients most need a clear, confident answer. That risk echoes a broader pattern researchers have flagged elsewhere: a Berkeley study found that leaning on AI for help can quietly erode a person’s own persistence and judgment over time, which is worth keeping in mind any time a difficult, ambiguous decision gets outsourced to a chatbot by default.

    The study did not measure real-world costs, but the shape of the risk is easy to see: if similar errors occur often enough in actual chatbot use, they could add up to unnecessary follow-up scans, biopsies, and patient anxiety over findings that were never going to matter. That remains a hypothesis, not something this benchmark demonstrated.

    The Limits of a Multiple-Choice Test

    Choosing Wisely scenarios are, by design, close to a clean multiple-choice test: a well-defined clinical situation with an established “right” answer from a respected guideline body. Most real medical decisions are messier, shaped by a patient’s anxiety, family history, insurance coverage, and a dozen details a short text prompt cannot capture. The study’s own authors frame their 5.4% overuse rate as reassuring but incomplete — strong evidence that today’s leading AI models have absorbed the broad principle of “don’t test without a reason,” paired with a reminder that they still default to the crowd’s judgment, flaws included, in the cases where the crowd itself is unsure.

    For now, the practical takeaway is less about whether AI chatbots are generally “safe” for medical questions and more about where their blind spots sit. In this benchmark, the models rarely withheld care that was genuinely needed, but unnecessary testing remained a real problem in a handful of gray-area scenarios. Because the study used hypothetical vignettes rather than real patients, more research is needed before anyone can say how these patterns play out in an actual exam room — which is exactly why a chatbot’s answer on a borderline test is worth raising with an actual doctor rather than treating as a final word.

    Sources: American College of Physicians / Newswise press release, AuntMinnie, Annals of Internal Medicine (DOI)

  • [Science] What Is Cognitive Offloading? A Simple Guide to How It Works

    [Science] What Is Cognitive Offloading? A Simple Guide to How It Works

    Every time you set a phone reminder instead of memorizing an appointment, or ask a chatbot to summarize a report instead of reading it yourself, you are doing something psychologists call cognitive offloading. The habit itself is ancient, but a wave of recent research is asking a sharper question: what happens to our own performance in the moments when the tool isn’t there?

    Source Paper
    Title: Cognitive Offloading
    Authors: Evan F. Risko, Sam J. Gilbert
    Journal: Trends in Cognitive Sciences, Vol. 20, No. 9, pp. 676–688 (2016)
    DOI: 10.1016/j.tics.2016.07.002

    What Is Cognitive Offloading?

    Cognitive offloading is the use of physical action or an external tool to reduce the mental effort a task would otherwise require. Psychologists Evan Risko (University of Waterloo) and Sam Gilbert (University College London) published an influential review of the concept in 2016, in Trends in Cognitive Sciences, which remains the field’s clearest framework for the idea. Rather than solving a problem entirely “in your head,” you shift part of the work onto your body or your environment.

    It’s a far older behavior than smartphones. Tilting your head to read an upside-down sign, counting on your fingers, or jotting a phone number on a scrap of paper are all forms of cognitive offloading. What has changed is the scale and power of the tools available — calculators, GPS, search engines, and now generative AI assistants can take over not just small chores but entire chains of reasoning.

    The idea is closely related to the “extended mind” thesis, proposed by philosophers Andy Clark and David Chalmers in 1998. They argued that a notebook carried everywhere and consulted automatically can function, for practical purposes, as part of a person’s memory system — their famous example involved a man named Otto who used a notebook the way most people use their own recall. Risko and Gilbert’s later work gave this philosophical idea an empirical, testable framework: when exactly do people choose to offload, and what happens to their underlying skills when they do it repeatedly?

    How It Works: A Decision Loop, Not a Switch

    Imagine being asked to remember five errands on the way home. You could hold them all in mind, or you could write them down. Risko and Gilbert’s framework treats this as a running cost-benefit calculation rather than a one-time choice: the decision depends on how demanding the task feels, how confident you are in your own memory, and how much effort writing the list would save.

    The important detail is that this calculation runs on perceived demand and confidence, not necessarily on actual ability — and people’s self-assessments can be mistaken, so they sometimes reach for an external aid even when they could have managed without it. Whether repeated reliance on a tool gradually changes a person’s confidence or unaided performance over the longer term is a related but separate question, one that is harder to answer than the moment-to-moment decision itself. Offloading isn’t inherently a problem — compensating for a busy week with a written list is a perfectly sensible trade — but it is a trade worth noticing rather than making automatically.

    Diagram of the cognitive offloading decision loop
    Source: created based on Risko & Gilbert (2016), Trends in Cognitive Sciences

    Why It’s Back in the Spotlight Now

    Cognitive offloading has been studied for decades, but generative AI has pushed it from a niche academic topic into a mainstream concern. A widely discussed 2011 study in Science by Betsy Sparrow, Jenny Liu, and Daniel Wegner found that once people believed information would remain accessible online, they remembered where to find it far better than the information itself — a pattern nicknamed the “Google effect.” That study is a useful historical analogy rather than direct proof of what AI chatbots do to the brain: a search engine mainly stores and retrieves information, while an AI assistant can also perform reasoning-like tasks, draft text, and summarize on a person’s behalf, which is a different kind of delegation that researchers are only beginning to study on its own terms.

    The question researchers are now asking is not whether people offload thinking to AI, but how much of their own judgment they lose when the AI isn’t there.

    A 2025 study published in Societies by Michael Gerlich surveyed 666 participants and reported a statistically significant negative correlation between frequent AI tool use and scores on critical-thinking tasks, with younger, more AI-reliant participants scoring lowest. Because the study measured AI use and critical-thinking ability at a single point in time, it cannot determine whether frequent AI use weakens critical thinking, whether people with different existing abilities simply use AI differently, or whether some other factor explains both — but it adds to a growing body of research raising the same question from different angles.

    Everyday Examples You Already Use

    Cognitive offloading shows up in ordinary moments far more often than most people realize. A few familiar examples: writing a shopping list instead of memorizing items, letting a smartphone calendar track every appointment, typing a destination into GPS navigation instead of memorizing the route, and asking a search engine a factual question instead of trying to recall it. In most of these cases the trade is a fair one — a shopping list, for instance, tends to help you finish the errand without costing you much else. The more interesting question researchers are asking is which kinds of offloading are closer to that fair trade, and which carry a greater cost to independent performance later on.

    A 2020 study in Scientific Reports, led by Louisa Dahmani and Véronique Bohbot at McGill University, examined this trade-off directly with GPS. In a sample of 50 regular drivers, heavier GPS use was associated with weaker spatial memory on unaided navigation tasks. A smaller follow-up of 13 of the same participants about three years later found that people whose GPS use had grown the most over that period also showed the steepest declines in spatial memory — a dose-response pattern the authors say is more consistent with GPS use contributing to the decline than the reverse, though they are careful to flag their small sample size and note that the direction of cause and effect is not fully settled.

    Not every example involves memory. Risko and Gilbert’s framework also covers offloading onto the body itself — pointing at an object instead of holding its location in mind, or using a finger to keep a place while reading a long list — and “intention offloading,” where a sticky note or phone alert carries the job of remembering to act later, rather than the job of remembering the information itself. A related but distinct idea is automation, where a system completes part of a task with no decision from the user at all. A GPS app that both tracks your route and reroutes you around traffic without being asked is doing more than simple offloading; it raises its own, separate questions about what happens to a skill when a person is never required to exercise it in the first place.

    What Recent Research Shows

    The clearest recent demonstration comes from a UC Berkeley–led study published in 2026. Researchers ran three randomized trials with 1,222 participants who solved math and reading problems either with or without ChatGPT. Everyone then had to solve a final set of problems entirely unaided. Participants who had previously relied on the AI scored markedly lower on those unaided problems than participants who had worked alone from the start — in one trial, 57% versus 73% correct. Related article: AI Assistance Reduces Persistence and Hurts Independent Performance looks at this study’s full results and the researchers’ own caveats about how far the findings generalize.

    None of this means offloading is simply bad. In a self-reported, non-randomized part of the same study, participants who said they had copied AI-generated answers directly scored the lowest on the unaided follow-up problems (65% correct on average), while participants who said they had used the AI only for hints scored close to those who never used AI at all. Because that comparison is based on what participants reported about their own behavior rather than a randomized condition, it should be read as a suggestive pattern rather than a proven result. Still, it points toward a recurring theme across this research: the question is less whether to offload and more how — leaning on a tool for a finished answer looks different, in its effects, from using it as a scaffold while you still do the thinking yourself.

    Sources: Risko & Gilbert, “Cognitive Offloading,” Trends in Cognitive Sciences (2016), Gerlich, “AI Tools in Society,” Societies (2025), Columbia News on Sparrow et al., Science (2011), American Psychological Association Monitor (2026)

    See also: Why Letting AI Help Can Erode Your Grit and AI Summaries Can Distort Human Memory.

  • [Science] Why Letting AI Help Can Erode Your Grit, Berkeley Study Finds

    [Science] Why Letting AI Help Can Erode Your Grit, Berkeley Study Finds

    A chatbot that hands you the answer can make a hard problem feel easy — until the chatbot is gone. A new study from researchers at five universities found that people who solved problems with an AI assistant performed worse on the next problems, once that assistant was taken away, than people who had worked through the very same problems alone from the start. The gap showed up after roughly 10 to 15 minutes of AI-assisted problem-solving, and it held across three separate experiments involving 1,222 recruited participants — a pattern the authors say raises real questions about whether leaning on AI for answers can undermine people’s ability, or willingness, to solve problems on their own.

    Three Experiments, One Pattern

    The team — Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker and Rachit Dubey, drawing on UC Berkeley’s Center for Human-Compatible AI along with Carnegie Mellon University, MIT, the University of Oxford and UCLA — ran three randomized controlled trials totaling 1,222 recruited participants (307, 585 and 168 after some participants were excluded under the researchers’ prespecified criteria). In the first experiment, 354 people were recruited to solve 12 basic fraction problems; half worked alone throughout, while the other half could ask a ChatGPT assistant, built on GPT-5, for help — before everyone, AI group included, had to solve three further problems entirely on their own. A second experiment repeated that design with 667 people, and a third, with 201 participants, swapped the math problems for SAT-style reading comprehension questions.

    In every version of the study, the early pattern looked encouraging: while they still had access to the AI, the assisted group solved problems faster and scored higher than the group working alone — exactly what you would expect from a capable assistant. The twist came in the second half of each session, once the AI was removed and everyone, including the AI group, had to finish the rest of the problems unaided.

    The Numbers Behind the Finding

    Once AI access ended, the previously assisted group fell behind the group that had never used it, and the gap was consistent across all three experiments. In the first fraction experiment, the former AI group answered an average of 57% of the three final problems correctly, versus 73% for the no-AI group — a statistically significant difference (p < 0.001). The larger replication of the same design found a smaller but still reliable gap: 71% versus 77% (p = 0.02). In the reading-comprehension experiment, the difference widened again, to 76% versus 89% (p = 0.007).

    Bar chart comparing solve rates after AI was removed across three experiments
    Source: data from Liu, Christian, Dumbalska, Bakker & Dubey (arXiv:2604.04721) and UC Berkeley News

    Giving up altogether followed a similar trend. In the first experiment, 20% of the former AI group skipped a problem rather than attempt it, compared with 11% of the no-AI group. The pattern was weaker in the larger replication (10% versus 7%, not statistically significant) but reappeared clearly in the reading task, where 8% of the AI group skipped questions versus just 1% of the control group. A closer look at how participants said they had actually used the AI while it was available sharpened the picture further, though this part of the analysis was observational rather than randomized, since people were not assigned to copy answers or ask for hints: those who reported simply copying the AI’s finished answers fared worst afterward, averaging just 65% correct on their own, while those who said they used the AI mainly for hints showed no statistically significant gap from people who never touched AI at all.

    Participants were randomly assigned to the AI or no-AI condition, and the larger replication added a pretest to check the groups further: accuracy and the tendency to skip a problem did not differ significantly between groups beforehand, which strengthens the case that the later gap reflects the AI condition itself rather than pre-existing differences between participants. In that same larger replication, participants were also asked how they had actually used the AI while it was available. A small slice, about 12%, said they had barely used it despite having access, and this group performed about as well afterward as people who never had AI access at all — reinforcing the idea that it is the degree of reliance on AI, not mere availability of it, that predicts the later drop-off.

    “It’s a striking finding, but in a way it supplies ammunition for a story that a lot of us kind of feel in our gut.” — Brian Christian, UC Berkeley

    What Might Explain the Pattern

    The authors propose two explanations, while cautioning that their experiments were not designed to prove which one is doing the work. The first is a kind of expectation mismatch: once someone gets used to near-instant answers, grinding through a problem by hand can suddenly feel disproportionately slow, which may make people more willing to give up rather than push through. The second is the loss of what the researchers call “productive struggle” — the idea that difficulty itself is not simply an obstacle to route around, but part of how people practice and strengthen the skills a problem requires. Bypass that effort often enough, the argument goes, and there may be fewer chances for those skills to get built in the first place.

    It’s a different mechanism, but this study joins a related question this site has covered before: Georgetown University researchers recently found that AI-generated summaries of events can distort people’s memory of what actually happened. That study is about memory rather than problem-solving, but together the two point to a broader pattern worth watching — handing a mental task to AI can carry a hidden cost to accuracy or independent ability, even when it feels like pure convenience in the moment.

    “Not Just Fractions and SAT Problems”

    Christian has been blunt about how far he thinks the implications reach beyond a psychology lab. “This is not a story about fractions and SAT problems,” he said in comments accompanying the study’s release. “This is something that is happening to human knowledge and human expertise.” He described the irony directly: widely used AI assistants, he said, are “often helping us in ways that are kind of unhelpful” — lifting performance on the task directly in front of us while, the data suggest, working against the independent competence we will need the next time no AI is around.

    The caveats matter here. The study measured performance minutes after AI access ended, not months or years later, so it cannot say how long this effect lasts or whether frequent users eventually adapt and recover. The breakdown by how people used the AI — copying answers versus asking for hints — was observational rather than randomized, so it shows an association rather than definitive proof of cause and effect. And all three experiments involved short, one-off tasks completed by paid online participants, a different setting from a student using AI across a semester of coursework or an employee weaving it into daily work.

    What This Means for Using AI

    None of this is an argument for avoiding AI assistance altogether — the same data show it clearly boosted short-term performance while it was available, and for many real-world tasks, getting the right answer quickly is exactly the point. The more useful takeaway from this one study may be about how AI gets used rather than whether: in the self-reported usage data, the steepest drop-off belonged to people who let the AI hand them a finished answer, while those who said they used it mainly for hints or partial guidance looked much closer to the group that never used AI at all. Treating a chatbot more like a tutor that nudges you toward a solution, rather than a vending machine that dispenses the finished one, looks like a reasonable bet for keeping your own problem-solving sharp — though it’s one study, in a narrow setting, and confirming how far that advice travels beyond fractions and reading quizzes will take further research.

    Sources: UC Berkeley News, arXiv:2604.04721, “AI Assistance Reduces Persistence and Hurts Independent Performance”