Xsciens

WorldMAP NOW

WorldMAP NOW

Tag: Artificial Intelligence

  • [Science] What Is Backpropagation? A Simple Guide to How AI Learns

    [Science] What Is Backpropagation? A Simple Guide to How AI Learns

    Backpropagation is the algorithm that lets artificial neural networks learn from their own mistakes, and it quietly powers much of today’s AI — from chatbots that answer your questions to the apps that recognize a face in a photo. Understanding it, even at a basic level, is one of the clearest ways to understand how modern deep-learning systems actually learn.

    Original Paper

    Title: Learning representations by back-propagating errors
    Authors: David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams
    Journal: Nature, Vol. 323, pp. 533–536 (1986)
    DOI: 10.1038/323533a0

    What Is Backpropagation, Exactly?

    In plain terms, backpropagation — short for “backward propagation of errors,” also known as error backpropagation — is a method for training a neural network by repeatedly working out how wrong its answer was, and in which direction each connection should adjust. A neural network is built from layers of simple, interconnected units (loosely inspired by neurons) joined by adjustable connection strengths called weights. When a network is first created, its weights are usually set to small, carefully chosen random values, so its first guesses are generally poor. Backpropagation is the procedure that turns that rough starting point into a network that can recognize a cat in a photo, translate a sentence, or predict the next word in a reply.

    There is one important distinction worth keeping straight: backpropagation calculates how much each connection contributed to the error, while a separate step called gradient descent uses that information to actually adjust the connections. People often use “backpropagation” loosely to describe the whole training process, but technically the two are different steps working together.

    How the Algorithm Actually Works

    Training with backpropagation happens in two passes. In the forward pass, an input — a photo, a sentence, a row of sensor data — moves through the network’s layers in order, and the network produces a prediction. That prediction is then compared with the correct answer using a loss function, a numerical measure of how far the prediction was from the desired result.

    Backpropagation doesn’t tell a network what to think. It tells the network how wrong it was, and in which direction each connection should move to be a little less wrong next time.

    A simple analogy helps before the math: imagine a product moving through several stages of a factory line, and the finished item turns out defective. To find the cause, managers work backward from the end of the line, and each stage reports how much it may have contributed to the problem, passing that information to the stage before it. The backward pass in backpropagation does something similar, except it uses calculus to work this out exactly. Starting at the output layer and moving back toward the input, the algorithm applies the chain rule — a technique for tracking how a change at one stage ripples through a chain of dependent calculations. Each layer combines information about how its own calculations affected the next layer with the error information passed back from that layer, producing a gradient: a measure of how sensitive the loss is to a small change in each weight, at its current value.

    Diagram of the forward pass and backward pass in backpropagation
    The forward pass produces a prediction and a loss; the backward pass applies the chain rule, layer by layer, to compute gradients and update the weights.

    Once every weight has a gradient, gradient descent uses it to actually update the weights, nudging each one a small amount in the direction that reduces the error. Repeat this forward-pass/backward-pass/update cycle over thousands or millions of examples, and a network that started with randomly initialized connections gradually comes to reflect real structure in the data. The appeal of backpropagation is efficiency: it calculates gradients for every weight in a network in roughly the time needed for a small number of forward and backward passes, rather than requiring a separate experiment for each individual weight.

    Why Backpropagation Is Getting Attention Again

    Backpropagation is not new, and its mathematical core had already been explored before it became famous. Reverse-mode automatic differentiation — the calculus technique at the heart of the algorithm — traces back to Finnish researcher Seppo Linnainmaa’s 1970 master’s thesis, and American researcher Paul Werbos proposed applying a related idea to neural networks in his 1974 doctoral thesis. The method became particularly influential after David Rumelhart, Geoffrey Hinton and Ronald Williams published “Learning representations by back-propagating errors” in Nature in 1986, demonstrating that it could let multilayer networks automatically discover useful internal features rather than relying entirely on engineers to design them by hand.

    For years afterward, backpropagation-trained networks were mostly a research curiosity, limited by available data and computing power. That changed dramatically in 2012, when Alex Krizhevsky and Ilya Sutskever, working with their advisor Geoffrey Hinton, combined backpropagation, the large labeled ImageNet dataset, and graphics-card computing power in a network known as AlexNet. It dramatically outperformed competing approaches in that year’s ImageNet image-recognition competition, a result widely credited with triggering the modern deep learning boom. The large language models behind today’s generative AI chatbots are, at their core, still trained with variations of the same algorithm.

    Timeline of key milestones in the history of backpropagation, from 1970 to the 2020s
    Source: created based on Linnainmaa (1970), Werbos (1974), Rumelhart, Hinton & Williams, Nature (1986), and the AlexNet/ImageNet milestone (2012).

    What has put backpropagation back in the spotlight recently is a different question: does the brain do anything like this? In October 2026, a team at MIT’s McGovern Institute, led by Qianli Liao, Liu Ziyin and Yulu Gan with senior authors Mark Harnett and Tomaso Poggio, published a study in the Proceedings of the National Academy of Sciences proposing a computational model in which randomly connected artificial networks can self-organize their own forward and feedback pathways, using only local learning rules inspired by biological synapses and no global error signal — and still learn nearly as well as backpropagation on the tasks tested. The researchers are careful to frame this as a candidate mechanism and a testable hypothesis, not proof that real brains work this way, but it reopens a question that has nagged neuroscientists for roughly 40 years: backpropagation is clearly powerful enough to drive today’s AI, yet nothing in a biological brain has been shown to do precisely what it requires. (Related article: A Simulated Brain Circuit Builds Its Own AI-Like Learning System, MIT Finds.)

    Where You’ve Already Met Backpropagation

    Even though the math stays behind the scenes, the results of backpropagation-trained networks are part of ordinary life. Many modern face-unlock features and the photo apps that automatically group pictures of the same person run on neural networks trained with backpropagation or related gradient-based methods. Voice assistants that transcribe speech or answer spoken questions generally rely on networks trained the same way. Generative AI chatbots are also built using neural networks trained with backpropagation — during an early stage called pretraining, for example, a language model repeatedly tries to predict the next word in a passage of text, compares its guess with the actual word, and adjusts its weights to reduce the error; later training stages then refine how it follows instructions and responds to feedback.

    It’s worth keeping one distinction in mind: backpropagation mainly happens during training, not every time you use these tools. Once a network has learned, it typically uses its already-adjusted weights to make predictions, without recalculating gradients or updating itself each time — the heavy lifting happened earlier, often across billions of training examples.

    What to Watch Next

    Backpropagation itself is a mature, well-understood technique, so the open questions now sit in its limitations and the alternatives researchers are exploring. One is scientific: whether the MIT team’s self-organizing, locally-plausible learning circuits — or related approaches sometimes called feedback alignment or predictive coding — can be tested directly against real neural tissue, and whether they can match backpropagation’s performance on problems much larger than the ones tested so far. Another is practical: as AI models keep growing, researchers continue to look for training methods that need less computing power than standard backpropagation while keeping its accuracy. The next breakthrough may not come from making backpropagation itself faster, but from discovering how networks can learn without depending on it so completely — which is exactly the question AI researchers and neuroscientists are now asking together.

    Xsciens Editorial Note

    Backpropagation is best understood as a precise bookkeeping trick for calculating how a network should adjust itself, not as a theory of how any brain, artificial or biological, actually “thinks.”

    • A common misconception. “Neural network” learning is often described as though it copies how the brain learns, since artificial “neurons” were loosely inspired by biological ones. In practice, standard backpropagation’s backward calculation relies on precise weight-related information tied to the exact connections used for forward signaling — something no real neuron is known to reproduce or transmit in the same way. (Some alternative methods, such as feedback alignment, relax this requirement, which is itself a sign of how specific the original objection is.) The resemblance between artificial and biological “neurons” is a loose analogy, not a working description of what either one computes.
    • Limits and open questions. A common misreading is that backpropagation always needs a human-provided “correct answer,” like a teacher grading homework. That’s not quite right: it needs a differentiable training objective, which can come from supervised labels, but also from self-supervised tasks (such as predicting the next word in a sentence) or reinforcement-style rewards. What backpropagation does not explain, on its own, is why some network architectures generalize well to new data and others don’t, or how an AI system might decide what is worth learning in the first place — those remain active, only partly answered research questions, not settled ones.
    • What to watch next. The MIT team’s October 2026 study proposes, as a specific and testable hypothesis, that a four-synapse wiring motif linking ascending and descending pathways in the cortex could let real circuits self-organize a similar learning ability. This is a model prediction, not a confirmed feature of brain tissue — whether neuroscientists actually find that wiring pattern, and whether locally-plausible learning rules can match backpropagation’s performance on problems far larger than the ones tested so far, are the two developments worth watching.
    • From the editors — If brain-inspired, locally-learning AI matures the way this MIT team hopes, the assistants people use every day could stop feeling like blank tools that need re-explaining each time and start quietly adapting to how each person actually works. Getting there would likely mean systems that keep adjusting themselves after they leave the lab, learning from ordinary daily use instead of one big retraining cycle. That same ability to keep learning on its own is exactly what to watch: without a way to check what a system has quietly picked up, it could just as easily drift toward learning the wrong lesson from unusual or misleading experience. This is still an early computational demonstration, not a finished product — so if an assistant ever starts to feel like it “gets” you a little faster, it’s worth asking, every so often, what it actually learned to get there.

    References for this note: Quanta Magazine — Artificial Neural Nets Finally Yield Clues to How Brains Learn, arXiv — Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment

    Sources: IBM — What Is Backpropagation?, TechTarget — What Is a Backpropagation Algorithm?, Rumelhart, Hinton & Williams, Nature (1986), Seppo Linnainmaa, Computer History Museum — AlexNet Source Code, Xsciens — A Simulated Brain Circuit Builds Its Own AI-Like Learning System, MIT Finds

  • [Science] AI Mostly Avoids Unnecessary Medical Tests — But Stumbles Where Doctors Disagree

    [Science] AI Mostly Avoids Unnecessary Medical Tests — But Stumbles Where Doctors Disagree

    Ask a popular AI chatbot whether you need a thyroid ultrasound for a small nodule your doctor found by accident, and there is a real chance it will tell you to get one anyway — even though medical guidelines say not to bother. A large new benchmark study finds that commercial AI models mostly avoid recommending unnecessary medical tests and treatments when restraint is the right call. But in the small slice of cases where human doctors themselves disagree about what counts as “necessary,” the AI stumbles right alongside them.

    原論文情報 / Original Paper
    Title: Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios
    Authors: Dean Ariel, MD, MPH, et al.
    Journal: Annals of Internal Medicine
    DOI: 10.7326/ANNALS-26-01667

    What the Study Actually Tested

    The research team, led by Dean Ariel, MD, MPH, of Clalit Health Services and Ben-Gurion University of the Negev in Israel, built 225 clinical scenarios from 15 recommendations issued by Choosing Wisely, a long-running U.S. campaign that lists tests and treatments doctors and patients should question because they rarely help and sometimes cause harm. Each scenario described a specific patient situation, such as a healthy adult requesting a routine heart scan or a patient with an incidentally discovered small thyroid nodule.

    For each of the 15 recommendations, the researchers wrote 10 scenarios in which the right move was restraint (skipping the test or treatment) and 5 scenarios in which action was actually warranted. They then put all 225 scenarios to six commercial large language models, with each model answering every scenario three times, for 4,050 responses in total — 2,700 on the restraint scenarios and 1,350 on the action scenarios. That split matters: the 150 restraint scenarios (times six models and three runs) are where “unnecessary care” can be measured, while the 75 action scenarios are where “wrongly withholding care” can be measured. The two error rates below come from those two different slices, not from one shared pool of 4,050 responses.

    Choosing Wisely has been running since 2012, built around a simple idea: ask medical specialty societies themselves to name the tests and treatments their own members order too often. The campaign’s lists now cover hundreds of recommendations, from skipping antibiotics for a simple cold to avoiding imaging for uncomplicated lower-back pain. Using it as the study’s yardstick means the “right answer” for each scenario was not the researchers’ own judgment call — it came from the specialists most familiar with each procedure’s actual value.

    Low Overuse, With Exceptions

    Across the scenarios where restraint was the right call, the AI models recommended potentially unnecessary care in just 5.4% of responses. For eight of the fifteen Choosing Wisely recommendations tested, the models never once suggested the unnecessary test or treatment — a result that held steady across scenarios involving imaging, lab work, and routine screening alike.

    In scenarios where restraint was the right answer, AI chatbots still recommended unnecessary care 5.4% of the time overall — but that average hid a couple of scenarios where they got it wrong nearly half the time.

    On the other side of the ledger, the models were also cautious about swinging too far toward withholding care altogether. In the scenarios where testing or treatment was actually warranted, they failed to recommend it — a pattern the researchers call “false restraint” — in only 0.6% of responses. In other words, the dominant failure mode by far was ordering too much, not too little, but neither failure was common.

    Where the AI Models Stumbled

    The 5.4% average mask two restraint scenarios where the models struggled far more than usual. The worst was thyroid ultrasound imaging for a small, incidentally discovered nodule with no worrying features: the AI models recommended the unnecessary scan in 42.8% of their responses to that scenario. The second-worst was routine electrocardiogram (ECG) screening for low-risk adults with no symptoms, where models recommended the unneeded test in 28.3% of their responses.

    On the false-restraint side, the single riskiest category was prostate-specific antigen (PSA) testing, where the models wrongly withheld a test that was actually appropriate in 3.3% of their responses to that scenario — still low, but the highest rate among the categories studied.

    Chart showing AI chatbots recommended unnecessary care in 5.4% of responses overall, 42.8% for thyroid ultrasound, 28.3% for routine ECG, and 3.3% false restraint for PSA testing
    Source: Data from Ariel et al., Annals of Internal Medicine (2026)

    Why Doctors Themselves Can’t Agree

    The authors point to a possible explanation that has nothing to do with AI being broken: on thyroid nodules specifically, medical societies themselves do not agree. Different professional guidelines draw the line for “worth scanning” at different nodule sizes and risk profiles, and primary care doctors in the real world follow that same disputed judgment call in different directions. One plausible reading is that large language models, trained heavily on published medical literature and real clinical text, picked up on that same disagreement — though the study itself does not prove which training material caused the errors, only that the errors cluster exactly where human guidance is split.

    That is a notably different story from an AI simply failing to learn the rules. It suggests these models are not making random errors; their mistakes line up with the same contested judgment calls that divide human doctors.

    What This Means for Patients and Doctors

    An editorial published alongside the study, written by Dr. Deborah Korenstein of the Icahn School of Medicine at Mount Sinai and Dr. Daniel Morgan of the VA Maryland Health Care System and University of Maryland, cautions against reading too much into a benchmark built from hypothetical scenarios. Real clinical decisions involve a patient’s full history, a physical exam, and a conversation — not a paragraph of text handed to a chatbot. The editorial argues that any claim about AI’s real clinical value needs to be tested with actual patient encounters, not written vignettes.

    Even so, the stakes are real and rising. Patients increasingly ask AI chatbots for a second opinion before or after seeing a doctor, and some clinicians already lean on AI tools to help triage test requests. If an AI model inherits a human blind spot rather than correcting for it, it can quietly reinforce the same overtesting it was supposed to help reduce — particularly in exactly the gray-area cases where patients most need a clear, confident answer. That risk echoes a broader pattern researchers have flagged elsewhere: a Berkeley study found that leaning on AI for help can quietly erode a person’s own persistence and judgment over time, which is worth keeping in mind any time a difficult, ambiguous decision gets outsourced to a chatbot by default.

    The study did not measure real-world costs, but the shape of the risk is easy to see: if similar errors occur often enough in actual chatbot use, they could add up to unnecessary follow-up scans, biopsies, and patient anxiety over findings that were never going to matter. That remains a hypothesis, not something this benchmark demonstrated.

    The Limits of a Multiple-Choice Test

    Choosing Wisely scenarios are, by design, close to a clean multiple-choice test: a well-defined clinical situation with an established “right” answer from a respected guideline body. Most real medical decisions are messier, shaped by a patient’s anxiety, family history, insurance coverage, and a dozen details a short text prompt cannot capture. The study’s own authors frame their 5.4% overuse rate as reassuring but incomplete — strong evidence that today’s leading AI models have absorbed the broad principle of “don’t test without a reason,” paired with a reminder that they still default to the crowd’s judgment, flaws included, in the cases where the crowd itself is unsure.

    For now, the practical takeaway is less about whether AI chatbots are generally “safe” for medical questions and more about where their blind spots sit. In this benchmark, the models rarely withheld care that was genuinely needed, but unnecessary testing remained a real problem in a handful of gray-area scenarios. Because the study used hypothetical vignettes rather than real patients, more research is needed before anyone can say how these patterns play out in an actual exam room — which is exactly why a chatbot’s answer on a borderline test is worth raising with an actual doctor rather than treating as a final word.

    Sources: American College of Physicians / Newswise press release, AuntMinnie, Annals of Internal Medicine (DOI)

  • [Economy] Just 3 in 10 Say They’ve Mastered AI as Job-Loss Fears Spread to Over-40s

    [Economy] Just 3 in 10 Say They’ve Mastered AI as Job-Loss Fears Spread to Over-40s

    Generative AI is everywhere in conversation, yet fewer than 3 in 10 people say they’ve actually mastered using it. According to “Quality of Society Index 2026,” an annual survey released on October 19, 2026 by Dentsu Soken, a large generational gap separates confident AI users from everyone else — and over the past year, anxiety about AI taking people’s jobs has spread beyond young workers to reach people in their 40s, 50s, 60s and 70s as well, narrowing a gap that used to be much wider. Here’s what the numbers show, why the gap between confidence and anxiety matters, and what it suggests about how workplaces should be handling AI training right now.

    原論文情報 / Original Paper
    Title: “Quality of Society Index 2026” (クオリティ・オブ・ソサエティ指標2026)
    Authors: Dentsu Soken Inc., Humanology Innovation Division, Quality of Society Center
    Journal: Dentsu Soken Press Release
    DOI: https://www.dentsusoken.com/news/release/2026/1019.html

    Dentsu Soken’s think-tank arm, the Quality of Society Center, has run this survey every year since 2019 to track how people’s attitudes are reshaping Japanese society — covering everything from views on work and community to broader questions about what a “good society” looks like. The 2026 edition was fielded online from June 23 to 30, 2026, among 12,000 men and women aged 18 to 79 across the country, including high school students, through the research firm Dentsu Macromill Insight. With a sample that large, the margin of error works out to at most ±1.3 percentage points at a 95% confidence level. For the first time since the survey began, this year’s edition added questions about how well people feel they use AI, and how comfortable they are letting AI agents handle everyday tasks on their behalf — a timely addition, given how quickly AI tools have moved from novelty to daily habit for many people over the past two years.

    Confidence in AI Still Lags Behind the Hype

    Asked whether they agreed with the statement “I feel I use AI well,” only 29.8% of respondents said yes. Confidence rose with youth, but even the most confident group — 18-to-29-year-olds — topped out at 46.6%, short of a majority. In other words, even among the demographic that grew up with smartphones and is usually assumed to be most comfortable with new technology, most people still don’t feel they’ve got AI figured out. Dentsu Soken reads this as a sign that while conversational generative AI has become familiar — most people have at least tried a chatbot by now — actually using it well remains a trial-and-error process for the majority. It’s worth being precise about what the 29.8% figure actually measures: it’s self-assessed confidence, gathered by asking people how they feel about their own AI use, not an independent test of how skillfully they actually use it. Someone who rarely uses AI at all and someone who uses it daily but still feels unsure could both end up in the “not confident” category.

    Confident Users Trust AI Agents More

    The survey also asked a more forward-looking question: would people accept letting an AI agent — an automated assistant that acts on a person’s behalf rather than simply answering questions — handle three everyday situations? The scenarios were managing money, such as investment decisions; helping spark a new friendship or romantic connection; and booking restaurants or travel. Across all three, people who said they use AI well were consistently more accepting of handing the task to an AI agent than people who didn’t feel confident. That’s a fairly intuitive result — people who already trust their own ability to work with AI are, unsurprisingly, more willing to extend that trust further — but it’s worth noting precisely what the data can and can’t support. The two measures clearly move together, but a single survey like this can only show that confidence and acceptance are correlated among respondents, not that one causes the other. It’s equally possible that people who are generally early adopters of new technology simply score higher on both measures at once.

    Anxiety Jumps 7.8 Points, Gap Narrows

    Regardless of how confident people felt about using AI themselves, anxiety about losing a job to AI grew across the board. Among currently employed respondents (n=5,734 in 2026, n=5,708 in 2025), 30.9% said they felt anxious about AI or automation taking their job, up 7.8 points from 23.1% the year before — a sizable one-year jump for a tracking survey like this. The shift wasn’t evenly distributed by age, though, and that’s arguably the more interesting part of the finding. A year earlier, anxiety about AI displacing jobs was concentrated among younger workers. In the 2026 wave, that anxiety rose among people in their 40s, 50s, 60s and 70s as well, pulling every age bracket toward roughly 30%, and narrowing what had been a fairly wide generational gap. Dentsu Soken points to a year of expanding AI adoption inside Japanese workplaces as a likely factor behind the broader spread of concern. That’s a reasonable reading of the trend, but it’s the survey organization’s own interpretation of the pattern — this kind of survey can show that anxiety rose and spread across ages, but it can’t, on its own, establish workplace AI adoption as the specific cause.

    30.9% of workers now say they’re anxious about losing their job to AI — up 7.8 points in a year, with the generational gap narrowing as older workers caught up to younger ones.
    Bar charts showing AI confidence by age group and the year-over-year rise in job-loss anxiety
    Source: created from data in Dentsu Soken’s “Quality of Society Index 2026.”

    Echoes of an Earlier AI Persistence Study

    Put the two findings together and a pattern emerges: confidence in actually using AI well remains a minority experience, while anxiety about AI’s effect on jobs has outrun it, spreading across generations faster than comfort with the technology has. That pattern echoes something we covered previously on this site: a set of randomized experiments by researchers at UC Berkeley and several partner institutions, involving 1,222 participants across three separate trials, found that people who leaned on AI assistance to solve math or reading problems performed measurably worse afterward — once the AI was taken away — than people who had worked unaided from the start. In the first experiment, people who had used AI got only 57% of follow-up questions right on their own, versus 73% for the group that never had AI help. It’s important to be precise about what kind of evidence each study offers, though. The Berkeley work was a controlled experiment measuring task performance under lab conditions; the Dentsu Soken survey is a broad measure of public sentiment about jobs and confidence. They’re different kinds of evidence answering different questions, and one doesn’t prove the other. But read side by side, they point toward a broader unease about how people are relating to AI as it becomes more capable and more embedded in daily life and work.

    The Real Risk: A Training Gap

    If AI adoption keeps expanding at work — and the survey’s own data suggests it already has, given how sharply anxiety spread this year — companies will likely need to give employees concrete, hands-on ways to learn how to use AI tools well, along with room to experiment without fear of getting it wrong in front of colleagues or managers. This survey alone can’t say precisely what’s driving the rise in anxiety, or whether workers currently have enough opportunities to build real confidence with these tools. But the combination it does show is hard to ignore: confidence in using AI well remains rare, at well under a third of the population even among the young, while anxiety about AI’s effect on jobs has become common across every age group. That gap points to a practical, near-term challenge for employers: treating AI rollout and workforce training as a single connected problem, rather than assuming that giving people access to AI tools is the same thing as teaching them how to use those tools well.

    Sources: Dentsu Soken, “Quality of Society Index 2026” press release, full survey report (PDF)

  • [Science] What Is Cognitive Offloading? A Simple Guide to How It Works

    [Science] What Is Cognitive Offloading? A Simple Guide to How It Works

    Every time you set a phone reminder instead of memorizing an appointment, or ask a chatbot to summarize a report instead of reading it yourself, you are doing something psychologists call cognitive offloading. The habit itself is ancient, but a wave of recent research is asking a sharper question: what happens to our own performance in the moments when the tool isn’t there?

    Source Paper
    Title: Cognitive Offloading
    Authors: Evan F. Risko, Sam J. Gilbert
    Journal: Trends in Cognitive Sciences, Vol. 20, No. 9, pp. 676–688 (2016)
    DOI: 10.1016/j.tics.2016.07.002

    What Is Cognitive Offloading?

    Cognitive offloading is the use of physical action or an external tool to reduce the mental effort a task would otherwise require. Psychologists Evan Risko (University of Waterloo) and Sam Gilbert (University College London) published an influential review of the concept in 2016, in Trends in Cognitive Sciences, which remains the field’s clearest framework for the idea. Rather than solving a problem entirely “in your head,” you shift part of the work onto your body or your environment.

    It’s a far older behavior than smartphones. Tilting your head to read an upside-down sign, counting on your fingers, or jotting a phone number on a scrap of paper are all forms of cognitive offloading. What has changed is the scale and power of the tools available — calculators, GPS, search engines, and now generative AI assistants can take over not just small chores but entire chains of reasoning.

    The idea is closely related to the “extended mind” thesis, proposed by philosophers Andy Clark and David Chalmers in 1998. They argued that a notebook carried everywhere and consulted automatically can function, for practical purposes, as part of a person’s memory system — their famous example involved a man named Otto who used a notebook the way most people use their own recall. Risko and Gilbert’s later work gave this philosophical idea an empirical, testable framework: when exactly do people choose to offload, and what happens to their underlying skills when they do it repeatedly?

    How It Works: A Decision Loop, Not a Switch

    Imagine being asked to remember five errands on the way home. You could hold them all in mind, or you could write them down. Risko and Gilbert’s framework treats this as a running cost-benefit calculation rather than a one-time choice: the decision depends on how demanding the task feels, how confident you are in your own memory, and how much effort writing the list would save.

    The important detail is that this calculation runs on perceived demand and confidence, not necessarily on actual ability — and people’s self-assessments can be mistaken, so they sometimes reach for an external aid even when they could have managed without it. Whether repeated reliance on a tool gradually changes a person’s confidence or unaided performance over the longer term is a related but separate question, one that is harder to answer than the moment-to-moment decision itself. Offloading isn’t inherently a problem — compensating for a busy week with a written list is a perfectly sensible trade — but it is a trade worth noticing rather than making automatically.

    Diagram of the cognitive offloading decision loop
    Source: created based on Risko & Gilbert (2016), Trends in Cognitive Sciences

    Why It’s Back in the Spotlight Now

    Cognitive offloading has been studied for decades, but generative AI has pushed it from a niche academic topic into a mainstream concern. A widely discussed 2011 study in Science by Betsy Sparrow, Jenny Liu, and Daniel Wegner found that once people believed information would remain accessible online, they remembered where to find it far better than the information itself — a pattern nicknamed the “Google effect.” That study is a useful historical analogy rather than direct proof of what AI chatbots do to the brain: a search engine mainly stores and retrieves information, while an AI assistant can also perform reasoning-like tasks, draft text, and summarize on a person’s behalf, which is a different kind of delegation that researchers are only beginning to study on its own terms.

    The question researchers are now asking is not whether people offload thinking to AI, but how much of their own judgment they lose when the AI isn’t there.

    A 2025 study published in Societies by Michael Gerlich surveyed 666 participants and reported a statistically significant negative correlation between frequent AI tool use and scores on critical-thinking tasks, with younger, more AI-reliant participants scoring lowest. Because the study measured AI use and critical-thinking ability at a single point in time, it cannot determine whether frequent AI use weakens critical thinking, whether people with different existing abilities simply use AI differently, or whether some other factor explains both — but it adds to a growing body of research raising the same question from different angles.

    Everyday Examples You Already Use

    Cognitive offloading shows up in ordinary moments far more often than most people realize. A few familiar examples: writing a shopping list instead of memorizing items, letting a smartphone calendar track every appointment, typing a destination into GPS navigation instead of memorizing the route, and asking a search engine a factual question instead of trying to recall it. In most of these cases the trade is a fair one — a shopping list, for instance, tends to help you finish the errand without costing you much else. The more interesting question researchers are asking is which kinds of offloading are closer to that fair trade, and which carry a greater cost to independent performance later on.

    A 2020 study in Scientific Reports, led by Louisa Dahmani and Véronique Bohbot at McGill University, examined this trade-off directly with GPS. In a sample of 50 regular drivers, heavier GPS use was associated with weaker spatial memory on unaided navigation tasks. A smaller follow-up of 13 of the same participants about three years later found that people whose GPS use had grown the most over that period also showed the steepest declines in spatial memory — a dose-response pattern the authors say is more consistent with GPS use contributing to the decline than the reverse, though they are careful to flag their small sample size and note that the direction of cause and effect is not fully settled.

    Not every example involves memory. Risko and Gilbert’s framework also covers offloading onto the body itself — pointing at an object instead of holding its location in mind, or using a finger to keep a place while reading a long list — and “intention offloading,” where a sticky note or phone alert carries the job of remembering to act later, rather than the job of remembering the information itself. A related but distinct idea is automation, where a system completes part of a task with no decision from the user at all. A GPS app that both tracks your route and reroutes you around traffic without being asked is doing more than simple offloading; it raises its own, separate questions about what happens to a skill when a person is never required to exercise it in the first place.

    What Recent Research Shows

    The clearest recent demonstration comes from a UC Berkeley–led study published in 2026. Researchers ran three randomized trials with 1,222 participants who solved math and reading problems either with or without ChatGPT. Everyone then had to solve a final set of problems entirely unaided. Participants who had previously relied on the AI scored markedly lower on those unaided problems than participants who had worked alone from the start — in one trial, 57% versus 73% correct. Related article: AI Assistance Reduces Persistence and Hurts Independent Performance looks at this study’s full results and the researchers’ own caveats about how far the findings generalize.

    None of this means offloading is simply bad. In a self-reported, non-randomized part of the same study, participants who said they had copied AI-generated answers directly scored the lowest on the unaided follow-up problems (65% correct on average), while participants who said they had used the AI only for hints scored close to those who never used AI at all. Because that comparison is based on what participants reported about their own behavior rather than a randomized condition, it should be read as a suggestive pattern rather than a proven result. Still, it points toward a recurring theme across this research: the question is less whether to offload and more how — leaning on a tool for a finished answer looks different, in its effects, from using it as a scaffold while you still do the thinking yourself.

    Sources: Risko & Gilbert, “Cognitive Offloading,” Trends in Cognitive Sciences (2016), Gerlich, “AI Tools in Society,” Societies (2025), Columbia News on Sparrow et al., Science (2011), American Psychological Association Monitor (2026)

    See also: Why Letting AI Help Can Erode Your Grit and AI Summaries Can Distort Human Memory.

  • A Simulated Brain Circuit Builds Its Own AI-Like Learning System, MIT Finds

    A Simulated Brain Circuit Builds Its Own AI-Like Learning System, MIT Finds

    For decades, neuroscientists have faced a nagging puzzle. Modern AI owes its power to backpropagation, an algorithm that adjusts a network’s connections by propagating error information backward through precisely coordinated connections — a computation with no clearly established equivalent in biological circuitry. Yet some studies have found that activity patterns in trained artificial networks resemble those recorded in the brain’s own visual system, which only sharpens the question of how biological neurons, using only local signals and changing synapses, might approximate something similar. A new study from MIT’s McGovern Institute for Brain Research, published October 9 in the Proceedings of the National Academy of Sciences (PNAS), offers a candidate answer: when simple neuron-like units are wired randomly and left to follow local, biologically plausible plasticity rules, they can spontaneously self-organize into circuits that learn nearly as well as backpropagation does on the tasks tested — without anyone telling them how.

    原論文情報 / Original Paper
    Title: How Biological Synapses Self-Assemble Gradient Learning
    Authors: Qianli Liao, Liu Ziyin, Yulu Gan, et al.
    Journal: Proceedings of the National Academy of Sciences (PNAS)
    DOI: 10.1073/pnas.2606245123

    A 40-Year Mismatch Between Brains and AI

    Modern deep learning owes almost all of its success to backpropagation: a network makes a prediction, compares it with the correct answer, and then propagates that error backward, layer by layer, nudging every connection weight in the direction that would have reduced the mistake. It works brilliantly, but standard implementations depend on precise mathematical relationships between forward and backward connections, plus coordinated transmission of error information through the network. There is no established evidence that cortical circuits implement anything equivalent in its conventional form.

    The puzzle deepened as deep learning matured. Comparisons between the internal activity of backpropagation-trained networks and recordings from biological visual systems have repeatedly found intriguing similarities — not identical, but notably closer than chance would predict. Such similarities do not prove the brain runs backpropagation, but they raise an obvious question: if it does not, what local process could still produce computations that end up looking so much like it?

    Local Rules, No Blueprint

    The MIT team, led by Qianli Liao, Liu Ziyin and Yulu Gan with senior authors Mark Harnett and Tomaso Poggio (and co-author Brian Cheung), approached the question differently from most prior attempts to make backpropagation “more biological.” Instead of designing a circuit by hand and checking whether it could plausibly exist in a brain, they started from almost nothing: layers of neuron-like units connected at random, with no predetermined wiring diagram at all.

    The system runs on local synaptic plasticity rules, including heterosynaptic mechanisms that let a connection change in response to activity beyond that single synapse — a more specific relative of the familiar Hebbian idea that “cells that fire together wire together.” Critically, these rules only need information available locally, at or near a given synapse (the junction where one neuron signals to the next). No unit needs to know what is happening elsewhere in the network, and no single global error signal is broadcast to the whole system.

    Diagram showing random wiring self-organizing into a learning circuit
    Source: created based on findings from MIT McGovern Institute / PNAS (2026).

    Circuits That Wire Themselves

    Left to run under those local rules, the randomly connected units did not stay random for long. Over repeated rounds of activity, the network developed small, recurring wiring patterns the researchers call “self-assembling motifs.” The model’s analysis points to a specific four-synapse motif linking the brain’s ascending, bottom-up pathways with its descending, top-down pathways — a concrete structural prediction that experimentalists can now go looking for in real neural tissue, not a pattern already confirmed to exist there.

    What these motifs do next is the more remarkable part: networks of them self-organize further, so that their combined dynamics approximate a generalized form of stochastic gradient descent — the family of optimization methods that includes backpropagation — without a hand-designed, globally coordinated algorithm directing them. In other words, before the network learned anything from its training data, it first had to build the machinery that would let it learn.

    Starting from random connectivity, local synaptic plasticity rules were enough for the network to assemble circuit motifs whose combined dynamics approximate gradient-descent-style learning.

    Holding Its Own Against Backpropagation

    A biologically plausible circuit is only interesting if it actually works, so the researchers tested their self-organized networks on image pattern recognition and classification tasks. According to MIT, the self-assembled circuits achieved performance comparable to conventional networks trained directly with backpropagation on the tasks tested. The MIT News announcement does not report specific accuracy figures or benchmark datasets — those details sit in the full PNAS paper — so the result is best read as evidence that self-organization carried no obvious cost on this particular test, not as a claim that it matches backpropagation across deep learning in general.

    That matters because it suggests biological plausibility and computational competence are not necessarily in tension. For years, biologically inspired alternatives to backpropagation have often underperformed it on real tasks. A circuit that assembles itself from random wiring and still competes with backpropagation on the tasks it was tried on narrows that gap, at least in this one demonstration.

    A Prediction Neuroscientists Can Go Test

    The researchers are careful to note that a self-organizing network learning like this does not prove the human brain learns this way — it is a candidate mechanism, not a confirmed one. But unlike many theoretical proposals in computational neuroscience, this one comes with a concrete, falsifiable prediction: if the brain really does something like this, cortical circuits should contain that specific four-synapse motif connecting ascending and descending pathways. Experimental neuroscientists can now search actual brain tissue for exactly that wiring signature, which is what turns an elegant simulation into a testable scientific hypothesis.

    Why It Matters Beyond the Lab

    If the prediction holds up, the payoff would extend in both directions. For neuroscience, a confirmed biologically plausible learning circuit would help explain how the brain adapts throughout life, and could sharpen our understanding of what goes wrong in conditions involving altered learning or memory, as well as inform the design of future brain-computer interfaces that need to interact with the brain’s own learning rules rather than fight against them.

    For AI, the lesson cuts the other way: today’s networks are built top-down, with every connection and training rule specified by engineers before a single example is seen. A system that can bootstrap its own learning machinery from random wiring and local plasticity rules hints at an alternative path — architectures that configure themselves as they encounter data, rather than ones that are handed a finished blueprint. Liao, Ziyin, Gan and their colleagues frame their result as an early step in that direction, not a finished product, but their work gives both neuroscientists and AI researchers a concrete new question to test: not just how networks learn once they are built, but how learning itself can come preinstalled.

    For more on where artificial and biological intelligence still diverge, see our earlier coverage of how MIT neuroscientists found the brain re-files information the instant you make a decision, a separate finding that is likewise reshaping how AI researchers think about adaptive, flexible learning.

    Sources: MIT News, Proceedings of the National Academy of Sciences (PNAS), arXiv preprint

  • [Science] Why Letting AI Help Can Erode Your Grit, Berkeley Study Finds

    [Science] Why Letting AI Help Can Erode Your Grit, Berkeley Study Finds

    A chatbot that hands you the answer can make a hard problem feel easy — until the chatbot is gone. A new study from researchers at five universities found that people who solved problems with an AI assistant performed worse on the next problems, once that assistant was taken away, than people who had worked through the very same problems alone from the start. The gap showed up after roughly 10 to 15 minutes of AI-assisted problem-solving, and it held across three separate experiments involving 1,222 recruited participants — a pattern the authors say raises real questions about whether leaning on AI for answers can undermine people’s ability, or willingness, to solve problems on their own.

    Three Experiments, One Pattern

    The team — Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker and Rachit Dubey, drawing on UC Berkeley’s Center for Human-Compatible AI along with Carnegie Mellon University, MIT, the University of Oxford and UCLA — ran three randomized controlled trials totaling 1,222 recruited participants (307, 585 and 168 after some participants were excluded under the researchers’ prespecified criteria). In the first experiment, 354 people were recruited to solve 12 basic fraction problems; half worked alone throughout, while the other half could ask a ChatGPT assistant, built on GPT-5, for help — before everyone, AI group included, had to solve three further problems entirely on their own. A second experiment repeated that design with 667 people, and a third, with 201 participants, swapped the math problems for SAT-style reading comprehension questions.

    In every version of the study, the early pattern looked encouraging: while they still had access to the AI, the assisted group solved problems faster and scored higher than the group working alone — exactly what you would expect from a capable assistant. The twist came in the second half of each session, once the AI was removed and everyone, including the AI group, had to finish the rest of the problems unaided.

    The Numbers Behind the Finding

    Once AI access ended, the previously assisted group fell behind the group that had never used it, and the gap was consistent across all three experiments. In the first fraction experiment, the former AI group answered an average of 57% of the three final problems correctly, versus 73% for the no-AI group — a statistically significant difference (p < 0.001). The larger replication of the same design found a smaller but still reliable gap: 71% versus 77% (p = 0.02). In the reading-comprehension experiment, the difference widened again, to 76% versus 89% (p = 0.007).

    Bar chart comparing solve rates after AI was removed across three experiments
    Source: data from Liu, Christian, Dumbalska, Bakker & Dubey (arXiv:2604.04721) and UC Berkeley News

    Giving up altogether followed a similar trend. In the first experiment, 20% of the former AI group skipped a problem rather than attempt it, compared with 11% of the no-AI group. The pattern was weaker in the larger replication (10% versus 7%, not statistically significant) but reappeared clearly in the reading task, where 8% of the AI group skipped questions versus just 1% of the control group. A closer look at how participants said they had actually used the AI while it was available sharpened the picture further, though this part of the analysis was observational rather than randomized, since people were not assigned to copy answers or ask for hints: those who reported simply copying the AI’s finished answers fared worst afterward, averaging just 65% correct on their own, while those who said they used the AI mainly for hints showed no statistically significant gap from people who never touched AI at all.

    Participants were randomly assigned to the AI or no-AI condition, and the larger replication added a pretest to check the groups further: accuracy and the tendency to skip a problem did not differ significantly between groups beforehand, which strengthens the case that the later gap reflects the AI condition itself rather than pre-existing differences between participants. In that same larger replication, participants were also asked how they had actually used the AI while it was available. A small slice, about 12%, said they had barely used it despite having access, and this group performed about as well afterward as people who never had AI access at all — reinforcing the idea that it is the degree of reliance on AI, not mere availability of it, that predicts the later drop-off.

    “It’s a striking finding, but in a way it supplies ammunition for a story that a lot of us kind of feel in our gut.” — Brian Christian, UC Berkeley

    What Might Explain the Pattern

    The authors propose two explanations, while cautioning that their experiments were not designed to prove which one is doing the work. The first is a kind of expectation mismatch: once someone gets used to near-instant answers, grinding through a problem by hand can suddenly feel disproportionately slow, which may make people more willing to give up rather than push through. The second is the loss of what the researchers call “productive struggle” — the idea that difficulty itself is not simply an obstacle to route around, but part of how people practice and strengthen the skills a problem requires. Bypass that effort often enough, the argument goes, and there may be fewer chances for those skills to get built in the first place.

    It’s a different mechanism, but this study joins a related question this site has covered before: Georgetown University researchers recently found that AI-generated summaries of events can distort people’s memory of what actually happened. That study is about memory rather than problem-solving, but together the two point to a broader pattern worth watching — handing a mental task to AI can carry a hidden cost to accuracy or independent ability, even when it feels like pure convenience in the moment.

    “Not Just Fractions and SAT Problems”

    Christian has been blunt about how far he thinks the implications reach beyond a psychology lab. “This is not a story about fractions and SAT problems,” he said in comments accompanying the study’s release. “This is something that is happening to human knowledge and human expertise.” He described the irony directly: widely used AI assistants, he said, are “often helping us in ways that are kind of unhelpful” — lifting performance on the task directly in front of us while, the data suggest, working against the independent competence we will need the next time no AI is around.

    The caveats matter here. The study measured performance minutes after AI access ended, not months or years later, so it cannot say how long this effect lasts or whether frequent users eventually adapt and recover. The breakdown by how people used the AI — copying answers versus asking for hints — was observational rather than randomized, so it shows an association rather than definitive proof of cause and effect. And all three experiments involved short, one-off tasks completed by paid online participants, a different setting from a student using AI across a semester of coursework or an employee weaving it into daily work.

    What This Means for Using AI

    None of this is an argument for avoiding AI assistance altogether — the same data show it clearly boosted short-term performance while it was available, and for many real-world tasks, getting the right answer quickly is exactly the point. The more useful takeaway from this one study may be about how AI gets used rather than whether: in the self-reported usage data, the steepest drop-off belonged to people who let the AI hand them a finished answer, while those who said they used it mainly for hints or partial guidance looked much closer to the group that never used AI at all. Treating a chatbot more like a tutor that nudges you toward a solution, rather than a vending machine that dispenses the finished one, looks like a reasonable bet for keeping your own problem-solving sharp — though it’s one study, in a narrow setting, and confirming how far that advice travels beyond fractions and reading quizzes will take further research.

    Sources: UC Berkeley News, arXiv:2604.04721, “AI Assistance Reduces Persistence and Hurts Independent Performance”

  • AI Summaries Can Distort Human Memory — A Warning for Police and Witnesses

    AI Summaries Can Distort Human Memory — A Warning for Police and Witnesses

    You watch a short video of a car approaching an intersection, notice the traffic sign, and see it strike a pedestrian. The next day, you read an AI-generated summary of that video, one that happens to get the sign wrong. Asked afterward what you actually saw, you become measurably less likely to answer correctly. That is the core result of a new study from Georgetown University and the University of Washington: a misleading AI summary can distort people’s memory of an event they witnessed directly, which matters because AI-generated summaries are already spreading into newsrooms, offices, and policing.

    Illustration: video summarized by AI, with details breaking off before reaching memory
    From video to AI summary to memory: small details can break off and go missing along the way — an illustration of the distortion effect described in this article.

    AI Summaries Are Already Everywhere

    Large language models like ChatGPT and Gemini are now routinely used to condense meeting recordings, news footage, and surveillance video into a few bullet points. Some U.S. police departments have gone further, adopting separate tools that draft incident-report narratives from transcribed body-camera audio. One such product, Axon’s Draft One, generates a draft within minutes of a call ending, which an officer must then review and approve; the Fort Collins, Colorado police department has reported an 82% drop in time spent writing reports after adopting it. That tool drafts text from audio transcripts rather than summarizing video, and it was not tested in this study — but it illustrates how quickly AI-written accounts are entering settings where accuracy carries real stakes.

    That broader shift is what worries the study’s authors. “AI is a new method of delivering misinformation,” said lead author Mattea Sim, an assistant research professor at Georgetown’s McCourt School of Public Policy. Co-author Yael Eiger, a Ph.D. candidate at the University of Washington who studies technology in policing, put it more bluntly: she said it concerns her that police departments may already be relying on video-summarization technology without the kind of testing this study attempted.

    Testing Memory With a Car-Crash Video

    The researchers ran their test in two stages. First, they asked ChatGPT-5.5 and Gemini 2.5 Flash-Lite to summarize two short animated videos of a car-pedestrian collision. The two videos were nearly identical except for one detail: one showed the car approaching a stop sign, the other a yield sign. Each model summarized each video five times, producing 20 summaries in total.

    Second, 360 people were recruited to watch one of the two videos themselves. One to two days later, the researchers had each participant read a summary of what they had just seen — either an accurate one or one with a single planted error that reversed the key sign detail — and told them, at random, that the summary had been written by either an AI system or a human transcriber. Of the 360 who started, 328 completed both stages and passed an attention check.

    Most Summaries Omitted the Key Event

    Before testing human memory, the researchers first graded the 20 AI-written summaries against a list of details the researchers had identified as central to the event. The results were not reassuring: on average, the summaries omitted 51.6% of those central details, and 19 of the 20 summaries failed to mention the single most important one — the car hitting the pedestrian.

    The two models differed in other ways. Every summary from both ChatGPT and Gemini contained at least one error. Gemini’s summaries included a critical inaccuracy — one that reversed a core fact, such as the traffic sign, rather than just dropping a minor detail — in all of its summaries, compared with 30% for ChatGPT. Both models also occasionally invented details that were never in the video at all, a pattern researchers call hallucination. These figures come from a small, controlled test of two models summarizing two short videos, so they should be read as evidence that serious errors can occur rather than as a general failure rate for AI video summaries.

    “AI has the potential to generate misinformation, even absent any adversarial intent,” the study authors write.

    A Wrong Summary Cut Recall Nearly in Half

    The second stage showed these errors don’t stay confined to the summary — they appear to bleed into people’s own memory of the event. Participants who read an accurate summary correctly identified the sign 83.6% of the time. Among those who read the summary with the planted error, accuracy dropped to 44.8%, a gap the researchers found to be statistically significant and unlikely to be explained by chance alone.

    Bar chart comparing recall accuracy

    That 38.8-percentage-point gap is the study’s central finding: a single misleading detail, embedded in an otherwise ordinary-looking summary, substantially reduced participants’ ability to correctly recall a detail they had directly observed just one or two days earlier. The experiment shows a clear misinformation effect; it doesn’t by itself prove that the original memory was erased or permanently replaced, only that recall of this one detail became much less reliable.

    Source Labels Alone Didn’t Help

    One of the more striking results is what didn’t matter. Participants told the summary came from an AI were no more accurate, and no less likely to be misled, than those told a human wrote it: 46.3% versus 43.4%, a difference too small to be statistically meaningful. How much participants said they trusted AI in general, or how often they used it, also made no measurable difference.

    In this experiment, simply labeling a summary as AI-generated rather than human-written did not reduce the misleading effect. The researchers say this challenges a common assumption in AI-safety discussions — that keeping a “human in the loop,” meaning a person who reviews the AI’s output, is enough to catch its mistakes. If a misleading summary quietly shapes the reviewer’s own recollection of the source material, that reviewer may be less able to notice anything is wrong. The study did not test other safeguards, such as requiring reviewers to check specific claims against the original recording, so it can’t yet say which interventions would actually help.

    Why This Matters for Courts and Policing

    The experiment used an animated video, student volunteers, and a lab setting — not real police officers, real witnesses, or an actual criminal case. Its relevance to policing is more general: when an AI-generated account becomes part of how someone reconstructs what happened, errors in that account may shape what they later remember. If an officer or witness reads an inaccurate AI-generated recap of body-camera footage before giving a formal statement, that recap could influence their later recollection — a real concern, though not one this particular study tested directly.

    This isn’t only a human blind spot, either. A separate study covered previously on this site found that AI vision systems fail to reproduce the same motion illusions that trick the human eye — one more reminder that machine and human perception can diverge in either direction, so neither can simply be trusted to check the other without independent verification (see our earlier coverage of AI vision and motion illusions).

    A condensed version of this research was presented this week at the Ninth AAAI/ACM Conference on AI, Ethics, and Society (AIES 2026), held October 12–14 in Malmö, Sweden, after first appearing as a preprint in September. The authors aren’t calling for AI summaries to be banned from sensitive uses, but they argue error rates like these need to be measured and disclosed before such tools are trusted with decisions that affect legal outcomes.

    The researchers are also upfront about what they didn’t test. They used only consumer-facing versions of ChatGPT and Gemini, not the enterprise or custom-tuned models that vendors like Axon actually deploy, and the AI error-rate analysis covered just two short animated videos. Eiger says that gap is itself part of the problem: outside researchers generally have little visibility into which exact models or settings police vendors use, which makes independent auditing difficult. Sim says a natural next step is to study real body-camera footage and real police reports directly, to see whether AI-generated summaries measurably shift how people judge a police-civilian interaction — not just a simple traffic-sign detail in an animated clip.

    Xsciens Editorial Note

    The finding is not simply that “AI makes mistakes”; it is that a single wrong detail can slip into a person’s memory and quietly bend the judgment of the human who is supposed to be checking.

    • Read the 38.8-point gap in context. Accuracy was 83.6% after an accurate summary and 44.8% after a misleading one. But this comes from two short animated videos, two AI models, and 328 analyzed participants, so it is not an error rate for AI summaries in general.
    • An “AI-written” label did not help. Accuracy was 46.3% when people were told an AI wrote the summary and 43.4% when told a human did, with no meaningful difference. The study did not test checking steps such as comparing against the original video, so it does not show that human review is useless, only that review has to be designed carefully.
    • Not a test of real police tools. The participants were volunteers watching animations, not officers or witnesses in real cases. The classic Loftus and Palmer experiments had already shown that the wording of a question can change what people remember, so the new element here is the AI-generated text as the source of the distortion.
    • Our imagination (speculation) — We wonder whether future AI summaries that show the exact moment of the video each sentence is based on, so you can jump to it in one click, might make errors easier to catch. A meeting recording that lets you tap a line and hear the original remark is one example. But if people skip the check even when the evidence is one tap away, the same problem could remain, so this is only our speculation.

    References for this note: Georgetown University press release, the paper (arXiv:2609.28820), Loftus & Palmer (1974), explained (SimplyPsychology)

    Sources: Georgetown University, “New Research Finds That Misleading AI-Generated Summaries Can Distort Human Memory”, Sim, Eiger & Kohno, “AI-Enabled Human Memory Manipulation” (arXiv:2609.28820), Axon, “Axon Reimagines Report Writing With Draft One” (via Officer.com)

    See also: our explainer on cognitive offloading, the habit of handing memory and thinking to tools.