Tag: Computer Vision

  • AI Vision Misses an Illusion the Brain Shows — A Gap Worth Testing

    AI Vision Misses an Illusion the Brain Shows — A Gap Worth Testing

    A new study finds that after watching something move, people briefly misjudge exactly where a now-still object is sitting — and a matching shift shows up in signals recorded from monkeys’ brains. The seven AI vision models the researchers tested, built to mimic human sight, did not reproduce that shift at all. Published October 2, 2026 in the journal Current Biology, the work from York University in Toronto does not prove that this gap causes real-world AI vision systems to fail. What it does offer is a precise, repeatable benchmark that engineers building self-driving cars, robots, and other systems that must track a changing scene could use to check whether their software handles recent visual history the way biological vision does.

    Static-image benchmarks, how often a system correctly names or locates an object in a single photo, have long been a mainstay of computer-vision research. But the study’s authors point out that real cameras rarely see the world as a single frozen frame; they see a stream of images shaped by whatever happened a moment earlier. Notably, even a video-based network in the test, built specifically to use multiple frames over time, did not reproduce the shift seen in biological vision. That suggests the gap is not simply about feeding AI more frames, but about a specific kind of recalibration that today’s architectures do not learn on their own.

    A Trick of Motion Fools the Brain

    The effect the researchers studied is related to the classic motion aftereffect, often called the waterfall illusion because it was first described by people who stared at flowing water. After watching motion in one direction — stripes sliding left, for example — a stationary object shown afterward can appear displaced in the opposite direction, even though its pixels have not moved at all. The York University team used this positional bias as a precise, measurable probe into how the brain represents where an object is, rather than treating it as a mere curiosity.

    Testing Monkeys, Humans, and Machines

    The research team, led by senior author Kohitij Kar and first author Elizaveta Yakubovskaya, ran related versions of the same basic test on three very different visual systems. Seventy-nine human volunteers, 35 in a no-motion baseline group and 22 in each of two adaptation-direction groups, briefly saw an object flash on screen for a tenth of a second and clicked where they thought it was. Two rhesus macaques viewed similar sequences while chronically implanted electrode arrays recorded signals from more than 150 neurons in the inferior temporal (IT) cortex, a region of the brain’s object-recognition pathway whose activity also happens to carry information about where an object sits. The same images were then fed into seven widely used computer-vision models, ranging from classic image classifiers such as AlexNet and VGG-16 to a vision transformer and a video-based action-recognition network called SlowFast.

    AI Could Find Objects, Not the Illusion

    On ordinary, non-adapted images, both the brain recordings and the AI models could track an object’s true horizontal position: linear readouts from several of the networks correlated with true position about as strongly as readouts from the monkeys’ own IT neurons did, in the range of roughly 0.59 to 0.84. So the models were not simply bad at locating objects. The difference showed up only after adaptation. Human volunteers reported an object’s position shifted by an average of 0.33 degrees of visual angle, opposite the direction of the preceding motion. The position decoded from the monkeys’ IT population activity shifted even further, by 0.84 degrees in the same direction, though that neural shift is a decoded signal rather than a direct report of what the animal consciously saw. Every AI model tested, including SlowFast, shifted by two-hundredths of a degree or less, in practice no meaningful shift at all.

    Human observers reported their perception shifting by 0.33 degrees after watching motion, and the position decoded from monkeys’ IT neurons shifted by 0.84 degrees — while every AI vision model tested shifted by two-hundredths of a degree or less.
    Bar chart comparing object-position shift after motion adaptation across human perception, macaque IT cortex, and seven AI vision models, on a log scale in degrees.
    Source: created based on data from Yakubovskaya et al., Current Biology (2026).

    Copying the Brain’s Adaptation Trick

    The gap did not mean the AI models were incapable of representing position-shifting information at all. When the researchers applied a mathematical transformation, derived from how IT neurons’ responses changed during adaptation, directly onto the models’ internal feature representations, the models began producing human-like position shifts. That shows the models’ feature spaces can support the effect once an appropriate transformation is imposed from outside. It does not show that any of the networks learned to generate that adaptation on their own during ordinary training, which is the gap the authors highlight. This is not the first time researchers have argued that borrowing the brain’s own computations, not just its final answers, could inform AI design; a separate study on how the brain re-files decision options made a related case for mining neural codes rather than just outputs.

    A Gap Worth Testing, Not Yet a Flaw

    The researchers are careful to frame this as a demonstrated difference in representations, not proof that AI vision performs worse in practice. Still, they argue it is a difference worth testing. Self-driving cars, delivery robots, and surveillance systems all judge where things are from a stream of frames in which recent motion could, in principle, matter. Consider a robot tracking a cyclist crossing its camera’s field of view: researchers could test whether the robot’s position estimate shifts the way a human’s perception would after similar recent motion, even though the current image alone has not changed. The York study does not show that reproducing this illusion would make such a system safer or more accurate; it offers a concrete, repeatable way to ask the question. Kar has argued more broadly that judging AI only by whether it reaches the right final answer can miss gaps like this one, and that matching the computations behind human perception, not only matching outputs, may help researchers evaluate whether an AI vision system handles a changing scene the way people expect.

    What the Study Doesn’t Yet Show

    The authors are explicit about the limits of their work. The experiments involved only two macaques and one specific illusion, so it is unclear how far the pattern generalizes to other forms of perceptual adaptation, to more animals, or to other stages of the visual pathway beyond IT cortex. The AI models tested were standard research networks for image and video recognition, not the newer multimodal AI assistants that combine vision with language, which the study did not evaluate. The researchers also do not claim that the perceptual bias they measured is actually beneficial for tracking objects in everyday life, only that it reflects a real computation in primate vision that current AI architectures do not reproduce on their own. They suggest the motion aftereffect could now serve as a simple, repeatable benchmark that other labs and companies use to test how their own vision models handle a changing visual world, and Kar’s group is already pursuing that direction, building what it calls more primate-like artificial neural networks for tasks such as continuous object tracking.

    Sources: Yakubovskaya et al., “The macaque IT cortex but not current artificial vision networks encode object position in perceptually aligned coordinates,” Current Biology (2026), Yakubovskaya et al., arXiv preprint (2026), York University press release coverage, via Neuroscience News