- Scale Labs and Elorian built a benchmark of 522 questions on what images and video imply. People scored 93.1%. The strongest model the authors tested scored 53.6%.
- Passing expert-level tests does not show that a model reads a scene the way a person does at a glance.
- Before deploying AI on camera or video judgments, test it on your own scenes and ask vendors for error rates.
A glance is not in the data
A driver looks at the gap between two parked cars and knows the van will fit. Nobody wrote that down as a rule. The driver just sees it.
Researchers at Scale Labs, working with the AI startup Elorian, built a test around exactly this kind of judgment. They call it Humanity's Sixth Sense (HSS). Their finding is simple. Frontier AI models handle many hard tasks, yet they fall well short of people on these quick, intuitive reads of a scene.
The lesson for leaders is that a model's score on exams does not tell you how it performs when a decision depends on what a picture implies. Those are different skills, and only one of them has been widely measured.
How the test works
HSS has 522 questions in total. Import AI reports that they draw on 288 still images and 234 short video clips, and that the clips run 17.6 hours in all. Humans wrote the prompts.
The questions sit in four groups. Temporal and causal questions ask what happened before a scene and what comes next. Physical and spatial questions test practical judgments: whether something fits, whether it is reachable, and what another person could see from their position. Social questions ask about intent, attention and belonging. Abstract questions ask about the rules a scene implies.
Import AI gives two examples. One asks whether a bookcase can hold more books. The other shows horses pulling a plough and asks whether the chains carry the plough's drag weight. Neither answer is printed in the image. Both must be inferred.
Scores are pass@1, which means the answer was right on the first try. The 53.6% is the top score among the models the authors evaluated, not a ranking of every model. People scored 94.1 on images and 91.9 on video. GPT-6-astra scored 55.6 and 51.1. Import AI also reports two Chinese open-weight models: GLM-5.3-flash at 26.8 and Kimi K3 at 25.5.
Where models go wrong
The researchers looked at the failures. The two most common single mistakes were missing a detail that decides the answer, at 21% of failures, and misidentifying an object, person or role, at 20%.
That matters in practice. A model can miss one decisive detail and still return an answer. The benchmark does not tell us how sure the models sounded when they were wrong. Even so, a reviewer should not assume a wrong answer will look wrong.
The authors also tried an agentic setup, where the model can manipulate the visual input as it works. It narrowed the gap but did not close it.
What this does and does not show
This is one benchmark of 522 questions, built by the groups that released it. It does not rate any product for any specific job. It says nothing about how a model performs on your cameras, your documents or your customers.
Import AI offers a second reading. The newsletter notes that some people argued true visual reasoning would need separate 2D and 3D world models. It says the fact that language-model systems like GPT and Claude score as well as they do is something of a counter to that view. Scoring 53.6% is still far from human level, but the newsletter's point is about the route, not the destination.
So there are two true statements at once. These models do better than many expected. They are also far from a person's quick read of a scene.
What to ask your team
The practical risk is not that models are weak. It is that strong results on familiar tests may be taken as proof of a skill nobody tested.
Ask any team proposing AI for image or video judgment these questions:
First, what decision does the model make from a scene, and does the answer depend on something implied rather than visible? Safety checks, damage assessment and monitoring may do. Check each one.
Second, has the vendor tested on your own images and footage, with a human baseline, rather than quoting a public benchmark?
Third, what is the measured error rate, and which errors are most common? HSS suggests missed details and misidentified objects or roles are good places to look.
Fourth, who reviews the output, and does that person know the model can miss a detail that decides the answer?
HSS gives a name and a number to a gap that practitioners will recognise. Before you hand a model a glance-level decision, measure whether it can make one.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Scale Labs (Scale AI), with Elorian.
