- VentureBeat Intelligence found 64% of surveyed respondents traced at least one confident, wrong AI agent answer to missing or inconsistent business context in six months; model errors were excluded.
- Respondents at firms running, piloting or building a semantic layer reported more failures: 78% versus 37% for the rest (38 respondents). The survey did not ask why.
- Leaders should ask which source defines each key business term for their agents, who owns it, and how wrong answers are counted.
Talk about AI agent errors usually starts with the model. A VentureBeat survey asked a narrower question: how often did a wrong answer trace back to the company's own data and definitions? Model errors were set aside, so the survey cannot say which cause is bigger.
In August 2026, VentureBeat Intelligence polled people at companies with at least 100 employees. One question asked whether an agent had given a confident but wrong answer because business context was missing or inconsistent. Examples were a wrong metric definition or stale data. Respondents were told to leave out the model's own mistakes.
About half of those (32% of all respondents) traced more than one. Another 31% of all respondents reported none. The researchers call these cases context failures.
How an agent gets the number wrong
An agent answering a business question needs two things: the data and the definition behind it. What counts as revenue? Which customer table is current? Suppose a definition is absent, or two systems disagree. The reply still reads well, yet the number underneath is wrong.
That makes these errors hard to catch. Nothing in the answer signals a problem. The report describes the cost downstream: a staff member may repeat the bad figure to a customer, or make a call using out-of-date data.
The usual remedy is a semantic layer. Think of it as one rulebook of business terms and metrics. Agents and dashboards both read from it, so two tools stop giving different answers to the same question. It also gives a team a correct reference to test an agent against. The report adds that it needs ongoing work, because someone must agree each definition and keep it current.
The odd result: more investment, more failures reported
Respondents at companies already investing in a semantic layer were much likelier to report a failure than those at firms still weighing one or with no plans.
The researchers did not ask why. They offer two explanations they cannot tell apart. Teams with a semantic layer may spot more failures because they hold a correct definition to compare against. Or firms that suffered failures may be the ones that went on to build one.
The groups are small. A separate July sample showed a similar gap, at 89% versus 35%. The report says the gap stays large when respondents who do not run agents on enterprise data or do not track root cause are removed.
Here is the lesson for executives. A hospital that logs many incidents may be no less safe than one that logs few, if the second cannot see its own mistakes. Apply that to the 37%. If teams without a semantic layer lack a reference to check against, some failures may simply go unspotted. That reading is untested.
The second explanation fits the data just as well. Firms that were burned by wrong answers may be the ones that built a layer, which would push their reported failures up. The survey cannot separate the two. Either way, the practical question is the same: how does your firm count wrong answers?
Where agents actually look
Owning a semantic layer does not mean agents use it. A firm can run one for its dashboards while agents draw context from elsewhere. Retrieval over documents, known as RAG, was named the primary source by 32% of respondents. Direct queries to live systems, such as databases, APIs and Model Context Protocol servers, drew 21%. The report calls that gap too close to call. A governed semantic layer was named by 13%.
So at roughly three in four of those firms, the agreed definitions are not the agent's main source. Retrieval, live queries and loading large inputs straight into the model each fetch information. None of them necessarily carries the company's agreed meaning of a term. The agent must also consult the semantic layer to get it.
Ease of ingestion outranks accuracy as the top selection factor
Among the 117 respondents running retrieval in production, 36% said ease of loading data in mattered most when they chose a platform. Only 11% named retrieval accuracy first. Ingestion ranks high for a practical reason. If a document never entered the system, or went stale, the agent cannot find it. Accuracy is the other half. The report warns that a system serving the wrong passage can leave an agent answering confidently from it.
Two cautions apply. The 36% for ingestion and 22% for access control and permissions are too close to call. And the survey records which factor came first for each person. It cannot say how much weight buyers gave accuracy.
The market is also in motion. Separately, 65% said they intend to add or swap a retrieval platform within a year.
Questions to put to your team
First, for the five numbers your leaders quote most, which source does each agent use to define them? Second, who owns each definition, and who updates it when the business changes? Third, how do you count wrong agent answers today? If the answer is that users complain, you may be measuring only what is visible.
Fourth, when you evaluate retrieval tools, ask for accuracy tests on your own data, not only ingestion speed. Fifth, what would it cost to rebuild your context stack if you changed model providers? The report notes that moving agents to another model can mean exactly that. In the survey, 37% named standalone tools as their most likely direction over 12 months, and 22% named consolidating on one provider's stack.
An agent can be fluent and wrong at the same time. The fix starts with knowing which of your numbers it was never taught.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: VentureBeat.
