- Two New York Life speakers studied 1,368 AI failures and found that weak deployment controls, such as human review and monitoring, were most closely tied to harm.
- Their point is that a model's safety claim means little until you know who checks its output and how many customers it reaches.
A safety claim about an AI model tells you little on its own. Vishal Srivastava and Tanmay Sah of New York Life made this case on the AI Engineer show. They argued that the same model carries different risk in different companies. What changes it is whether anyone checks the output, whether anyone is watching, and how many customers it reaches.
What was said
The pair studied 1,368 incidents from a public AI incident database. They sorted the failures into three areas: technical strength, deployment controls and business impact. Thirteen sub-areas, they said, cover almost 98% of the incidents. On average, about six of those sub-areas showed up in each failure.
Not all sub-areas count equally. According to the show notes, weak deployment controls, such as human review and monitoring, were the most closely tied to harm. A speaker on the show put it this way: "all these sub-dimensions are important, but deployment controls are far more important."
They gave a plain example of exposure. If a system serves internal staff, employees can see an error and contain it. If it faces outside customers, they said, any small issue can spread fast. Their business-impact area also asks how many users an error would touch.
They then split risk into four parts: how often the model fails, whether the failure reaches a real decision, whether it goes uncaught, and how severe the result is. A speaker on the show summed it up: "the same model can create different risk depending on how we deploy it".
Why it matters
Our reading: "Is this model safe?" is the wrong question for a buyer or a manager. A better one is who reads the output before a customer does. A second is what would flag a bad answer in production. A third is how many people one mistake could reach.
The speakers drew a practical step from this. If you know which use cases carry the most at stake, you can put more testing effort there. They also said testing should cover the whole system, including retrieval, tool failures, permission limits and human review, not only the model.
The other side
The evidence has limits. The data is a public incident database, so it can only show cases that made it into that database. The speakers described links between sub-areas and harm. In the excerpts, they did not show that any one control causes less harm.
Technical strength is still one of their three areas, so they did not say the model itself is unimportant. The excerpts also do not show how a company would score each sub-area.
The speakers also conceded a weakness in their own approach. System tests are sometimes written or judged by another AI, and that judge can make mistakes and miss important failures. Checking the evaluator is therefore another job to do.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- AI Engineer: 1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life (2026-10-22)
