Skip to content
Security & Trust Talking point

Credit Karma's Harish Gaggar: many agents fail in production when controls are missing

Harish Gaggar of Intuit Credit Karma argued that gateways, audit and secret filtering come before the agent.

W
WebPulse Newsroom
AI-assisted · 2 min read
Share on X LinkedIn
Credit Karma's Harish Gaggar: many agents fail in production when controls are missing
In brief
  • Harish Gaggar of Intuit Credit Karma argued that many agents fail in production because basic controls around them are missing.
  • He named a gateway, reviewed tools, monitoring, audit, secret filtering and human sign-off on risky actions as the paved path.

Harish Gaggar, who works at Intuit Credit Karma, argued on the AI Engineer show that many AI agents fail in production because basic controls around them are missing. His advice to teams was to build an enterprise framework, not just an agent.

What was said

Gaggar opened by describing why agents break once they go live. He said many fail because their "foundation controls are missing." He then listed what he meant.

First, a gateway. An agent should prove who it is before its request is routed to the right tool server. Second, tool servers that expose only reviewed tools. Third, tracking and evaluation of every query. Fourth, a way to keep secrets and personal data from reaching the model, the traces or the agent's memory store. Last, an audit record of what the agent actually did.

He was firm on access. Nothing should reach an endpoint directly. Every call goes through the gateway first. He also said monitoring is not optional: "the AI agent cannot be trusted without continuous monitoring, evaluation, and feedback." His examples were questions such as which tool was slow and whether authentication rejections are spiking.

His closing advice was not to rush an agent out the door, and to keep a human in the loop for high-risk actions.

Why it matters

Our reading: the boring platform work deserves at least as much attention as the agent. Gaggar did not rank the two. But his list suggests a buyer or manager should ask different questions. A polished agent demo shows little. Better questions are who can call it, which tools it may use, what is logged, and where secrets go.

It also bears on who pays when things go wrong. Gaggar's list includes keeping secrets and personal data out of model calls, traces and memory. Without that control, the likely losers are the people whose data flows through the agent, not the team that shipped it. That inference is ours, not his.

The other side

In the portions we reviewed, Gaggar gave no figures on how often agents fail, and no incident showing what a missing control cost. "Many agents" is his impression, not a measured rate. His only figures in the excerpts concerned token savings.

The talk describes one company's setup, built on specific open-source tools. In the portions we reviewed, it did not address how smaller teams without a platform group could afford a gateway, review process and monitoring stack.

His advice to avoid rushing and to keep humans in the loop also implies slower shipping and more staff time. He did not discuss that trade-off in the portions we reviewed.

Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.

The conversation this talking point comes from

Share this insight