- Uber routes every AI model request through one internal gateway, so safety checks, audit logs and spend limits apply to all projects at once.
- The setup grew from years of platform work done before AI, and the excerpts do not show how well the checks perform.
When every AI model request passes through one gateway, risk stops being scattered across teams. It can be checked, recorded and billed in one place. Uday Kiran Medisetty, a Distinguished Engineer at Uber, described this setup on The Pragmatic Engineer podcast, in an episode published on October 14, 2026.
What was said
Medisetty said a team at Uber needs very little to use a model. It creates a client and supplies a project ID and a caller token. In return it gets one API that matches either OpenAI's or Anthropic's format. Models from all vendors, open or closed, sit behind that gateway.
The first feature added was called AI Guard. Medisetty said the aim was to stop sensitive content going back to users. Because it is middleware at one entry point, it applies the same way to every project. Uber's security team can improve its own models there. He said these run on every request and every response, under strict speed limits.
Since every project is tracked, Uber can attribute usage to each user and caller. Later it added spend limits for projects and users, plus audit logs and session traces. Medisetty put it this way: "every model request at uber goes through this so there's no model request that doesn't go through this". The host asked whether billing works the same way, and Medisetty agreed.
Why it matters
Our reading: this is a design choice that makes AI risk smaller before it grows. If each team wired up models by itself, each would need its own safety checks, logs and cost controls. With one gateway, a fix made once reaches every project.
For managers and buyers, the question is whether you could answer three things today. Which models are being called? Who called them? What did it cost? At Uber, the answer sits in one system. Medisetty also said the gateway could become the place to add model routing, which was still a topic of discussion.
The other side
This took years of groundwork. A speaker on the show noted that Uber built its developer tools long before AI. Another said the companies gaining most from AI tend to have had developer platform teams before it. A smaller firm may not have that base to build on.
The excerpts also leave questions open. They give no figures on what AI Guard catches or misses, or on how much delay it adds. They also do not show how Uber controls what its agents are allowed to access. A gateway for model requests does not answer that by itself. It is a strong control point, but the excerpts do not show it is enough on its own.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- The Pragmatic Engineer: Uber’s AI infra, from the inside (2026-10-14)
