- Brzak said a tenant on a shared H100 GPU can infer a neighbor's model depth and decoding speed just by timing its own memory reads.
- He said workload type showed up only with speculative decoding on and the model already known. He also said the method cannot recover prompts or answers.
A leak does not have to expose your data to cost you. Branislav Brzak of Stealthium argued this on the AI Engineer show. He said a tenant sharing an NVIDIA H100 GPU can infer which model a neighbor runs, how deep it is and how fast it generates text. With one optional setting on, it can also infer the kind of work. It never reads prompts or answers.
What was said
Brzak studied GPUs split into isolated slices, a feature NVIDIA calls MIG. Each slice has its own compute units and memory. He said GPU-wide memory barriers still reach every slice. In his words, "Timing is the one thing that isn't isolated."
The attacker only times reads from its own small buffer, every 5 microseconds. It never touches the victim's memory. To the naked eye, an idle and a busy neighbor look the same. Brzak put the median gap at about 3 microseconds. For the attacker, he said, that is enough.
From that trace he reported identifying the model's architecture and depth with 97% accuracy, and its decoding speed with 99.3%.
He said the method is not a trained classifier. It runs in fewer than 30 lines of code. It tests candidate layer counts and keeps the best fit. Because it fits rather than looks up, he argued it also works on proprietary and unreleased models. The excerpts do not show an unreleased model being tested.
Why it matters
Our reading: model identity, size and speed are business facts, even when every prompt stays private. A rival with a slice on the same physical GPU could learn which model powers your product.
Workload type is a narrower risk. If speculative decoding is on and the attacker already knows the model, a rival might estimate how much of your work is code, prose or structured output. Speculative decoding is a speed-up that drafts tokens ahead.
For buyers of shared GPU capacity, the question shifts from whether data is safe to who shares the hardware. Brzak advised keeping an untrusted tenant's slice away from the sensitive model. Ask your provider whether you control that placement.
The other side
Brzak set clear limits himself. The attack cannot recover prompts, answers or contents. The workload result held only with speculative decoding on, which is an opt-in in vLLM, not the default. Even then, 31 of 40 half-second windows were right.
Signal strength varied widely across the 11 configurations he tested. He answered that a "worse signal-to-noise ratio isn't the same as safety." Still, the real threat depends on whether your model choice is actually secret.
A second variant works across GPUs linked in one server. There, he said, the layer-count signal falls below the noise. He credited other researchers with the original MIG leak. The talk covered H100 hardware only.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- AI Engineer: Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium (2026-10-12)
