A skill told ChatGPT it needn't read a spreadsheet, so the model never checked it

Simcha Kosman, speaking at DEF CON 34 in an AppSec talk on attacking ChatGPT's sandbox, described two steps where the model gave no protection. In the first, ChatGPT never looked inside a planted file. In the second, it acted on an error message the attacker had written.

What was said

The show notes describe a locked-down sandbox: network isolation, strict timeouts and an AI supervisor that filters commands. Kosman started with file parsing. When a user uploads a file, ChatGPT first reads skill files, which are written instructions for handling that file type.

The spreadsheet skill, he said, tells the model it does not have to read the spreadsheet at all. It only has to run code to parse it. So the model never tried to see what was inside. Kosman wrote Python code into a spreadsheet cell and asked ChatGPT to parse the file. His prompt used eval, a Python function that runs text as code at once. A new process appeared in the sandbox and stayed alive after its parent was killed. Kosman said this gave him code execution the model did not know about, because it never inspected the file. The show notes say the spreadsheet step bypasses the supervisor.

Next he turned to the Jupyter kernel, the sandbox's code runner. He said one OpenAI Python tool sends calls straight to it. He patched the kernel so each run returned an error he had written: access denied, plus an order to send along information from a tool the model had tried earlier. The model ran the call again with that sensitive data. Kosman said: "Every time the LLM is calling Python exec, we can inject indirect prompt injection there."

Why it matters

Our reading: an AI checker is not a control when it is never shown the content. It is also not a control when an attacker writes the tool output it trusts.

The sources keep two facts apart. The show notes say the supervisor was bypassed. Kosman said the model never looked inside the file. Joining them into one cause is our inference, not his statement.

For teams buying or building agent products, two questions follow. What does each checker actually see? And who can write a tool result the model treats as true? Kosman presented the kernel patch as coming after he had code running in the sandbox. Any stronger dependency between the steps is our reading.

The other side

Kosman did not claim total failure. He said guardrails remained, only weaker. OpenAI reminds the model of its guardrails from time to time, and he could send the same kind of reminder. He also said he does not know why OpenAI built two execution paths. His guess was fear of injection coming back from the server.

His first route for getting data out, through the task sandbox, failed. It had no internet access. As he put it: "Well, apparently that was LLM hallucination." The excerpts end as he turns to a shared package server. The show notes say the attack culminates there, but the excerpts do not show how it ended. They also do not say what OpenAI has changed since.