Skip to content
Security & Trust Talking point

Weak lab security makes a safety pause costly, so Christiano wants algorithm work paused

He argues labs should stop improving training algorithms until security improves, because a rival could copy them.

W
WebPulse Newsroom
AI-assisted · 2 min read
Share on X LinkedIn
Weak lab security makes a safety pause costly, so Christiano wants algorithm work paused
In brief
  • Paul Christiano argued that US AI labs should pause improvements to training algorithms until their security improves, because a rival could copy that work.
  • He said weak security makes slowing down costly. He also said the debate is unsettled and that a lab acting alone would lose ground.

Paul Christiano, who leads the Alignment Research Center, said on the Dwarkesh Podcast that US AI labs should pause work on better training algorithms until their security improves. His reasoning: if a lab ever needs to slow down for safety, weak security makes that pause costly, because a rival could copy its work. He spoke as a private citizen, though he recently joined OpenAI's board.

What was said

Christiano's recommendation was narrow. Labs should stop improving pre-training and reinforcement learning, the methods used to train models, because such gains spread easily and could be stolen fast. In his words: "Basically, the Frontier Labs should pause." In his view, adding more such advances now hurts the country's security.

That view rests on a risk. Christiano's concern is a period when labs may need to slow down because they cannot control increasingly capable systems. If a rival has already stolen the training methods by then, the lab cannot afford to wait.

His evidence was weak defences. He said Anthropic and OpenAI recently faced unsophisticated actors who seemed able to extract most of their algorithmic progress. The host, Dwarkesh, described two cases. A security startup called Hacktron gained access to OpenAI's shared code store, including the power to propose changes, within days. Another, Cathedral, reached all of Anthropic's Slack within 12 hours. Dwarkesh concluded: "In reality, we're 12 hours ahead of china, right?"

Why it matters

Our reading: a lead may count for little if the methods leak. Dwarkesh argued that a rival would then be "right at the frontier" without doing the research. That would shrink the lab's lead, so time spent waiting may not help it.

Christiano called security the biggest hurdle to slowing down without massive costs. For managers and buyers, the practical question is who can reach your chat workspace and shared code store. In the cases Dwarkesh described, those were the two places attackers got in.

The other side

Christiano said a pause can mean many things, and that labs do plenty of good work that should continue.

He conceded that a lab acting alone is in a rough spot. If it slows down to fix security while a competitor's leaks cost it market share, nothing improves. He said the debate has not reached a robust answer and that others hold different views.

He noted that most people credit copying techniques, not theft, for Chinese developers sitting close behind. Dwarkesh suggested foreign states may already have taken such secrets. The excerpts offer no evidence either way.

Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.

The conversation this talking point comes from