- Francois named open-weight models in the hands of rogue actors and nation states as his biggest AI concern, not a machine with its own will.
- He said banks, hospitals, financial services and governments should invest heavily in cyber defense, starting with basics like changing default passwords.
The AI scenario that worries Francois most is not a machine with its own will. It is a dull one. Capable open-weight models, whose files anyone can download and run, give bad actors more power. He made the case on Y Combinator's "AI Safety Fact vs. Fiction" Paper Club episode. His fix was plain: strengthen defenses, starting with the basics.
What was said
Francois set aside the idea that AI gains consciousness and turns on us within ten or twenty years. His biggest concern was different: "really great open weight models allows for a lot more of the same from bad, rogue actors or nation states."
He pointed to attacks already happening. He described a friend's local hospital in Poland, where Russian hackers took control and shut it down for three weeks. He said they may have used AI, and that AI makes such attacks easier.
He thought the usual remedies were weak. Banning open weights would fail, he said, because he had already downloaded models himself. Registering GPUs would fail too, since people can book chips in another country.
His answer was to invest in defense. He said banks, hospitals, financial services and governments should spend heavily. Drawing on his years in retail, he recalled antiquated systems untouched for a long time, where trying the "admin-admin" login still got attackers in. He blamed that unchanged default for most of the leaked social security numbers.
Why it matters
Our reading: the argument shifts attention from the model to the target. If attackers get cheaper tools, the weakest systems become the easiest wins. That points to unglamorous work, not new AI products.
For managers, the checklist is short. Find the old systems nobody owns. Replace factory-set passwords. Count that work as security spending.
Later in the session, a speaker on the show described a two-sided threat. AI may produce mountains of buggy software, and it may also raise hackers' power against legacy systems. That second point came from this speaker, not from Francois.
The other side
The evidence here is thin. The Poland case was secondhand, and Francois only said the attackers may have used AI. His claim about leaked social security numbers was a sweeping remark, not a measured figure. He gave no odds for any scenario.
His plan also rests on a trust call. He urged that frontier labs go faster, presuming they are the good people. He said it would be worse if the roles were flipped, but he offered no proof.
Another speaker offered a counterweight: AI can attack, but it can also defend. The talk did not settle who pays for defenders to catch up, or how fast they can.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- Y Combinator: AI Safety Fact vs. Fiction | YC Paper Club (2026-10-17)
