ESLint paid for effort, and AI has made effort hard to verify

A polished pull request used to signal understanding

Rust contributor Jynn Nelson, as Socket reported, made a point about code contributions. A polished, well-tested pull request suggested its author had taken the time to understand the codebase. A maintainer could then expect more work from that person, so a review doubled as mentoring.

Nelson argued that AI has made this signal less reliable. A contributor can now produce an impressive patch without knowing why it works.

This article's argument builds on that point. ESLint paid for effort. Effort is exactly what AI has made hard to verify.

What ESLint did

Socket reported that ESLint creator Nicholas C. Zakas announced the end of the project's contributor pool on November 13. The pool began in 2022. It paid people outside the core team for code, documentation, Discord answers and other useful work.

By Socket's account, payouts to outsiders added up to $51,180 over four years. Of that, $24,680 went out in 2025 alone. By the team's own assessment, the program also helped bring in people who later joined the team. Zakas said the decision was unrelated to the program's cost.

The problem was proof. In 2025, ESLint ruled that work made mainly with AI tools would not earn payment. Contributors were asked to say so in their pull request descriptions. Some did not. As models improved, maintainers found it harder to separate human work from AI output.

Then people Zakas called "AI monitors" began checking pull requests and payment reports. Some publicly accused contributors of hiding AI use. Zakas also received private messages naming suspect accounts. "All of this took the joy out of the contributor pool," he wrote.

ESLint did not ban AI. Contributors may still use it if they disclose it, review the output and stay responsible for it. The policy adds one more condition: AI-written pull requests must tackle issues the team has already accepted.

How two AI systems can agree on the wrong problem

The tldraw project shows how this goes wrong. Founder Steve Ruiz built a Claude Code command that turned rough notes into detailed GitHub issues. With clear notes it worked. With vague ones, it sometimes invented bugs.

Outside contributors then pointed their own coding assistants at those issues. The resulting pull requests looked sound, and some passed automated tests. But they solved problems the project never meant to solve.

Socket's summary is that neither AI had enough context to judge whether the work made sense. The second tool implemented what the first produced without questioning the premise. Passing tests showed the code did what the tests checked, not that the change was wanted. tldraw now closes external pull requests automatically. It still accepts issues and discussions.

Review time is the strained resource

Socket pointed to GitHub's October report. It says an AI agent wrote the first draft of over 40% of pull requests on the platform by September. In March, that share was 17%. GitHub also counted 110 million pull requests begun by agents in September. Many are legitimate work.

The Godot Foundation said in June that writing a pull request had become far easier, while reviewing one had not. It now bars autonomous AI agents and substantial AI-generated code. Contributors who have merged no more than three pull requests must ask a maintainer before suggesting new features.

LLVM took a lighter approach. It allows AI-assisted work with human oversight, but keeps "good first issue" tasks for human learners. Node.js added extra requirements for pull requests over 5,000 lines.

In the Linux kernel, Lorenzo Stoakes moved on November 10 to step down from several reviewer roles. He cited the volume of AI-generated patch series. In the later mailing-list thread, he said such submissions went from occasional to weekly to every few days. He put them at about half of the patches he sees, or more. He said he mainly wanted to cut inbox noise.

The Linux Foundation published a study in October. Of the maintainers it surveyed, 59% said contributions linked to generative AI were growing. Respondents put low-quality AI-written pull requests at about a quarter of what arrives. That is a survey estimate, not a count.

These figures measure different things. One counts agent-started pull requests across GitHub. One is a single developer's estimate of his own inbox. One is a survey estimate. None should be added to the others.

Money faces similar pressure. Socket reported that curl ended its bug bounty in January after a rise in low-quality reports, many AI-assisted. In July it paused vulnerability reporting for a month.

The hidden cost is the next maintainer

Closing the door protects today's reviewers. It also narrows the route by which newcomers become reviewers. Godot's announcement raised this concern. Feedback pasted straight into an AI assistant teaches the contributor little.

Zakas was frank about the unknown. Whether AI-powered contributions will build the same dedication as human work, he said, "is yet to be seen."

What leaders should ask

Many companies rely on open source packages they do not pay for. The people who review those packages are limited in number. Put three questions to your engineering and security teams.

First, which of our key dependencies changed their contribution or bug reporting rules this year? Second, when we send fixes upstream, do we disclose AI use and have a person who understands each change? Third, do we fund or staff any of the projects we rely on?

Review time is the scarce supply. A company that sends AI-written patches upstream spends someone else's attention.

Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Socket.