In Cohere's coding tests, a stronger lead set the ceiling and cheap workers cut cost

In Cohere's tests of coding-agent teams, a stronger lead set the best result the team could reach. Cheaper workers cut the cost. Nouha Dziri, a senior research scientist at Cohere, presented the findings on the AI Engineer show.

What was said

Dziri tested teams in which a lead agent splits a coding job and workers handle the parts. She paired strong and cheap models in different roles.

In one setup the lead put more agents on planning. She swapped the strong lead's workers for cheap ones. Performance stayed almost the same and cost fell 60%, she said, with a small loss in productivity.

The reverse failed. Cheap workers could not make up for a weak lead. A weaker lead with cheap workers scored 51%. The strong model working alone scored 57%. Dziri summed it up: "the leader sets the ceiling, and the employees set the price."

Where the workers sat mattered. A team of 16 cheap planners cost 38% less than a team of only the strong model, she said, and results kept improving. More agents in implementation or validation did not help. In her words, that "doesn't really help much."

The agents also behaved like a hierarchy. Dziri said about 84% of messages went from worker to leader.

Why it matters

Our reading: the lead's seat is a judging job. Dziri did not use that term. She measured a stronger model in the lead role, which gave a higher ceiling.

For anyone buying or running agents, that suggests asking who leads the work, not only how many agents are on it. In her tests, the savings came from cheap planning workers, not from every stage.

The same logic may apply to people. A worker who can only produce output fills the cheap seat. The scarcer skill is deciding what to hand off and whether the result is good.

The other side

These results come from coding benchmarks. The leaders were AI models, not human managers. Applying them to people is our step, not Dziri's.

Cheap workers also had limits. Dziri said about half of one cheap model's messages were malformed and rejected by the system.

Budget stopped her from scaling the teams further, and she said there is a limit. She also said that "we now have teams of agents who can work autonomously 24/7 without supervision." The excerpts do not say who judges the lead when no person is watching.