Skip to content
The AI-First Web

Sonnet 5.5 costs ~50% more per task at max effort, tests find

Artificial Analysis found the token price unchanged, but the model writes far more, lifting benchmark cost per task.

W
WebPulse Newsroom
AI-assisted · 4 min read
Share on X LinkedIn
Sonnet 5.5 costs ~50% more per task at max effort, tests find
In brief
  • Artificial Analysis found Claude Sonnet 5.5 at max effort cost $7.60 per task, about 50% more than Sonnet 5, even though Anthropic kept the per-token price unchanged.
  • The cause is volume: it used about 193,000 output tokens per task, the most the firm has measured, so token price alone hides the real cost of the work.
  • Leaders should ask teams to track cost per completed task, check which effort setting is used, and re-measure their own workloads when a model updates.

The price per token is not the price of the work

AI vendors sell by the token, which is a small chunk of text. Most buyers budget the same way. They multiply the price per token by an estimate of use.

A new evaluation shows why that shortcut can mislead. The price is only half of the sum. The other half is how many tokens the model chooses to spend on each job.

This is like paying a contractor a fixed hourly rate. The rate looks the same on every quote. The final invoice depends on how many hours the contractor decides the job needs.

What Artificial Analysis found

Artificial Analysis, an independent AI benchmarking firm, tested Anthropic's Claude Sonnet 5.5. At max effort, the top of five effort settings, it gained 18 points over Sonnet 5 on the firm's Intelligence Index. It now ranks second, behind only Opus 5.5 at max effort.

Anthropic kept the price the same as Sonnet 5: $2 per million input tokens and $10 per million output tokens. The per-token price is a single list price. Yet in Artificial Analysis testing, the cost was $7.60 per task at max effort. That is about 50% higher than Sonnet 5's cost per task.

The reason is volume. Sonnet 5.5 at max effort used about 193,000 output tokens per Intelligence Index task. The researchers say that is the highest token use they have measured.

That is around 60% more than Opus 5.5 or Sonnet 5 at max effort. It is roughly seven times GPT-6 Astra at max effort.

What the extra spending buys

The gains are real. On Terminal-Bench 4.0, which tests AI agents working at a computer command line, Sonnet 5.5 scored 64%. Opus 5.5 and GPT-6 Astra (at xhigh effort) scored 60%.

On three knowledge-work tests, it reached parity with Opus 5.5. One example is GDPval-AA, where the scores were 1844 and 1846. The researchers add that it needed significantly more tokens to get there.

There are limits. Sonnet 5.5 scored 54% on factual accuracy in AA-Omniscience, against 66% for Opus 5.5. Its hallucination rate was lower, at 47% against 59%. Hallucination here means giving a wrong answer instead of admitting doubt.

Value depends on the setting you pick

At its list price, the researchers place Sonnet 5.5 off the Pareto frontier. That is the line of options where no rival gives more intelligence for the same cost. At lower effort settings, GPT-6 Astra or Sol configurations delivered equivalent performance for less money.

One setting looks better. At high effort, Sonnet 5.5 came within a small margin of GPT-6 Sol on intelligence. The researchers put the cost per task of the two at about the same level. Each setting sits at a different point on cost and performance, so the setting matters as much as the model.

One caveat comes from the researchers. They tested a pre-release version with a bug that can degrade answers to requests using structured outputs. Anthropic fixed it for public release. Artificial Analysis plans to re-run the relevant tests.

What leaders should ask their teams

The lesson here is that model choice is a cost decision as well as a quality decision, and this model shows it clearly. A flat price list can hide a change in what each task costs.

Put these questions to the team that runs your AI tools:

1. Do we track cost per completed task, or only price per token? Ask for both numbers.

2. Which effort setting do we run, and who chose it? Each setting sits at a different cost and performance point.

3. When a vendor updates a model at the same price, do we re-measure our own workloads? A benchmark is not your bill.

4. Does the task need top accuracy on facts? The gap in factual accuracy suggests testing before you assume a cheaper class is enough.

A model that thinks longer can be worth the money. Make sure someone has decided that on purpose. The price per token tells you what a word costs, not how many words the job will use.

Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Artificial Analysis.