What will your workload actually cost?

Methodology & Data Sources

Data last changed July 27, 2026; each score carries its own as-of date below (no-change sync runs do not bump this date, and upstream leaderboards publish on their own schedules). Prices are our marketplace rates, fetched live. Benchmark scores come from public publications and leaderboards — never our own measurements, and never fabricated: where no direct public score exists, we show nothing.

Benchmarks used

Quality scores normalize each raw benchmark value with fixed anchors — norm = clamp((raw − anchor_low) / (anchor_high − anchor_low), 0, 1) × 100 — then take the category’s weighted average. When a model is missing a benchmark, the remaining weights are re-normalized; a model with no scored benchmark in a category gets no quality score at all.

SWE-V-adjusted cost per job

For models with a SWE-bench Verified score we also show swev_adjusted_cost = cost_per_job ÷ (SWE-bench Verified pass rate / 100) — the estimated cost per solved task if failures were blindly retried. Only the SWE-bench Verified pass rate is ever used as the denominator; Arena WebDev and LiveBench scores have no probability semantics and are never substituted. Models without a SWE-V score show “—” — we never estimate one.

• The 1/p formula assumes independent retries at a constant success rate.
• It assumes each retry uses the same tokens as the first attempt.
• Real failures often need human or tool intervention rather than a blind retry — that overhead is not priced in.
• Solve rates come from the SWE-bench Verified task set, which differs from your task distribution.

Capability tiers

Models with a quality score are grouped into 1-3 tiers (Tier 1 Frontier / Tier 2 Balanced / Tier 3 Budget) by a deterministic natural-gap rule: scores are rounded to 1 decimal, then any gap between adjacent models (in descending order) of at least max(score range × 25%, 8 points) is a candidate tier boundary; only the largest two candidate gaps become boundaries (ties go to the higher-ranked position). No qualifying gap means a single tier. Tier boundaries are pinned by a build-time snapshot test, so a data update can never silently move a model between tiers — every tier change is human-reviewed. The “Cheapest in Tier 1” highlight marks the lowest cost-per-job model inside the top tier; tier membership only requires a quality score, so a model can hold a tier while its SWE-V-adjusted cost column shows “—”.

Why most categories have no quality ranking

We only publish a quality ranking for a category when at least 5 of our models have a direct public score — exact model version, exact benchmark version, eval mode noted. Right now that bar is met only for Coding / Frontend & UI. For every other category the public leaderboards either don’t cover our catalog or have stopped updating, so we show the cost calculator only. We would rather show you nothing than a made-up number.

Task profiles behind the calculator