AI model benchmarks · the judgment layer
Every new AI model’s benchmark chart
says “we win.”
The maker draws it, and most of the gaps on it are noise. We read each release and tell you which differences are real — and which you should ignore.
85%
independently verified
41
AI models tracked
14
benchmarks
No composite score. Ever.
Three things a leaderboard won’t tell you — live from today’s data
HLE
8.1
Real gap
Claude Sonnet 5.555.0
Kimi K346.9
Claude Sonnet 5.5 leads by 8.1 points on Humanity's Last Exam — outside the noise band, worth acting on.
DeepSWE
9.0
Tie
Gemini 3.8 Flash74.0
Gemini 3.7 Flash65.0
A 9.0-point gap on DeepSWE (n=113) is inside the noise band (±9.5) — call it a tie.
GLM-5.3 · Terminal-Bench 2.1
16.7
Maker’s chart ran above
Z.ai's launch chart88.2
Artificial Analysis83.9
vals.ai (archived)71.5
Z.ai’s own chart is 4.3 points above Artificial Analysis’s run of the same model — inside this benchmark’s ±10.6 band. A second independent run has it at 71.5, 16.7 above the vendor’s chart and further than the band from Artificial Analysis. Neither independent run reproduces the other, and neither reaches the maker's number.
Scores & pricing
All models →Why no single score? →Beam 1.0
Reflection
$0.70 / $2.10
2026-10-16
HLE
34.7independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetMuse Spark Open
Meta (Meta Superintelligence Labs)
$1.25 / $4.25
2026-10-15
HLE
43.9independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetNemotron 3.5 Super VL
NVIDIA
$0.23 / $0.57
2026-10-15
HLE
30.7independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetMuse Wonder
Meta (Meta Superintelligence Labs)
$4.00 / $20.00
2026-10-12
HLE
50.1independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
52.0independent
OSWorld 2.0
Not sourced yetClaude Haiku 5.5
Anthropic
$0.10 / $0.50
2026-10-07
HLE
44.4independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
72.1independent
AnalystAgent
53.8independent
Terminal-Bench 4.0
35.4independent
OSWorld 2.0
Not sourced yetMistral Large 4
Mistral AI
$0.68 / $2.09
2026-10-06
HLE
35.0independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
61.7self-reported
Toolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetGemini 4 Argon
Google DeepMind
$4.00 / $20.00
2026-09-30
HLE
57.1independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
48.8independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetGPT-6.1 Sol
OpenAI
$2.00 / $10.00
2026-09-29
HLE
52.9independent
Terminal-Bench 2.1
Not sourced yetDeepSWE
69.6independent
Toolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
94.2independent
LiveBench
81.6independent
AnalystAgent
50.0independent
Terminal-Bench 4.0
55.0independent
OSWorld 2.0
Not sourced yetClaude Sonnet 5.5
Anthropic
$2.00 / $10.00
2026-09-28
HLE
55.0independent
Terminal-Bench 2.1
83.2independent
DeepSWE
71.0self-reported
Toolathlon-Verified
77.8self-reported
Agents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
77.8independent
AnalystAgent
57.5independent
Terminal-Bench 4.0
64.1independent
OSWorld 2.0
Not sourced yetMiMo-V2.6-Pro
Xiaomi
$0.43 / $0.87
2026-09-22
HLE
49.4independent
Terminal-Bench 2.1
67.8independent
DeepSWE
71.9self-reported
Toolathlon-Verified
76.9self-reported
Agents' Last Exam
31.6self-reported
ARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
37.5independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetMiMo-V2.6-Flash
Xiaomi
$0.14 / $0.28
2026-09-22
HLE
Not sourced yetTerminal-Bench 2.1
76.4independent
DeepSWE
67.9self-reported
Toolathlon-Verified
73.6self-reported
Agents' Last Exam
27.6self-reported
ARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetClaude Opus 5.5
Anthropic
$4.00 / $20.00
2026-09-22
HLE
61.4independent
Terminal-Bench 2.1
87.6independent
DeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
38.2independent
ARC-AGI-2
Not sourced yetLiveBench
83.2independent
AnalystAgent
56.3independent
Terminal-Bench 4.0
65.2independent
OSWorld 2.0
Not sourced yetGPT-6 Luna
OpenAI
$0.10 / $0.50
2026-09-22
HLE
38.5independent
Terminal-Bench 2.1
73.0independent
DeepSWE
66.6self-reported
Toolathlon-Verified
Not sourced yetAgents' Last Exam
25.0independent
ARC-AGI-2
59.3independent
LiveBench
72.0independent
AnalystAgent
43.8independent
Terminal-Bench 4.0
13.6independent
OSWorld 2.0
Not sourced yetGrok 4.7
xAI (SpaceXAI)
$2.00 / $6.00
2026-09-21
HLE
43.1independent
Terminal-Bench 2.1
73.4independent
DeepSWE
71.0self-reported
Toolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
77.4independent
AnalystAgent
52.5independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetStep 5 Preview
StepFun
$1.00 / $2.70
2026-09-20
HLE
46.5independent
Terminal-Bench 2.1
85.0self-reported
DeepSWE
67.7self-reported
Toolathlon-Verified
74.1self-reported
Agents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
35.0independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetDeepSeek V4.1 Flash
DeepSeek
$0.30 / $1.20
2026-09-10
HLE
39.2independent
Terminal-Bench 2.1
74.5independent
DeepSWE
74.2self-reported
Toolathlon-Verified
Not sourced yetAgents' Last Exam
31.8self-reported
ARC-AGI-2
Not sourced yetLiveBench
81.1independent
AnalystAgent
35.0independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetGPT-6 Astra
OpenAI
$10.00 / $50.00
2026-09-03
HLE
54.7independent
Terminal-Bench 2.1
87.4independent
DeepSWE
74.0independent
Toolathlon-Verified
Not sourced yetAgents' Last Exam
34.2independent
ARC-AGI-2
95.0independent
LiveBench
Not sourced yetAnalystAgent
51.3independent
Terminal-Bench 4.0
59.6independent
OSWorld 2.0
Not sourced yetGemini 3.8 Flash
Google DeepMind
$0.75 / $3.75
2026-09-02
HLE
47.8independent
Terminal-Bench 2.1
87.6independent
DeepSWE
74.0independent
Toolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
75.8independent
AnalystAgent
46.3independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetMuse Spark 1.3
Meta (Meta Superintelligence Labs)
$1.25 / $4.25
2026-09-02
HLE
48.7independent
Terminal-Bench 2.1
84.3independent
DeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
32.2independent
ARC-AGI-2
Not sourced yetLiveBench
81.6independent
AnalystAgent
37.5independent
Terminal-Bench 4.0
24.8independent
OSWorld 2.0
Not sourced yetClaude Fable 5.1
Anthropic
$10.00 / $50.00
2026-09-01
HLE
59.1independent
Terminal-Bench 2.1
91.4independent
DeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
90.0independent
LiveBench
83.4independent
AnalystAgent
57.5independent
Terminal-Bench 4.0
58.1independent
OSWorld 2.0
Not sourced yetTencent Hy4 preview
Tencent
$0.83 / $2.50
2026-08-28
HLE
43.4self-reported
Terminal-Bench 2.1
55.1independent
DeepSWE
64.3self-reported
Toolathlon-Verified
74.1self-reported
Agents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
Not sourced yetAnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetGLM-5.3-Flash
Zhipu AI (Z.ai)
$0.15 / $0.50
2026-08-26
HLE
39.9independent
Terminal-Bench 2.1
84.3independent
DeepSWE
63.0independent
Toolathlon-Verified
78.4independent
Agents' Last Exam
26.3self-reported
ARC-AGI-2
65.8independent
LiveBench
71.6independent
AnalystAgent
25.0independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetQwen3.8-Flash-Next
Alibaba
—
2026-08-26
HLE
38.0independent
Terminal-Bench 2.1
86.1independent
DeepSWE
58.7self-reported
Toolathlon-Verified
73.5self-reported
Agents' Last Exam
24.3self-reported
ARC-AGI-2
Not sourced yetLiveBench
76.2independent
AnalystAgent
Not sourced yetTerminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetGLM-5.3
Zhipu AI (Z.ai)
$1.40 / $4.40
2026-08-14
HLE
42.3independent
Terminal-Bench 2.1
83.9independent
DeepSWE
69.0independent
Toolathlon-Verified
73.0self-reported
Agents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
76.1independent
AnalystAgent
38.8independent
Terminal-Bench 4.0
38.9independent
OSWorld 2.0
Not sourced yetDeepSeek V4 Pro (0813)
DeepSeek
$1.32 / $3.96
2026-08-13
HLE
41.0independent
Terminal-Bench 2.1
78.7independent
DeepSWE
62.7self-reported
Toolathlon-Verified
74.4independent
Agents' Last Exam
25.7self-reported
ARC-AGI-2
61.3independent
LiveBench
77.4independent
AnalystAgent
Not sourced yetTerminal-Bench 4.0
14.1independent
OSWorld 2.0
Not sourced yetQwen3.8-Max
Alibaba
$2.00 / $6.00
2026-08-03
HLE
43.1independent
Terminal-Bench 2.1
88.8independent
DeepSWE
57.0independent
Toolathlon-Verified
72.5self-reported
Agents' Last Exam
27.0independent
ARC-AGI-2
Not sourced yetLiveBench
78.5independent
AnalystAgent
45.0independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetKimi K3
Moonshot AI
$3.00 / $15.00
2026-07-16
HLE
46.9independent
Terminal-Bench 2.1
80.9independent
DeepSWE
69.0independent
Toolathlon-Verified
76.5independent
Agents' Last Exam
28.3independent
ARC-AGI-2
60.4independent
LiveBench
79.2independent
AnalystAgent
38.8independent
Terminal-Bench 4.0
17.2independent
OSWorld 2.0
Not sourced yetMiniMax M3
MiniMax
$0.30 / $1.20
2026-06-01
HLE
39.0independent
Terminal-Bench 2.1
53.6independent
DeepSWE
Not sourced yetToolathlon-Verified
Not sourced yetAgents' Last Exam
Not sourced yetARC-AGI-2
Not sourced yetLiveBench
67.3independent
AnalystAgent
10.0independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
22.3independent
Gemini 3.1 Pro Preview
Google DeepMind
$2.00 / $12.00
2026-02-19
HLE
47.0independent
Terminal-Bench 2.1
65.8independent
DeepSWE
Not sourced yetToolathlon-Verified
61.1independent
Agents' Last Exam
16.4independent
ARC-AGI-2
Not sourced yetLiveBench
77.0independent
AnalystAgent
41.3independent
Terminal-Bench 4.0
Not sourced yetOSWorld 2.0
Not sourced yetINDEPENDENT = a third party ran the benchmark itself (tbench.ai, vals.ai, Artificial Analysis, Snorkel…).
SELF-REPORTED = the vendor’s own number, not yet replicated.
Every cell links to its source — methodology.
Showing 10 of 14 benchmarks. The other 4 are not ranked here because this site grades them unfit for ranking — 3 are saturated (too many models crowd the ceiling, so a gap between them is not a finding), and 1 is contaminated. All 14 benchmarks, with the grading.
Showing 29 of 41 models with sourced scores. 12 are previous versions of a line we track — a newer one in the same line and price tier shipped since — and are not ranked here for that reason: Claude Opus 4.8, GPT-5.6 Sol, GLM-5.2, Claude Fable 5, Claude Opus 5, Claude Sonnet 5, Grok 4.6, Gemini 3.7 Flash, DeepSeek V4 Flash (0731), Muse Spark 1.2, GPT-5.6 Luna, GPT-6 Sol. Every one of them keeps their own page, recorded price and scores — the full catalog has every model.
There is no total score. We won’t collapse 14 different benchmarks into one number that pretends a 0.5-point gap means something — see a head-to-head for which gaps here are real.
Analysis
- 2026-08-28Models with 10M token context windows 2026None of the 41 models we track ships a native 10M-token window. 29 run flat-rate to their max; 11 price-step at or above a threshold. Every claim sourced.
- 2026-08-27GLM-5.3-Flash vs DeepSeek Flash vs Qwen3.8GLM-5.3-Flash vs Qwen3.8-Flash-Next vs DeepSeek V4 Flash: independent-score counts ran 4/6, 0/4, 8/8 at launch — two days later the boards moved.