AI Agents & Models benchmarks
How performance is measured at the frontier · as of 2026-06-15 · 4 benchmarks
Capability
- #011,474
- #02
Gemini-2.5-Pro-Preview-05-06 1,446 - #031,443
- #04
ChatGPT-4o-latest (2025-03-26) 1,429 - #051,428
- #061,424
- #071,423
- #081,417
- #091,413
- #10
Gemini-2.5-Flash-Preview-04-17 1,397 - #11
Qwen3-235B-A22B-no-thinking 1,392 - #121,384
SWE-bench Verified
% resolvedShare of real GitHub issues the model resolves end-to-end.
↑ higher is better·SWE-bench
- #0171.8%
- #02
Moatless Tools + Claude 4 Sonnet 70.8% - #03
OpenHands + Claude 4 Sonnet 70.4% - #04
SWE-agent + Claude 4 Sonnet 66.6% - #0565.8%
- #0665.4%
- #07
SWE-agent + Claude 3.7 Sonnet w/ Review Heavy 62.4% - #08
OpenHands + 4x Scaled (2024-02-03) 60.8% - #0958.2%
- #10
OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022) 53.0% - #11
Composio SWE-Kit (2024-10-25) 48.6% - #1247.2%
Economics
Blended cost
$ / 1M tokensPublished price, 3:1 input:output blend — lower is cheaper.
↓ lower is better·LiteLLM (open pricing)
- #01$0.32
- #02$0.42
- #03$0.85
- #04$3.44
- #05$3.44
- #06$3.50
- #07$4.50
- #08$6.00
- #09$6.00
- #10$10.00
Context
- #011M
- #021M
- #031M
- #041M
- #05272K
- #06200K
- #07200K
- #08131.1K
- #09131.1K
- #10128K
