Source: Artificial Analysis β€” Composite average pass@1 across DeepSWE v1.1, Terminal-Bench 4.0, and SWE-Atlas-QnA.
Auto-updated weekly Β· Last update: 2026-10-12 12:00 UTC

#AgentModelProviderScore
1Claude CodeSonnet 5.5Anthropic68
2Claude CodeOpus 5.5Anthropic66
3Antigravity CLIGemini 4 ArgonGoogle64
4CodexGPT-6.1 SolOpenAI63
5Claude CodeFable 5.1Anthropic62
6CodexGPT-6 AstraOpenAI62
7Grok BuildGrok 4.7SpaceXAI56
8Muse CodeMuse Spark 1.3Meta54

⏱ Time per Task

Mean wall clock time per task (lower is better)

#AgentWall Time
1Codex - GPT-6.1 Sol (xhigh) ({'reasoning_effort': 'xhigh'}) (OpenAI)15.5m
2Muse Code - Muse Spark 1.3 (max) (Meta)18.4m
3Codex - GPT-6 Astra (max) (OpenAI)29.4m
4Antigravity CLI - Gemini 4 Argon (high) (Google)34.5m
5Claude Code - Fable 5.1 (max) (with fallback) (Anthropic)34.8m
6Grok Build - Grok 4.7 (xhigh) (SpaceXAI)39.2m
7Claude Code - Opus 5.5 (max) (Anthropic)1.1h
8Claude Code - Sonnet 5.5 (max) (Anthropic)1.5h

πŸ’° Cost per Task

Mean API cost per task in USD (lower is better)

#AgentCost (USD)
1Codex - GPT-6.1 Sol (xhigh) ({'reasoning_effort': 'xhigh'}) (OpenAI)$1.04
2Muse Code - Muse Spark 1.3 (max) (Meta)$3.98
3Antigravity CLI - Gemini 4 Argon (high) (Google)$5.84
4Codex - GPT-6 Astra (max) (OpenAI)$7.47
5Grok Build - Grok 4.7 (xhigh) (SpaceXAI)$8.82
6Claude Code - Fable 5.1 (max) (with fallback) (Anthropic)$12.39
7Claude Code - Opus 5.5 (max) (Anthropic)$13.04
8Claude Code - Sonnet 5.5 (max) (Anthropic)$14.19

About the Benchmarks

  • DeepSWE v1.1 β€” Software engineering tasks, 113 tasks (Datacurve)
  • Terminal-Bench 4.0 β€” Agentic terminal use, 66 tasks (Laude Institute)
  • SWE-Atlas-QnA β€” Technical Q&A, 124 questions (Scale AI)

The index represents the average pass@1 across 3 runs of each benchmark.

Data scraped weekly by an AI Agent. For the latest results, visit the original page.