TheAgentCompany
A simulated software company: browse, code, message coworkers.
Agent Index weight
0%
shown for context only
Models tested
14
from the benchmark’s leaderboard and reports
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | DeepSeek V3.2 Supported DeepSeek | 42.9% | source | $0.40 | Oct 13, 2025 | |
| 2 | Gemini 2.5 Flash Estimated Google | 41.1% | source | — | Oct 13, 2025 | |
| 3 | Claude Sonnet 4 Supported Anthropic | 33.1% | source | $1.63 | Jun 14, 2025 | |
| 4 | Claude 3.7 Sonnet Supported Anthropic | 30.9% | source | $3.70 | Jun 14, 2025 | |
| 5 | Gemini 2.5 Pro Supported Google | 30.3% | source | $4.23 | May 10, 2025 | |
| 6 | Claude-Sonnet-3.5 Estimated Anthropic | 24.0% | source | $6.34 | Dec 17, 2024 | |
| 7 | Gemini-2.0-Flash Google | 11.4% | source | $0.79 | Dec 17, 2024 | |
| 8 | GPT-4o (2024-08-06) Supported OpenAI | 8.6% | source | $1.29 | Dec 17, 2024 | |
| 9 | Llama 3.1-405B Meta | 7.4% | source | $3.21 | Dec 17, 2024 | |
| 10 | Llama 3.1-70B Meta | 6.9% | source | $0.93 | Dec 17, 2024 | |
| 11 | Qwen2.5-72B Estimated Alibaba (Qwen) | 5.7% | source | $1.53 | Dec 17, 2024 | |
| 12 | Gemini 1.5 Pro (Feb 2024) Google | 3.4% | source | $6.78 | Dec 17, 2024 | |
| 13 | Amazon Nova Pro Amazon | 1.7% | source | $1.55 | Dec 17, 2024 | |
| 14 | Qwen2-72B Alibaba (Qwen) | 1.1% | source | $0.28 | Dec 17, 2024 |