SShogo Bench

METR Time Horizons

How long a task (in human expert time) the model completes with 50% reliability. Scored on METR’s task-suite average.

Agent Index weight
10%
in the Long-horizon autonomy pillar
Models tested
35
from the benchmark’s leaderboard and reports
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1Claude Mythos Preview
Anthropic
85.2%source——
2Claude Opus 4.6 Supported
Anthropic
78.9%source——
3Gemini 3.1 Pro Preview Supported
Google
77.0%source——
4GPT-5.2 Supported
OpenAI
75.3%source——
5Claude Opus 4.5 (latest) Supported
Anthropic
75.0%source——
6GPT-5.3 Codex Supported
OpenAI
74.5%source——
7GPT-5.4 Supported
OpenAI
74.3%source——
8Gemini 3 Pro Preview Supported
Google
71.0%source——
9GPT-5.1 Codex Max Estimated
OpenAI
70.8%source——
10GPT-5 Supported
OpenAI
69.6%source——
11Claude Sonnet 4.5 (latest) Supported
Anthropic
67.4%source——
12Claude Opus 4.1 (latest) Supported
Anthropic
66.8%source——
13Grok 4 Supported
xAI
66.6%source——
14o3 Supported
OpenAI
65.4%source——
15Claude-Opus-4 Supported
Anthropic
63.9%source——
16o4-mini Estimated
OpenAI
63.9%source——
17Claude Sonnet 4 Supported
Anthropic
62.0%source——
18Claude 3.7 Sonnet Supported
Anthropic
60.0%source——
19Kimi K2 Thinking Estimated
Moonshot AI
59.2%source——
20GPT OSS 120B Supported
OpenAI
56.6%source——
21Gemini 2.5 Pro Supported
Google
55.4%source——
22DeepSeek-R1 Estimated
DeepSeek
53.8%source——
23o1 Estimated
OpenAI
51.1%source——
24DeepSeek V3 0324
DeepSeek
49.6%source——
25DeepSeek-V3
DeepSeek
47.4%source——
26Claude-Sonnet-3.5 Estimated
Anthropic
45.2%source——
27GPT-4o (2024-08-06) Supported
OpenAI
40.8%source——
28GPT-4 Turbo
OpenAI
36.7%source——
29GPT-4
OpenAI
36.1%source——
30Qwen2.5-72B Estimated
Alibaba (Qwen)
35.8%source——
31Qwen2-72B
Alibaba (Qwen)
29.9%source——
32Claude 3 Opus Estimated
Anthropic
29.5%source——
33GPT-3.5 Turbo Instruct
OpenAI
21.5%source——
34GPT-3 175B (davinci)
OpenAI
16.2%source——
35GPT-2 (1.5B)
OpenAI
10.1%source——

For models without a METR Time Horizons result, the index estimates one from Epoch’s Capabilities Index using a logistic fit on 29 models (residual SD 0.13 on the logit scale). Estimates are labelled and widen the model’s range.