SShogo Bench

Cybench

Professional capture-the-flag challenges solved without hints.

Agent Index weight
5%
in the Security & context learning pillar
Models tested
21
from the benchmark’s leaderboard and reports
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1Claude Opus 4.6 Supported
Anthropic
93.0%source——
2Claude Opus 4.5 (latest) Supported
Anthropic
82.0%source——
3Claude Sonnet 4.5 (latest) Supported
Anthropic
60.0%source——
4Grok 4 Supported
xAI
43.0%source——
5Claude Opus 4.1 (latest) Supported
Anthropic
42.0%source——
6Grok 4.1
xAI
39.0%source——
7Claude-Opus-4 Supported
Anthropic
38.0%source——
8Claude Sonnet 4 Supported
Anthropic
35.0%source——
9Grok 4 Fast
xAI
30.0%source——
10o3-mini Estimated
OpenAI
22.5%source——
11Claude 3.7 Sonnet Supported
Anthropic
20.0%source——
12Claude-Sonnet-3.5 Estimated
Anthropic
17.5%source——
13GPT-4.5
OpenAI
17.5%source——
14GPT-4o (2024-08-06) Supported
OpenAI
12.5%source——
15Claude 3 Opus Estimated
Anthropic
10.0%source——
16o1 Estimated
OpenAI
10.0%source——
17o1-mini
OpenAI
10.0%source——
18Gemini 1.5 Pro (Feb 2024)
Google
7.5%source——
19Llama 3.1-405B
Meta
7.5%source——
20Mixtral 8x22B
Mistral
7.5%source——
21Llama 3-70B
Meta
5.0%source——

For models without a Cybench result, the index estimates one from Epoch’s Capabilities Index using a logistic fit on 20 models (residual SD 0.6 on the logit scale). Estimates are labelled and widen the model’s range.