SShogo Bench

BALROG

Agentic play across text and grid games (NetHack, Crafter, BabyAI).

Agent Index weight
0%
shown for context only
Models tested
37
from the benchmark’s leaderboard and reports
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1GPT-6 Astra Supported
OpenAI
68.3%source—Sep 18, 2026
2Claude Opus 5 Supported
Anthropic
63.4%source—Sep 20, 2026
3GPT-5.6 Sol Supported
OpenAI
60.0%source—Sep 18, 2026
4Gemini 3 Pro Preview Supported
Google
58.1%source—Feb 3, 2026
5Gemini 3.1 Pro Preview Supported
Google
57.0%source—Feb 25, 2026
6GPT-5.6 Terra Estimated
OpenAI
53.2%source—Sep 20, 2026
7Gemini 3 Flash Preview Supported
Google
48.1%source—Feb 13, 2026
8GPT-5.6 Luna Estimated
OpenAI
45.6%source—Sep 18, 2026
9Grok 4 Supported
xAI
43.6%source—Jul 23, 2025
10Claude Opus 4.5 (latest) Supported
Anthropic
43.5%source—Feb 24, 2026
11Gemini 2.5 Pro Supported
Google
43.3%source—Apr 25, 2025
12DeepSeek-R1 Estimated
DeepSeek
34.9%source—Apr 10, 2025
13Gemini 2.5 Flash Estimated
Google
33.5%source—Jul 22, 2025
14GPT-5 Supported
OpenAI
32.8%source—Aug 19, 2025
15Claude-Sonnet-3.5 Estimated
Anthropic
32.6%source—Nov 11, 2024
16GPT-4o (2024-08-06) Supported
OpenAI
32.3%source—Nov 11, 2024
17Claude Haiku 4.5 Estimated
Anthropic
31.2%source—Feb 24, 2026
18Grok 3
xAI
29.5%source—Apr 25, 2025
19Reka Flash 3
rekaai
29.2%source—Mar 13, 2025
20Llama 3.1-70B
Meta
27.9%source—Nov 11, 2024
21Llama 3.2 90B
Meta
27.3%source—Nov 11, 2024
22Llama 3.3 70B
Meta
23.0%source—Dec 9, 2024
23Gemini 1.5 Pro (Feb 2024)
Google
21.0%source—Nov 11, 2024
24DeepSeek-R1-Distill-Qwen-32B
DeepSeek
19.5%source—Jan 26, 2025
25Claude 3.5 Haiku
Anthropic
19.3%source—Dec 11, 2024
26Mistral Nemo
Mistral
17.6%source—Dec 9, 2024
27GPT-4o mini
OpenAI
17.4%source—Nov 11, 2024
28Llama 3.2 11B
Meta
16.8%source—Nov 11, 2024
29Qwen2.5-72B Estimated
Alibaba (Qwen)
16.2%source—Nov 25, 2024
30Llama 3.1-8B
Meta
15.1%source—Nov 11, 2024
31Gemini 1.5 Flash (Sep 2024)
Google
14.6%source—Nov 11, 2024
32Qwen2-VL-72B-Instruct
Unknown
12.8%source—Nov 25, 2024
33Phi 4
Microsoft
11.6%source—Jan 13, 2025
34Llama 3.2 3B Instruct
Meta
10.1%source—Nov 11, 2024
35Qwen2.5-7B
Alibaba (Qwen)
7.8%source—Nov 25, 2024
36Llama 3.2 1B Instruct
Meta
6.6%source—Nov 11, 2024
37Qwen2-VL-7B-Instruct
Unknown
3.7%source—Nov 25, 2024