SShogo Bench

ARC-AGI-2

Abstract visual puzzles that are easy for people and hard for models.

Agent Index weight
0%
shown for context only
Models tested
96
from the benchmark’s leaderboard and reports
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1GPT-6 Astra Supported
OpenAI
95.0%source——
2Gemini 4 Carbon Supported
Google
94.2%source——
3GPT-6.1 Sol Estimated
OpenAI
94.2%source——
4Claude Fable 5.2 Supported
Anthropic
93.8%source——
5Claude Opus 5.5 Supported
Anthropic
93.3%source——
6GPT-5.6 Sol Supported
OpenAI
92.5%source$1.44—
7Gemini 4 Argon Supported
Google
91.3%source——
8Claude Opus 5 Supported
Anthropic
90.4%source——
9Claude Fable 5.1 Supported
Anthropic
90.0%source——
10GPT-6 Sol Estimated
OpenAI
89.6%source——
11Claude Fable 5 Supported
Anthropic
89.2%source——
12Gemini 3.8 Flash Supported
Google
89.2%source——
13GPT-5.5 Supported
OpenAI
85.0%source$1.87—
14Gemini 3.7 Flash Estimated
Google
84.6%source——
15Gemini 3 Deep Think
Google
84.6%source——
16GPT-5.5 Pro
OpenAI
84.6%source$10.51—
17GPT-5.6 Terra Estimated
OpenAI
83.9%source$1.09—
18Claude Sonnet 5.5 Estimated
Anthropic
83.9%source——
19GPT-5.4 Pro
OpenAI
83.3%source$16.41—
20Inkling 2 Small Estimated
thinkingmachines
81.7%source——
21DeepSeek V4.1 Flash 1017 Estimated
DeepSeek
78.1%source——
22Gemini 3.1 Pro Preview Supported
Google
77.1%source$0.96—
23Claude Opus 4.7 Supported
Anthropic
75.8%source——
24GPT-5.4 Supported
OpenAI
74.0%source$1.52—
25DeepSeek V4.1 Flash Estimated
DeepSeek
72.9%source——
26Claude Opus 4.8 Supported
Anthropic
72.1%source$2.74—
27Gemini 3.5 Flash Supported
Google
72.1%source$0.85—
28Claude Opus 4.6 Supported
Anthropic
69.2%source——
29Grok 4.6 Supported
xAI
67.1%source——
30GLM-5.3-Flash Supported
Z.ai (Zhipu)
65.8%source——
31Grok 4.20 (Non-Reasoning) Estimated
xAI
65.1%source$0.92—
32DeepSeek V4 Flash 0731
DeepSeek
61.4%source——
33Grok 4.7 Estimated
xAI
61.4%source——
34Claude Haiku 5.5 Estimated
Anthropic
61.3%source——
35DeepSeek V4 Pro 0813
DeepSeek
61.3%source——
36Claude Sonnet 4.6 Supported
Anthropic
60.4%source$2.70—
37Gemini 3.6 Flash Estimated
Google
60.4%source——
38Kimi K3 Supported
Moonshot AI
60.4%source$1.59—
39GPT-5.6 Luna Estimated
OpenAI
59.5%source$0.67—
40GPT-6 Luna
OpenAI
59.3%source——
41Muse Spark Open
Meta
57.5%source——
42GPT-5.2 Pro
OpenAI
54.2%source$15.72—
43GPT-5.2 Supported
OpenAI
52.9%source$1.90—
44Grok 4.5 Estimated
xAI
52.6%source$0.78—
45Qwen3.8 27B
Alibaba (Qwen)
42.4%source——
46Inkling Small
thinkingmachines
40.1%source——
47Claude Opus 4.5 (latest) Supported
Anthropic
37.6%source$2.40—
48Inkling Estimated
thinkingmachines
36.5%source$0.64—
49Gemini 3 Flash Preview Supported
Google
33.6%source——
50Gemini 3 Pro Preview Supported
Google
31.1%source——
51Beam 1.0
Reflection
23.5%source——
52GLM-5.2 Supported
Z.ai (Zhipu)
22.8%source$0.25—
53GPT-5.4 mini
OpenAI
18.9%source$0.75—
54GPT-5 Pro
OpenAI
18.3%source$7.14—
55GPT-5.1 Supported
OpenAI
17.6%source$1.17—
56Grok 4 Supported
xAI
16.0%source——
57Claude Sonnet 4.5 (latest) Supported
Anthropic
13.6%source$0.76—
58Kimi K2.5 Supported
Moonshot AI
11.8%source$0.28—
59Gemini 3.5 Flash Lite
Google
10.3%source——
60GPT-5 Supported
OpenAI
9.9%source$0.73—
61Claude-Opus-4 Supported
Anthropic
8.6%source$1.93—
62o3 Supported
OpenAI
6.5%source$0.83—
63tiny-recursion-model
Unknown
6.3%source$2.10—
64o4-mini Estimated
OpenAI
6.1%source$0.86—
65Claude Sonnet 4 Supported
Anthropic
5.9%source$0.49—
66GPT-5.4 nano
OpenAI
5.7%source$0.16—
67Grok 4 Fast
xAI
5.3%source$0.06—
68Gemini 2.5 Pro Supported
Google
4.9%source$0.76—
69GLM-5 Supported
Z.ai (Zhipu)
4.9%source$0.27—
70MiniMax-M2.5 Estimated
MiniMax
4.9%source$0.17—
71o3-pro
OpenAI
4.9%source$7.55—
72GPT-5 Mini Estimated
OpenAI
4.4%source$0.20—
73Claude Haiku 4.5 Estimated
Anthropic
4.0%source$0.38—
74DeepSeek V3.2 Supported
DeepSeek
4.0%source$0.12—
75o3-mini Estimated
OpenAI
3.0%source$0.55—
76GPT-5 Nano
OpenAI
2.6%source$0.03—
77Gemini 2.5 Flash Estimated
Google
2.5%source$0.32—
78DeepSeek-R1 Estimated
DeepSeek
1.3%source$0.08—
79Gemini-2.0-Flash
Google
1.3%source$0.00—
80codex-mini-2025-05-16
Unknown
1.3%source$0.23—
81Qwen3 235B A22B Instruct 2507
Alibaba (Qwen)
1.3%source$0.00—
82Claude 3.7 Sonnet Supported
Anthropic
0.9%source$0.36—
83o1-mini
OpenAI
0.8%source$0.19—
84Gemini 1.5 Pro (Feb 2024)
Google
0.8%source$0.04—
85GPT-4.5
OpenAI
0.8%source$2.10—
86GPT-4.1
OpenAI
0.4%source$0.07—
87Grok-3 mini
xAI
0.4%source$0.01—
88GPT-4.1 mini
OpenAI
0.0%source$0.01—
89GPT-4.1 nano
OpenAI
0.0%source$0.00—
90GPT-4o (2024-08-06) Supported
OpenAI
0.0%source$0.08—
91GPT-4o mini
OpenAI
0.0%source$0.01—
92Grok 3
xAI
0.0%source$0.14—
93Llama 4 Maverick
Meta
0.0%source$0.01—
94Llama 4 Scout
Meta
0.0%source$0.01—
95Magistral Medium 1.1
Mistral
0.0%source——
96Magistral Small 1.0
Mistral
0.0%source$0.05—