SShogo Bench

CritPt

Unpublished research-level physics problems.

Agent Index weight
0%
shown for context only
Models tested
162
from the benchmark’s leaderboard and reports
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1Gemini 4 Carbon Supported
Google
36.0%source——
2Claude Fable 5.2 Supported
Anthropic
35.7%source——
3GPT-5.6 Sol Supported
OpenAI
32.3%source——
4Claude Opus 5.5 Supported
Anthropic
31.7%source——
5GPT-6.1 Sol Estimated
OpenAI
31.7%source——
6GPT-6 Astra Supported
OpenAI
31.7%source——
7Claude Sonnet 5.5 Estimated
Anthropic
31.4%source——
8Claude Fable 5.1 Supported
Anthropic
31.1%source——
9GPT-6 Sol Estimated
OpenAI
30.9%source——
10GPT-5.5 Pro
OpenAI
30.6%source——
11GPT-5.4 Pro
OpenAI
30.0%source——
12GPT-5.6 Terra Estimated
OpenAI
30.0%source——
13Claude Opus 5 Supported
Anthropic
29.1%source——
14Claude Fable 5 Supported
Anthropic
28.6%source——
15Muse Wonder Estimated
Meta
28.3%source——
16GPT-6.1 Terra Estimated
OpenAI
28.0%source——
17Gemini 4 Argon Supported
Google
27.1%source——
18GPT-5.5 Supported
OpenAI
27.1%source——
19MiMo-V2.6-Pro
Xiaomi
26.6%source——
20Muse Spark 1.3 Estimated
Meta
26.0%source——
21Gemini 3 Deep Think
Google
25.7%source——
22Kimi K3.1 Pro
Moonshot AI
24.6%source——
23DeepSeek V4.1 Flash 1017 Estimated
DeepSeek
24.0%source——
24GPT-5.4 Supported
OpenAI
23.4%source——
25Kimi K3 Supported
Moonshot AI
23.4%source——
26Claude Haiku 5.5 Estimated
Anthropic
22.6%source——
27Claude Opus 4.8 Supported
Anthropic
20.9%source——
28GLM-5.2 Supported
Z.ai (Zhipu)
20.9%source——
29Step 5 Preview
stepfun
20.9%source——
30GPT-5.6 Luna Estimated
OpenAI
20.6%source——
31Qwen3.8 Max Preview Estimated
Alibaba (Qwen)
20.0%source——
32Grok 4.6 Supported
xAI
19.7%source——
33GPT-6 Luna
OpenAI
19.4%source——
34GLM-5.3 Supported
Z.ai (Zhipu)
19.1%source——
35DeepSeek V4 Pro 0813
DeepSeek
18.6%source——
36Gemini 3.8 Flash Supported
Google
18.3%source——
37Grok 4.7 Estimated
xAI
18.0%source——
38Gemini 3.1 Pro Preview Supported
Google
17.7%source——
39Muse Spark 1.2 Estimated
Meta
17.7%source——
40Qwen3.8 Max 0902
Alibaba (Qwen)
17.7%source——
41Claude Sonnet 5 Supported
Anthropic
16.9%source——
42GPT-5.3 Codex Supported
OpenAI
16.9%source——
43DeepSeek V4 Flash 0731
DeepSeek
16.6%source——
44GLM-5.3-Flash Supported
Z.ai (Zhipu)
15.4%source——
45Grok 4.5 Estimated
xAI
15.4%source——
46Muse Spark 1.1 Estimated
Meta
15.1%source——
47Muse Spark Open
Meta
14.9%source——
48DeepSeek V4.1 Flash Estimated
DeepSeek
14.3%source——
49Gemini 3.7 Flash Estimated
Google
14.3%source——
50Qwen3.7 Max
Alibaba (Qwen)
13.4%source——
51Beam 1.0
Reflection
13.1%source——
52Gemini 3.5 Flash Supported
Google
13.1%source——
53DeepSeek V4 Pro Estimated
DeepSeek
12.9%source——
54GPT-5 Supported
OpenAI
12.6%source——
55Claude Opus 4.7 Supported
Anthropic
12.0%source——
56MiMo-V2.6-Flash
Xiaomi
12.0%source——
57Muse Spark
Meta
11.3%source——
58Mistral Large 4
Mistral
11.1%source——
59Gemini 3.6 Flash Estimated
Google
10.6%source——
60GPT-5.4 mini
OpenAI
10.0%source——
61Kimi K2.7 Code Estimated
Moonshot AI
10.0%source——
62GPT-5.4 nano
OpenAI
9.3%source——
63Grok Build 0.1
xAI
9.1%source——
64Qwen3.7 Plus
Alibaba (Qwen)
9.1%source——
65Inkling Small
thinkingmachines
8.3%source——
66Grok 4.3 Beta
xAI
8.0%source——
67Kimi K2.6 Estimated
Moonshot AI
8.0%source——
68DeepSeek V4 Flash
DeepSeek
7.1%source——
69Gemini 3 Pro Preview Supported
Google
6.9%source——
70Inkling Estimated
thinkingmachines
5.4%source——
71Qwen3.8 27B
Alibaba (Qwen)
5.4%source——
72GPT-5.1 Supported
OpenAI
4.9%source——
73GLM-5.1 Estimated
Z.ai (Zhipu)
4.6%source——
74MiMo-V2.5-Pro
Xiaomi
4.0%source——
75MiMo-V2.5
Xiaomi
3.7%source——
76MiniMax-M3 Estimated
MiniMax
3.7%source——
77Ring-2.6-1T
Ant Group
3.7%source——
78Claude Sonnet 4.6 Supported
Anthropic
3.1%source——
79Kimi K2.5 Supported
Moonshot AI
3.1%source——
80Nemotron 3 Super
NVIDIA
3.1%source——
81Nemotron 3 Ultra
NVIDIA
3.1%source——
82DeepSeek V3.2 Supported
DeepSeek
2.9%source——
83Qwen3.6 Plus Estimated
Alibaba (Qwen)
2.9%source——
84Muse Glimmer
Meta
2.6%source——
85Step 3.7 Flash
stepfun
2.3%source——
86Gemini 2.5 Pro Supported
Google
2.0%source——
87DeepSeek V3.1 Terminus
DeepSeek
1.7%source——
88GLM-4.7 Estimated
Z.ai (Zhipu)
1.7%source——
89Gemma 4 31B IT
Google
1.4%source——
90GPT OSS 20B
OpenAI
1.4%source——
91Claude Sonnet 4.5 (latest) Supported
Anthropic
1.1%source——
92Gemini 3.1 Flash Lite
Google
1.1%source——
93GLM-4.6
Z.ai (Zhipu)
1.1%source——
94GPT OSS 120B Supported
OpenAI
1.1%source——
95o3 Supported
OpenAI
1.1%source——
96DeepSeek-R1 Estimated
DeepSeek
1.1%source——
97Gemini 2.5 Flash Estimated
Google
1.1%source——
98Qwen3.5 122B-A10B
Alibaba (Qwen)
0.9%source——
99Qwen3.5 397B-A17B
Alibaba (Qwen)
0.9%source——
100Qwen3.6 27B
Alibaba (Qwen)
0.9%source——
101Trinity Large Thinking
arcee-ai
0.9%source——
102Mercury 2
inception
0.8%source——
103o4-mini Estimated
OpenAI
0.6%source——
104MiniMax-M2.7
MiniMax
0.6%source——
105Qwen3.5 35B-A3B
Alibaba (Qwen)
0.6%source——
106Qwen3.5 9B
Alibaba (Qwen)
0.6%source——
107Claude-Opus-4 Supported
Anthropic
0.3%source——
108Claude Sonnet 4 Supported
Anthropic
0.3%source——
109Command A+
Cohere
0.3%source——
110Magistral Medium 1.2
Mistral
0.3%source——
111Magistral Small 1.2
Mistral
0.3%source——
112o3-mini Estimated
OpenAI
0.3%source——
113Qwen3-30B-A3B-Thinking (Jul 2025)
Alibaba (Qwen)
0.3%source——
114Qwen3 32B
Alibaba (Qwen)
0.3%source——
115Qwen3.6 35B-A3B
Alibaba (Qwen)
0.3%source——
116Amazon Nova Micro
Amazon
0.0%source——
117Claude 3.5 Haiku
Anthropic
0.0%source——
118Claude Haiku 4.5 Estimated
Anthropic
0.0%source——
119Cogito v2.1
Deep Cogito
0.0%source——
120Cohere Command A
Cohere
0.0%source——
121DeepSeek V3 0324
DeepSeek
0.0%source——
122DeepSeek-V3
DeepSeek
0.0%source——
123Devstral Small 2
Mistral
0.0%source——
124Gemini 3.5 Flash Lite
Google
0.0%source——
125Gemma 3 12B
Google
0.0%source——
126Gemma 3 270M
Google
0.0%source——
127Gemma 3 27B
Google
0.0%source——
128Gemma 4 26B A4B
Google
0.0%source——
129GPT-4.1 mini
OpenAI
0.0%source——
130GPT-4.1 nano
OpenAI
0.0%source——
131GPT-4o (2024-08-06) Supported
OpenAI
0.0%source——
132GPT Chat Latest
OpenAI
0.0%source——
133GPT-5 Mini Estimated
OpenAI
0.0%source——
134granite-4.0-350m
Unknown
0.0%source——
135granite-4.0-micro
Unknown
0.0%source——
136Granite 4.1 30B
IBM
0.0%source——
137Llama 3.1-405B
Meta
0.0%source——
138Llama 3.1-8B
Meta
0.0%source——
139Llama 3.2 11B
Meta
0.0%source——
140Llama 3.3 70B
Meta
0.0%source——
141Llama 4 Maverick
Meta
0.0%source——
142Llama 4 Scout
Meta
0.0%source——
143Mercury 2.5
inception
0.0%source——
144Mimo-V2-Flash
Xiaomi
0.0%source——
145Mistral Large 3
Mistral
0.0%source——
146Mistral Medium 3.1
Mistral
0.0%source——
147Mistral Medium 3.5
Mistral
0.0%source——
148Mistral Small 3.2
Mistral
0.0%source——
149Mistral Small 3.1
Mistral
0.0%source——
150Nvidia Nemotron Nano 9B V2
NVIDIA
0.0%source——
151Nova 2.0 Pro Preview
Amazon
0.0%source——
152Phi 4
Microsoft
0.0%source——
153Phi 4 Mini
Microsoft
0.0%source——
154Qwen3 14B
Alibaba (Qwen)
0.0%source——
155Qwen3 235B A22B Thinking 2507
Alibaba (Qwen)
0.0%source——
156Qwen3.5 0.8B
Alibaba (Qwen)
0.0%source——
157Qwen3.5 2B
Alibaba (Qwen)
0.0%source——
158Qwen3.5 4B
Alibaba (Qwen)
0.0%source——
159Qwen3 8B
Alibaba (Qwen)
0.0%source——
160Qwen3 Coder Next
Alibaba (Qwen)
0.0%source——
161Reka Flash 3
rekaai
0.0%source——
162solar-pro3
upstage
0.0%source——