SShogo Bench

SciCode

Research-grade scientific coding problems.

Agent Index weight
0%
shown for context only
Models tested
139
from the benchmark’s leaderboard and reports
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1Claude Fable 5.2 Supported
Anthropic
67.7%source——
2Claude Opus 5.5 Supported
Anthropic
66.9%source——
3Gemini 4 Carbon Supported
Google
64.2%source——
4Claude Fable 5.1 Supported
Anthropic
63.1%source——
5Gemini 4 Argon Supported
Google
61.8%source——
6Claude Fable 5 Supported
Anthropic
61.0%source——
7Claude Sonnet 5.5 Estimated
Anthropic
61.0%source——
8MiMo-V2.6-Pro
Xiaomi
60.9%source——
9Kimi K3.1 Pro
Moonshot AI
60.8%source——
10Muse Wonder Estimated
Meta
60.3%source——
11Gemini 3.7 Flash Estimated
Google
59.8%source——
12Muse Spark 1.3 Estimated
Meta
59.7%source——
13Kimi K3 Supported
Moonshot AI
59.5%source——
14GLM-5.3 Supported
Z.ai (Zhipu)
59.0%source——
15Step 5 Preview
stepfun
58.9%source——
16Muse Spark 1.1 Estimated
Meta
58.8%source——
17Gemini 3.1 Pro Preview Supported
Google
58.7%source——
18Muse Spark Open
Meta
58.4%source——
19GPT-5.6 Sol Supported
OpenAI
57.8%source——
20Grok 4.7 Estimated
xAI
57.8%source——
21GPT-6 Sol Estimated
OpenAI
57.6%source——
22Muse Spark 1.2 Estimated
Meta
57.4%source——
23Gemini 3.8 Flash Supported
Google
56.6%source——
24GPT-5.4 Supported
OpenAI
56.6%source——
25GPT-6 Astra Supported
OpenAI
56.5%source——
26Grok 4.6 Supported
xAI
56.5%source——
27Claude Opus 5 Supported
Anthropic
56.4%source——
28GPT-5.5 Supported
OpenAI
56.1%source——
29DeepSeek V4.1 Flash 1017 Estimated
DeepSeek
55.9%source——
30GPT-6.1 Sol Estimated
OpenAI
55.8%source——
31GPT-6.1 Terra Estimated
OpenAI
55.1%source——
32Claude Haiku 5.5 Estimated
Anthropic
55.0%source——
33GPT-5.6 Terra Estimated
OpenAI
55.0%source——
34Grok 4.5 Estimated
xAI
55.0%source——
35GPT-6 Luna
OpenAI
54.6%source——
36Claude Opus 4.7 Supported
Anthropic
54.5%source——
37Claude Sonnet 5 Supported
Anthropic
54.3%source——
38Mistral Large 4
Mistral
54.2%source——
39Gemini 3.5 Flash Supported
Google
53.9%source——
40GPT-5.6 Luna Estimated
OpenAI
53.6%source——
41Claude Opus 4.8 Supported
Anthropic
53.5%source——
42Kimi K2.6 Estimated
Moonshot AI
53.5%source——
43Gemini 3.6 Flash Estimated
Google
53.4%source——
44Qwen3.8 Max Preview Estimated
Alibaba (Qwen)
53.2%source——
45GPT Chat Latest
OpenAI
52.5%source——
46Inkling 2 Small Estimated
thinkingmachines
52.3%source——
47GPT-5.4 mini
OpenAI
52.1%source——
48Qwen3.8 Max 0902
Alibaba (Qwen)
52.1%source——
49DeepSeek V4.1 Flash Estimated
DeepSeek
51.9%source——
50DeepSeek V4 Pro 0813
DeepSeek
51.7%source——
51GLM-5.3-Flash Supported
Z.ai (Zhipu)
51.6%source——
52Muse Spark
Meta
51.5%source——
53MiMo-V2.6-Flash
Xiaomi
51.3%source——
54GLM-5.2 Supported
Z.ai (Zhipu)
51.2%source——
55Grok Build 0.1
xAI
50.2%source——
56MiMo-V2.5-Pro
Xiaomi
50.2%source——
57MiniMax-M2.7
MiniMax
50.1%source——
58DeepSeek V4 Pro Estimated
DeepSeek
50.0%source——
59DeepSeek V4 Flash 0731
DeepSeek
49.9%source——
60Inkling Small
thinkingmachines
49.7%source——
61Qwen3.7 Max
Alibaba (Qwen)
49.5%source——
62Kimi K2.5 Supported
Moonshot AI
49.0%source——
63Beam 1.0
Reflection
48.5%source——
64Grok 4.3 Beta
xAI
48.3%source——
65Kimi K2.7 Code Estimated
Moonshot AI
47.8%source——
66GPT-5.4 nano
OpenAI
47.2%source——
67MiniMax-M3 Estimated
MiniMax
47.1%source——
68Inkling Estimated
thinkingmachines
47.0%source——
69Claude Sonnet 4.6 Supported
Anthropic
46.8%source——
70Qwen3.8 27B
Alibaba (Qwen)
46.6%source——
71Gemini 2.5 Pro Supported
Google
46.3%source——
72Qwen3.7 Plus
Alibaba (Qwen)
46.1%source——
73Claude Sonnet 4.5 (latest) Supported
Anthropic
45.7%source——
74Gemma 4 31B IT
Google
45.5%source——
75GLM-4.7 Estimated
Z.ai (Zhipu)
45.1%source——
76Ring-2.6-1T
Ant Group
45.0%source——
77DeepSeek V4 Flash
DeepSeek
44.9%source——
78Muse Glimmer
Meta
44.9%source——
79GLM-5.1 Estimated
Z.ai (Zhipu)
44.8%source——
80Gemini 3.1 Flash Lite
Google
43.4%source——
81GPT-5.1 Supported
OpenAI
43.3%source——
82MiMo-V2.5
Xiaomi
43.1%source——
83GPT-5 Supported
OpenAI
42.9%source——
84o3-mini Estimated
OpenAI
42.8%source——
85Nova 2.0 Pro Preview
Amazon
42.7%source——
86Claude Haiku 4.5 Estimated
Anthropic
42.2%source——
87Qwen3 235B A22B Thinking 2507
Alibaba (Qwen)
41.4%source——
88Gemini 3.5 Flash Lite
Google
41.3%source——
89Cogito v2.1
Deep Cogito
41.0%source——
90Qwen3.6 Plus Estimated
Alibaba (Qwen)
40.7%source——
91Trinity Large Thinking
arcee-ai
40.6%source——
92GPT-4.1 mini
OpenAI
40.4%source——
93Nemotron 3 Ultra
NVIDIA
40.3%source——
94Mistral Medium 3.5
Mistral
40.2%source——
95Claude Sonnet 4 Supported
Anthropic
40.0%source——
96Gemma 4 26B A4B
Google
40.0%source——
97Step 3.7 Flash
stepfun
40.0%source——
98GPT-5 Mini Estimated
OpenAI
39.2%source——
99Magistral Medium 1.2
Mistral
39.2%source——
100DeepSeek V3 0324
DeepSeek
39.0%source——
101DeepSeek V3.2 Supported
DeepSeek
38.9%source——
102GPT OSS 20B
OpenAI
38.9%source——
103Mercury 2
inception
38.7%source——
104Command A+
Cohere
38.5%source——
105Mercury 2.5
inception
38.5%source——
106GLM-4.6
Z.ai (Zhipu)
38.4%source——
107DeepSeek-R1 Estimated
DeepSeek
38.3%source——
108DeepSeek V3.1 Terminus
DeepSeek
38.0%source——
109Qwen3.6 27B
Alibaba (Qwen)
37.3%source——
110Mistral Large 3
Mistral
36.6%source——
111Qwen3.6 35B-A3B
Alibaba (Qwen)
36.6%source——
112Qwen3 Coder Next
Alibaba (Qwen)
36.2%source——
113GPT OSS 120B Supported
OpenAI
36.0%source——
114Nemotron 3 Super
NVIDIA
36.0%source——
115DeepSeek-V3
DeepSeek
35.8%source——
116Qwen3.5 122B-A10B
Alibaba (Qwen)
35.6%source——
117Qwen3 32B
Alibaba (Qwen)
35.4%source——
118Magistral Small 1.2
Mistral
35.2%source——
119Qwen3-30B-A3B-Thinking (Jul 2025)
Alibaba (Qwen)
33.0%source——
120Mistral Medium 3.1
Mistral
32.4%source——
121Llama 4 Maverick
Meta
31.7%source——
122Qwen3 14B
Alibaba (Qwen)
31.6%source——
123Qwen3.5 9B
Alibaba (Qwen)
29.5%source——
124Qwen3.5 35B-A3B
Alibaba (Qwen)
29.3%source——
125Devstral Small 2
Mistral
28.8%source——
126Mistral Small 3.1
Mistral
27.8%source——
127Claude 3.5 Haiku
Anthropic
27.4%source——
128Mistral Small 3.2
Mistral
26.4%source——
129Llama 3.3 70B
Meta
26.0%source——
130GPT-4.1 nano
OpenAI
25.9%source——
131Mimo-V2-Flash
Xiaomi
25.9%source——
132Granite 4.1 30B
IBM
25.8%source——
133solar-pro3
upstage
25.5%source——
134Gemma 3 27B
Google
23.3%source——
135Qwen3 8B
Alibaba (Qwen)
22.6%source——
136Llama 4 Scout
Meta
21.3%source——
137Gemma 3 12B
Google
16.4%source——
138Llama 3.1-8B
Meta
13.2%source——
139Phi 4 Mini
Microsoft
10.8%source——