SciCode
Research-grade scientific coding problems.
Agent Index weight
0%
shown for context only
Models tested
139
from the benchmark’s leaderboard and reports
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 Supported Anthropic | 67.7% | source | — | — | |
| 2 | Claude Opus 5.5 Supported Anthropic | 66.9% | source | — | — | |
| 3 | Gemini 4 Carbon Supported Google | 64.2% | source | — | — | |
| 4 | Claude Fable 5.1 Supported Anthropic | 63.1% | source | — | — | |
| 5 | Gemini 4 Argon Supported Google | 61.8% | source | — | — | |
| 6 | Claude Fable 5 Supported Anthropic | 61.0% | source | — | — | |
| 7 | Claude Sonnet 5.5 Estimated Anthropic | 61.0% | source | — | — | |
| 8 | MiMo-V2.6-Pro Xiaomi | 60.9% | source | — | — | |
| 9 | Kimi K3.1 Pro Moonshot AI | 60.8% | source | — | — | |
| 10 | Muse Wonder Estimated Meta | 60.3% | source | — | — | |
| 11 | Gemini 3.7 Flash Estimated Google | 59.8% | source | — | — | |
| 12 | Muse Spark 1.3 Estimated Meta | 59.7% | source | — | — | |
| 13 | Kimi K3 Supported Moonshot AI | 59.5% | source | — | — | |
| 14 | GLM-5.3 Supported Z.ai (Zhipu) | 59.0% | source | — | — | |
| 15 | Step 5 Preview stepfun | 58.9% | source | — | — | |
| 16 | Muse Spark 1.1 Estimated Meta | 58.8% | source | — | — | |
| 17 | Gemini 3.1 Pro Preview Supported Google | 58.7% | source | — | — | |
| 18 | Muse Spark Open Meta | 58.4% | source | — | — | |
| 19 | GPT-5.6 Sol Supported OpenAI | 57.8% | source | — | — | |
| 20 | Grok 4.7 Estimated xAI | 57.8% | source | — | — | |
| 21 | GPT-6 Sol Estimated OpenAI | 57.6% | source | — | — | |
| 22 | Muse Spark 1.2 Estimated Meta | 57.4% | source | — | — | |
| 23 | Gemini 3.8 Flash Supported Google | 56.6% | source | — | — | |
| 24 | GPT-5.4 Supported OpenAI | 56.6% | source | — | — | |
| 25 | GPT-6 Astra Supported OpenAI | 56.5% | source | — | — | |
| 26 | Grok 4.6 Supported xAI | 56.5% | source | — | — | |
| 27 | Claude Opus 5 Supported Anthropic | 56.4% | source | — | — | |
| 28 | GPT-5.5 Supported OpenAI | 56.1% | source | — | — | |
| 29 | DeepSeek V4.1 Flash 1017 Estimated DeepSeek | 55.9% | source | — | — | |
| 30 | GPT-6.1 Sol Estimated OpenAI | 55.8% | source | — | — | |
| 31 | GPT-6.1 Terra Estimated OpenAI | 55.1% | source | — | — | |
| 32 | Claude Haiku 5.5 Estimated Anthropic | 55.0% | source | — | — | |
| 33 | GPT-5.6 Terra Estimated OpenAI | 55.0% | source | — | — | |
| 34 | Grok 4.5 Estimated xAI | 55.0% | source | — | — | |
| 35 | GPT-6 Luna OpenAI | 54.6% | source | — | — | |
| 36 | Claude Opus 4.7 Supported Anthropic | 54.5% | source | — | — | |
| 37 | Claude Sonnet 5 Supported Anthropic | 54.3% | source | — | — | |
| 38 | Mistral Large 4 Mistral | 54.2% | source | — | — | |
| 39 | Gemini 3.5 Flash Supported Google | 53.9% | source | — | — | |
| 40 | GPT-5.6 Luna Estimated OpenAI | 53.6% | source | — | — | |
| 41 | Claude Opus 4.8 Supported Anthropic | 53.5% | source | — | — | |
| 42 | Kimi K2.6 Estimated Moonshot AI | 53.5% | source | — | — | |
| 43 | Gemini 3.6 Flash Estimated Google | 53.4% | source | — | — | |
| 44 | Qwen3.8 Max Preview Estimated Alibaba (Qwen) | 53.2% | source | — | — | |
| 45 | GPT Chat Latest OpenAI | 52.5% | source | — | — | |
| 46 | Inkling 2 Small Estimated thinkingmachines | 52.3% | source | — | — | |
| 47 | GPT-5.4 mini OpenAI | 52.1% | source | — | — | |
| 48 | Qwen3.8 Max 0902 Alibaba (Qwen) | 52.1% | source | — | — | |
| 49 | DeepSeek V4.1 Flash Estimated DeepSeek | 51.9% | source | — | — | |
| 50 | DeepSeek V4 Pro 0813 DeepSeek | 51.7% | source | — | — | |
| 51 | GLM-5.3-Flash Supported Z.ai (Zhipu) | 51.6% | source | — | — | |
| 52 | Muse Spark Meta | 51.5% | source | — | — | |
| 53 | MiMo-V2.6-Flash Xiaomi | 51.3% | source | — | — | |
| 54 | GLM-5.2 Supported Z.ai (Zhipu) | 51.2% | source | — | — | |
| 55 | Grok Build 0.1 xAI | 50.2% | source | — | — | |
| 56 | MiMo-V2.5-Pro Xiaomi | 50.2% | source | — | — | |
| 57 | MiniMax-M2.7 MiniMax | 50.1% | source | — | — | |
| 58 | DeepSeek V4 Pro Estimated DeepSeek | 50.0% | source | — | — | |
| 59 | DeepSeek V4 Flash 0731 DeepSeek | 49.9% | source | — | — | |
| 60 | Inkling Small thinkingmachines | 49.7% | source | — | — | |
| 61 | Qwen3.7 Max Alibaba (Qwen) | 49.5% | source | — | — | |
| 62 | Kimi K2.5 Supported Moonshot AI | 49.0% | source | — | — | |
| 63 | Beam 1.0 Reflection | 48.5% | source | — | — | |
| 64 | Grok 4.3 Beta xAI | 48.3% | source | — | — | |
| 65 | Kimi K2.7 Code Estimated Moonshot AI | 47.8% | source | — | — | |
| 66 | GPT-5.4 nano OpenAI | 47.2% | source | — | — | |
| 67 | MiniMax-M3 Estimated MiniMax | 47.1% | source | — | — | |
| 68 | Inkling Estimated thinkingmachines | 47.0% | source | — | — | |
| 69 | Claude Sonnet 4.6 Supported Anthropic | 46.8% | source | — | — | |
| 70 | Qwen3.8 27B Alibaba (Qwen) | 46.6% | source | — | — | |
| 71 | Gemini 2.5 Pro Supported Google | 46.3% | source | — | — | |
| 72 | Qwen3.7 Plus Alibaba (Qwen) | 46.1% | source | — | — | |
| 73 | Claude Sonnet 4.5 (latest) Supported Anthropic | 45.7% | source | — | — | |
| 74 | Gemma 4 31B IT Google | 45.5% | source | — | — | |
| 75 | GLM-4.7 Estimated Z.ai (Zhipu) | 45.1% | source | — | — | |
| 76 | Ring-2.6-1T Ant Group | 45.0% | source | — | — | |
| 77 | DeepSeek V4 Flash DeepSeek | 44.9% | source | — | — | |
| 78 | Muse Glimmer Meta | 44.9% | source | — | — | |
| 79 | GLM-5.1 Estimated Z.ai (Zhipu) | 44.8% | source | — | — | |
| 80 | Gemini 3.1 Flash Lite Google | 43.4% | source | — | — | |
| 81 | GPT-5.1 Supported OpenAI | 43.3% | source | — | — | |
| 82 | MiMo-V2.5 Xiaomi | 43.1% | source | — | — | |
| 83 | GPT-5 Supported OpenAI | 42.9% | source | — | — | |
| 84 | o3-mini Estimated OpenAI | 42.8% | source | — | — | |
| 85 | Nova 2.0 Pro Preview Amazon | 42.7% | source | — | — | |
| 86 | Claude Haiku 4.5 Estimated Anthropic | 42.2% | source | — | — | |
| 87 | Qwen3 235B A22B Thinking 2507 Alibaba (Qwen) | 41.4% | source | — | — | |
| 88 | Gemini 3.5 Flash Lite Google | 41.3% | source | — | — | |
| 89 | Cogito v2.1 Deep Cogito | 41.0% | source | — | — | |
| 90 | Qwen3.6 Plus Estimated Alibaba (Qwen) | 40.7% | source | — | — | |
| 91 | Trinity Large Thinking arcee-ai | 40.6% | source | — | — | |
| 92 | GPT-4.1 mini OpenAI | 40.4% | source | — | — | |
| 93 | Nemotron 3 Ultra NVIDIA | 40.3% | source | — | — | |
| 94 | Mistral Medium 3.5 Mistral | 40.2% | source | — | — | |
| 95 | Claude Sonnet 4 Supported Anthropic | 40.0% | source | — | — | |
| 96 | Gemma 4 26B A4B Google | 40.0% | source | — | — | |
| 97 | Step 3.7 Flash stepfun | 40.0% | source | — | — | |
| 98 | GPT-5 Mini Estimated OpenAI | 39.2% | source | — | — | |
| 99 | Magistral Medium 1.2 Mistral | 39.2% | source | — | — | |
| 100 | DeepSeek V3 0324 DeepSeek | 39.0% | source | — | — | |
| 101 | DeepSeek V3.2 Supported DeepSeek | 38.9% | source | — | — | |
| 102 | GPT OSS 20B OpenAI | 38.9% | source | — | — | |
| 103 | Mercury 2 inception | 38.7% | source | — | — | |
| 104 | Command A+ Cohere | 38.5% | source | — | — | |
| 105 | Mercury 2.5 inception | 38.5% | source | — | — | |
| 106 | GLM-4.6 Z.ai (Zhipu) | 38.4% | source | — | — | |
| 107 | DeepSeek-R1 Estimated DeepSeek | 38.3% | source | — | — | |
| 108 | DeepSeek V3.1 Terminus DeepSeek | 38.0% | source | — | — | |
| 109 | Qwen3.6 27B Alibaba (Qwen) | 37.3% | source | — | — | |
| 110 | Mistral Large 3 Mistral | 36.6% | source | — | — | |
| 111 | Qwen3.6 35B-A3B Alibaba (Qwen) | 36.6% | source | — | — | |
| 112 | Qwen3 Coder Next Alibaba (Qwen) | 36.2% | source | — | — | |
| 113 | GPT OSS 120B Supported OpenAI | 36.0% | source | — | — | |
| 114 | Nemotron 3 Super NVIDIA | 36.0% | source | — | — | |
| 115 | DeepSeek-V3 DeepSeek | 35.8% | source | — | — | |
| 116 | Qwen3.5 122B-A10B Alibaba (Qwen) | 35.6% | source | — | — | |
| 117 | Qwen3 32B Alibaba (Qwen) | 35.4% | source | — | — | |
| 118 | Magistral Small 1.2 Mistral | 35.2% | source | — | — | |
| 119 | Qwen3-30B-A3B-Thinking (Jul 2025) Alibaba (Qwen) | 33.0% | source | — | — | |
| 120 | Mistral Medium 3.1 Mistral | 32.4% | source | — | — | |
| 121 | Llama 4 Maverick Meta | 31.7% | source | — | — | |
| 122 | Qwen3 14B Alibaba (Qwen) | 31.6% | source | — | — | |
| 123 | Qwen3.5 9B Alibaba (Qwen) | 29.5% | source | — | — | |
| 124 | Qwen3.5 35B-A3B Alibaba (Qwen) | 29.3% | source | — | — | |
| 125 | Devstral Small 2 Mistral | 28.8% | source | — | — | |
| 126 | Mistral Small 3.1 Mistral | 27.8% | source | — | — | |
| 127 | Claude 3.5 Haiku Anthropic | 27.4% | source | — | — | |
| 128 | Mistral Small 3.2 Mistral | 26.4% | source | — | — | |
| 129 | Llama 3.3 70B Meta | 26.0% | source | — | — | |
| 130 | GPT-4.1 nano OpenAI | 25.9% | source | — | — | |
| 131 | Mimo-V2-Flash Xiaomi | 25.9% | source | — | — | |
| 132 | Granite 4.1 30B IBM | 25.8% | source | — | — | |
| 133 | solar-pro3 upstage | 25.5% | source | — | — | |
| 134 | Gemma 3 27B Google | 23.3% | source | — | — | |
| 135 | Qwen3 8B Alibaba (Qwen) | 22.6% | source | — | — | |
| 136 | Llama 4 Scout Meta | 21.3% | source | — | — | |
| 137 | Gemma 3 12B Google | 16.4% | source | — | — | |
| 138 | Llama 3.1-8B Meta | 13.2% | source | — | — | |
| 139 | Phi 4 Mini Microsoft | 10.8% | source | — | — |