SShogo Bench

o1

O-series reasoning model for hard analysis, math, coding, and planning

Shogo Agent Index

Estimated#88 of 92
16.7range 11.7–21.7 · 15% of weight measured
Agentic coding
16
estimated
Long-horizon autonomy
28
50% measured
Knowledge-work agents
16
estimated
Computer use
6
estimated
Security & context learning
11
50% measured

Price

Input
$14.00/M
Output
$54.00/M
Cache read
—/M
3:1 blend
$24.00/M
Agent-effective
$15.90/M

Agent-effective = 20 input tokens per output token, 85% cache hits. Why.

Agentic benchmarks

BenchmarkScoreNormalisedIndex weightHarness / source
SWE-bench Verifiedest.8%predicted 48
Terminal-Bench 2.0est.8%predicted 7
DeepSWEest.6%predicted 8
FrontierSWEest.5%predicted 1
CursorBenchest.4%predicted 16
GSO-Benchest.4%predicted 1
METR Time Horizons51.1%10%source
Vending-Bench 2est.6%predicted 2
PostTrainBenchest.4%predicted 10
APEX-Agentsest.8%predicted 9
GDPvalest.7%predicted 15
Remote Labor Indexest.5%predicted 0
DeepResearch Benchest.5%predicted 41
OSWorld 2.0est.6%predicted 0
OSWorldest.4%predicted 13
Cybench10.0%5%source
CL-benchest.5%predicted 12

Other benchmarks

BenchmarkCategoryScoreSource
Aider PolyglotCoding61.7%source
ARC-AGIReasoning30.7%source
Chess PuzzlesReasoning11.0%source
DTBenchKnowledge74.7%source
Fiction.LiveBenchLong context83.3%source
FrontierMath Tiers 1-3Math14.7%source
GeoBenchMultimodal80.0%source
GPQA DiamondReasoning76.8%source
Humanity's Last ExamReasoning8.0%source
LMCAKnowledge22.3%source
MATH Level 5Math94.7%source
OTIS Mock AIMEMath73.3%source
SimpleBenchReasoning41.7%source
SimpleQA VerifiedKnowledge41.1%source
VPCTMultimodal37.0%source