SShogo Bench

o3

Deliberate o-series reasoner for hard math, coding, and multi-step analysis

Shogo Agent Index

Supported#67 of 92
31.4range 22.5–40.2 · 43% of weight measured
Agentic coding
34
34% measured
Long-horizon autonomy
42
50% measured
Knowledge-work agents
27
48% measured
Computer use
11
40% measured
Security & context learning
35
50% measured

Price

Input
$1.82/M
Output
$7.27/M
Cache read
$0.45/M
3:1 blend
$3.18/M
Agent-effective
$0.97/M

Cheapest host for agents: OpenAI at $1.07/M.

Agent-effective = 20 input tokens per output token, 85% cache hits. Why.

Agentic benchmarks

BenchmarkScoreNormalisedIndex weightHarness / source
SWE-bench Verified62.3%8%source
Terminal-Bench 2.0est.8%predicted 41
DeepSWEest.6%predicted 28
FrontierSWEest.5%predicted 5
CursorBenchest.4%predicted 28
GSO-Bench8.8%4%OpenHands
METR Time Horizons65.4%10%source
Vending-Bench 2est.6%predicted 17
PostTrainBenchest.4%predicted 18
APEX-Agentsest.8%predicted 28
GDPval30.8%7%source
Remote Labor Indexest.5%predicted 1
DeepResearch Bench45.2%5%source
OSWorld 2.0est.6%predicted 3
OSWorld23.0%4%o3 (100 steps)
Cybenchest.5%predicted 53
CL-bench17.8%5%source

Other benchmarks

BenchmarkCategoryScoreSource
Aider PolyglotCoding81.3%source
ARC-AGIReasoning60.8%source
ARC-AGI-2Reasoning6.5%source
Chess PuzzlesReasoning37.0%source
CritPtReasoning1.1%source
DTBenchKnowledge84.8%source
Fiction.LiveBenchLong context88.9%source
FrontierMath Tiers 1-3Math33.3%source
GeoBenchMultimodal74.0%source
GPQA DiamondReasoning81.8%source
Humanity's Last ExamReasoning20.3%source
LMCAKnowledge39.7%source
MATH Level 5Math97.8%source
OTIS Mock AIMEMath84.4%source
SimpleBenchReasoning53.1%source
SimpleQA VerifiedKnowledge49.4%source
VPCTMultimodal52.0%source

API hosts

Live prices and uptime for every host serving o3 via OpenRouter.

HostInputOutputCache readAgent $/MUptime 24hQuant
OpenAI$2.00$8.00$0.50$1.07100.0%—