whatmodel.sh / o3-pro
o3-pro
↗ share
+ contribute
o3-pro is a closed-weights language model from OpenAI, released June 2025. Licence: proprietary.
Catalogue identity notes
OpenAI o3-pro (dated snapshot o3-pro-2025-06-10, the only o3-pro snapshot; API alias o3-pro), a higher-compute version of o3 served through the Responses API. Distinct from o3 and from GPT-5 Pro. Evaluated in this lane's sources as 'o3-pro (high)'.
$ how it compares
The LLMs ranked either side of o3-pro, out of 828 with a score, then more from OpenAI.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 85 |  Grok 4.3 | 70±1 | api $1.56/M | ||
| 86 |  Gemini 2.5 Pro Preview (06-05) | 70±2 | comparison only | ||
| 87 | o3-pro | 70±2 | api $35.00/M | ||
| 88 |  KAT-Coder-V2.5-Dev (35B-A3B) | 70±2 | local | ||
| 89 |  Qwen3.6 Plus | 70±1 | api $1.13/M | ||
| more from OpenAI | |||||
| 1 | GPT-6 Astra | 87±1 | api $20.00/M | ||
| 3 | GPT-6.1 Sol | 85±1 | api $4.00/M | ||
| 7 | GPT-6 Sol | 83±1 | api $4.00/M | ||
$ what it's good at
agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #120 of 300 * · 9.4% of its score · 1 test
Trails SmolLM3 3B, Jamba 1.5 Large, Nova 2 Pro (Preview) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Thinkingbox-bench pass@1 (507 stateful business-workflow agent tasks, N=20) | 19.3% ✎ | ▅··▃·▃·█▃▅██ beats 3/15 | 9.4% |
coding▁▁▁▃▆██▇▆▄▃▁ #69 of 458 * · 41% of its score · 1 test
Beats DeepSeek V4 Flash Vision (Exp), GPT-5.2, GPT-5 here, though they rank higher overall. Trails Gemini 3.5 Flash-Lite, MiMo-V2.5 (310B-A15B), MiMo-V2.5-Pro (1T-A42B) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) | 84.9% ✎ | ▂·▂▃▅▂▃█▆▄▂▃ beats 48/49 | 41% |
math▁▂▃▅▆▅▄█▄▁▁▁ #65 of 398 * · 15% of its score · 1 test
Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.7 Plus here, though they rank higher overall. Trails INTELLECT-3, Qwen3-Coder-Next (80B-A3B), GPT-5 mini here, though they rank lower overall.
other tests2 not in a task · 34% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond lab | 84.0% ✎ | ▁▁▁▁▂▄▅▅███▄ beats 222/310 | 34% |
| Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate) | 76.7% ✎ | ▃▂▁▂▅█▂▃▄▃▁▁ beats 2/98 | 0.0% |
$ where to run it
Through an API
api · $35.00/M · $20 in / $80 out per 1M tokens · 200k context · developers.openai.com ↗ · price sourceUSD per 1M text tokens, standard uncached input and output. LiteLLM record 'o3-pro-2025-06-10' (litellm_provider openai, mode responses): input_cost_per_token 2e-05, output_cost_per_token 8e-05; Batch and search-context charges excluded. Currency: USD.
$ how the score was worked out · OpenAI vendor-reported benchm… 48%, Aider polyglot benchmark runs 41%, One Success Isn't Reliability… 9% and 1 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · lab-reported | 2 | 48% |
 Aider polyglot benchmark runs | 1 | 41% |
 One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4) | 1 | 9% |
 Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository) | 1 | 0% |
| prior · pull toward the middle | 1% | |
| weighted average | = 70.2 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json