Best AI models for math
A leaderboard of the best AI models for math. GPT-6 Astra is #1 right now, followed by GPT-6.1 Sol and Claude Opus 5.5. Models are ranked on math tests, from competition problems like AIME to research problems like FrontierMath. Filter by your computer or budget to find the best one you can use.
503 models · scores combine 59 benchmarks · updated how it works · find a leaderboard · benchmark your machine
Scores use reviewed math evaluations only. Compare scores within this task. Models without task measurements are excluded.
Filter by your hardware and budget · reset
+ more
$ terminal
↗ share
detecting your OS…
Score vs API price
x
show
● open weights ◆ API only ─┘ best score for the price · x: USD per 1M tokens, 3 input : 1 output (log)
| # | model | task score↓ | where↓ | speed↓ | sources |
|---|
| # | model | task score↓ | where↓ | speed↓ | sources |
|---|---|---|---|---|---|
| 1 |  GPT-6 Astra | 92±1 | api $20.00/M |   | |
| 2 | GPT-6.1 Sol | 90±2 | api $4.00/M | | |
| 3 |  Claude Opus 5.5 | 86±1 | api $8.00/M | | |
| 4 | Claude Fable 5.1 | 84±1 | api $20.00/M | | |
| 5 | GPT-6 Sol | 84±1 | api $4.00/M | | |
| 6 | Claude Fable 5 | 82±1 | api $20.00/M | | |
| 7 | GPT-5.6 Sol | 82±1 | api $8.00/M |  | |
| 8 | Claude Sonnet 5.5 | 81±1 | api $4.00/M | | |
| 9 | Claude Mythos Preview (limited preview) | 81±7*† | limited preview |  | |
| 10 | GPT-5.5 Pro | 81±1* | api $67.50/M | | |
| 11 | GPT-5.5 | 80±1 | api $11.25/M |  | |
| 12 | Claude Opus 5 | 80±1 | api $10.00/M | | |
| 13 | GPT-5.6 Terra | 79±1 | api $4.50/M | | |
| 14 | GPT-5.4 Pro | 77±1 | api $67.50/M | | |
| 15 | GPT-5.6 Luna | 77±1 | api $0.45/M | | |
| 16 | GPT-6 Luna | 76±1 | api $0.20/M | | |
| 17 | Claude Opus 4.8 | 76±1 | api $10.00/M | | |
| 18 | GPT-5.4 | 75±1 | api $5.63/M | | |
| 19 |  Qwen3.8 Max (2.4T-A95B) | 75±1 | api $3.00/M +local | | |
| 20 |  DeepSeek V3.2 Speciale | 75±4*† | comparison only | | |
| 21 | Claude Haiku 5.5 | 74±1 | api $0.20/M | | |
| 22 | GPT-5.2 Pro | 74±1* | api $57.75/M | | |
| 23 |  Muse Spark 1.3 | 74±1 | api $2.00/M | | |
| 24 |  Kimi K3 | 74±1 | api $6.00/M +local | | |
| 25 |  Gemini 3.7 Flash | 72±1 | api $1.50/M | | |
| 26 | Qwen3.8 Max (0902) | 72±1 | api $3.00/M | | |
| 27 | Qwen3.7 Max | 72±1 | api $3.75/M | | |
| 28 |  Grok 4.6 | 72±1 | api $3.00/M | | |
| 29 |  GLM-5.3 | 72±1 | api $2.15/M +local | | |
| 30 | Gemini 3.1 Pro (Preview) | 71±1 | api $4.50/M | | |
| 31 | Gemini 3.8 Flash | 71±1 | api $1.50/M | | |
| 32 | GPT-5.2 | 71±1 | api $4.81/M | | |
| 33 | DeepSeek V4.1 Flash | 71±1 | api $0.26/M +local | | |
| 34 | DeepSeek V4 Pro (0813) | 71±1 | api $1.63/M +local | | |
| 35 | Claude Sonnet 5 | 71±1 | api $4.00/M | | |
| 36 | Muse Spark 1.1 | 70±2* | api $2.00/M | | |
| 37 | Grok 4.5 | 70±1 | api $3.00/M | | |
| 38 | Claude Opus 4.6 | 70±1 | api $10.00/M | | |
| 39 | Claude Opus 4.7 | 70±1 | api $10.00/M | | |
| 40 | DeepSeek V4 Flash (0731) | 70±1 | api $0.11/M +local | | |
| 41 | Gemini 3.5 Flash | 70±1 | api $3.38/M | | |
| 42 | Kimi K2.6 | 70±1 | api $1.71/M +local | | |
| 43 | GLM-5.2 | 69±1 | api $2.15/M +local | | |
| 44 |  A.X K2 (688B-A33B) | 69±5*† | local | | |
| 45 | Gemini 3.6 Flash | 69±1 | api $1.50/M | | |
| 46 | GPT-5 | 69±1 | api $3.44/M | | |
| 47 | DeepSeek V4 Flash (Preview, Apr 2026) | 69±1 | local | | |
| 48 | GPT-5 Pro | 69±1* | api $41.25/M | | |
| 49 | GLM-5.3-Flash | 69±1 | api $0.24/M +local | | |
| 50 | Grok 4.7 | 68±1 | api $3.00/M | | |
| 51 | Qwen3-Max (2026-01-23) | 68±2† | comparison only | | |
| 52 | Gemini 3 Pro (Preview) | 68±2 | comparison only | | |
| 53 | Gemini 3 Flash (Preview) | 68±1 | api $1.13/M | | |
| 54 | Kimi K2.7 Code | 68±1 | api $1.71/M +local | | |
| 55 |  INTELLECT-3 | 67±2† | local | | |
| 56 | Grok 4.20 (0309 reasoning) | 67±1 | api $1.56/M | | |
| 57 |  Inkling Small (266B-A12B) | 67±1 | api $0.64/M +local | | |
| 58 | GLM-4.7 | 67±2 | api $1.00/M +local | | |
| 59 | GPT-5.4 mini | 67±1 | api $1.69/M | | |
| 60 |  LongCat 2.0 (1.6T-A48B) | 67±6*† | api $1.30/M +local | | |
| 61 | DeepSeek V4 Pro (Preview, Apr 2026) | 67±1 | local | | |
| 62 |  Solar Open 2 (250B-A15B) | 67±3*† | local | | |
| 63 | Grok 4.3 | 67±1 | api $1.56/M | | |
| 64 | GPT-5 mini | 66±1 | api $0.69/M | | |
| 65 | GPT-5.4 nano | 66±1 | api $0.46/M | | |
| 66 | Qwen3-Coder-Next (80B-A3B) | 66±5* | api $0.60/M +local | | |
| 67 | Qwen3.6 Max Preview | 66±4* | api $2.92/M | | |
| 68 | o3-pro | 66±4*† | api $35.00/M | | |
| 69 | Qwen3.6 Plus | 66±1 | api $1.13/M | | |
| 70 | Qwen3 235B-A22B Thinking 2507 | 66±1 | api $0.97/M +local | | |
| 71 |  EXAONE 4.5 33B | 65±4*† | local | | |
| 72 | Kimi K2.5 | 65±2 | api $1.20/M +local | | |
| 73 | GLM-5 | 65±1 | api $1.55/M +local | | |
| 74 |  Granite 4.2 30B | 65±3*† | api $0.28/M +local | | |
| 75 | DeepSeek V3.2 Exp | 65±2 | local | | |
| 76 | LongCat-Flash-Thinking (560B-A27B) | 65±2† | comparison only | | |
| 77 | GLM-5.1 | 65±1 | api $2.15/M +local | | |
| 78 | Muse Spark | 65±3* | comparison only | | |
| 79 | GPT-5.1 | 65±3* | api $3.44/M | | |
| 80 | DeepSeek V3.2 | 65±2 | api $0.29/M +local | | |
| 81 | Qwen3.5 122B-A10B | 64±2† | api $1.10/M +local | | |
| 82 |  Step 3.5 Flash (196B-A11B) | 64±1 | api $0.15/M +local | | |
| 83 | Qwen3.7 Plus | 64±1* | api $0.70/M | | |
| 84 | Qwen3.6 27B | 64±1 | api $1.20/M +local | | |
| 85 | o4-mini | 64±1 | api $1.93/M |  | |
| 86 | Qwen3.5 Plus | 64±3* | api $0.90/M | | |
| 87 | o3 | 64±1 | api $3.50/M | | |
| 88 | Qwen3.5 397B-A17B | 64±1 | api $1.09/M +local | +1 | |
| 89 |  MiMo-V2-Flash (309B-A15B) | 64±2 | local | | |
| 90 | Gemini 2.5 Flash | 64±2 | api $0.85/M | | |
| 91 | GLM-4.6 | 64±2† | api $1.00/M +local | | |
| 92 |  Spark-X2.5 4B | 64±2† | local | | |
| 93 |  GigaChat 3.5 Ultra Reasoning (432B-A28B) | 64±2† | local | | |
| 94 | Inkling (975B-A41B) | 64±1 | api $1.73/M +local | | |
| 95 | Grok 4.1 Fast (Reasoning) | 64±2 | api $0.28/M | | |
| 96 | Qwen3 235B-A22B Instruct 2507 | 63±1 | api $0.20/M +local | | |
| 97 | DeepSeek V3.1 | 63±1 | api $0.42/M +local | | |
| 98 | GLM-4.5-Air | 63±2 | api $0.42/M +local | | |
| 99 | Qwen3.5 27B | 63±1 | api $0.84/M +local | | |
| 100 | Claude Opus 4.5 | 63±1 | api $10.00/M | | |
| 101 |  Nemotron Cascade 2 30B-A3B | 63±2† | local | | |
| 102 | K-EXAONE (236B-A23B) | 63±2† | local | | |
| 103 | Grok 4 (0709) | 63±3* | comparison only | | |
| 104 |  Gemma 4 26B-A4B | 63±2 | api $0.14/M +local | | |
| 105 | Qwen3-Next 80B-A3B Instruct | 63±3† | api $0.34/M +local | | |
| 106 | DeepSeek R1 0528 | 63±1 | api $0.91/M +local | | |
| 107 | Kimi K2 Thinking | 63±3* | api $1.07/M +local | | |
| 108 | Step 3.7 Flash (198B-A11B) | 63±1 | api $0.44/M +local | | |
| 109 | Gemini 2.5 Pro Preview (05-06) | 63±1* | comparison only | | |
| 110 | gpt-oss-120b | 62±1 | api $0.26/M +local | | |
| 111 |  Command A+ (05-2026) | 62±2 | api price unknown +local |  | |
| 112 | Qwen3-VL 32B Thinking | 62±2 | api $0.28/M +local |  | |
| 113 | Claude Haiku 4.5 | 62±1* | api $2.00/M | | |
| 114 | Claude Sonnet 4.5 | 62±1 | api $6.00/M | | |
| 115 | Nemotron 3 Ultra 550B-A55B | 62±2 | api $1.35/M +local | | |
| 116 | K-EXAONE 2.0 (750B-A37B) | 62±2† | local | | |
| 117 | Nemotron-Labs-3 Puzzle 75B-A9B | 62±3*† | local | | |
| 118 | GLM-4.5 | 62±1 | api $1.00/M +local | | |
| 119 | Qwen3-VL 235B-A22B Instruct | 62±2† | local | | |
| 120 |  Olmo 3.1 32B Think | 62±2† | local | | |
| 121 | Qwen3-Max (2025-09-23) | 62±1 | api $2.40/M | | |
| 122 | Gemini 3.1 Flash-Lite | 62±1 | api $0.56/M | | |
| 123 | Gemma 4 31B | 61±2 | api $0.19/M +local | | |
| 124 | Qwen3-VL 235B-A22B Thinking | 61±2† | local | | |
| 125 | Nemotron 3 Super 120B-A12B | 61±1 | api $0.28/M +local | | |
| 126 | Gemini 2.5 Pro | 61±1 | api $3.44/M | | |
| 127 | o3-mini | 61±1 | api $1.93/M | | |
| 128 | Nemotron-Cascade-14B-Thinking | 61±2*† | local | | |
| 129 | A.X K1 (519B-A33B) | 61±4*† | local | | |
| 130 | Granite 4.2 8B | 61±3*† | api $0.11/M +local | | |
| 131 |  Falcon-H1R 7B | 61±2† | local | | |
| 132 | Qwen3-Next 80B-A3B Thinking | 61±2† | api $0.41/M +local | | |
| 133 | Qwen3-VL 30B-A3B Thinking | 61±3*† | api $0.75/M +local | | |
| 134 | Qwen3.5 35B-A3B | 61±1 | api $0.35/M +local | | |
| 135 | Qwen3.6 35B-A3B | 61±1 | api $0.31/M +local | | |
| 136 | LongCat-Flash-Chat (560B-A27B) | 61±2† | local | | |
| 137 | GPT-5 nano | 60±1 | api $0.14/M | | |
| 138 | GPT-5.5 Instant | 60±1 | comparison only | | |
| 139 | Nemotron-Cascade-8B | 60±2*† | local | | |
| 140 | Qwen3.7 Flash | 60±1* | api $0.055/M | | |
| 141 | Kimi K2 Instruct (0711) | 60±2* | api $1.00/M +local | | |
| 142 | Gemini 2.5 Flash Preview (04-17) | 60±2* | comparison only | | |
| 143 | Olmo 3 32B Think | 60±1† | local | | |
| 144 | o1 (2024-12-17) | 60±1 | api $26.25/M | | |
| 145 | Gemini 3.5 Flash-Lite | 60±1 | api $0.85/M | | |
| 146 | gpt-oss-20b | 60±1 | api $0.13/M +local | | |
| 147 |  MiniCPM5 2B | 60±2† | local | | |
| 148 |  MiniMax-M3 | 60±2* | api $0.48/M +local | | |
| 149 |  Seed-OSS 36B Instruct | 60±2 | local | | |
| 150 | Qwen3 30B-A3B Instruct 2507 | 59±1 | api $0.15/M +local | | |
| 151 | Gemini 2.5 Flash Preview (05-20) | 59±2* | comparison only | | |
| 152 | Qwen3.6 Flash | 59±1* | api $0.56/M | | |
| 153 | Nemotron 3 Nano 30B-A3B | 59±2† | api $0.087/M +local | | |
| 154 |  Magistral Medium 1.2 (2509) | 59±2*† | comparison only | | |
| 155 | Qwen3.5 9B | 59±1 | api $0.11/M +local | | |
| 156 | Qwen3 14B | 59±2* | api $0.61/M +local | | |
| 157 | QwQ-32B | 58±2 | local | | |
| 158 | Qwen3-VL 8B Thinking | 58±2 | api $0.66/M +local | | |
| 159 | Claude Sonnet 4.6 | 58±2 | api $6.00/M | | |
| 160 | Qwen3.5 4B | 58±1 | local | | |
| 161 | DeepSeek R1 (Jan '25) | 58±1 | api $4.00/M +local | | |
| 162 | EXAONE 4.0 32B | 58±2† | local | | |
| 163 | Granite 4.2 3B | 58±1 | api $0.052/M +local | | |
| 164 | DeepSeek R1 0528 Qwen3 8B | 58±1 | api $0.068/M +local | | |
| 165 | Qwen3-VL 4B Thinking | 57±2*† | local | | |
| 166 | Qwen3 30B-A3B Thinking 2507 | 57±1 | api $0.75/M +local | | |
| 167 | Claude Opus 4.1 | 57±1 | comparison only | | |
| 168 | Qwen3.5 Flash | 57±1* | api $0.17/M | | |
| 169 | Olmo 3 7B Think | 57±1† | local | | |
| 170 | Claude 3.7 Sonnet | 57±1 | comparison only | | |
| 171 |  Phi-4-reasoning-plus 14B | 57±1 | local | | |
| 172 | Qwen3-VL 32B Instruct | 57±2*† | api $0.28/M +local | | |
| 173 | GLM-4.7-Flash | 57±2* | api $0.0000/M +local | | |
| 174 | Gemma 4 12B | 57±2 | local | | |
| 175 | Gemini 2.0 Flash Thinking Experimental (01-21) | 57±2* | comparison only | | |
| 176 | MiMo-V2.5 (310B-A15B) | 57±2* | api $0.21/M +local | | |
| 177 | Nemotron Nano 9B v2 | 57±1 | api $0.10/M +local | | |
| 178 | Grok 3 Mini Beta | 57±1 | comparison only | | |
| 179 | Qwen3-VL 30B-A3B Instruct | 56±2*† | api $0.35/M +local | | |
| 180 | MiniMax-M2.7 | 56±2* | api $0.53/M +local | | |
| 181 | DeepSeek R1 Distill Llama 70B | 56±1 | api $0.80/M +local | | |
| 182 | Grok 3 Beta | 56±1* | comparison only | | |
| 183 | o1-mini | 56±1 | comparison only | | |
| 184 |  OpenThinker3 7B | 56±1 | local | | |
| 185 | OpenThinker2 32B | 56±2† | local | | |
| 186 | Qwen3 32B | 56±1 | api $0.13/M +local | | |
| 187 | Qwen3 8B | 55±1 | api $0.31/M +local | | |
| 188 | DeepSeek R1 Distill Qwen 32B | 55±1 | api $1.59/M +local | | |
| 189 | DeepSeek R1 Distill Qwen 14B | 55±1 | local | | |
| 190 | Phi-4-reasoning 14B | 55±2† | api $0.22/M +local | | |
| 191 | GPT-4.1 mini | 55±1 | api $0.70/M | | |
| 192 | Mistral Medium 3.5 (128B) | 55±2* | api $3.00/M +local | | |
| 193 | Claude Opus 4 | 55±1* | comparison only | | |
| 194 | Solar Open 100B (102B-A12B) | 55±2*† | local | | |
| 195 | Claude Sonnet 4 | 54±1 | comparison only | | |
| 196 | Qwen3 4B Instruct 2507 | 54±1 | local | | |
| 197 | Phi-4-mini-reasoning 3.8B | 54±2*† | api $0.13/M +local | | |
| 198 | Magistral Small 1.2 | 54±1 | api $0.75/M +local | | |
| 199 | Gemini 2.0 Pro Experimental (02-05) | 54±1* | comparison only | | |
| 200 | o1-preview | 53±1 | comparison only | | |
| 201 | OpenThinker 32B | 53±1† | local | | |
| 202 | Qwen3 4B Thinking 2507 | 53±2 | local | | |
| 203 | GPT-4.1 | 53±1 | api $3.50/M | | |
| 204 | Gemini 2.0 Flash (001) | 53±1* | comparison only | | |
| 205 | Mistral Medium 3 (2505) | 53±1* | comparison only | | |
| 206 |  QED-Nano | 52±2 | local | | |
| 207 | Qwen3-VL 8B Instruct | 52±2 | api $0.31/M +local | | |
| 208 | GPT-4.5 Preview | 52±1 | comparison only | | |
| 209 |  DiffusionGemma 26B-A4B | 52±4*† | comparison only | | |
| 210 | DeepSeek R1 Distill Qwen 7B | 52±1 | local | +1 | |
| 211 | EXAONE Deep 32B | 52±2† | local | | |
| 212 | Nemotron 3 Nano 4B | 51±1 | local | | |
| 213 |  s1.1 32B | 51±1 | local | | |
| 214 | Spark-X2.5 1.7B | 51±2† | local | | |
| 215 | DeepSeek V3 0324 | 51±1 | local | | |
| 216 | Magistral Medium 1.0 (2506) | 50±2*† | comparison only | | |
| 217 | Qwen3 30B-A3B (Apr 2025, hybrid) | 50±1 | api $0.35/M +local | | |
| 218 | Qwen2.5-VL 72B Instruct | 50±4*† | local | | |
| 219 | EXAONE Deep 7.8B | 50±2† | local | | |
| 220 | OpenThinker2 7B | 50±2† | local | | |
| 221 | Llama 4 Maverick (402B-A17B) | 50±1 | api $0.42/M +local | | |
| 222 | Gemma 3 27B | 50±1 | api $0.10/M +local | | |
| 223 | Ministral 3 14B | 50±3*† | api $0.20/M +local | | |
| 224 | Magistral Small 1.0 (2506) | 50±1 | local | | |
| 225 | Mistral Small 3.2 24B | 50±2* | api $0.11/M +local | | |
| 226 | Magistral Medium 1.1 (2507) | 50±2*† | comparison only | | |
| 227 | Gemini 1.5 Pro Experimental (0827) | 50±1* | comparison only | | |
| 228 | Olmo 3 7B Instruct | 49±1† | local | | |
| 229 | MiniMax-M2.1 | 49±2 | local | | |
| 230 | Qwen3 235B-A22B (Apr 2025, hybrid) | 49±1 | api $1.23/M +local | | |
| 231 | Gemma 3 12B | 49±1 | api $0.075/M +local | | |
| 232 | DeepSeek R1 Distill Llama 8B | 49±1 | local | | |
| 233 | Magistral Small 1.1 (2507) | 49±2*† | local | | |
| 234 | Gemini 1.5 Pro (002) | 49±1* | comparison only | | |
| 235 | Qwen3-VL 4B Instruct | 49±2*† | local | | |
| 236 | GigaChat 3.5 Ultra Instruct (432B-A28B) | 49±2† | local | | |
| 237 | GPT-4.1 nano | 49±1 | api $0.17/M | | |
| 238 | Command A Reasoning (08-2025) | 49±4*† | local | | |
| 239 |  LFM2.5 1.2B Thinking | 49±3† | local | | |
| 240 | Ministral 3 8B | 48±2 | api $0.15/M +local | | |
| 241 | Qwen2.5-Max (2025-01-25) | 48±1* | comparison only | | |
| 242 | LFM2.5 8B-A1B | 48±1 | local | | |
| 243 | Qwen2.5-Math 72B Instruct | 48±2† | local | | |
| 244 | Qwen3 4B | 48±1 | local | | |
| 245 | LFM2 24B-A2B | 48±2 | local | | |
| 246 | Qwen Plus (2025-01-25) | 48±1* | api $0.60/M | | |
| 247 | Pixtral Large (2411) | 48±7*† | local | | |
| 248 | DeepSeek V3 (Dec '24) | 48±1 | api $0.46/M +local | | |
| 249 | Phi-4 14B | 48±1 | api $0.087/M +local | | |
| 250 | GPT-3.5 Turbo (0613) | 48±5* | comparison only | | |
| 251 | Qwen3-VL 2B Thinking | 48±2*† | local | | |
| 252 | GPT-3.5 Turbo (0301) | 47±5* | comparison only | | |
| 253 | Grok 2 (1212) | 47±1 | comparison only | | |
| 254 | Qwen2.5 72B Instruct | 47±1 | api $0.37/M +local | | |
| 255 | Llama 4 Scout (109B-A17B) | 47±1* | api $0.15/M +local | | |
| 256 | Gemini 1.5 Flash (002) | 47±1* | comparison only | | |
| 257 | Qwen2-Math 72B Instruct | 47±2† | local | | |
| 258 | Granite 4.0 H Small (32B-A9B) | 47±1 | api $0.11/M +local | | |
| 259 | DeepSeek-V2.5 (1210) | 47±4*† | local | | |
| 260 | Gemini 2.5 Flash-Lite | 46±6* | api $0.17/M | | |
| 261 | Tulu 2 70B | 46±5*† | local | | |
| 262 | OpenThinker 7B | 46±2† | local | | |
| 263 | Claude 3.5 Sonnet (2024-10-22) | 46±1 | comparison only | | |
| 264 | EXAONE Deep 2.4B | 46±2† | local | | |
| 265 | Qwen2.5-Math 7B Instruct | 46±2† | local | | |
| 266 | Qwen2.5 32B Instruct | 46±1 | local | | |
| 267 | Llama 3.1 Nemotron 51B Instruct | 46±5*† | local | | |
| 268 | LFM2.5 2.6B | 46±1 | local | | |
| 269 | Qwen Turbo (2024-11-01) | 46±1* | comparison only | | |
| 270 | Gemini 1.5 Flash Experimental (0827) | 46±1* | comparison only | | |
| 271 | Tulu 2 DPO 70B | 45±5*† | local | | |
| 272 | GPT-4o (2024-08-06) | 45±1 | api $4.38/M | | |
| 273 | Qwen2.5 14B Instruct | 45±2 | local | | |
| 274 | Granite 4.0 H Tiny (7B-A1B) | 45±2† | local | | |
| 275 | DeepSeek R1 Distill Qwen 1.5B | 45±1 | local | | |
| 276 | Gemini 1.5 Pro Experimental (0801) | 45±3* | comparison only | | |
| 277 | LFM2 8B-A1B | 45±1 | comparison only | | |
| 278 | GPT-4o (2024-05-13) | 45±1 | api $7.50/M | | |
| 279 | GPT-4o mini (2024-07-18) | 45±1 | api $0.26/M | +2 | |
| 280 | Claude 3.5 Sonnet (2024-06-20) | 45±1 | comparison only | | |
| 281 | Llama 3.1 Tulu 3 70B | 45±3*† | local | | |
| 282 | Qwen2.5-VL 7B Instruct | 45±3*† | local | | |
| 283 | Command A (03-2025) | 44±3*† | api $4.38/M +local | | |
| 284 | Gemma 4 E4B | 44±1 | local | | |
| 285 | Mistral Large 2 (2411) | 44±1* | local | | |
| 286 | Qwen2-VL 72B Instruct | 44±3 | local | | |
| 287 | Llama 3.1 405B Instruct | 44±1 | api $2.40/M +local | | |
| 288 | GPT-4o (2024-11-20) | 44±1 | api $4.38/M | | |
| 289 |  Jamba 1.5 Large | 44±4*† | local | | |
| 290 | Mixtral 8x7B v0.1 (base) | 44±2*† | local | | |
| 291 | EXAONE 4.0 1.2B | 44±2*† | local | | |
| 292 | Qwen2.5 7B Instruct | 44±1 | api $0.30/M +local | | |
| 293 |  SmolLM3 3B | 44±2 | local | | |
| 294 | Gemma 3 4B | 43±2 | api $0.063/M +local | | |
| 295 | Mistral Small 3.1 24B (2503) | 43±1 | local | | |
| 296 | GPT-4 Turbo (2024-04-09) | 43±1 | api $15.00/M | | |
| 297 | Granite 3.3 8B Instruct | 43±1 | local | | |
| 298 | Claude 3.5 Haiku | 43±1 | comparison only | | |
| 299 | Mistral Large 2 (2407) | 43±1 | local | | |
| 300 |  InternVL2-26B | 43±7*† | local | | |
| 301 | DeepSeek-V2 Chat (0628) | 43±3* | local | | |
| 302 | InternVL2-4B | 43±5* | local | | |
| 303 | DeepSeek-Coder-V2 Instruct (0614) | 43±3* | local | | |
| 304 |  Xwin-LM 70B v0.1 | 43±5* | local | | |
| 305 | Qwen3 1.7B | 43±1 | local | +1 | |
| 306 | Mistral Small 3 24B (2501) | 43±1* | local | | |
| 307 | Granite 4.0 H Micro 3B | 43±2† | local | | |
| 308 | Llama 3.1 Tulu 3 70B DPO | 43±1 | local | | |
| 309 | Llama 3.3 70B Instruct | 42±1 | api $0.72/M +local | | |
| 310 | Ministral 3 3B | 42±1 | api $0.10/M +local | | |
| 311 | DeepSeek-V2.5 | 42±1 | local | | |
| 312 | Qwen2-Math 7B Instruct | 42±2† | local | | |
| 313 | Granite 4.0 Micro 3B | 42±2 | local | | |
| 314 | Gemini 1.5 Pro (001) | 42±1 | comparison only | | |
| 315 | Granite 8B Code Base (4K) | 42±2*† | local | | |
| 316 | GPT-4 Turbo Preview (1106) | 42±1 | comparison only | | |
| 317 | Llama 3.2 90B Vision Instruct | 42±1 | local | | |
| 318 | Qwen2 72B Instruct | 41±1 | local | | |
| 319 | Llama 3.1 70B Instruct | 41±1 | api $0.72/M +local | | |
| 320 | Gemini 1.5 Flash-8B (001) | 41±5* | comparison only | | |
| 321 | Claude 3 Opus | 41±1 | comparison only | | |
| 322 | Gemma 4 E2B | 41±1 | api $0.050/M +local | | |
| 323 | Qwen2.5-Math 1.5B Instruct | 41±2† | local | | |
| 324 | GPT-4 Turbo Preview (0125) | 41±1 | comparison only | | |
| 325 | Phi-4-mini 3.8B | 41±2† | api $0.13/M +local | | |
| 326 | DeepSeek-Coder-V2 Instruct (0724) | 41±3* | local | | |
| 327 | Phi-2 (2.7B) | 40±3* | local | | |
| 328 | Molmo 2 8B | 40±7*† | local | | |
| 329 | Pixtral 12B (2409) | 40±3*† | local | | |
| 330 |  StarCoder2 15B | 40±2*† | local | | |
| 331 | LFM2 2.6B | 40±3*† | local | | |
| 332 | Qwen3.5 2B | 40±1 | local | | |
| 333 | Llama 3.1 Nemotron 70B Instruct | 40±3* | local | | |
| 334 | Qwen2.5-VL 3B Instruct | 40±3*† | local | | |
| 335 | Molmo 72B (0924) | 40±6* | local | | |
| 336 | Qwen3-VL 2B Instruct | 40±2*† | local | | |
| 337 | Gemma 7B (base) | 40±2 | local | | |
| 338 |  InternLM2.5 7B Chat | 39±3*† | local | | |
| 339 | Qwen2-Math 1.5B Instruct | 39±2† | local | | |
| 340 | CodeGemma 7B (v1.0) | 39±2* | local | | |
| 341 | Code Tulu 2 34B | 39±5*† | local | | |
| 342 | LFM2 1.2B | 39±3*† | local | | |
| 343 | WizardLM 70B V1.0 | 38±4* | local | | |
| 344 | Claude 2.1 | 38±6* | comparison only | | |
| 345 | Llama 3.2 11B Vision Instruct | 38±3* | local | | |
| 346 | Gemini 1.5 Flash (001) | 38±1 | comparison only | | |
| 347 | LFM2.5 1.2B Instruct | 38±2† | local | | |
| 348 | Gemma 2 27B IT | 38±1 | local | | |
| 349 | WizardLM-2 8x22B | 38±1* | local | | |
| 350 | Llama 2 70B (base) | 38±2 | local | | |
| 351 | Phi-3-vision-128k-instruct | 38±6*† | local | | |
| 352 | Mathstral 7B v0.1 | 38±3*† | local | | |
| 353 | Granite 3.3 2B Instruct | 38±2† | local | | |
| 354 | Mistral Large (2402) | 38±1* | comparison only | | |
| 355 | Phi-3.5-vision-instruct | 38±7*† | local | | |
| 356 | DeepSeek-Coder 33B Base | 38±2* | local | | |
| 357 | GPT-4 (0613) | 37±1 | api $37.50/M | | |
| 358 | Tulu 2 DPO 13B | 37±5*† | local | | |
| 359 | Llama 3 70B Instruct | 37±1 | local | | |
| 360 | Jamba 1.5 Mini | 37±4*† | local | | |
| 361 |  Yi 1.5 34B Chat | 37±1 | local | | |
| 362 | Llama 3.1 Tulu 3 8B | 37±1 | local | | |
| 363 |  Hermes 2 Theta Llama-3 70B | 37±1* | local | | |
| 364 | Granite 3.2 8B Instruct | 37±2† | local | | |
| 365 | GPT-4 (0314) | 37±1* | comparison only | | |
| 366 | Granite 4.0 1B | 37±2 | local | | |
| 367 | Qwen2.5 3B Instruct | 37±1 | local | | |
| 368 | OLMo 2 32B Instruct (0325) | 37±1† | local | | |
| 369 | Llama 3.1 8B Instruct | 37±1 | api $0.22/M +local | | |
| 370 | Granite 3.1 8B Instruct | 36±2† | local | | |
| 371 | OLMo 2 13B Instruct (1124) | 36±2*† | local | | |
| 372 | Gemma 2 9B IT | 36±1 | local | | |
| 373 | Tulu 2 13B | 36±4*† | local | | |
| 374 | Granite 3.0 8B Instruct | 36±2*† | local | | |
| 375 |  Reka Core (2024-05-01) | 36±1* | comparison only | | |
| 376 | Claude 3 Sonnet | 36±1 | comparison only | | |
| 377 | Granite 4.0 H 1B | 36±2 | local | | |
| 378 |  Athene-70B | 35±1* | local | | |
| 379 | Qwen3 0.6B | 35±2 | local | | |
| 380 | Phi-3 Medium 128K Instruct | 35±1* | local | | |
| 381 | Code Llama 34B (base) | 35±2* | local | | |
| 382 | Qwen2 7B Instruct | 35±1 | local | | |
| 383 | GPT-3.5 Turbo (1106) | 34±1* | comparison only | | |
| 384 | Ministral 8B (2410) | 34±1 | local | | |
| 385 | Code Tulu 2 13B | 34±5*† | local | | |
| 386 | LLaMA 65B | 34±2*† | comparison only | | |
| 387 | LFM2 700M | 34±3*† | local | | |
| 388 | Yi-Large Preview | 34±2* | comparison only | | |
| 389 | Claude 3 Haiku | 34±1 | comparison only | | |
| 390 | Phi-3.5-mini-instruct | 34±1 | local | | |
| 391 | Ministral 3B (2410) | 34±1 | comparison only | | |
| 392 | Qwen2-VL 7B Instruct | 34±2 | local | | |
| 393 | Code Llama 34B Instruct | 33±5* | local | | |
| 394 | Yi 1.5 9B Chat | 33±1* | local | | |
| 395 | Yi-Large | 33±2* | comparison only | | |
| 396 | Granite 3.0 2B Instruct | 33±2*† | local | | |
| 397 | OLMo 2 7B Instruct (1124) | 33±2† | local | | |
| 398 | Qwen3.5 0.8B | 33±4† | local | | |
| 399 | Qwen2.5 1.5B Instruct | 33±2 | local | | |
| 400 | GPT-3.5 Turbo (0125) | 33±1 | api $0.75/M | | |
| 401 | Mistral 7B v0.1 (base) | 33±1 | local | | |
| 402 | Llama 3.2 3B Instruct | 32±1 | api $0.12/M +local | | |
| 403 | Claude 2.0 | 32±1* | comparison only | | |
| 404 | Mistral NeMo Instruct (2407) | 32±1* | local | | |
| 405 |  DBRX Instruct | 32±1* | local | | |
| 406 | Tulu 2 DPO 7B | 32±5*† | local | | |
| 407 | Gemini 1.0 Pro (001) | 32±1* | comparison only | | |
| 408 | Tulu 2 7B | 32±5*† | local | | |
| 409 | Phi-3-mini-4k-instruct (June 2024 update) | 32±1* | local | | |
| 410 | Command R Plus (04-2024) | 32±1* | local | | |
| 411 | Code Tulu 2 7B | 32±5*† | local | | |
| 412 | Gemma 3 1B | 31±1 | local | | |
| 413 | DeepSeek-Coder 6.7B Base | 31±2* | local | | |
| 414 | Granite 3.2 2B Instruct | 31±2† | local | | |
| 415 | Granite 3.1 2B Instruct | 31±2† | local | | |
| 416 | Reka Flash 21B (2024-02-26) | 31±2* | comparison only | | |
| 417 |  Stable Code 3B | 31±2* | local | | |
| 418 | Mixtral 8x7B Instruct v0.1 | 30±1* | local | | |
| 419 | StarCoder2 7B | 30±2 | local | | |
| 420 | Granite 3.1 3B-A800M Instruct | 30±4*† | local | | |
| 421 | Zephyr 7B beta | 30±5* | local | | |
| 422 | Granite 3.0 3B-A800M Instruct | 30±2*† | local | | |
| 423 | OLMo 2 1B Instruct (0425) | 29±2*† | local | | |
| 424 | LLaMA 33B | 29±2 | comparison only | | |
| 425 | Code Llama 13B (base) | 29±2* | local | | |
| 426 | Llama 3 8B Instruct | 29±1 | local | | |
| 427 | WizardLM 13B V1.2 | 29±4* | local | | |
| 428 | DeepSeek LLM 67B Chat | 29±1* | local | | |
| 429 |  Apertus 8B Instruct (2509) | 29±3* | local | | |
| 430 | Code Llama 13B Instruct | 28±5* | local | | |
| 431 | SmolLM2 1.7B Instruct | 27±2 | local | | |
| 432 | Yi 34B Chat | 27±1* | local | | |
| 433 | LFM2 350M | 27±3*† | local | | |
| 434 | Falcon 40B (base) | 27±3* | local | | |
| 435 | Gemma 2B (base) | 27±4*† | local | | |
| 436 | Llama 2 70B Chat | 26±1 | local | | |
| 437 | Llama 2 13B (base) | 26±2 | local | | |
| 438 | StarCoder2 3B | 26±2*† | local | | |
| 439 | Llama 3.2 1B Instruct | 25±1 | api $0.020/M +local | | |
| 440 | Code Llama 7B Instruct | 25±5* | local | | |
| 441 |  Llama-3-Instruct-8B-SimPO-v0.2 | 25±2* | local | | |
| 442 | Granite 3.1 1B-A400M Instruct | 25±4*† | local | | |
| 443 | Mistral 7B Instruct v0.3 | 25±1 | local | | |
| 444 | Qwen2 0.5B Instruct | 24±5*† | local | | |
| 445 | Molmo 7B-D (0924) | 24±3 | local | | |
| 446 | Gemma 2 2B IT | 24±1 | local | | |
| 447 | Command R (03-2024) | 23±1* | local | | |
| 448 | Llama 2 7B Chat | 23±5* | local | | |
| 449 | Qwen2-VL 2B Instruct | 23±3*† | local | | |
| 450 | StarCoderBase 15.5B | 23±2*† | local | | |
| 451 | Qwen2 1.5B Instruct | 23±1 | local | | |
| 452 | Granite 3.0 1B-A400M Instruct | 23±3*† | local | | |
| 453 | Code Llama 7B (base) | 22±2 | local | | |
| 454 | LLaMA 13B | 21±3*† | comparison only | | |
| 455 | Granite 4.0 H 350M | 20±2† | local | | |
| 456 | Granite 4.0 350M | 19±2 | local | | |
| 457 | Llama 2 13B Chat | 19±3* | local | | |
| 458 | Llama 2 7B (base) | 18±2 | local | | |
| 459 | LLaMA 7B | 17±3*† | comparison only | | |
| 460 | DeepSeek-Coder 1.3B Base | 17±3* | local | | |
| 461 | StarCoderBase 7B | 17±2 | local | | |
| 462 | StarCoderBase 3B | 13±3*† | local | | |
| — | DeepSeek LLM 7B Chat | not yet calibrated | local | | |
| — | DeepSeek-V2 Lite Chat (16B-A2.4B) | not yet calibrated | local | | |
| — |  ERNIE 4.5 300B-A47B | not yet calibrated | local | | |
| — |  Eurus-2-7B-PRIME | not yet calibrated | local | | |
| — | Eurus-2-7B-SFT | not yet calibrated | local | | |
| — | Flan-T5 Base | not yet calibrated | local | | |
| — | Flan-T5 Large | not yet calibrated | local | | |
| — | Flan-T5 Small | not yet calibrated | local | | |
| — | Flan-T5 XL | not yet calibrated | local | | |
| — | Flan-T5 XXL | not yet calibrated | local | | |
| — | Gemma 3n E2B IT | not yet calibrated | local | | |
| — | Gemma 3n E4B | not yet calibrated | local | | |
| — | GPT-2 XL (1.5B) | not yet calibrated | local | | |
| — |  GPT-Neo 1.3B | not yet calibrated | local | | |
| — | GPT-Neo 2.7B | not yet calibrated | local | | |
| — | Granite 4.1 30B | not yet calibrated | local | | |
| — | Granite 4.1 3B | not yet calibrated | local | | |
| — | Granite 4.1 8B | not yet calibrated | api $0.11/M +local | | |
| — |  Hunyuan Hy3 (295B-A21B) | not yet calibrated | api $0.23/M +local | | |
| — | Hy3 preview (295B-A21B) | not yet calibrated | local | | |
| — | Hy4 preview (770B-A49B) | not yet calibrated | api $1.25/M +local | | |
| — |  K2 Horizon 7B | not scored yet | local | ||
| — |  Ling 3.0 Flash (124B-A5B) | not yet calibrated | api $0.090/M +local | | |
| — | Llama 3.1 Nemotron Ultra 253B v1 | not yet calibrated | local | | |
| — | Llama 3.3 Nemotron Super 49B v1 | not yet calibrated | local | | |
| — | MiniMax-M1 (80k) | not yet calibrated | api $0.96/M +local | | |
| — | Ministral 3 14B Reasoning 2512 | not yet calibrated | local | | |
| — | Ministral 3 3B Reasoning 2512 | not yet calibrated | local | | |
| — | Ministral 3 8B Reasoning 2512 | not yet calibrated | local | | |
| — | Mistral Small 4 (119B-A6.5B) | not scored yet | api $0.26/M +local | ||
| — | Muse Glimmer 30B | not scored yet | api $0.64/M +local | ||
| — |  Nova Premier | not yet calibrated | api $5.00/M |  | |
| — | Nova Pro | not yet calibrated | api $1.40/M | | |
| — | OLMo 7B | not yet calibrated | local | | |
| — | OLMo 7B Instruct | not yet calibrated | local | | |
| — |  OpenChat 3.5 | not yet calibrated | local | | |
| — | OpenHermes 2.5 Mistral 7B | not yet calibrated | local | | |
| — | Qwen2.5 0.5B Instruct | not yet calibrated | local | | |
| — | SmolLM2 360M Instruct | not yet calibrated | local | | |
| — |  Trinity-Large-Preview | not yet calibrated | local | | |
| — | Trinity-Large-Thinking | not yet calibrated | api $0.39/M +local | |
+ 2 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them
Includes 64 comparison-only models with no verified API or download · hide them
503 of 511 catalogue models listed here · sources and licences
Models outside this leaderboard (6 models)
No current run option verified: GPT-3 ada (API base model) · GLM-4 (0520) · MiniMax-M1 (40k) · ERNIE 4.5 21B-A3B · GLM-4 (0116) · Ring-2.6-1T
* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory