whatmodel.sh / how far behind is your GPU
How to use it
- Find your line by your graphics card's memory: 8 GB, 24 GB, or any size if you rent a server.
- Point at any month to see the best model on each line then, and how far behind closed models it was.
- Click a model's name for its scores, every source, and the command to run it.
- For your exact machine, enter its GPU memory and RAM on the leaderboard: 24 GB card · 8 GB card. Each model shows the version that fits and the command to paste.
How it's worked out
- The black line is the best closed model you could use each month (API or app only). The coloured lines are the best open-weights model: at any size, one that fits a 24 GB graphics card, and one that fits in 8 GB.
- Each dashed line runs back to the month closed models first scored the same. Today that is 8 months for open weights (Kimi K3.1 Pro), 13 months for a 24 GB card (Qwen3.8 27B) and 26 months for 8 GB (Qwen3.5 9B).
- Half of new closed bests were matched by an open model within 8 months, by one that fits a 24 GB card within 14 months, and by one that fits in 8 GB within 24 months. Bests still waiting count as “at least this long so far” (5, 7 and 13 of 18), so the waits aren't made to look shorter by leaving them out.
- Fits means a download at 4-bit precision or better that runs entirely on the card, by the same sizing as the leaderboard: the file's size as its host lists it, an 8,000-token conversation and working memory, with half a gigabyte left for the card itself. Mac-only builds and setups that spill into system memory don't count. A quantised download keeps its model's score.
- Only models with 5 or more independently measured results are on the lines, closed or open (536 models). Makers' own numbers run higher than independent ones, and small models are often scored only by their makers.
- Today's scores combine published tests on one scale; 50 is average. Matching a score is not matching every ability: a small model can equal an older frontier model overall and still be weaker on long, open-ended work. Scoring method.
Each new best closed model, and how long until you could run its equal
| Closed model | released | score | open weights, any size | fits a 24 GB GPU | fits in 8 GB |
|---|---|---|---|---|---|
| GPT-4 (0314) | Mar 2023 | 53.6 | 15 months · DeepSeek-Coder-V2 Instruct (0614) | 18 months · Qwen2.5 32B Instruct | 24 months · Qwen2.5-Omni 7B |
| GPT-4 Turbo Preview (1106) | Nov 2023 | 54.6 | 8 months · Llama 3.1 405B Instruct | 10 months · Qwen2.5 32B Instruct | 16 months · Qwen2.5-Omni 7B |
| GPT-4 Turbo (2024-04-09) | Apr 2024 | 56.1 | 7 months · Qwen2.5 Coder 32B Instruct | 7 months · Qwen2.5 Coder 32B Instruct | 11 months · Qwen2.5-Omni 7B |
| GPT-4o (2024-05-13) | May 2024 | 57.6 | 7 months · DeepSeek V3 (Dec '24) | 10 months · QwQ-32B | 22 months · Qwen3.5 9B |
| GPT-4o (2024-08-06) | Aug 2024 | 57.7 | 4 months · DeepSeek V3 (Dec '24) | 7 months · QwQ-32B | 19 months · Qwen3.5 9B |
| o1-preview | Sep 2024 | 62.9 | 8 months · DeepSeek R1 0528 | 17 months · Qwen3.5 27B | not yet (26 months so far) |
| o1 (2024-12-17) | Dec 2024 | 65.3 | 7 months · Qwen3 235B-A22B Thinking 2507 | 14 months · Qwen3.5 27B | not yet (23 months so far) |
| Gemini 2.5 Pro Experimental (03-25) | Mar 2025 | 67 | 9 months · GLM-4.7 | 11 months · Qwen3.5 27B | not yet (20 months so far) |
| o3 | Apr 2025 | 68.1 | 9 months · Kimi K2.5 | 16 months · Qwen3.8 27B | not yet (19 months so far) |
| Gemini 2.5 Pro Preview (06-05) | Jun 2025 | 69.1 | 10 months · Kimi K2.6 | 14 months · Qwen3.8 27B | not yet (17 months so far) |
| GPT-5 | Aug 2025 | 70.8 | 8 months · Kimi K2.6 | 12 months · Qwen3.8 27B | not yet (15 months so far) |
| GPT-5 Pro | Oct 2025 | 71.7 | 9 months · Kimi K3 | not yet (13 months so far) | not yet (13 months so far) |
| GPT-5.2 Pro | Dec 2025 | 75.7 | 8 months · Qwen3.8 Max (2.4T-A95B) | not yet (11 months so far) | not yet (11 months so far) |
| GPT-5.4 Pro | Mar 2026 | 78.2 | not yet (8 months so far) | not yet (8 months so far) | not yet (8 months so far) |
| Claude Fable 5 | Jun 2026 | 78.6 | not yet (5 months so far) | not yet (5 months so far) | not yet (5 months so far) |
| Claude Opus 5 | Jul 2026 | 79.8 | not yet (4 months so far) | not yet (4 months so far) | not yet (4 months so far) |
| GPT-6 Astra | Sep 2026 | 84.4 | not yet (2 months so far) | not yet (2 months so far) | not yet (2 months so far) |
| Claude Fable 5.2 | Nov 2026 | 88.5 | not yet (0 months so far) | not yet (0 months so far) | not yet (0 months so far) |
Every result links to its source on the model's page. The data (CC BY-SA 4.0) and code (MIT) are on GitHub. See also the open vs closed race.