whatmodel.sh / open vs closed
How it's worked out
- Each line is the best model you could use that month: closed (API or app only) or open-weights (downloadable).
- Colour is where the maker is based. Never-released research models (PaLM, Chinchilla) aren't on the lines.
- Today's scores combine published tests on one scale; 50 is average. Scoring method.
- 7 months is the median wait for an open model to match each new closed best (17 matched, 4 still waiting).
All models that led the chart
Closed leaders
- Jun 2020 GPT-3 davinci (API base model) 31.6 · US
- Aug 2021 Codex code-davinci-001 32.8 · US
- Jan 2022 GPT-3 text-davinci-001 33.9 · US
- Mar 2022 Codex code-davinci-002 (GPT-3.5 base) 47.3 · US
- Mar 2023 GPT-4 (0314) 53.9 · US
- Nov 2023 GPT-4 Turbo Preview (1106) 55 · US
- Apr 2024 GPT-4 Turbo (2024-04-09) 56.5 · US
- May 2024 GPT-4o (2024-05-13) 58 · US
- Aug 2024 GPT-4o (2024-08-06) 58.3 · US
- Sep 2024 o1-preview 63.5 · US
- Dec 2024 o1 (2024-12-17) 66.1 · US
- Mar 2025 Gemini 2.5 Pro Experimental (03-25) 67.8 · US
- Apr 2025 o3 68.8 · US
- Jun 2025 o3-pro 69.9 · US
- Aug 2025 GPT-5 71.7 · US
- Oct 2025 GPT-5 Pro 71.9 · US
- Dec 2025 GPT-5.2 Pro 75.9 · US
- Mar 2026 GPT-5.4 Pro 79.4 · US
- Apr 2026 GPT-5.5 Pro 80.7 · US
- Jul 2026 Claude Opus 5 81.2 · US
- Sep 2026 GPT-6 Astra 85.6 · US
Open-weights leaders
- Jun 2020 T5 Large 16.8 · US
- Mar 2021 GPT-Neo 2.7B 18 · international
- Jun 2021 GPT-J 6B 24.6 · international
- Oct 2021 T0 11B 34.6 · international
- Oct 2022 Flan-T5 XXL 39.1 · US
- Feb 2023 LLaMA 65B 40 · US
- Mar 2023 Flan-UL2 20B 47.8 · US
- Aug 2023 Phind CodeLlama 34B v2 48.2 · US
- Oct 2023 speechless-codellama-34b-v2.0 50.4 · China
- Nov 2023 DeepSeek-Coder 33B Instruct 51.3 · China
- Feb 2024 OpenCodeInterpreter-DS-33B 51.6 · international
- Apr 2024 CodeQwen1.5 7B Chat 51.7 · China
- May 2024 Llama-3-Instruct-8B-SimPO 52.1 · US
- Jun 2024 DeepSeek-Coder-V2 Instruct (0614) 54.2 · China
- Jul 2024 Gemma 2 9B IT SimPO 60.1 · US
- Jan 2025 DeepSeek R1 (Jan '25) 63.5 · China
- May 2025 DeepSeek R1 0528 65.1 · China
- Jul 2025 Qwen3 235B-A22B Thinking 2507 66.1 · China
- Sep 2025 LongCat-Flash-Thinking (560B-A27B) 68.1 · China
- Jan 2026 Kimi K2.5 69.2 · China
- Feb 2026 Qwen3.5 397B-A17B 69.8 · China
- Apr 2026 Kimi K2.6 71.9 · China
- Jun 2026 GLM-5.2 72.6 · China
- Jul 2026 Kimi K3 76 · China
- Aug 2026 Hy4 preview (770B-A49B) 77.7 · China
Every result links to its source on the model's page. The data (CC BY-SA 4.0) and code (MIT) are on GitHub.