whatmodel.sh / maker vs independent
How it's worked out
- Each dot is one model on one test where its maker published a score and someone independent (a leaderboard, a paper, another lab running it) measured it too. 611 pairs, 180 models, 103 tests. Gaps are in percentage points on each test's own scale, not the whatmodel score.
- Most pairs roughly agree: overall the maker's number is higher 61% of the time (95% interval 57%–66%), by a median of +0.8 points. But when the two are 5+ points apart, the maker's is higher 72% of the time: 138 pairs against 55 (95% interval 63%–79%). Intervals resample whole models, since one model's pairs aren't independent.
- It depends on the test. Standard tests with a fixed format (MMLU, WinoGrande, ARC Challenge, HumanEval) barely move; tests where the setup is part of the result, like the agent harness on SWE-bench Verified, sampling and reasoning budget on AIME, or few-shot prompting on GSM8K and MBPP, show the biggest gaps.
- It doesn't seem to be rivals marking each other down. Against neutral testers (leaderboards, Epoch AI, papers that don't release a model) the maker is higher 55% of 236 pairs, 57% of its 63 big gaps; against rival labs' own release documents, 63% of 446, 76% of 150. Rivals test more of the setup-sensitive tests. Where a rival and a neutral tester measured the same model on the same test (272 cases), the rival was lower 119 times and higher 119 times (median 0.0).
- Gaps aren't all inflation: settings often differ (reasoning effort, agent harness, few-shot prompts, test subset or version), and labs choose which tests to publish, so these pairs aren't a random sample. When a side has several results for the same model and test, the dot uses their median. Accuracy tests only; arena ratings have no maker-reported twin. Hover a dot for both sources.
By test
Tests with 13 or more pairs.
| Test | pairs | median gap | maker higher |
|---|---|---|---|
| SWE-bench Verified | 13 | +9.0 | 92% |
| MBPP | 13 | +9.0 | 77% |
| ARC Easy | 14 | +4.8 | 79% |
| AIME 2024 | 14 | +3.8 | 86% |
| GSM8K | 19 | +3.3 | 84% |
| AIME 2025 | 20 | +2.6 | 70% |
| MMLU-Pro | 29 | +2.0 | 69% |
| GPQA Diamond | 51 | +1.1 | 67% |
| IFEval (aggregation unspecified) | 13 | +1.0 | 69% |
| PIQA | 16 | +0.5 | 75% |
| MMLU | 20 | +0.1 | 50% |
| HumanEval | 34 | 0.0 | 47% |
| WinoGrande | 25 | -0.1 | 40% |
| ARC Challenge | 14 | -0.6 | 50% |
| τ²-bench Telecom | 18 | -1.3 | 39% |
By maker
Makers with 10 or more pairs. Small groups swing a lot: read these as rough.
| Maker | pairs | median gap | maker higher |
|---|---|---|---|
| Alibaba Cloud | 148 | +1.0 | 61% |
| Meta | 89 | +0.7 | 62% |
| EleutherAI | 48 | 0.0 | 35% |
| 48 | +1.7 | 60% | |
| DeepSeek | 46 | +0.8 | 70% |
| OpenAI | 36 | +0.5 | 56% |
| Z.ai | 27 | +0.9 | 67% |
| Mistral AI | 20 | +1.6 | 75% |
| BigScience | 20 | -11.9 | 10% |
| BigCode | 20 | +2.2 | 95% |
| Liquid AI | 15 | -0.7 | 47% |
| Ai2 | 14 | +8.8 | 93% |
| Microsoft | 14 | 0.0 | 50% |
| IBM | 13 | +1.7 | 77% |
| Anthropic | 12 | +1.6 | 75% |
| All makers | 611 | +0.8 | 61% |
All 611 pairs, biggest maker advantage first
| Model | Test | maker | independent | gap | Sources |
|---|---|---|---|---|---|
| Molmo 7B-D (0924) Ai2 | ChartQA | 84.1 | 27.0 | +57.1 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Qwen2-VL 7B Instruct Alibaba Cloud | ChartQA | 83.0 | 41.2 | +41.8 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Molmo 7B-D (0924) Ai2 | MathVista (testmini) | 51.6 | 12.3 | +39.3 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| gpt-oss-120b OpenAI | SWE-bench Verified | 62.4 | 31.0 | +31.4 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ServiceNow-AI/Apriel-1.6-15b-Thinker model card |
| Qwen3.5 397B-A17B Alibaba Cloud | BrowseComp | 69.0 | 40.5 | +28.5 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Molmo 7B-D (0924) Ai2 | VQAv2 (val) | 85.6 | 57.1 | +28.5 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith) | Terminal-Bench 2.1 | 64.2 | 36.0 | +28.2 | ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| OPT-30B Meta | OpenBookQA | 57.2 | 29.2 | +28.0 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy — authors' own runs |
| DeepSeek R1 0528 DeepSeek | SWE-bench Verified | 57.6 | 29.6 | +28.0 | deepseek-ai/DeepSeek-R1-0528 model card · ServiceNow-AI/Apriel-1.6-15b-Thinker model card |
| LLaMA 7B Meta | OpenBookQA | 57.2 | 31.4 | +25.8 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Ministral 3 14B Reasoning 2512 Mistral AI | GPQA Diamond | 71.2 | 46.0 | +25.2 | mistralai/Ministral-3-14B-Reasoning-2512 model card · JetBrains/Mellum2-12B-A2.5B-Thinking model card |
| SmolLM3 3B Hugging Face | GSM-Plus | 83.4 | 58.9 | +24.5 | HuggingFaceTB/SmolLM3-3B model card (Evaluation section) · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation |
| OPT-1.3B Meta | SuperGLUE WSC (prompted) | 62.2 | 38.5 | +23.7 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| OPT-1.3B Meta | OpenBookQA | 46.8 | 23.4 | +23.4 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs |
| Magistral Small 1.2 Mistral AI | GPQA Diamond | 70.1 | 47.6 | +22.5 | mistralai/Magistral-Small-2509 model card · Epoch AI — internal evaluations |
| Nemotron 3 Nano 4B NVIDIA | AIME 2025 | 78.5 | 56.3 | +22.2 | nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 model card · openbmb/MiniCPM5-2B model card |
| Qwen2-VL 72B Instruct Alibaba Cloud | ChartQA | 88.3 | 66.6 | +21.7 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| LLaMA 33B Meta | OpenBookQA | 58.6 | 37.5 | +21.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Molmo 7B-D (0924) Ai2 | MMMU (val) | 45.3 | 24.3 | +21.0 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Gemma 3 1B Google | GSM8K | 62.8 | 42.2 | +20.6 | Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| LLaMA 65B Meta | ARC Easy | 78.9 | 58.8 | +20.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| gpt-oss-120b OpenAI | AIME 2024 | 95.8 | 75.8 | +20.0 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| DeepSeek V3.1 DeepSeek | AIME 2025 | 88.4 | 68.6 | +19.8 | deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Qwen3 Coder 30B-A3B Instruct Alibaba Cloud | SWE-bench Verified | 51.6 | 31.8 | +19.8 | Qwen/Qwen3-Coder-30B-A3B-Instruct model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith) | SWE-bench Verified | 75.6 | 55.8 | +19.8 | ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| GLM-4.7-Flash Z.ai | SWE-bench Verified | 59.2 | 39.7 | +19.5 | zai-org/GLM-4.7-Flash model card · SERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3) |
| Qwen3.5 2B Alibaba Cloud | MMLU-Pro | 66.5 | 47.1 | +19.4 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| LLaMA 13B Meta | OpenBookQA | 56.4 | 37.2 | +19.2 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith) | SWE-bench Multilingual | 69.3 | 51.7 | +17.6 | ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| Gemma 3 1B Google | IFEval (aggregation unspecified) | 80.2 | 62.9 | +17.3 | Google DeepMind Gemma 3 270M IT model card · LiquidAI/LFM2-1.2B model card |
| LFM2.5 2.6B Liquid AI | LiveCodeBench v6 (window unspecified) | 59.4 | 42.1 | +17.3 | LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card |
| Mistral 7B v0.1 (base) Mistral AI | GSM8K | 51.1 | 34.3 | +16.8 | Mistral 7B — authors' own runs · Gemma report — original evaluations |
| Claude Sonnet 4.5 Anthropic | τ²-bench Retail | 86.2 | 69.8 | +16.4 | τ-bench leaderboard: Anthropic's own submissions · ServiceNow-AI/Apriel-1.6-15b-Thinker model card |
| Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith) | SWE-bench Pro (split unspecified) | 50.4 | 34.5 | +15.9 | ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| SmolLM3 3B Hugging Face | GPQA Diamond | 41.7 | 26.3 | +15.4 | HuggingFaceTB/SmolLM3-3B model card (Evaluation section) · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation |
| StarCoder2 3B BigCode | MBPP | 57.4 | 42.4 | +15.0 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| StarCoder2 15B BigCode | MBPP | 66.2 | 51.2 | +15.0 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| gpt-oss-120b OpenAI | AIME 2025 | 92.5 | 77.7 | +14.8 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| LLaMA 33B Meta | BoolQ | 83.1 | 68.4 | +14.7 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| GPT-5.1 OpenAI | τ²-bench Telecom | 95.6 | 80.9 | +14.7 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Ministral 8B (2410) Mistral AI | MATH (subset unspecified) | 54.5 | 40.0 | +14.5 | Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards) · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| DeepSeek R1 Distill Qwen 14B DeepSeek | GPQA Diamond | 59.1 | 44.7 | +14.4 | deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations |
| DeepSeek R1 0528 DeepSeek | AIME 2025 | 87.5 | 73.4 | +14.1 | deepseek-ai/DeepSeek-R1-0528 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Qwen3 Coder 30B-A3B Instruct Alibaba Cloud | SWE-bench Multilingual | 34.7 | 20.7 | +14.0 | Qwen/Qwen3-Coder-30B-A3B-Instruct model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | τ²-bench Airline | 50.0 | 36.0 | +14.0 | Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Qwen3.5 2B Alibaba Cloud | LongBench v2 | 38.7 | 24.9 | +13.8 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| Qwen3-VL 32B Thinking Alibaba Cloud | τ²-bench Telecom | 46.9 | 33.3 | +13.6 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Llama 3.1 Tulu 3 8B Ai2 | BIG-Bench Hard | 69.0 | 55.5 | +13.5 | 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card |
| StarCoderBase 3B BigCode | MBPP | 42.6 | 29.4 | +13.2 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| StarCoderBase 15.5B BigCode | MBPP | 50.6 | 37.4 | +13.2 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| CodeGemma 2B (v1.0) Google | MBPP | 43.6 | 30.4 | +13.2 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| StarCoder2 7B BigCode | GSM8K | 40.4 | 27.2 | +13.2 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| Nemotron Nano 9B v2 NVIDIA | AIME 2025 | 72.1 | 58.9 | +13.2 | nvidia/NVIDIA-Nemotron-Nano-9B-v2 model card · Olmo 3 |
| Ministral 3 3B Mistral AI | MATH (subset unspecified) | 83.0 | 70.2 | +12.8 | mistralai/Ministral-3-3B-Instruct-2512 model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| StarCoderBase 7B BigCode | MBPP | 47.4 | 34.8 | +12.6 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| Qwen3.5 35B-A3B Alibaba Cloud | SWE-bench Multilingual | 60.3 | 47.7 | +12.6 | Qwen/Qwen3.6-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| Magistral Small 1.0 (2506) Mistral AI | GPQA Diamond | 68.2 | 56.1 | +12.1 | mistralai/Magistral-Small-2509 model card · Epoch AI — internal evaluations |
| Claude Sonnet 4.5 Anthropic | τ²-bench Airline | 70.0 | 58.0 | +12.0 | τ-bench leaderboard: Anthropic's own submissions · ServiceNow-AI/Apriel-1.6-15b-Thinker model card |
| Qwen3 4B Alibaba Cloud | MMLU-Pro | 64.2 | 52.3 | +11.9 | Qwen/Qwen3-4B-Thinking-2507 model card · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation |
| Molmo 72B (0924) Ai2 | ChartQA | 87.3 | 75.6 | +11.7 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Qwen3.6 27B Alibaba Cloud | OmniDocBench 1.5 overall (higher is better) | 89.4 | 77.8 | +11.6 | Qwen/Qwen3.8-27B model card · meta-models/Muse-Glimmer-30B model card (speed and benchmark tables) |
| Qwen3.5 27B Alibaba Cloud | SWE-bench Multilingual | 69.3 | 57.7 | +11.6 | Qwen/Qwen3.6-27B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| EXAONE 4.5 33B LG AI Research | MMLU-Pro | 83.3 | 72.0 | +11.3 | LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · SqueezeBits — original EXAONE 4.5 BF16 evaluations |
| Molmo 72B (0924) Ai2 | VQAv2 (val) | 86.5 | 75.2 | +11.3 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| StarCoderBase 7B BigCode | GSM8K | 14.9 | 3.8 | +11.1 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| Qwen3 8B Alibaba Cloud | MMLU-Pro | 74.6 | 63.5 | +11.1 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Qwen3.5 2B Alibaba Cloud | CountBench | 91.4 | 80.5 | +10.9 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Granite 4.2 3B IBM | LiveCodeBench v6 (window unspecified) | 69.7 | 58.9 | +10.8 | ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card |
| DeepSeek R1 0528 DeepSeek | LiveCodeBench v6 (Aug 2024–May 2025) | 73.3 | 62.5 | +10.8 | deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Qwen3-VL 2B Instruct Alibaba Cloud | OCRBench | 85.8 | 75.1 | +10.7 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Qwen3.5 35B-A3B Alibaba Cloud | SWE-bench Verified | 69.2 | 58.6 | +10.6 | Qwen/Qwen3.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| Qwen3 1.7B Alibaba Cloud | MMLU-Pro | 56.5 | 45.9 | +10.6 | Qwen/Qwen3.5-2B model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| OPT-1.3B Meta | ARC Challenge | 33.9 | 23.3 | +10.6 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| Qwen3 1.7B Alibaba Cloud | MMMLU | 57.0 | 46.5 | +10.5 | Qwen/Qwen3.5-2B model card · LiquidAI/LFM2-1.2B model card |
| Qwen3.5 2B Alibaba Cloud | MMStar | 71.7 | 61.4 | +10.3 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| gpt-oss-20b OpenAI | GPQA Diamond | 71.5 | 61.2 | +10.3 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Epoch AI — internal evaluations |
| Qwen3.6 35B-A3B Alibaba Cloud | SWE-bench Multilingual | 67.2 | 57.0 | +10.2 | Qwen/Qwen3.6-27B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| GPT-Neo 1.3B EleutherAI | HellaSwag | 48.9 | 38.7 | +10.2 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs |
| LFM2.5 2.6B Liquid AI | AIME 2025 | 51.9 | 41.9 | +10.0 | LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card |
| Gemma 4 E2B Google | MMLU-Pro | 60.0 | 50.1 | +9.9 | Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card |
| Nemotron Nano 9B v2 NVIDIA | AIME 2024 | 81.9 | 72.1 | +9.8 | NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models · Olmo 3 |
| Qwen3-VL 2B Instruct Alibaba Cloud | ChartQA | 79.1 | 69.3 | +9.8 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| DeepSeek R1 0528 DeepSeek | MMLU-Pro | 85.0 | 75.3 | +9.7 | deepseek-ai/DeepSeek-R1-0528 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| LLaMA 13B Meta | BoolQ | 78.0 | 68.5 | +9.5 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | τ-bench Retail | 67.8 | 58.7 | +9.1 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Gemma 4 26B-A4B Google | GPQA Diamond | 82.3 | 73.2 | +9.1 | Gemma Team — Gemma 4 Technical Report · Epoch AI — internal evaluations |
| Qwen3.6 35B-A3B Alibaba Cloud | SWE-bench Verified | 73.4 | 64.4 | +9.0 | Qwen/Qwen3.6-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| StarCoder2 7B BigCode | MBPP | 54.4 | 45.4 | +9.0 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| Devstral Small 2 24B Mistral AI | SWE-bench Verified | 68.0 | 59.1 | +8.9 | mistralai/Devstral-Small-2-24B-Instruct-2512 model card · SERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3) |
| Gemma 4 31B Google | Terminal-Bench Hard | 36.0 | 27.1 | +8.9 | Gemma Team — Gemma 4 Technical Report · NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party models |
| LLaMA 13B Meta | ARC Easy | 76.1 | 67.2 | +8.9 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Gemma 3 4B Google | MMLU-Pro | 43.6 | 34.8 | +8.8 | Google DeepMind Gemma 3 270M IT model card · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation |
| LLaMA 33B Meta | Natural Questions (closed book) | 32.9 | 24.1 | +8.8 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| Claude Sonnet 5.5 Anthropic | Terminal-Bench 4.0 | 70.6 | 61.8 | +8.8 | Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions |
| GLM-5.1 Z.ai | BrowseComp | 68.0 | 59.4 | +8.6 | zai-org/GLM-5.1 model card · NVIDIA — Nemotron 3 Ultra technical report |
| LFM2.5 8B-A1B Liquid AI | Multi-IF (aggregation unspecified) | 79.9 | 71.4 | +8.5 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| Gemma 4 31B Google | GPQA Diamond | 84.3 | 75.8 | +8.5 | Gemma Team — Gemma 4 Technical Report · Epoch AI — internal evaluations |
| OPT-30B Meta | ARC Challenge | 43.1 | 34.6 | +8.5 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| LLaMA 33B Meta | ARC Challenge | 57.8 | 49.5 | +8.3 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| DeepSeek R1 0528 DeepSeek | AIME 2024 | 91.4 | 83.2 | +8.2 | deepseek-ai/DeepSeek-V3.1 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| InternVL2-4B OpenGVLab | TextVQA | 74.4 | 66.2 | +8.2 | OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table |
| Qwen3.5 2B Alibaba Cloud | AI2D test | 83.3 | 75.2 | +8.1 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| EXAONE 4.0 32B LG AI Research | τ²-bench Retail | 67.5 | 59.5 | +8.0 | LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Nemotron Nano 9B v2 NVIDIA | GPQA (subset unspecified) | 64.0 | 56.2 | +7.8 | nvidia/NVIDIA-Nemotron-Nano-9B-v2 model card · Olmo 3 |
| Gemma 3 1B Google | MATH (subset unspecified) | 48.0 | 40.3 | +7.7 | Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| Qwen3-VL 2B Instruct Alibaba Cloud | MMStar | 58.3 | 50.6 | +7.7 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| GLM-4.5-Air Z.ai | MMLU-Pro | 81.4 | 73.9 | +7.5 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Qwen3.5 2B Alibaba Cloud | MMBench-EN dev v1.1 | 83.3 | 76.0 | +7.3 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Gemma 3 27B Google | MATH (subset unspecified) | 89.0 | 81.8 | +7.2 | Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| Gemini 2.5 Flash Google | CharXiv (reasoning questions) | 63.7 | 56.5 | +7.2 | Google DeepMind Gemini model page and model cards (lab-reported scores) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card |
| GPT-Neo 2.7B EleutherAI | HellaSwag | 55.8 | 48.6 | +7.2 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs |
| gpt-oss-120b OpenAI | GPQA Diamond | 80.1 | 72.9 | +7.2 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Epoch AI — internal evaluations |
| DeepSeek R1 Distill Llama 8B DeepSeek | AIME 2024 | 50.4 | 43.3 | +7.1 | deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card |
| Qwen3.5 2B Alibaba Cloud | GPQA (subset unspecified) | 51.6 | 44.6 | +7.0 | Qwen/Qwen3.5-2B model card · XHToken/Spark-X2.5-4B model card |
| Qwen3.5 9B Alibaba Cloud | BFCL V4 (December 2025 archive) | 66.1 | 59.3 | +6.8 | Qwen/Qwen3.5-9B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card |
| Phi-4-reasoning-plus 14B Microsoft | AIME 2025 | 78.0 | 71.2 | +6.8 | microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Pythia 1B EleutherAI | ARC Easy | 56.9 | 50.2 | +6.7 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Pythia 12B EleutherAI | ARC Easy | 70.2 | 63.7 | +6.5 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| LLaMA 7B Meta | ARC Easy | 74.1 | 67.6 | +6.5 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Molmo 72B (0924) Ai2 | MathVista (testmini) | 58.6 | 52.2 | +6.4 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| EXAONE 4.0 32B LG AI Research | τ²-bench Airline | 52.0 | 45.6 | +6.4 | LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Qwen3 1.7B Alibaba Cloud | GPQA (subset unspecified) | 34.0 | 27.7 | +6.3 | Qwen/Qwen3.5-2B model card · LiquidAI/LFM2-1.2B model card |
| Qwen3-VL 2B Instruct Alibaba Cloud | MMLU-Pro | 49.0 | 42.8 | +6.2 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Mistral 7B v0.1 (base) Mistral AI | MBPP | 48.9 | 42.7 | +6.2 | Mistral 7B — authors' own runs · Gemma report — original evaluations |
| StarCoderBase 7B BigCode | HumanEval | 30.5 | 24.4 | +6.1 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · EvalPlus / EvalPerf published evaluation runs |
| Qwen2-VL 7B Instruct Alibaba Cloud | MMMU (val) | 54.1 | 48.1 | +6.0 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Qwen3.5 2B Alibaba Cloud | IFEval (aggregation unspecified) | 78.6 | 72.7 | +5.9 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| WizardCoder Python 34B V1.0 WizardLM | HumanEval | 73.2 | 67.4 | +5.8 | WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card) · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | LiveCodeBench v6 (Feb–May 2025) | 66.0 | 60.3 | +5.7 | Qwen/Qwen3-30B-A3B-Thinking-2507 model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| GLM-5 Z.ai | SWE-bench Verified | 77.8 | 72.1 | +5.7 | zai-org/GLM-5 model card · Epoch AI — internal evaluations |
| Qwen3 32B Alibaba Cloud | τ-bench Retail | 46.5 | 40.9 | +5.6 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| LFM2.5 8B-A1B Liquid AI | IFBench (aggregation unspecified) | 56.5 | 51.0 | +5.5 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| Pythia 6.9B EleutherAI | ARC Easy | 67.0 | 61.5 | +5.5 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Gemma 4 E4B Google | GPQA Diamond | 58.6 | 53.1 | +5.5 | Gemma Team — Gemma 4 Technical Report · JetBrains/Mellum2.1-12B-A2.5B-Thinking model card |
| Gemma 2B (base) Google | TriviaQA | 53.2 | 47.8 | +5.4 | Gemma report — original evaluations · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results) |
| Granite 4.0 H 1B IBM | MMLU-Pro | 32.9 | 27.6 | +5.3 | ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| Qwen3.5 2B Alibaba Cloud | RealWorldQA | 74.5 | 69.3 | +5.2 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Qwen3.5 27B Alibaba Cloud | SWE-bench Verified | 73.7 | 68.6 | +5.1 | Qwen/Qwen3.5-27B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card |
| InternVL2-4B OpenGVLab | MathVista (testmini) | 58.6 | 53.7 | +4.9 | OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table |
| GLM-4.5-Air Z.ai | AIME 2024 | 89.4 | 84.6 | +4.8 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Gemma 3 27B Google | GSM8K | 95.9 | 91.1 | +4.8 | Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| Gemma 3 12B Google | MMLU-Pro | 60.6 | 55.8 | +4.8 | Google DeepMind Gemma 3 270M IT model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| GPT-J 6B EleutherAI | ARC Easy | 67.0 | 62.2 | +4.8 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Gemma 3 12B Google | GSM8K | 94.4 | 89.7 | +4.7 | Google DeepMind Gemma 3 270M IT model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| LLaMA 33B Meta | ARC Easy | 80.0 | 75.3 | +4.7 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| DeepSeek R1 Distill Qwen 32B DeepSeek | MATH-500 | 94.3 | 89.7 | +4.6 | deepseek-ai/DeepSeek-R1 model card · open-thoughts/OpenThinker2-32B model card |
| Qwen2-VL 7B Instruct Alibaba Cloud | MathVista (testmini) | 58.2 | 53.7 | +4.5 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | τ²-bench Telecom | 26.3 | 21.9 | +4.4 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · LiquidAI/LFM2.5-8B-A1B model card |
| Gemma 3 4B Google | GPQA Diamond | 30.8 | 26.4 | +4.4 | Google DeepMind Gemma 3 270M IT model card · Epoch AI — internal evaluations |
| CodeGemma 2B (v1.0) Google | HumanEval | 31.1 | 26.8 | +4.3 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · EvalPlus / EvalPerf published evaluation runs |
| gpt-oss-120b OpenAI | Humanity's Last Exam | 14.9 | 10.6 | +4.3 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Qwen2 7B Instruct Alibaba Cloud | MATH (subset unspecified) | 52.9 | 48.6 | +4.3 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · InternLM GitHub repository model cards (Shanghai AI Laboratory) |
| gpt-oss-20b OpenAI | AIME 2025 | 91.7 | 87.4 | +4.3 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| DeepSeek R1 Distill Llama 70B DeepSeek | GPQA Diamond | 65.2 | 61.0 | +4.2 | deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations |
| GLM-5.1 Z.ai | Terminal-Bench 2.1 | 63.5 | 59.3 | +4.2 | zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen2-VL 72B Instruct Alibaba Cloud | MMMU (val) | 64.5 | 60.3 | +4.2 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Mistral Small 3.1 24B (2503) Mistral AI | GPQA Diamond | 46.0 | 41.9 | +4.1 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 model card · Epoch AI — internal evaluations |
| DeepSeek R1 0528 DeepSeek | GPQA Diamond | 81.0 | 76.9 | +4.1 | deepseek-ai/DeepSeek-R1-0528 model card · Epoch AI — internal evaluations |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | τ²-bench Retail | 74.6 | 70.5 | +4.1 | Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Phi-4-reasoning-plus 14B Microsoft | AIME 2024 | 81.3 | 77.2 | +4.1 | microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| gpt-oss-20b OpenAI | AIME 2024 | 92.1 | 88.0 | +4.1 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Qwen3.5 9B Alibaba Cloud | IFBench (aggregation unspecified) | 64.5 | 60.5 | +4.0 | Qwen/Qwen3.5-9B model card · XHToken/Spark-X2.5-4B model card |
| OPT-1.3B Meta | Story Cloze Test | 74.7 | 70.8 | +3.9 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| Gemma 3 27B Google | BIG-Bench Hard | 87.6 | 83.7 | +3.9 | Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| Nemotron 3 Nano 4B NVIDIA | MATH-500 | 95.4 | 91.6 | +3.8 | nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 model card · openbmb/MiniCPM5-2B model card |
| DeepSeek V3.1 DeepSeek | HMMT 2025 (edition unspecified) | 84.2 | 80.4 | +3.8 | deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | AIME 2025 | 85.0 | 81.3 | +3.7 | Qwen/Qwen3-30B-A3B-Thinking-2507 model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | τ²-bench Retail | 71.9 | 68.2 | +3.7 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Qwen3.5 397B-A17B Alibaba Cloud | MultiChallenge | 67.6 | 63.9 | +3.7 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| InternVL2-4B OpenGVLab | MMMU (val) | 47.9 | 44.2 | +3.7 | OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table |
| Inkling (975B-A41B) Thinking Machines Lab | SimpleQA Verified | 43.9 | 40.3 | +3.6 | thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations |
| StarCoderBase 3B BigCode | HumanEval | 21.3 | 17.7 | +3.6 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · EvalPlus / EvalPerf published evaluation runs |
| Gemma 3 12B Google | GPQA Diamond | 40.9 | 37.3 | +3.6 | Google DeepMind Gemma 3 270M IT model card · Epoch AI — internal evaluations |
| GPT-NeoX-20B EleutherAI | ARC Easy | 72.3 | 68.7 | +3.6 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| gpt-oss-20b OpenAI | MMLU | 85.3 | 81.7 | +3.6 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Olmo 3 7B Think Ai2 | IFEval (aggregation unspecified) | 88.2 | 84.7 | +3.5 | Olmo 3 · JetBrains/Mellum2-12B-A2.5B-Thinking model card |
| SmolLM2 1.7B Instruct Hugging Face | GSM8K | 48.8 | 45.3 | +3.5 | SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| GLM-4.5 Z.ai | AIME 2024 | 91.0 | 87.6 | +3.4 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Claude 1.3 Anthropic | GSM8K | 85.2 | 81.8 | +3.4 | Anthropic system cards and model announcements (lab-reported scores) · Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance — authors' own runs |
| GLM-5.1 Z.ai | Humanity's Last Exam | 31.0 | 27.6 | +3.4 | zai-org/GLM-5.1 model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | τ²-bench Telecom | 32.5 | 29.1 | +3.4 | Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Flan-T5 XXL Google | BIG-Bench Hard | 45.3 | 41.9 | +3.4 | Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| LLaMA 7B Meta | ARC Challenge | 44.8 | 41.4 | +3.4 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Granite 4.0 H Small (32B-A9B) IBM | GSM8K | 87.3 | 83.9 | +3.4 | ibm-granite/granite-4.0-h-small model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| Qwen3.5 397B-A17B Alibaba Cloud | HMMT February 2026 | 87.9 | 84.6 | +3.3 | Qwen/Qwen3.6-27B model card · LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card |
| LLaMA 13B Meta | GSM8K | 17.8 | 14.5 | +3.3 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs |
| Llama 3.1 Tulu 3 8B Ai2 | GSM8K | 87.6 | 84.3 | +3.3 | 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card |
| Qwen3 1.7B Alibaba Cloud | AIME 2025 | 36.8 | 33.5 | +3.3 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · LiquidAI/LFM2.5-1.2B-Thinking model card |
| LFM2.5 2.6B Liquid AI | Multi-IF (aggregation unspecified) | 80.1 | 76.8 | +3.3 | LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card |
| DeepSeek V3.1 DeepSeek | MMLU-Redux | 93.7 | 90.5 | +3.2 | deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Llama 3.1 Tulu 3 8B Ai2 | MMLU | 68.2 | 65.0 | +3.2 | 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card |
| Flan-T5 Large Google | MMLU | 45.1 | 41.9 | +3.2 | Scaling Instruction-Finetuned Language Models — authors' own runs · Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runs |
| CodeGemma 7B (v1.0) Google | MBPP | 56.2 | 53.0 | +3.2 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| LLaMA 33B Meta | WinoGrande | 76.0 | 72.8 | +3.2 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| OPT-30B Meta | Story Cloze Test | 80.3 | 77.1 | +3.2 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| OPT-175B Meta | Story Cloze Test | 82.9 | 79.7 | +3.2 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| LLaMA 7B Meta | Social IQA | 48.9 | 45.7 | +3.2 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs |
| Qwen3 1.7B Alibaba Cloud | IFBench (aggregation unspecified) | 26.7 | 23.6 | +3.1 | Qwen/Qwen3.5-2B model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| GLM-4.5 Z.ai | MMLU-Pro | 84.6 | 81.5 | +3.1 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| LLaMA 7B Meta | WinoGrande | 70.0 | 66.9 | +3.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| WizardCoder 33B V1.1 WizardLM | HumanEval | 79.9 | 76.9 | +3.0 | WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card) · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Granite 4.0 1B IBM | GSM8K | 76.4 | 73.4 | +3.0 | ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| LLaMA 13B Meta | ARC Challenge | 49.6 | 46.7 | +2.9 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloud | BFCL v3 | 70.8 | 67.9 | +2.9 | Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Berkeley Function Calling Leaderboard result archive |
| Claude Opus 4.6 Anthropic | GPQA Diamond | 91.3 | 88.4 | +2.9 | Anthropic system cards and model announcements (lab-reported scores) · Epoch AI — internal evaluations |
| Nemotron Nano 9B v2 NVIDIA | IFBench (aggregation unspecified) | 37.4 | 34.6 | +2.8 | NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models · Olmo 3 |
| Qwen2 7B Instruct Alibaba Cloud | GSM8K | 85.7 | 82.9 | +2.8 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · InternLM GitHub repository model cards (Shanghai AI Laboratory) |
| Qwen3-VL 2B Instruct Alibaba Cloud | CharXiv descriptive questions | 62.3 | 59.5 | +2.8 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| GPT-5 mini OpenAI | τ²-bench Telecom | 74.1 | 71.3 | +2.8 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · SEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages (arXiv:2606.28715v2) |
| Qwen3.5 397B-A17B Alibaba Cloud | SWE-bench Verified | 76.3 | 73.6 | +2.7 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | GPQA Diamond | 73.4 | 70.7 | +2.7 | Qwen/Qwen3.5-4B model card · Epoch AI — internal evaluations |
| DeepSeek R1 0528 Qwen3 8B DeepSeek | AIME 2024 | 86.0 | 83.3 | +2.7 | deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Qwen3.5 4B Alibaba Cloud | IFEval (aggregation unspecified) | 89.8 | 87.1 | +2.7 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| Qwen3.5 4B Alibaba Cloud | LongBench v2 | 50.0 | 47.3 | +2.7 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| InternVL2-4B OpenGVLab | ChartQA | 81.5 | 78.8 | +2.7 | OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table |
| LLaMA 13B Meta | WinoGrande | 72.8 | 70.1 | +2.7 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| GPT-4o mini (2024-07-18) OpenAI | CharXiv (reasoning questions) | 36.8 | 34.1 | +2.7 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · CharXiv leaderboard (princeton-nlp/CharXiv README) |
| Qwen3 4B Instruct 2507 Alibaba Cloud | MMLU-Pro | 69.6 | 66.9 | +2.7 | Qwen/Qwen3-4B-Instruct-2507 model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| DeepSeek V3 (Dec '24) DeepSeek | GPQA Diamond | 59.1 | 56.5 | +2.6 | deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations |
| OPT-30B Meta | SuperGLUE WSC (prompted) | 62.2 | 59.6 | +2.6 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Phi-1.5 (1.3B) Microsoft | ARC Easy | 75.6 | 73.1 | +2.5 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| GPT-5.6 Sol OpenAI | FrontierMath tier 4 v2 | 83.0 | 80.5 | +2.5 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| gpt-oss-120b OpenAI | Aider polyglot (225 exercises, two attempts) | 44.4 | 41.9 | +2.5 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Aider polyglot benchmark runs |
| GLM-4.7 Z.ai | GPQA Diamond | 85.7 | 83.3 | +2.4 | GLM-5 Team (Zhipu AI) — GLM-5 technical report · Epoch AI — internal evaluations |
| Granite 4.0 1B IBM | BFCL v3 | 54.8 | 52.4 | +2.4 | ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| Qwen3.7 Plus Alibaba Cloud | GPQA Diamond | 90.3 | 87.9 | +2.4 | Qwen/Qwen3.8-27B model card · Epoch AI — internal evaluations |
| Olmo 3 7B Instruct Ai2 | IFEval (aggregation unspecified) | 85.6 | 83.2 | +2.4 | Olmo 3 · JetBrains/Mellum2-12B-A2.5B-Instruct model card |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | MMLU-Redux | 93.8 | 91.4 | +2.4 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| LLaMA 7B Meta | MMLU | 35.1 | 32.8 | +2.3 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| Qwen3.5 9B Alibaba Cloud | GPQA Diamond | 81.7 | 79.4 | +2.3 | Qwen/Qwen3.5-9B model card · Epoch AI — internal evaluations |
| DeepSeek R1 Distill Qwen 7B DeepSeek | MATH-500 | 92.8 | 90.5 | +2.3 | deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card |
| Qwen2-VL 72B Instruct Alibaba Cloud | MathVista (testmini) | 70.5 | 68.2 | +2.3 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Kimi K2 Instruct (0711) Moonshot AI | τ²-bench Airline | 56.5 | 54.2 | +2.3 | τ²-bench leaderboard: Moonshot AI's own submission · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Qwen3.6 35B-A3B Alibaba Cloud | GPQA Diamond | 86.0 | 83.8 | +2.2 | Qwen/Qwen3.6-27B model card · Epoch AI — internal evaluations |
| Flan-T5 XL Google | MMLU | 52.4 | 50.2 | +2.2 | Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| DeepSeek R1 Distill Llama 8B DeepSeek | MATH-500 | 89.1 | 86.9 | +2.2 | deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | AIME 2025 | 70.3 | 68.3 | +2.0 | Qwen/Qwen3-235B-A22B-Instruct-2507 model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloud | τ²-bench Telecom | 23.3 | 21.3 | +2.0 | Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table) |
| Qwen3.5 35B-A3B Alibaba Cloud | IMO-AnswerBench | 76.8 | 74.8 | +2.0 | Qwen/Qwen3.6-35B-A3B model card · NVIDIA — Nemotron-Cascade 2 technical report |
| Granite 4.2 3B IBM | MMLU-Pro | 67.8 | 65.8 | +2.0 | ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card |
| GLM-4.5 Z.ai | MATH-500 | 98.2 | 96.2 | +2.0 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| BLOOM 7.1B BigScience | COPA | 69.0 | 67.0 | +2.0 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1) |
| OPT-30B Meta | COPA | 84.0 | 82.0 | +2.0 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| OPT-175B Meta | ARC Challenge | 44.1 | 42.1 | +2.0 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| GLM-5.1 Z.ai | Humanity's Last Exam with tools | 52.3 | 50.4 | +1.9 | zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3.6 27B Alibaba Cloud | GPQA Diamond | 87.8 | 85.9 | +1.9 | Qwen/Qwen3.6-27B model card · Epoch AI — internal evaluations |
| LLaMA 33B Meta | HellaSwag | 82.8 | 80.9 | +1.9 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| LLaMA 65B Meta | PIQA | 82.8 | 80.9 | +1.9 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| DeepSeek R1 0528 DeepSeek | Humanity's Last Exam | 17.7 | 15.9 | +1.8 | deepseek-ai/DeepSeek-R1-0528 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Mistral 7B v0.1 (base) Mistral AI | PIQA | 82.6 | 80.8 | +1.8 | Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Qwen3 8B Alibaba Cloud | BFCL v3 | 68.1 | 66.3 | +1.8 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Berkeley Function Calling Leaderboard result archive |
| DeepSeek R1 Distill Llama 8B DeepSeek | GPQA Diamond | 49.0 | 47.3 | +1.7 | deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card |
| Qwen3.5 9B Alibaba Cloud | IFEval (aggregation unspecified) | 91.5 | 89.8 | +1.7 | Qwen/Qwen3.5-9B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card |
| Granite 4.0 H Small (32B-A9B) IBM | MMLU | 78.4 | 76.7 | +1.7 | ibm-granite/granite-4.0-h-small model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| LLaMA 7B Meta | BoolQ | 75.8 | 74.1 | +1.7 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Claude 2.0 Anthropic | HumanEval | 71.2 | 69.5 | +1.7 | Anthropic system cards and model announcements (lab-reported scores) · EvalPlus / EvalPerf published evaluation runs |
| GPT-4o mini (2024-07-18) OpenAI | CharXiv descriptive questions | 76.6 | 74.9 | +1.7 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · CharXiv leaderboard (princeton-nlp/CharXiv README) |
| Pythia 1B EleutherAI | PIQA | 70.7 | 69.1 | +1.6 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| InternVL2-4B OpenGVLab | AI2D test | 78.9 | 77.3 | +1.6 | OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table |
| T0 11B BigScience | WinoGrande | 60.5 | 58.9 | +1.6 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation — authors' own runs |
| Inkling Small (266B-A12B) Thinking Machines Lab | SimpleQA Verified | 20.6 | 19.1 | +1.5 | thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations |
| Qwen3 8B Alibaba Cloud | AIME 2025 | 67.3 | 65.8 | +1.5 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Qwen3 1.7B Alibaba Cloud | BFCL v3 | 56.6 | 55.1 | +1.5 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Berkeley Function Calling Leaderboard result archive |
| OPT-30B Meta | WinoGrande | 69.7 | 68.2 | +1.5 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| Claude Opus 5.5 Anthropic | Terminal-Bench 4.0 | 66.4 | 64.9 | +1.5 | Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions |
| LLaMA 33B Meta | TriviaQA | 69.9 | 68.5 | +1.4 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| Claude Instant 1.2 Anthropic | HumanEval | 58.7 | 57.3 | +1.4 | Anthropic system cards and model announcements (lab-reported scores) · EvalPlus / EvalPerf published evaluation runs |
| Molmo 72B (0924) Ai2 | MMMU (val) | 54.1 | 52.7 | +1.4 | Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2) |
| Ministral 3 8B Mistral AI | MATH (subset unspecified) | 87.6 | 86.2 | +1.4 | mistralai/Ministral-3-8B-Instruct-2512 model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| Qwen3.5 397B-A17B Alibaba Cloud | GPQA (subset unspecified) | 88.4 | 87.1 | +1.3 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloud | τ²-bench Retail | 52.7 | 51.4 | +1.3 | Qwen/Qwen3-235B-A22B-Thinking-2507 model card · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Code Llama 34B Python Meta | HumanEval | 53.7 | 52.4 | +1.3 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Phi-1.5 (1.3B) Microsoft | BoolQ | 75.8 | 74.5 | +1.3 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| DeepSeek V3.1 DeepSeek | LiveCodeBench (version unspecified) | 74.8 | 73.5 | +1.3 | DeepSeek-V3.1-Terminus model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| LLaMA 65B Meta | ARC Challenge | 56.0 | 54.8 | +1.2 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| OPT-175B Meta | WinoGrande | 73.7 | 72.5 | +1.2 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| Phi-4-reasoning-plus 14B Microsoft | GPQA Diamond | 69.1 | 67.9 | +1.2 | microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| DeepSeek V4.1 Flash DeepSeek | GPQA Diamond | 90.9 | 89.8 | +1.1 | deepseek-ai/DeepSeek-V4.1-Flash model card · Epoch AI — internal evaluations |
| Gemma 4 E4B Google | MMLU-Pro | 69.4 | 68.3 | +1.1 | Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card |
| Qwen3.5 4B Alibaba Cloud | MMLU-Pro | 79.1 | 78.0 | +1.1 | Qwen/Qwen3.5-4B model card · openbmb/MiniCPM5-2B model card |
| DeepSeek R1 (Jan '25) DeepSeek | AIME 2024 | 79.8 | 78.7 | +1.1 | deepseek-ai/DeepSeek-R1-0528 model card · Microsoft — Phi-4-reasoning Technical Report |
| LLaMA 7B Meta | PIQA | 79.3 | 78.2 | +1.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Qwen3.5 397B-A17B Alibaba Cloud | LiveCodeBench v6 (Feb–May 2025) | 83.6 | 82.6 | +1.0 | Qwen/Qwen3.5-397B-A17B model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1 |
| Inkling Small (266B-A12B) Thinking Machines Lab | GPQA Diamond | 89.5 | 88.5 | +1.0 | thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations |
| LFM2.5 8B-A1B Liquid AI | IFEval (aggregation unspecified) | 91.8 | 90.8 | +1.0 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | τ-bench Airline | 48.0 | 47.0 | +1.0 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| GLM-4.5 Z.ai | AIME 2025 | 85.4 | 84.4 | +1.0 | zai-org/GLM-4.6 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| LLaMA 13B Meta | HumanEval | 15.8 | 14.8 | +1.0 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| GPT-4o mini (2024-07-18) OpenAI | HumanEval (simple-evals) | 87.2 | 86.2 | +1.0 | OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | IFEval (aggregation unspecified) | 88.9 | 88.0 | +0.9 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Phi-1.5 (1.3B) Microsoft | WinoGrande | 73.4 | 72.5 | +0.9 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| GPT-5 OpenAI | τ²-bench Telecom | 96.7 | 95.8 | +0.9 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs |
| GLM-5.1 Z.ai | CritPt | 4.6 | 3.7 | +0.9 | zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report |
| Flan-T5 Base Google | MMLU | 35.9 | 35.0 | +0.9 | Scaling Instruction-Finetuned Language Models — authors' own runs · Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runs |
| OPT-1.3B Meta | BoolQ | 60.5 | 59.6 | +0.9 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs |
| LLaMA 65B Meta | GSM8K | 50.9 | 50.0 | +0.9 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | GPQA Diamond | 81.1 | 80.2 | +0.9 | Qwen/Qwen3.5-27B model card · Epoch AI — internal evaluations |
| GLM-5.1 Z.ai | τ³-bench | 70.6 | 69.7 | +0.9 | zai-org/GLM-5.1 model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3.5 397B-A17B Alibaba Cloud | GPQA Diamond | 88.4 | 87.6 | +0.8 | Qwen/Qwen3.6-27B model card · Epoch AI — internal evaluations |
| DeepSeek V3 0324 DeepSeek | GPQA Diamond | 68.4 | 67.6 | +0.8 | deepseek-ai/DeepSeek-V3.1 model card · Epoch AI — internal evaluations |
| Qwen3.5 397B-A17B Alibaba Cloud | Humanity's Last Exam | 28.7 | 27.9 | +0.8 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | ZebraLogic | 95.0 | 94.2 | +0.8 | Qwen/Qwen3-235B-A22B-Instruct-2507 model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| GPT-Neo 2.7B EleutherAI | PIQA | 73.0 | 72.2 | +0.8 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs |
| GPT-J 6B EleutherAI | LAMBADA (OpenAI version), last-word accuracy | 69.0 | 68.2 | +0.8 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Mistral 7B v0.1 (base) Mistral AI | WinoGrande | 74.8 | 74.0 | +0.8 | Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Mistral 7B v0.1 (base) Mistral AI | ARC Easy | 80.3 | 79.5 | +0.8 | Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Flan-T5 XL Google | BIG-Bench Hard | 41.0 | 40.2 | +0.8 | Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| LLaMA 13B Meta | PIQA | 79.7 | 78.9 | +0.8 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| LLaMA 65B Meta | HellaSwag | 84.2 | 83.4 | +0.8 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| Qwen3.5 35B-A3B Alibaba Cloud | GPQA Diamond | 84.2 | 83.5 | +0.7 | Qwen/Qwen3.5-35B-A3B model card · Epoch AI — internal evaluations |
| DeepSeek V4 Pro (0813) DeepSeek | GPQA Diamond | 92.4 | 91.7 | +0.7 | deepseek-ai/DeepSeek-V4.1-Flash model card · Epoch AI — internal evaluations |
| DeepSeek R1 Distill Qwen 7B DeepSeek | GPQA Diamond | 49.1 | 48.4 | +0.7 | deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card |
| DeepSeek R1 Distill Llama 70B DeepSeek | AIME 2024 | 70.0 | 69.3 | +0.7 | deepseek-ai/DeepSeek-R1 model card · Microsoft — Phi-4-reasoning Technical Report |
| StarCoder2 15B BigCode | CRUXEval-I | 48.1 | 47.4 | +0.7 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| LLaMA 33B Meta | PIQA | 82.3 | 81.6 | +0.7 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| SmolLM2 1.7B Instruct Hugging Face | MATH (subset unspecified) | 21.0 | 20.3 | +0.7 | SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| Gemma 3 1B Google | MMLU-Pro | 14.7 | 14.0 | +0.7 | Google DeepMind Gemma 3 270M IT model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| GLM-4.5 Z.ai | IFEval prompt-level strict | 86.1 | 85.4 | +0.7 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Qwen2.5 Coder 32B Instruct Alibaba Cloud | HumanEval | 92.7 | 92.1 | +0.6 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs |
| StarCoder2 3B BigCode | CRUXEval-I | 32.7 | 32.1 | +0.6 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| GPT-Neo 2.7B EleutherAI | MathQA | 24.7 | 24.1 | +0.6 | EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs |
| StarCoder2 15B BigCode | HumanEval | 46.3 | 45.7 | +0.6 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| OPT-1.3B Meta | PIQA | 72.3 | 71.7 | +0.6 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| Code Llama 13B Python Meta | HumanEval | 43.3 | 42.7 | +0.6 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| DeepSeek R1 0528 Qwen3 8B DeepSeek | AIME 2025 | 76.3 | 75.8 | +0.5 | deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Qwen3.5 4B Alibaba Cloud | MMLU-Redux | 88.8 | 88.3 | +0.5 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| GPT-J 6B EleutherAI | WinoGrande | 64.7 | 64.2 | +0.5 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs |
| GPT-NeoX-20B EleutherAI | PIQA | 77.9 | 77.4 | +0.5 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Granite 4.0 1B IBM | MMLU-Pro | 34.0 | 33.5 | +0.5 | ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| Code Llama 13B (base) Meta | MBPP | 47.0 | 46.5 | +0.5 | Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs |
| GPT-4o mini (2024-07-18) OpenAI | MGSM (simple-evals) | 87.0 | 86.5 | +0.5 | OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report |
| Claude Opus 5 Anthropic | Terminal-Bench 4.0 | 52.3 | 51.8 | +0.5 | Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions |
| Qwen3.5 397B-A17B Alibaba Cloud | WMT24++ | 78.9 | 78.5 | +0.4 | Qwen/Qwen3.5-397B-A17B model card · Cohere — North Small Translate (arXiv:2609.13916v1) |
| Pythia 1B EleutherAI | WinoGrande | 53.7 | 53.3 | +0.4 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| GPT-4o mini (2024-07-18) OpenAI | DROP F1, 3-shot (simple-evals) | 79.7 | 79.3 | +0.4 | OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | OJBench | 32.5 | 32.1 | +0.4 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| StarCoder2 7B BigCode | CRUXEval-I | 34.6 | 34.2 | +0.4 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| StarCoder2 15B BigCode | CRUXEval-O | 47.1 | 46.7 | +0.4 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| DeepSeek V3.1 DeepSeek | MMLU-Pro | 84.8 | 84.4 | +0.4 | deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| GPT-J 6B EleutherAI | PIQA | 75.9 | 75.6 | +0.3 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs |
| StarCoder2 3B BigCode | HumanEval | 31.7 | 31.4 | +0.3 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| LLaMA 7B Meta | GSM8K | 11.0 | 10.7 | +0.3 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs |
| DeepSeek V3 (Dec '24) DeepSeek | IFEval prompt-level strict | 86.1 | 85.8 | +0.3 | deepseek-ai/DeepSeek-R1 model card · OpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file |
| GLM-4.5-Air Z.ai | MATH-500 | 98.1 | 97.8 | +0.3 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Pythia 6.9B EleutherAI | LAMBADA (OpenAI version), last-word accuracy | 67.3 | 67.0 | +0.3 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Phi-1.5 (1.3B) Microsoft | PIQA | 76.6 | 76.3 | +0.3 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Mistral 7B v0.1 (base) Mistral AI | HumanEval | 28.4 | 28.1 | +0.3 | Mistral 7B — authors' own runs · MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.) |
| StarCoder2 7B BigCode | HumanEval | 35.4 | 35.1 | +0.3 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| LLaMA 13B Meta | MMLU | 46.9 | 46.6 | +0.3 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| Code Llama 13B (base) Meta | CRUXEval-I | 42.5 | 42.2 | +0.3 | CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| DeepSeek R1 Distill Qwen 7B DeepSeek | AIME 2024 | 55.5 | 55.3 | +0.2 | deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | IFEval (aggregation unspecified) | 88.7 | 88.5 | +0.2 | Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| Qwen3.5 4B Alibaba Cloud | IFBench (aggregation unspecified) | 59.2 | 59.0 | +0.2 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| GLM-4.5-Air Z.ai | SWE-bench Verified | 57.6 | 57.4 | +0.2 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · SERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3) |
| StarCoder2 3B BigCode | CRUXEval-O | 34.2 | 34.0 | +0.2 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| StarCoder2 7B BigCode | CRUXEval-O | 36.0 | 35.8 | +0.2 | StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| LFM2.5 2.6B Liquid AI | IFBench (aggregation unspecified) | 59.2 | 59.0 | +0.2 | LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card |
| Granite 4.0 H 1B IBM | GSM8K | 69.8 | 69.6 | +0.2 | ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| Code Llama 7B (base) Meta | CRUXEval-O | 34.2 | 34.0 | +0.2 | CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| OPT-30B Meta | MMLU | 26.1 | 25.9 | +0.2 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy — authors' own runs |
| GPT-4o mini (2024-07-18) OpenAI | MMLU (simple-evals, zero-shot CoT) | 82.0 | 81.8 | +0.2 | OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report |
| Gemma 7B (base) Google | MATH | 24.3 | 24.1 | +0.2 | Gemma report — original evaluations · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| LLaMA 65B Meta | HumanEval | 23.7 | 23.5 | +0.2 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs |
| DeepSeek R1 Distill Qwen 1.5B DeepSeek | GPQA Diamond | 33.8 | 33.6 | +0.2 | deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations |
| Granite 4.2 3B IBM | BFCL V4 (December 2025 archive) | 52.4 | 52.2 | +0.2 | ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card |
| DeepSeek R1 0528 DeepSeek | Aider polyglot (225 exercises, two attempts) | 71.6 | 71.4 | +0.2 | deepseek-ai/DeepSeek-V3.1 model card · Aider polyglot benchmark runs |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | HMMT 2025 (edition unspecified) | 83.9 | 83.8 | +0.1 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Pythia 6.9B EleutherAI | PIQA | 75.2 | 75.1 | +0.1 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Mistral 7B v0.1 (base) Mistral AI | HellaSwag | 81.2 | 81.1 | +0.1 | Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Qwen3.5 4B Alibaba Cloud | SuperGPQA | 52.9 | 52.8 | +0.1 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| LLaMA 65B Meta | MMLU | 63.4 | 63.3 | +0.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs |
| Nemotron 3 Ultra 550B-A55B NVIDIA | Humanity's Last Exam | 26.7 | 26.6 | +0.1 | NVIDIA — Nemotron 3 Ultra technical report · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1 |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | MMLU-Pro | 84.4 | 84.4 | 0.0 | Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| DeepSeek V3 0324 DeepSeek | Aider polyglot (225 exercises, two attempts) | 55.1 | 55.1 | 0.0 | deepseek-ai/DeepSeek-V3.1 model card · Aider polyglot benchmark runs |
| Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloud | Aider polyglot (225 exercises, two attempts) | 59.6 | 59.6 | 0.0 | Qwen/Qwen3-30B-A3B-Instruct-2507 model card · Aider polyglot benchmark runs |
| Qwen3 32B Alibaba Cloud | Aider polyglot (225 exercises, two attempts) | 40.0 | 40.0 | 0.0 | Qwen/Qwen3-Next-80B-A3B-Instruct model card · Aider polyglot benchmark runs |
| Qwen3 4B Instruct 2507 Alibaba Cloud | τ²-bench Telecom | 13.2 | 13.2 | 0.0 | Qwen/Qwen3-4B-Instruct-2507 model card · Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2) |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | τ²-bench Airline | 58.0 | 58.0 | 0.0 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Qwen3.5 2B Alibaba Cloud | BFCL V4 (December 2025 archive) | 43.6 | 43.6 | 0.0 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| Qwen3.5 397B-A17B Alibaba Cloud | Humanity's Last Exam with tools | 48.3 | 48.3 | 0.0 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3.5 9B Alibaba Cloud | MMLU-Redux | 91.1 | 91.1 | 0.0 | Qwen/Qwen3.5-9B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card |
| GPT-Neo 1.3B EleutherAI | WinoGrande | 55.0 | 55.0 | 0.0 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · What Language Model to Train if You Have One Million GPU Hours? — authors' own runs |
| GPT-Neo 2.7B EleutherAI | LAMBADA (OpenAI version), last-word accuracy | 62.2 | 62.2 | 0.0 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · RedPajama — original zero-shot 3B comparison |
| GPT-Neo 1.3B EleutherAI | MathQA | 24.1 | 24.1 | 0.0 | EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs · What Language Model to Train if You Have One Million GPU Hours? — authors' own runs |
| GPT-NeoX-20B EleutherAI | LAMBADA (OpenAI version), last-word accuracy | 72.0 | 72.0 | 0.0 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| GPT-NeoX-20B EleutherAI | WinoGrande | 66.1 | 66.1 | 0.0 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Pythia 2.8B EleutherAI | LAMBADA (OpenAI version), last-word accuracy | 64.7 | 64.7 | 0.0 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · RedPajama — original zero-shot 3B comparison |
| Pythia 12B EleutherAI | SciQ | 90.2 | 90.2 | 0.0 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Magicoder-S-DS 6.7B ise-uiuc | HumanEval | 76.8 | 76.8 | 0.0 | Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs · EvalPlus / EvalPerf published evaluation runs |
| Nova Pro Amazon | BFCL v3 | 60.8 | 60.8 | 0.0 | Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports) · Berkeley Function Calling Leaderboard result archive |
| Qwen2.5 Coder 32B Instruct Alibaba Cloud | HumanEval+ | 87.2 | 87.2 | 0.0 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs |
| CodeQwen1.5 7B Chat Alibaba Cloud | HumanEval | 83.5 | 83.5 | 0.0 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs |
| CodeQwen1.5 7B Chat Alibaba Cloud | HumanEval+ | 78.7 | 78.7 | 0.0 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs |
| Command A+ (05-2026) Cohere | Terminal-Bench Hard | 25.0 | 25.0 | 0.0 | Cohere vendor-reported benchmark scores for its own models (cohere.com blog) · Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2) |
| Command R Plus (08-2024) Cohere | RULER 128k | 64.6 | 64.6 | 0.0 | Cohere — Command A technical report · NVIDIA RULER repository README results table |
| Gemma 3 12B Google | MATH (subset unspecified) | 83.8 | 83.8 | 0.0 | Google DeepMind Gemma 3 270M IT model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| Flan-T5 XXL Google | MMLU | 55.1 | 55.1 | 0.0 | Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| CodeGemma 7B (v1.0) Google | HumanEval | 44.5 | 44.5 | 0.0 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · EvalPlus / EvalPerf published evaluation runs |
| Code Llama 7B (base) Meta | HumanEval | 33.5 | 33.5 | 0.0 | Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs |
| gpt-oss-20b OpenAI | SimpleQA | 6.7 | 6.7 | 0.0 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| GPT-5.6 Sol OpenAI | Terminal-Bench 4.0 | 37.3 | 37.3 | 0.0 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Terminal-Bench 4.0 official leaderboard submissions |
| Seed-OSS 36B Instruct ByteDance Seed | GPQA Diamond | 71.4 | 71.5 | -0.1 | ByteDance-Seed/Seed-OSS-36B-Instruct model card · Epoch AI — internal evaluations |
| Qwen3.5 35B-A3B Alibaba Cloud | AIME 2026 | 91.0 | 91.1 | -0.1 | Qwen/Qwen3.6-35B-A3B model card · NVIDIA — Nemotron-Cascade 2 technical report |
| Pythia 12B EleutherAI | LAMBADA (OpenAI version), last-word accuracy | 70.5 | 70.6 | -0.1 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Pythia 2.8B EleutherAI | PIQA | 73.9 | 74.0 | -0.1 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · RedPajama — original zero-shot 3B comparison |
| OPT-30B Meta | PIQA | 77.5 | 77.6 | -0.1 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| LLaMA 65B Meta | WinoGrande | 77.0 | 77.1 | -0.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| GPT-5.6 Sol OpenAI | FrontierMath tiers 1–3 v2 | 89.0 | 89.1 | -0.1 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| Pythia 12B EleutherAI | WinoGrande | 63.9 | 64.0 | -0.1 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| LLaMA 33B Meta | MMLU | 57.8 | 57.9 | -0.1 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| Code Llama 34B (base) Meta | CRUXEval-O | 42.4 | 42.5 | -0.1 | CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| GLM-5.1 Z.ai | AIME 2026 | 95.3 | 95.4 | -0.1 | zai-org/GLM-5.1 model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1 |
| DeepSeek R1 (Jan '25) DeepSeek | IFEval prompt-level strict | 83.3 | 83.4 | -0.1 | deepseek-ai/DeepSeek-R1 model card · OpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file |
| Code Llama 13B (base) Meta | CRUXEval-O | 39.7 | 39.9 | -0.2 | CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| GPT-J 6B EleutherAI | OpenBookQA | 28.8 | 29.0 | -0.2 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| LLaMA 7B Meta | HumanEval | 10.5 | 10.7 | -0.2 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | AIME 2025 | 92.3 | 92.5 | -0.2 | Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Kimi K2 Instruct (0711) Moonshot AI | τ²-bench Retail | 70.6 | 70.8 | -0.2 | τ²-bench leaderboard: Moonshot AI's own submission · Meituan LongCat-Flash-Chat model card (evaluation table) |
| OPT-1.3B Meta | WinoGrande | 59.5 | 59.7 | -0.2 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| DeepSeek R1 0528 Qwen3 8B DeepSeek | GPQA Diamond | 61.1 | 61.4 | -0.3 | deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| GPT-NeoX-20B EleutherAI | SciQ | 92.8 | 93.1 | -0.3 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Pythia 12B EleutherAI | PIQA | 76.0 | 76.3 | -0.3 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Phi-3.5-vision-instruct Microsoft | ChartQA | 81.8 | 82.1 | -0.3 | Microsoft Phi-3.5-vision-instruct model card: Benchmarks table · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Gemma 3 1B Google | BIG-Bench Hard | 39.1 | 39.4 | -0.3 | Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| Gemma 3 4B Google | MMLU | 58.1 | 58.4 | -0.3 | Gemma Team — Gemma 3 Technical Report · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation |
| Code Llama 7B (base) Meta | CRUXEval-I | 35.9 | 36.2 | -0.3 | CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| DeepSeek V3 (Dec '24) DeepSeek | Aider polyglot (225 exercises, two attempts) | 49.6 | 50.0 | -0.4 | deepseek-ai/DeepSeek-R1 model card · Aider polyglot benchmark runs |
| Pythia 6.9B EleutherAI | WinoGrande | 60.9 | 61.3 | -0.4 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Phi-1.5 (1.3B) Microsoft | OpenBookQA | 37.2 | 37.6 | -0.4 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| LLaMA 33B Meta | GSM8K | 35.6 | 36.0 | -0.4 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| GLM-4.5 Z.ai | Humanity's Last Exam | 14.4 | 14.8 | -0.4 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| gpt-oss-20b OpenAI | Humanity's Last Exam | 10.9 | 11.3 | -0.4 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| DeepSeek R1 (Jan '25) DeepSeek | AIME 2025 | 70.0 | 70.4 | -0.4 | deepseek-ai/DeepSeek-R1-0528 model card · Microsoft — Phi-4-reasoning Technical Report |
| Qwen3.5 2B Alibaba Cloud | MMLU-Redux | 79.6 | 80.0 | -0.4 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| Qwen3.5 397B-A17B Alibaba Cloud | MMLU-Pro | 87.8 | 88.3 | -0.5 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | SuperGPQA | 56.8 | 57.3 | -0.5 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| GPT-J 6B EleutherAI | SciQ | 91.0 | 91.5 | -0.5 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Granite 4.0 H 1B IBM | BFCL v3 | 50.2 | 50.7 | -0.5 | ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card |
| GPT-5 OpenAI | GPQA Diamond | 85.7 | 86.2 | -0.5 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| Qwen3.5 4B Alibaba Cloud | GPQA Diamond | 76.2 | 76.8 | -0.6 | Qwen/Qwen3.5-4B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card |
| Qwen3 32B Alibaba Cloud | SuperGPQA | 48.7 | 49.3 | -0.6 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | MMMLU | 78.4 | 79.0 | -0.6 | Qwen/Qwen3.5-9B model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Code Llama 34B (base) Meta | CRUXEval-I | 47.2 | 47.8 | -0.6 | CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| Qwen3-VL 2B Instruct Alibaba Cloud | RealWorldQA | 63.9 | 64.6 | -0.7 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| GPT-4o mini (2024-07-18) OpenAI | GPQA Diamond (simple-evals) | 40.2 | 40.9 | -0.7 | OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report |
| Qwen3.5 4B Alibaba Cloud | OJBench | 24.1 | 24.8 | -0.7 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| GLM-5.2 Z.ai | GPQA Diamond | 91.2 | 91.9 | -0.7 | zai-org/GLM-5.2 model card · Epoch AI — internal evaluations |
| Pythia 2.8B EleutherAI | WinoGrande | 59.4 | 60.1 | -0.7 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · RedPajama — original zero-shot 3B comparison |
| LFM2.5 8B-A1B Liquid AI | BFCL V4 (December 2025 archive) | 48.5 | 49.2 | -0.7 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| Gemma 3 1B Google | MMLU | 38.8 | 39.5 | -0.7 | Gemma Team — Gemma 3 Technical Report · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| GPT-Neo 2.7B EleutherAI | PubMedQA | 57.5 | 58.3 | -0.8 | EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs |
| Pythia 6.9B EleutherAI | SciQ | 89.7 | 90.5 | -0.8 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| GPT-NeoX-20B EleutherAI | OpenBookQA | 29.0 | 29.8 | -0.8 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| DeepSeek V3.1 DeepSeek | AIME 2024 | 93.1 | 93.9 | -0.8 | deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| StarCoder 15.5B BigCode | HumanEval | 33.6 | 34.5 | -0.9 | StarCoder: may the source be with you! — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| DeepSeek R1 (Jan '25) DeepSeek | GPQA Diamond | 71.5 | 72.4 | -0.9 | deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations |
| Phi-3.5-vision-instruct Microsoft | AI2D test | 78.1 | 79.0 | -0.9 | Microsoft Phi-3.5-vision-instruct model card: Benchmarks table · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| o3 OpenAI | GPQA Diamond | 81.0 | 81.9 | -0.9 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | MMLU-Pro | 80.9 | 81.9 | -1.0 | Qwen/Qwen3-30B-A3B-Thinking-2507 model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Qwen3 32B Alibaba Cloud | τ²-bench Telecom | 26.3 | 27.3 | -1.0 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Qwen3.5 4B Alibaba Cloud | BFCL V4 (December 2025 archive) | 50.3 | 51.3 | -1.0 | Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card |
| Inkling (975B-A41B) Thinking Machines Lab | GPQA Diamond | 87.2 | 88.3 | -1.1 | thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations |
| DeepSeek V4 Flash (0731) DeepSeek | GPQA Diamond | 89.9 | 91.0 | -1.1 | deepseek-ai/DeepSeek-V4.1-Flash model card · Epoch AI — internal evaluations |
| GPT-5.6 Terra OpenAI | FrontierMath tiers 1–3 v2 | 84.9 | 86.0 | -1.1 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| Qwen3.5 2B Alibaba Cloud | SuperGPQA | 37.5 | 38.6 | -1.1 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| Granite 4.2 3B IBM | AIME 2025 | 78.3 | 79.4 | -1.1 | ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card |
| Qwen3 30B-A3B Thinking 2507 Alibaba Cloud | MMLU-ProX | 76.4 | 77.6 | -1.2 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · NVIDIA — Nemotron 3 Nano technical report |
| Qwen3.5 397B-A17B Alibaba Cloud | AIME 2026 | 91.3 | 92.5 | -1.2 | Qwen/Qwen3.5-397B-A17B model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1 |
| GPT-Neo 2.7B EleutherAI | WinoGrande | 56.5 | 57.7 | -1.2 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs |
| Jamba 1.5 Large AI21 Labs | MMLU-Pro | 48.3 | 49.5 | -1.2 | Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1) · MMLU-Pro author-released evaluation runs |
| Code Llama 7B Python Meta | HumanEval | 38.4 | 39.6 | -1.2 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Code Llama 34B (base) Meta | HumanEval | 48.8 | 50.0 | -1.2 | Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs |
| GLM-4.5 Z.ai | BFCL v3 | 77.8 | 79.1 | -1.3 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| LLaMA 33B Meta | MATH | 7.1 | 8.4 | -1.3 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| Gemma 7B (base) Google | HumanEval | 32.3 | 33.6 | -1.3 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Mistral 7B v0.1 (base) Mistral AI | MMLU | 61.3 | 62.7 | -1.4 | Mistral 7B — authors' own runs · MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.) |
| Gemma 2B (base) Google | HumanEval | 22.0 | 23.5 | -1.5 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Code Llama 70B Instruct Meta | HumanEval | 67.8 | 69.3 | -1.5 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| Qwen3 1.7B Alibaba Cloud | IFEval (aggregation unspecified) | 72.5 | 74.1 | -1.6 | Qwen/Qwen3.5-2B model card · HuggingFaceTB/SmolLM3-3B model card (Evaluation section) |
| Qwen3.5 2B Alibaba Cloud | OCRBench | 84.5 | 86.1 | -1.6 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| OPT-175B Meta | PIQA | 79.5 | 81.1 | -1.6 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| Granite 4.0 H Small (32B-A9B) IBM | MMLU-Pro | 55.5 | 57.1 | -1.6 | ibm-granite/granite-4.0-h-small model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| Qwen3.5 397B-A17B Alibaba Cloud | SWE-bench Multilingual | 69.3 | 70.9 | -1.6 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | τ²-bench Telecom | 45.6 | 47.3 | -1.7 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| Qwen3 4B Alibaba Cloud | BFCL v3 | 61.7 | 63.4 | -1.7 | Qwen/Qwen3-4B-Thinking-2507 model card · Berkeley Function Calling Leaderboard result archive |
| Qwen3.5 397B-A17B Alibaba Cloud | MMLU-ProX | 84.7 | 86.4 | -1.7 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3.6 27B Alibaba Cloud | IFBench (aggregation unspecified) | 69.1 | 70.8 | -1.7 | Qwen/Qwen3.8-27B model card · meta-models/Muse-Glimmer-30B model card (speed and benchmark tables) |
| BLOOM 560M BigScience | WinoGrande | 49.9 | 51.6 | -1.7 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| Nemotron Nano 9B v2 NVIDIA | MMLU | 82.6 | 84.3 | -1.7 | NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models · Olmo 3 |
| Kimi K2 Instruct (0711) Moonshot AI | τ²-bench Telecom | 65.8 | 67.5 | -1.7 | τ²-bench leaderboard: Moonshot AI's own submission · Meituan LongCat-Flash-Chat model card (evaluation table) |
| gpt-oss-20b OpenAI | MMMLU | 75.7 | 77.4 | -1.7 | OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Qwen3 235B-A22B Instruct 2507 Alibaba Cloud | MMLU-Pro | 83.0 | 84.8 | -1.8 | Qwen/Qwen3-235B-A22B-Instruct-2507 model card · Meituan LongCat-Flash-Chat model card (evaluation table) |
| GLM-5 Z.ai | GPQA Diamond | 86.0 | 87.8 | -1.8 | GLM-5 Team (Zhipu AI) — GLM-5 technical report · Epoch AI — internal evaluations |
| CodeQwen1.5 7B Chat Alibaba Cloud | MBPP+ | 67.2 | 69.0 | -1.8 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs |
| LLaMA 33B Meta | HumanEval | 21.7 | 23.5 | -1.8 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba Cloud | BFCL v3 | 63.9 | 65.7 | -1.8 | Qwen/Qwen3-4B-Thinking-2507 model card · Berkeley Function Calling Leaderboard result archive |
| Qwen2.5 Coder 32B Instruct Alibaba Cloud | MBPP+ | 75.1 | 77.0 | -1.9 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs |
| DeepSeek R1 Distill Qwen 32B DeepSeek | GPQA Diamond | 62.1 | 64.1 | -2.0 | deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations |
| Pythia 1B EleutherAI | SciQ | 84.0 | 86.0 | -2.0 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Qwen3 8B Alibaba Cloud | LiveCodeBench v6 (window unspecified) | 51.0 | 53.0 | -2.0 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Code Llama 34B (base) Meta | MBPP | 55.0 | 57.0 | -2.0 | Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs |
| OPT-1.3B Meta | COPA | 77.0 | 79.0 | -2.0 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs |
| GLM-5.1 Z.ai | GPQA Diamond | 86.2 | 88.3 | -2.1 | zai-org/GLM-5.1 model card · Epoch AI — internal evaluations |
| Claude Sonnet 5 Anthropic | Terminal-Bench 4.0 | 10.3 | 12.4 | -2.1 | Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions |
| Claude Fable 5.1 Anthropic | Terminal-Bench 4.0 | 55.8 | 57.9 | -2.1 | Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions |
| Qwen3.5 397B-A17B Alibaba Cloud | IMO-AnswerBench | 80.9 | 83.1 | -2.2 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| Qwen3.5 397B-A17B Alibaba Cloud | IFBench (aggregation unspecified) | 76.5 | 78.8 | -2.3 | Qwen/Qwen3.5-397B-A17B model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1 |
| Gemma 4 E2B Google | AIME 2026 | 37.5 | 39.8 | -2.3 | Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card |
| Gemma 2B (base) Google | Natural Questions (closed book) | 12.5 | 14.8 | -2.3 | Gemma report — original evaluations · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results) |
| EXAONE 4.0 32B LG AI Research | τ²-bench Telecom | 23.7 | 26.1 | -2.4 | LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Qwen3 32B Alibaba Cloud | MMLU-Pro | 75.5 | 77.9 | -2.4 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| GPT-5.6 Terra OpenAI | FrontierMath tier 4 v2 | 68.3 | 70.7 | -2.4 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| GPT-J 6B EleutherAI | ARC Challenge | 34.0 | 36.5 | -2.5 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Gemma 4 E4B Google | AIME 2026 | 42.5 | 45.0 | -2.5 | Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card |
| Llama 3.1 70B Instruct Meta | GSM8K | 92.0 | 94.5 | -2.5 | Puzzle: Distillation-Based NAS for Inference-Optimized LLMs (arXiv:2411.19146v5) · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| GPT-5.6 Luna OpenAI | FrontierMath tier 4 v2 | 58.5 | 61.0 | -2.5 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| LFM2 8B-A1B Liquid AI | GPQA Diamond | 29.3 | 31.9 | -2.6 | LFM2 Technical Report — Liquid AI own SmolLM3 evaluation · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| BLOOM 176B BigScience | COPA | 55.0 | 57.6 | -2.6 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1) |
| Claude Fable 5 Anthropic | Terminal-Bench 4.0 | 42.0 | 44.6 | -2.6 | Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions |
| Gemma 4 E2B Google | τ²-bench Telecom | 19.7 | 22.4 | -2.7 | Gemma Team — Gemma 4 Technical Report · LiquidAI/LFM2.5-8B-A1B model card |
| GLM-4.5-Air Z.ai | Humanity's Last Exam | 10.6 | 13.3 | -2.7 | GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs) |
| Qwen3 32B Alibaba Cloud | BFCL v3 | 66.6 | 69.3 | -2.7 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Berkeley Function Calling Leaderboard result archive |
| GPT-NeoX-20B EleutherAI | ARC Challenge | 38.0 | 40.8 | -2.8 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Phi-1.5 (1.3B) Microsoft | Social IQA | 52.6 | 55.4 | -2.8 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| GPT-4.1 OpenAI | τ²-bench Telecom | 34.0 | 36.9 | -2.9 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs |
| Gemma 3 1B Google | GPQA Diamond | 19.2 | 22.1 | -2.9 | Google DeepMind Gemma 3 270M IT model card · Epoch AI — internal evaluations |
| Mistral Small 3.2 24B Mistral AI | GPQA Diamond | 46.1 | 49.1 | -3.0 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 model card · Epoch AI — internal evaluations |
| GLM-5.1 Z.ai | IMO-AnswerBench | 83.8 | 86.8 | -3.0 | zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report |
| Code Llama 70B Python Meta | HumanEval | 57.3 | 60.4 | -3.1 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| DeepSeek R1 Distill Qwen 32B DeepSeek | AIME 2024 | 72.6 | 75.7 | -3.1 | deepseek-ai/DeepSeek-R1 model card · open-thoughts/OpenThinker2-32B model card |
| Phi-4-reasoning-plus 14B Microsoft | MMLU-Pro | 76.0 | 79.2 | -3.2 | microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| Pythia 12B EleutherAI | ARC Challenge | 31.8 | 35.1 | -3.3 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| LFM2.5 8B-A1B Liquid AI | AIME 2025 | 42.5 | 46.0 | -3.5 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| Llama 3.1 70B Instruct Meta | MMLU | 81.7 | 85.2 | -3.5 | Puzzle: Distillation-Based NAS for Inference-Optimized LLMs (arXiv:2411.19146v5) · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| GPT-5.6 Luna OpenAI | FrontierMath tiers 1–3 v2 | 78.6 | 82.1 | -3.5 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations |
| DeepSeek R1 (Jan '25) DeepSeek | Aider polyglot (225 exercises, two attempts) | 53.3 | 56.9 | -3.6 | deepseek-ai/DeepSeek-R1-0528 model card · Aider polyglot benchmark runs |
| Phi-1.5 (1.3B) Microsoft | ARC Challenge | 44.4 | 48.0 | -3.6 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| DeepSeek V3 (Dec '24) DeepSeek | SWE-bench Verified | 42.0 | 45.8 | -3.8 | deepseek-ai/DeepSeek-R1 model card · Cohere — Command A technical report |
| Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloud | τ²-bench Airline | 33.0 | 36.8 | -3.8 | Qwen/Qwen3-235B-A22B-Thinking-2507 model card · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1) |
| Qwen3 235B-A22B Thinking 2507 Alibaba Cloud | BFCL v3 | 71.9 | 75.7 | -3.8 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| BLOOM 1.1B BigScience | WinoGrande | 51.0 | 54.8 | -3.8 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| Qwen3 4B Alibaba Cloud | IFEval (aggregation unspecified) | 81.6 | 85.4 | -3.8 | Qwen/Qwen3-4B-Thinking-2507 model card · HuggingFaceTB/SmolLM3-3B model card (Evaluation section) |
| Mistral 7B v0.1 (base) Mistral AI | Natural Questions (closed book) | 26.0 | 30.1 | -4.1 | Mixtral of Experts — base-model evaluations · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results) |
| DeepSeek V3.1 DeepSeek | GPQA Diamond | 80.1 | 84.2 | -4.1 | deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| LFM2.5 2.6B Liquid AI | BFCL V4 (December 2025 archive) | 56.9 | 61.1 | -4.2 | LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card |
| Gemma 3 12B Google | MMLU | 71.9 | 76.2 | -4.3 | Gemma Team — Gemma 3 Technical Report · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| CodeGemma 7B IT (v1.0) Google | HumanEval | 56.1 | 60.4 | -4.3 | CodeGemma: Open Code Models Based on Gemma — authors' own runs · EvalPlus / EvalPerf published evaluation runs |
| Code Llama 13B (base) Meta | HumanEval | 36.0 | 40.3 | -4.3 | Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs |
| Qwen3 32B Alibaba Cloud | IFEval (aggregation unspecified) | 84.1 | 88.4 | -4.3 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| LFM2.5 8B-A1B Liquid AI | MATH-500 | 88.8 | 93.2 | -4.4 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| GLM-4.5V Z.ai | CharXiv (reasoning questions) | 58.4 | 63.0 | -4.6 | Z.ai GLM-V GitHub repository benchmark image (GLM-4.6V) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card |
| Qwen3.5 2B Alibaba Cloud | IFBench (aggregation unspecified) | 41.3 | 46.0 | -4.7 | Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card |
| Mistral 7B v0.1 (base) Mistral AI | TriviaQA | 66.2 | 70.9 | -4.7 | Mistral 7B — authors' own runs · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results) |
| Gemma 3 27B Google | MMLU | 76.9 | 81.8 | -4.9 | Gemma Team — Gemma 3 Technical Report · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| GPT-4.1 mini OpenAI | τ²-bench Telecom | 44.0 | 48.9 | -4.9 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs |
| DeepSeek V3.2 DeepSeek | τ²-bench aggregate | 80.3 | 85.2 | -4.9 | deepseek-ai/DeepSeek-V3.2 model card · stepfun-ai/Step-3.5-Flash model card |
| Qwen3-VL 2B Instruct Alibaba Cloud | IFEval (aggregation unspecified) | 68.2 | 73.4 | -5.2 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| Gemma 3 27B Google | GPQA Diamond | 42.4 | 47.7 | -5.3 | Gemma Team — Gemma 4 Technical Report · Epoch AI — internal evaluations |
| o4-mini OpenAI | τ²-bench Telecom | 40.5 | 46.1 | -5.6 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs |
| Qwen3.5 397B-A17B Alibaba Cloud | LongBench v2 | 63.2 | 68.9 | -5.7 | Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report |
| o3 OpenAI | τ²-bench Telecom | 58.2 | 64.0 | -5.8 | OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Meituan LongCat-Flash-Thinking model card (evaluation table) |
| BLOOM 560M BigScience | COPA | 55.0 | 61.0 | -6.0 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| OPT-175B Meta | ARC Easy | 67.0 | 73.0 | -6.0 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| Code Llama 13B Instruct Meta | HumanEval | 42.7 | 48.8 | -6.1 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| OPT-30B Meta | ARC Easy | 63.9 | 70.0 | -6.1 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| OPT-1.3B Meta | ARC Easy | 50.8 | 57.0 | -6.2 | OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| LLaMA 7B Meta | HellaSwag | 66.5 | 73.0 | -6.5 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| LFM2.5 8B-A1B Liquid AI | AIME 2026 | 50.0 | 56.7 | -6.7 | LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card |
| Gemma 7B (base) Google | GSM8K | 46.4 | 53.3 | -6.9 | Gemma report — original evaluations · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs |
| GPT-J 6B EleutherAI | HellaSwag | 59.0 | 66.2 | -7.2 | EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Mistral 7B v0.1 (base) Mistral AI | ARC Challenge | 55.2 | 62.5 | -7.3 | Mistral 7B — authors' own runs · MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.) |
| LFM2 8B-A1B Liquid AI | MMLU | 64.8 | 72.1 | -7.3 | LFM2 Technical Report — Liquid AI own SmolLM3 evaluation · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| Qwen3.5 2B Alibaba Cloud | HallusionBench | 58.0 | 65.5 | -7.5 | Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| SmolLM2 1.7B Instruct Hugging Face | BIG-Bench Hard | 32.2 | 39.8 | -7.6 | SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models |
| BLOOM 1.7B BigScience | WinoGrande | 49.6 | 57.2 | -7.6 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| LLaMA 13B Meta | HellaSwag | 69.6 | 77.6 | -8.0 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Pythia 6.9B EleutherAI | ARC Challenge | 31.6 | 39.7 | -8.1 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Gemma 4 E4B Google | τ²-bench Telecom | 18.4 | 26.8 | -8.4 | Gemma Team — Gemma 4 Technical Report · LiquidAI/LFM2.5-8B-A1B model card |
| Qwen3 32B Alibaba Cloud | LiveCodeBench v6 (Feb–May 2025) | 44.9 | 53.4 | -8.5 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Code Llama 7B Instruct Meta | HumanEval | 34.8 | 43.3 | -8.5 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| BLOOM 3B BigScience | WinoGrande | 50.0 | 58.6 | -8.6 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs |
| LLaMA 13B Meta | Social IQA | 50.4 | 59.0 | -8.6 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate — authors' own runs |
| Pythia 1B EleutherAI | ARC Challenge | 24.4 | 33.1 | -8.7 | Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Llama 3.1 Tulu 3 8B Ai2 | TruthfulQA MC2 | 55.0 | 64.1 | -9.1 | 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card |
| Pythia 6.9B EleutherAI | OpenBookQA | 25.2 | 35.4 | -10.2 | trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Code Llama 34B Instruct Meta | HumanEval | 41.5 | 51.8 | -10.3 | Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs |
| LLaMA 33B Meta | MBPP | 30.2 | 40.9 | -10.7 | LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations |
| Code Llama 7B (base) Meta | MBPP | 41.4 | 52.1 | -10.7 | Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs |
| BLOOM 1.1B BigScience | COPA | 57.0 | 68.0 | -11.0 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Qwen3.6 27B Alibaba Cloud | OSWorld-Verified | 63.9 | 75.6 | -11.7 | Qwen/Qwen3.8-27B model card · meta-models/Muse-Glimmer-30B model card (speed and benchmark tables) |
| BLOOM 560M BigScience | Story Cloze Test | 50.1 | 61.9 | -11.8 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| BLOOM 3B BigScience | COPA | 62.0 | 74.0 | -12.0 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| LFM2 8B-A1B Liquid AI | MMLU-Pro | 37.4 | 49.5 | -12.1 | LFM2 Technical Report — Liquid AI own SmolLM3 evaluation · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling |
| BLOOMZ 176B BigScience | WinoGrande | 58.4 | 72.5 | -14.1 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| Qwen3 32B Alibaba Cloud | τ-bench Airline | 23.0 | 38.0 | -15.0 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card |
| Pythia 6.9B EleutherAI | HellaSwag | 48.8 | 63.8 | -15.0 | trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs |
| Phi-1.5 (1.3B) Microsoft | HellaSwag | 47.6 | 62.6 | -15.0 | Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| BLOOM 7.1B BigScience | WinoGrande | 49.4 | 64.6 | -15.2 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Qwen3-VL 2B Instruct Alibaba Cloud | HallusionBench | 51.4 | 67.3 | -15.9 | Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table) |
| BLOOM 1.1B BigScience | Story Cloze Test | 47.3 | 63.3 | -16.0 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| BLOOM 1.7B BigScience | COPA | 54.0 | 70.0 | -16.0 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo) |
| Qwen3 4B Alibaba Cloud | AIME 2025 | 42.4 | 58.8 | -16.4 | Qwen/Qwen3-4B-Thinking-2507 model card · HuggingFaceTB/SmolLM3-3B model card (Evaluation section) |
| BLOOM 1.7B BigScience | Story Cloze Test | 48.5 | 65.4 | -16.9 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| GPT-NeoX-20B EleutherAI | HellaSwag | 53.5 | 71.4 | -17.9 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs |
| Qwen3 32B Alibaba Cloud | HMMT 2025 (edition unspecified) | 30.7 | 49.8 | -19.1 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| BLOOM 3B BigScience | Story Cloze Test | 47.5 | 67.8 | -20.3 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| BLOOM 176B BigScience | WinoGrande | 48.6 | 70.3 | -21.7 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs |
| GPT-J 6B EleutherAI | TriviaQA | 17.0 | 39.3 | -22.3 | GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results) |
| BLOOM 7.1B BigScience | Story Cloze Test | 49.4 | 72.0 | -22.6 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
| Qwen3 32B Alibaba Cloud | AIME 2025 | 46.6 | 72.2 | -25.6 | Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling |
| BLOOM 176B BigScience | Story Cloze Test | 51.0 | 76.9 | -25.9 | Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs |
Every result links to its source on the model's page. The data (CC BY-SA 4.0) and code (MIT) are on GitHub.