whatmodel.sh / lab vs independent benchmark

How it's worked out

By test

Tests with 13 or more pairs.

Testpairsmedian gapmaker higher
SWE-bench Verified13+9.092%
MBPP13+9.077%
ARC Easy14+4.779%
AIME 202414+3.886%
GSM8K19+3.384%
AIME 202520+2.670%
MMLU-Pro29+2.069%
GPQA Diamond51+1.167%
IFEval (aggregation unspecified)13+1.069%
PIQA16+0.575%
MMLU200.050%
HumanEval340.047%
WinoGrande25-0.140%
ARC Challenge14-0.650%
τ²-bench Telecom18-1.339%

By maker

Makers with 10 or more pairs. Small groups swing a lot: read these as rough.

Makerpairsmedian gapmaker higher
Alibaba Cloud148+1.161%
Meta89+0.762%
EleutherAI48-0.140%
Google48+1.660%
DeepSeek46+0.870%
OpenAI36+0.558%
Z.ai27+0.967%
Mistral AI20+1.675%
BigScience20-11.910%
BigCode20+2.295%
Liquid AI15-0.747%
Ai214+8.893%
Microsoft140.050%
IBM13+1.777%
Anthropic12+1.675%
All makers611+0.861%
All 611 pairs, biggest maker advantage first
ModelTestmakerindependentgapSources
Molmo 7B-D (0924) Ai2ChartQA84.127.0+57.1Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Qwen2-VL 7B Instruct Alibaba CloudChartQA83.041.2+41.8Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Molmo 7B-D (0924) Ai2MathVista (testmini)51.612.3+39.3Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
gpt-oss-120b OpenAISWE-bench Verified62.431.0+31.4OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ServiceNow-AI/Apriel-1.6-15b-Thinker model card
Qwen3.5 397B-A17B Alibaba CloudBrowseComp69.040.5+28.5Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Molmo 7B-D (0924) Ai2VQAv2 (val)85.657.1+28.5Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith)Terminal-Bench 2.164.235.96+28.24ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
OPT-30B MetaOpenBookQA57.229.2+28.0OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy — authors' own runs
DeepSeek R1 0528 DeepSeekSWE-bench Verified57.629.6+28.0deepseek-ai/DeepSeek-R1-0528 model card · ServiceNow-AI/Apriel-1.6-15b-Thinker model card
LLaMA 7B MetaOpenBookQA57.231.4+25.8LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Ministral 3 14B Reasoning 2512 Mistral AIGPQA Diamond71.246.0+25.2mistralai/Ministral-3-14B-Reasoning-2512 model card · JetBrains/Mellum2-12B-A2.5B-Thinking model card
SmolLM3 3B Hugging FaceGSM-Plus83.458.91+24.49HuggingFaceTB/SmolLM3-3B model card (Evaluation section) · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation
OPT-1.3B MetaSuperGLUE WSC (prompted)62.238.462+23.739OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
OPT-1.3B MetaOpenBookQA46.823.4+23.4OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs
Magistral Small 1.2 Mistral AIGPQA Diamond70.0747.601+22.469mistralai/Magistral-Small-2509 model card · Epoch AI — internal evaluations
Nemotron 3 Nano 4B NVIDIAAIME 202578.556.3+22.2nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 model card · openbmb/MiniCPM5-2B model card
Qwen2-VL 72B Instruct Alibaba CloudChartQA88.366.6+21.7Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
LLaMA 33B MetaOpenBookQA58.637.5+21.1LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Molmo 7B-D (0924) Ai2MMMU (val)45.324.3+21.0Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Gemma 3 1B GoogleGSM8K62.842.15+20.65Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
LLaMA 65B MetaARC Easy78.958.75+20.15LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
gpt-oss-120b OpenAIAIME 202495.875.8+20.0OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
DeepSeek V3.1 DeepSeekAIME 202588.468.585+19.815deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Chat model card (evaluation table)
Qwen3 Coder 30B-A3B Instruct Alibaba CloudSWE-bench Verified51.631.8+19.8Qwen/Qwen3-Coder-30B-A3B-Instruct model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith)SWE-bench Verified75.655.8+19.8ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
GLM-4.7-Flash Z.aiSWE-bench Verified59.239.7+19.5zai-org/GLM-4.7-Flash model card · SERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)
Qwen3.5 2B Alibaba CloudMMLU-Pro66.547.05+19.45Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
LLaMA 13B MetaOpenBookQA56.437.2+19.2LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith)SWE-bench Multilingual69.351.67+17.63ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
LFM2.5 2.6B Liquid AILiveCodeBench v6 (window unspecified)59.4142.1+17.31LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card
Gemma 3 1B GoogleIFEval (aggregation unspecified)80.262.9+17.3Google DeepMind Gemma 3 270M IT model card · LiquidAI/LFM2-1.2B model card
Mistral 7B v0.1 (base) Mistral AIGSM8K51.134.265+16.835Mistral 7B — authors' own runs · Gemma report — original evaluations
Claude Sonnet 4.5 Anthropicτ²-bench Retail86.269.8+16.4τ-bench leaderboard: Anthropic's own submissions · ServiceNow-AI/Apriel-1.6-15b-Thinker model card
Ornith 1.0 35B (35B-A3B) DeepReinforce (Ornith)SWE-bench Pro (split unspecified)50.434.47+15.93ornith-ai/Ornith-1.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
SmolLM3 3B Hugging FaceGPQA Diamond41.726.31+15.39HuggingFaceTB/SmolLM3-3B model card (Evaluation section) · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation
StarCoder2 3B BigCodeMBPP57.442.4+15.0StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
StarCoder2 15B BigCodeMBPP66.251.2+15.0StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
gpt-oss-120b OpenAIAIME 202592.577.7+14.8OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
GPT-5.1 OpenAIτ²-bench Telecom95.680.9+14.7OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
LLaMA 33B MetaBoolQ83.168.41+14.69LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Ministral 8B (2410) Mistral AIMATH (subset unspecified)54.540.0+14.5Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards) · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
DeepSeek R1 Distill Qwen 14B DeepSeekGPQA Diamond59.144.697+14.403deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations
DeepSeek R1 0528 DeepSeekAIME 202587.573.4+14.1deepseek-ai/DeepSeek-R1-0528 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Qwen3 Coder 30B-A3B Instruct Alibaba CloudSWE-bench Multilingual34.720.67+14.03Qwen/Qwen3-Coder-30B-A3B-Instruct model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
Qwen3 235B-A22B Instruct 2507 Alibaba Cloudτ²-bench Airline50.036.0+14.0Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table)
Qwen3.5 2B Alibaba CloudLongBench v238.724.9+13.8Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
Qwen3-VL 32B Thinking Alibaba Cloudτ²-bench Telecom46.933.3+13.6Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
Llama 3.1 Tulu 3 8B Ai2BIG-Bench Hard69.055.5+13.52 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card
StarCoderBase 3B BigCodeMBPP42.629.4+13.2StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
StarCoderBase 15.5B BigCodeMBPP50.637.4+13.2StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
StarCoder2 7B BigCodeGSM8K40.427.2+13.2StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
CodeGemma 2B (v1.0) GoogleMBPP43.630.4+13.2CodeGemma: Open Code Models Based on Gemma — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Nemotron Nano 9B v2 NVIDIAAIME 202572.158.9+13.2nvidia/NVIDIA-Nemotron-Nano-9B-v2 model card · Olmo 3
Ministral 3 3B Mistral AIMATH (subset unspecified)83.070.2+12.8mistralai/Ministral-3-3B-Instruct-2512 model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
Qwen3.5 35B-A3B Alibaba CloudSWE-bench Multilingual60.347.67+12.63Qwen/Qwen3.6-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
StarCoderBase 7B BigCodeMBPP47.434.8+12.6StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Magistral Small 1.0 (2506) Mistral AIGPQA Diamond68.1856.061+12.119mistralai/Magistral-Small-2509 model card · Epoch AI — internal evaluations
Claude Sonnet 4.5 Anthropicτ²-bench Airline70.058.0+12.0τ-bench leaderboard: Anthropic's own submissions · ServiceNow-AI/Apriel-1.6-15b-Thinker model card
Qwen3 4B Alibaba CloudMMLU-Pro64.252.31+11.89Qwen/Qwen3-4B-Thinking-2507 model card · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation
Molmo 72B (0924) Ai2ChartQA87.375.6+11.7Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Qwen3.5 27B Alibaba CloudSWE-bench Multilingual69.357.67+11.63Qwen/Qwen3.6-27B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
Qwen3.6 27B Alibaba CloudOmniDocBench 1.5 overall (higher is better)89.477.8+11.6Qwen/Qwen3.8-27B model card · meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)
EXAONE 4.5 33B LG AI ResearchMMLU-Pro83.371.96+11.34LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · SqueezeBits — original EXAONE 4.5 BF16 evaluations
Molmo 72B (0924) Ai2VQAv2 (val)86.575.2+11.3Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
StarCoderBase 7B BigCodeGSM8K14.93.8+11.1StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Qwen3 8B Alibaba CloudMMLU-Pro74.663.5+11.1Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen3.5 2B Alibaba CloudCountBench91.480.5+10.9Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Granite 4.2 3B IBMLiveCodeBench v6 (window unspecified)69.7158.9+10.81ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card
DeepSeek R1 0528 DeepSeekLiveCodeBench v6 (Aug 2024–May 2025)73.362.5+10.8deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Qwen3-VL 2B Instruct Alibaba CloudOCRBench85.875.1+10.7Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Qwen3.5 35B-A3B Alibaba CloudSWE-bench Verified69.258.6+10.6Qwen/Qwen3.5-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
OPT-1.3B MetaARC Challenge33.923.31+10.59OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Qwen3 1.7B Alibaba CloudMMLU-Pro56.545.94+10.56Qwen/Qwen3.5-2B model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
Qwen3 1.7B Alibaba CloudMMMLU57.046.51+10.49Qwen/Qwen3.5-2B model card · LiquidAI/LFM2-1.2B model card
Qwen3.5 2B Alibaba CloudMMStar71.761.4+10.3Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
gpt-oss-20b OpenAIGPQA Diamond71.561.2+10.3OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Epoch AI — internal evaluations
GPT-Neo 1.3B EleutherAIHellaSwag48.938.65+10.25EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs
Qwen3.6 35B-A3B Alibaba CloudSWE-bench Multilingual67.257.0+10.2Qwen/Qwen3.6-27B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
LFM2.5 2.6B Liquid AIAIME 202551.8741.9+9.97LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card
Gemma 4 E2B GoogleMMLU-Pro60.050.05+9.95Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card
Qwen3-VL 2B Instruct Alibaba CloudChartQA79.169.3+9.8Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Nemotron Nano 9B v2 NVIDIAAIME 202481.972.1+9.8NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models · Olmo 3
DeepSeek R1 0528 DeepSeekMMLU-Pro85.075.3+9.7deepseek-ai/DeepSeek-R1-0528 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
LLaMA 13B MetaBoolQ78.068.502+9.498LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Qwen3 30B-A3B Thinking 2507 Alibaba Cloudτ-bench Retail67.858.7+9.1Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Gemma 4 26B-A4B GoogleGPQA Diamond82.373.232+9.068Gemma Team — Gemma 4 Technical Report · Epoch AI — internal evaluations
StarCoder2 7B BigCodeMBPP54.445.4+9.0StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Qwen3.6 35B-A3B Alibaba CloudSWE-bench Verified73.464.4+9.0Qwen/Qwen3.6-35B-A3B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
Gemma 4 31B GoogleTerminal-Bench Hard36.027.08+8.92Gemma Team — Gemma 4 Technical Report · NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party models
Devstral Small 2 24B Mistral AISWE-bench Verified68.059.1+8.9mistralai/Devstral-Small-2-24B-Instruct-2512 model card · SERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)
LLaMA 13B MetaARC Easy76.167.21+8.89LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Gemma 3 4B GoogleMMLU-Pro43.634.76+8.84Google DeepMind Gemma 3 270M IT model card · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation
LLaMA 33B MetaNatural Questions (closed book)32.924.1+8.8LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
Claude Sonnet 5.5 AnthropicTerminal-Bench 4.070.661.82+8.78Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions
GLM-5.1 Z.aiBrowseComp68.059.4+8.6zai-org/GLM-5.1 model card · NVIDIA — Nemotron 3 Ultra technical report
Gemma 4 31B GoogleGPQA Diamond84.375.758+8.542Gemma Team — Gemma 4 Technical Report · Epoch AI — internal evaluations
LFM2.5 8B-A1B Liquid AIMulti-IF (aggregation unspecified)79.9371.4+8.53LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
OPT-30B MetaARC Challenge43.134.642+8.458OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
LLaMA 33B MetaARC Challenge57.849.464+8.336LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
DeepSeek R1 0528 DeepSeekAIME 202491.483.2+8.2deepseek-ai/DeepSeek-V3.1 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
InternVL2-4B OpenGVLabTextVQA74.466.2+8.2OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table
Qwen3.5 2B Alibaba CloudAI2D test83.375.2+8.1Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
EXAONE 4.0 32B LG AI Researchτ²-bench Retail67.559.5+8.0LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
Nemotron Nano 9B v2 NVIDIAGPQA (subset unspecified)64.056.2+7.8nvidia/NVIDIA-Nemotron-Nano-9B-v2 model card · Olmo 3
Qwen3-VL 2B Instruct Alibaba CloudMMStar58.350.6+7.7Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Gemma 3 1B GoogleMATH (subset unspecified)48.040.3+7.7Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GLM-4.5-Air Z.aiMMLU-Pro81.473.9+7.5GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Qwen3.5 2B Alibaba CloudMMBench-EN dev v1.183.376.0+7.3Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
GPT-Neo 2.7B EleutherAIHellaSwag55.848.555+7.245EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs
gpt-oss-120b OpenAIGPQA Diamond80.172.879+7.221OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Epoch AI — internal evaluations
Gemini 2.5 Flash GoogleCharXiv (reasoning questions)63.756.5+7.2Google DeepMind Gemini model page and model cards (lab-reported scores) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card
Gemma 3 27B GoogleMATH (subset unspecified)89.081.8+7.2Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
DeepSeek R1 Distill Llama 8B DeepSeekAIME 202450.443.3+7.1deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card
Qwen3.5 2B Alibaba CloudGPQA (subset unspecified)51.644.6+7.0Qwen/Qwen3.5-2B model card · XHToken/Spark-X2.5-4B model card
Phi-4-reasoning-plus 14B MicrosoftAIME 202578.071.2+6.8microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen3.5 9B Alibaba CloudBFCL V4 (December 2025 archive)66.159.315+6.785Qwen/Qwen3.5-9B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card
Pythia 1B EleutherAIARC Easy56.950.2+6.7Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Pythia 12B EleutherAIARC Easy70.263.721+6.479Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
LLaMA 7B MetaARC Easy74.0567.641+6.409LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
EXAONE 4.0 32B LG AI Researchτ²-bench Airline52.045.6+6.4LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
Molmo 72B (0924) Ai2MathVista (testmini)58.652.2+6.4Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Qwen3 1.7B Alibaba CloudGPQA (subset unspecified)34.027.72+6.28Qwen/Qwen3.5-2B model card · LiquidAI/LFM2-1.2B model card
Qwen3-VL 2B Instruct Alibaba CloudMMLU-Pro49.042.8+6.2Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Mistral 7B v0.1 (base) Mistral AIMBPP48.8542.7+6.15Mistral 7B — authors' own runs · Gemma report — original evaluations
StarCoderBase 7B BigCodeHumanEval30.524.4+6.1StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · EvalPlus / EvalPerf published evaluation runs
Qwen2-VL 7B Instruct Alibaba CloudMMMU (val)54.148.1+6.0Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Qwen3.5 2B Alibaba CloudIFEval (aggregation unspecified)78.672.7+5.9Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
WizardCoder Python 34B V1.0 WizardLMHumanEval73.267.4+5.8WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card) · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
GLM-5 Z.aiSWE-bench Verified77.872.078+5.722zai-org/GLM-5 model card · Epoch AI — internal evaluations
Qwen3 30B-A3B Thinking 2507 Alibaba CloudLiveCodeBench v6 (Feb–May 2025)66.060.3+5.7Qwen/Qwen3-30B-A3B-Thinking-2507 model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Qwen3 32B Alibaba Cloudτ-bench Retail46.4540.9+5.55Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Pythia 6.9B EleutherAIARC Easy67.061.485+5.515Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Gemma 4 E4B GoogleGPQA Diamond58.653.1+5.5Gemma Team — Gemma 4 Technical Report · JetBrains/Mellum2.1-12B-A2.5B-Thinking model card
LFM2.5 8B-A1B Liquid AIIFBench (aggregation unspecified)56.4751.0+5.47LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
Gemma 2B (base) GoogleTriviaQA53.247.77+5.43Gemma report — original evaluations · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
Granite 4.0 H 1B IBMMMLU-Pro32.8627.64+5.22ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card
Qwen3.5 2B Alibaba CloudRealWorldQA74.569.3+5.2Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Qwen3.5 27B Alibaba CloudSWE-bench Verified73.768.6+5.1Qwen/Qwen3.5-27B model card · Kwaipilot/KAT-Coder-V2.5-Dev model card
InternVL2-4B OpenGVLabMathVista (testmini)58.653.7+4.9OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table
GLM-4.5-Air Z.aiAIME 202489.484.6+4.8GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Gemma 3 12B GoogleMMLU-Pro60.655.8+4.8Google DeepMind Gemma 3 270M IT model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
Gemma 3 27B GoogleGSM8K95.991.1+4.8Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GPT-J 6B EleutherAIARC Easy67.062.247+4.753GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Gemma 3 12B GoogleGSM8K94.489.7+4.7Google DeepMind Gemma 3 270M IT model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
LLaMA 33B MetaARC Easy80.075.337+4.663LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
DeepSeek R1 Distill Qwen 32B DeepSeekMATH-50094.389.7+4.6deepseek-ai/DeepSeek-R1 model card · open-thoughts/OpenThinker2-32B model card
Qwen2-VL 7B Instruct Alibaba CloudMathVista (testmini)58.253.7+4.5Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Gemma 3 4B GoogleGPQA Diamond30.826.371+4.429Google DeepMind Gemma 3 270M IT model card · Epoch AI — internal evaluations
Qwen3 30B-A3B Thinking 2507 Alibaba Cloudτ²-bench Telecom26.321.93+4.37Qwen/Qwen3-Next-80B-A3B-Thinking model card · LiquidAI/LFM2.5-8B-A1B model card
gpt-oss-20b OpenAIAIME 202591.787.35+4.35OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen2 7B Instruct Alibaba CloudMATH (subset unspecified)52.948.6+4.3Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · InternLM GitHub repository model cards (Shanghai AI Laboratory)
CodeGemma 2B (v1.0) GoogleHumanEval31.126.8+4.3CodeGemma: Open Code Models Based on Gemma — authors' own runs · EvalPlus / EvalPerf published evaluation runs
gpt-oss-120b OpenAIHumanity's Last Exam14.910.6+4.3OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
DeepSeek R1 Distill Llama 70B DeepSeekGPQA Diamond65.260.972+4.228deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations
GLM-5.1 Z.aiTerminal-Bench 2.163.559.3+4.2zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report
Qwen2-VL 72B Instruct Alibaba CloudMMMU (val)64.560.3+4.2Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
gpt-oss-20b OpenAIAIME 202492.188.0+4.1OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen3 235B-A22B Instruct 2507 Alibaba Cloudτ²-bench Retail74.670.5+4.1Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table)
Phi-4-reasoning-plus 14B MicrosoftAIME 202481.377.2+4.1microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
DeepSeek R1 0528 DeepSeekGPQA Diamond81.076.913+4.087deepseek-ai/DeepSeek-R1-0528 model card · Epoch AI — internal evaluations
Mistral Small 3.1 24B (2503) Mistral AIGPQA Diamond45.9641.919+4.041mistralai/Mistral-Small-3.2-24B-Instruct-2506 model card · Epoch AI — internal evaluations
Qwen3.5 9B Alibaba CloudIFBench (aggregation unspecified)64.560.485+4.015Qwen/Qwen3.5-9B model card · XHToken/Spark-X2.5-4B model card
OPT-1.3B MetaStory Cloze Test74.770.78+3.92OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
Gemma 3 27B GoogleBIG-Bench Hard87.683.7+3.9Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
DeepSeek V3.1 DeepSeekHMMT 2025 (edition unspecified)84.280.4+3.8deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Nemotron 3 Nano 4B NVIDIAMATH-50095.491.6+3.8nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 model card · openbmb/MiniCPM5-2B model card
Qwen3.5 397B-A17B Alibaba CloudMultiChallenge67.663.9+3.7Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Qwen3 30B-A3B Thinking 2507 Alibaba CloudAIME 202585.081.3+3.7Qwen/Qwen3-30B-A3B-Thinking-2507 model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Qwen3 235B-A22B Thinking 2507 Alibaba Cloudτ²-bench Retail71.968.2+3.7Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
InternVL2-4B OpenGVLabMMMU (val)47.944.22+3.68OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table
GPT-NeoX-20B EleutherAIARC Easy72.368.687+3.613GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
gpt-oss-20b OpenAIMMLU85.381.7+3.6OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
StarCoderBase 3B BigCodeHumanEval21.317.7+3.6StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · EvalPlus / EvalPerf published evaluation runs
Inkling (975B-A41B) Thinking Machines LabSimpleQA Verified43.940.3+3.6thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations
Gemma 3 12B GoogleGPQA Diamond40.937.329+3.571Google DeepMind Gemma 3 270M IT model card · Epoch AI — internal evaluations
Olmo 3 7B Think Ai2IFEval (aggregation unspecified)88.284.7+3.5Olmo 3 · JetBrains/Mellum2-12B-A2.5B-Thinking model card
SmolLM2 1.7B Instruct Hugging FaceGSM8K48.845.3+3.5SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GLM-4.5 Z.aiAIME 202491.087.55+3.45GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Claude 1.3 AnthropicGSM8K85.281.8+3.4Anthropic system cards and model announcements (lab-reported scores) · Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance — authors' own runs
Qwen3 235B-A22B Instruct 2507 Alibaba Cloudτ²-bench Telecom32.529.1+3.4Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table)
GLM-5.1 Z.aiHumanity's Last Exam31.027.6+3.4zai-org/GLM-5.1 model card · NVIDIA — Nemotron 3 Ultra technical report
Granite 4.0 H Small (32B-A9B) IBMGSM8K87.2783.9+3.37ibm-granite/granite-4.0-h-small model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
LLaMA 7B MetaARC Challenge44.7541.38+3.37LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Flan-T5 XXL GoogleBIG-Bench Hard45.341.95+3.35Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
Qwen3 1.7B Alibaba CloudAIME 202536.833.485+3.315Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · LiquidAI/LFM2.5-1.2B-Thinking model card
Qwen3.5 397B-A17B Alibaba CloudHMMT February 202687.984.6+3.3Qwen/Qwen3.6-27B model card · LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card
LLaMA 13B MetaGSM8K17.814.5+3.3LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs
Llama 3.1 Tulu 3 8B Ai2GSM8K87.684.3+3.32 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card
LFM2.5 2.6B Liquid AIMulti-IF (aggregation unspecified)80.0776.8+3.27LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card
OPT-175B MetaStory Cloze Test82.979.66+3.24OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
LLaMA 33B MetaWinoGrande76.072.77+3.23LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
DeepSeek V3.1 DeepSeekMMLU-Redux93.790.5+3.2deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Llama 3.1 Tulu 3 8B Ai2MMLU68.265.0+3.22 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card
CodeGemma 7B (v1.0) GoogleMBPP56.253.0+3.2CodeGemma: Open Code Models Based on Gemma — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Flan-T5 Large GoogleMMLU45.141.9+3.2Scaling Instruction-Finetuned Language Models — authors' own runs · Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runs
LLaMA 7B MetaSocial IQA48.945.716+3.184LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs
OPT-30B MetaStory Cloze Test80.377.14+3.16OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
Qwen3 1.7B Alibaba CloudIFBench (aggregation unspecified)26.723.575+3.125Qwen/Qwen3.5-2B model card · LiquidAI/LFM2.5-1.2B-Thinking model card
GLM-4.5 Z.aiMMLU-Pro84.681.5+3.1GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Meituan LongCat-Flash-Thinking model card (evaluation table)
LLaMA 7B MetaWinoGrande70.066.93+3.07LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
WizardCoder 33B V1.1 WizardLMHumanEval79.976.85+3.05WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card) · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
Granite 4.0 1B IBMGSM8K76.3573.42+2.93ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card
Claude Opus 4.6 AnthropicGPQA Diamond91.388.384+2.916Anthropic system cards and model announcements (lab-reported scores) · Epoch AI — internal evaluations
Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba CloudBFCL v370.867.94+2.86Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Berkeley Function Calling Leaderboard result archive
LLaMA 13B MetaARC Challenge49.5546.725+2.825LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Qwen2 7B Instruct Alibaba CloudGSM8K85.782.9+2.8Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · InternLM GitHub repository model cards (Shanghai AI Laboratory)
Nemotron Nano 9B v2 NVIDIAIFBench (aggregation unspecified)37.434.6+2.8NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models · Olmo 3
Qwen3-VL 2B Instruct Alibaba CloudCharXiv descriptive questions62.359.5+2.8Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
GPT-5 mini OpenAIτ²-bench Telecom74.171.3+2.8OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · SEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages (arXiv:2606.28715v2)
Qwen3.5 397B-A17B Alibaba CloudSWE-bench Verified76.373.6+2.7Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
DeepSeek R1 0528 Qwen3 8B DeepSeekAIME 202486.083.3+2.7deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen3.5 4B Alibaba CloudIFEval (aggregation unspecified)89.887.1+2.7Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
Qwen3.5 4B Alibaba CloudLongBench v250.047.3+2.7Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
InternVL2-4B OpenGVLabChartQA81.578.8+2.7OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table
GPT-4o mini (2024-07-18) OpenAICharXiv (reasoning questions)36.834.1+2.7OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · CharXiv leaderboard (princeton-nlp/CharXiv README)
Qwen3 4B Instruct 2507 Alibaba CloudMMLU-Pro69.666.9+2.7Qwen/Qwen3-4B-Instruct-2507 model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
LLaMA 13B MetaWinoGrande72.870.128+2.672LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Qwen3 30B-A3B Thinking 2507 Alibaba CloudGPQA Diamond73.470.738+2.662Qwen/Qwen3.5-4B model card · Epoch AI — internal evaluations
OPT-30B MetaSuperGLUE WSC (prompted)62.259.615+2.585OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
DeepSeek V3 (Dec '24) DeepSeekGPQA Diamond59.156.534+2.566deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations
GPT-5.6 Sol OpenAIFrontierMath tier 4 v283.080.488+2.512OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
gpt-oss-120b OpenAIAider polyglot (225 exercises, two attempts)44.441.9+2.5OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Aider polyglot benchmark runs
Phi-1.5 (1.3B) MicrosoftARC Easy75.673.148+2.452Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3.7 Plus Alibaba CloudGPQA Diamond90.387.879+2.421Qwen/Qwen3.8-27B model card · Epoch AI — internal evaluations
Olmo 3 7B Instruct Ai2IFEval (aggregation unspecified)85.683.2+2.4Olmo 3 · JetBrains/Mellum2-12B-A2.5B-Instruct model card
Qwen3 235B-A22B Thinking 2507 Alibaba CloudMMLU-Redux93.891.4+2.4Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Granite 4.0 1B IBMBFCL v354.8252.43+2.39ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card
GLM-4.7 Z.aiGPQA Diamond85.783.333+2.367GLM-5 Team (Zhipu AI) — GLM-5 technical report · Epoch AI — internal evaluations
Qwen3.5 9B Alibaba CloudGPQA Diamond81.779.389+2.311Qwen/Qwen3.5-9B model card · Epoch AI — internal evaluations
DeepSeek R1 Distill Qwen 7B DeepSeekMATH-50092.890.5+2.3deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card
Qwen2-VL 72B Instruct Alibaba CloudMathVista (testmini)70.568.2+2.3Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Kimi K2 Instruct (0711) Moonshot AIτ²-bench Airline56.554.2+2.3τ²-bench leaderboard: Moonshot AI's own submission · Meituan LongCat-Flash-Chat model card (evaluation table)
LLaMA 7B MetaMMLU35.132.8+2.3LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
Flan-T5 XL GoogleMMLU52.450.2+2.2Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
DeepSeek R1 Distill Llama 8B DeepSeekMATH-50089.186.9+2.2deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card
Qwen3.6 35B-A3B Alibaba CloudGPQA Diamond86.083.838+2.162Qwen/Qwen3.6-27B model card · Epoch AI — internal evaluations
Granite 4.2 3B IBMMMLU-Pro67.8465.8+2.04ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card
Qwen3.5 35B-A3B Alibaba CloudIMO-AnswerBench76.874.8+2.0Qwen/Qwen3.6-35B-A3B model card · NVIDIA — Nemotron-Cascade 2 technical report
GLM-4.5 Z.aiMATH-50098.296.2+2.0GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
BLOOM 7.1B BigScienceCOPA69.067.0+2.0Crosslingual Generalization through Multitask Finetuning — authors' own runs · From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
OPT-30B MetaCOPA84.082.0+2.0OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
OPT-175B MetaARC Challenge44.142.12+1.98OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
Qwen3 235B-A22B Instruct 2507 Alibaba CloudAIME 202570.368.33+1.97Qwen/Qwen3-235B-A22B-Instruct-2507 model card · Meituan LongCat-Flash-Chat model card (evaluation table)
LLaMA 65B MetaPIQA82.880.85+1.95LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloudτ²-bench Telecom23.2521.3+1.95Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)
Qwen3.6 27B Alibaba CloudGPQA Diamond87.885.859+1.941Qwen/Qwen3.6-27B model card · Epoch AI — internal evaluations
GLM-5.1 Z.aiHumanity's Last Exam with tools52.350.4+1.9zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report
LLaMA 33B MetaHellaSwag82.880.925+1.875LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
Mistral 7B v0.1 (base) Mistral AIPIQA82.680.794+1.806Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
DeepSeek R1 0528 DeepSeekHumanity's Last Exam17.715.9+1.8deepseek-ai/DeepSeek-R1-0528 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Qwen3 8B Alibaba CloudBFCL v368.166.34+1.76Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Berkeley Function Calling Leaderboard result archive
Granite 4.0 H Small (32B-A9B) IBMMMLU78.4476.7+1.74ibm-granite/granite-4.0-h-small model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
DeepSeek R1 Distill Llama 8B DeepSeekGPQA Diamond49.047.3+1.7deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card
Qwen3.5 9B Alibaba CloudIFEval (aggregation unspecified)91.589.8+1.7Qwen/Qwen3.5-9B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card
Claude 2.0 AnthropicHumanEval71.269.5+1.7Anthropic system cards and model announcements (lab-reported scores) · EvalPlus / EvalPerf published evaluation runs
GPT-4o mini (2024-07-18) OpenAICharXiv descriptive questions76.674.92+1.68OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · CharXiv leaderboard (princeton-nlp/CharXiv README)
LLaMA 7B MetaBoolQ75.874.123+1.677LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Pythia 1B EleutherAIPIQA70.769.1+1.6Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
InternVL2-4B OpenGVLabAI2D test78.977.3+1.6OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tables · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table
T0 11B BigScienceWinoGrande60.4658.9+1.56Crosslingual Generalization through Multitask Finetuning — authors' own runs · NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation — authors' own runs
Claude Opus 5.5 AnthropicTerminal-Bench 4.066.464.85+1.55Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions
Qwen3 1.7B Alibaba CloudBFCL v356.655.055+1.545Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Berkeley Function Calling Leaderboard result archive
Qwen3 8B Alibaba CloudAIME 202567.365.8+1.5Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Inkling Small (266B-A12B) Thinking Machines LabSimpleQA Verified20.619.1+1.5thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations
OPT-30B MetaWinoGrande69.768.2+1.5OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Molmo 72B (0924) Ai2MMMU (val)54.152.7+1.4Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4) · Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
Claude Instant 1.2 AnthropicHumanEval58.757.3+1.4Anthropic system cards and model announcements (lab-reported scores) · EvalPlus / EvalPerf published evaluation runs
Ministral 3 8B Mistral AIMATH (subset unspecified)87.686.2+1.4mistralai/Ministral-3-8B-Instruct-2512 model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
LLaMA 33B MetaTriviaQA69.968.5+1.4LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
Qwen3.5 397B-A17B Alibaba CloudGPQA (subset unspecified)88.487.1+1.3Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
DeepSeek V3.1 DeepSeekLiveCodeBench (version unspecified)74.873.5+1.3DeepSeek-V3.1-Terminus model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Code Llama 34B Python MetaHumanEval53.752.4+1.3Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
Phi-1.5 (1.3B) MicrosoftBoolQ75.874.526+1.274Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloudτ²-bench Retail52.6551.4+1.25Qwen/Qwen3-235B-A22B-Thinking-2507 model card · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
LLaMA 65B MetaARC Challenge56.054.78+1.22LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
OPT-175B MetaWinoGrande73.772.5+1.2OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
Phi-4-reasoning-plus 14B MicrosoftGPQA Diamond69.167.9+1.2microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen3.5 4B Alibaba CloudMMLU-Pro79.178.0+1.1Qwen/Qwen3.5-4B model card · openbmb/MiniCPM5-2B model card
DeepSeek R1 (Jan '25) DeepSeekAIME 202479.878.7+1.1deepseek-ai/DeepSeek-R1-0528 model card · Microsoft — Phi-4-reasoning Technical Report
Gemma 4 E4B GoogleMMLU-Pro69.468.3+1.1Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card
LLaMA 7B MetaPIQA79.2578.183+1.067LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
DeepSeek V4.1 Flash DeepSeekGPQA Diamond90.989.836+1.064deepseek-ai/DeepSeek-V4.1-Flash model card · Epoch AI — internal evaluations
LFM2.5 8B-A1B Liquid AIIFEval (aggregation unspecified)91.8490.8+1.04LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
Qwen3.5 397B-A17B Alibaba CloudLiveCodeBench v6 (Feb–May 2025)83.682.6+1.0Qwen/Qwen3.5-397B-A17B model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
Qwen3 30B-A3B Thinking 2507 Alibaba Cloudτ-bench Airline48.047.0+1.0Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
GLM-4.5 Z.aiAIME 202585.484.4+1.0zai-org/GLM-4.6 model card · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
GPT-4o mini (2024-07-18) OpenAIHumanEval (simple-evals)87.286.2+1.0OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report
LLaMA 13B MetaHumanEval15.814.8+1.0LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
Inkling Small (266B-A12B) Thinking Machines LabGPQA Diamond89.588.51+0.99thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations
Flan-T5 Base GoogleMMLU35.934.95+0.95Scaling Instruction-Finetuned Language Models — authors' own runs · Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runs
Qwen3 30B-A3B Thinking 2507 Alibaba CloudIFEval (aggregation unspecified)88.988.0+0.9Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
GLM-5.1 Z.aiτ³-bench70.669.7+0.9zai-org/GLM-5.1 model card · NVIDIA — Nemotron 3 Ultra technical report
OPT-1.3B MetaBoolQ60.559.6+0.9OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs
GLM-5.1 Z.aiCritPt4.63.7+0.9zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report
LLaMA 65B MetaGSM8K50.950.0+0.9LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs
Qwen3 235B-A22B Thinking 2507 Alibaba CloudGPQA Diamond81.180.225+0.875Qwen/Qwen3.5-27B model card · Epoch AI — internal evaluations
GPT-5 OpenAIτ²-bench Telecom96.795.83+0.87OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs
Phi-1.5 (1.3B) MicrosoftWinoGrande73.472.534+0.866Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3.5 397B-A17B Alibaba CloudGPQA Diamond88.487.579+0.821Qwen/Qwen3.6-27B model card · Epoch AI — internal evaluations
GPT-Neo 2.7B EleutherAIPIQA73.072.19+0.81EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs
LLaMA 65B MetaHellaSwag84.283.39+0.81LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
Qwen3.5 397B-A17B Alibaba CloudHumanity's Last Exam28.727.9+0.8Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Flan-T5 XL GoogleBIG-Bench Hard41.040.2+0.8Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
DeepSeek V3 0324 DeepSeekGPQA Diamond68.467.614+0.786deepseek-ai/DeepSeek-V3.1 model card · Epoch AI — internal evaluations
Qwen3 235B-A22B Instruct 2507 Alibaba CloudZebraLogic95.094.22+0.78Qwen/Qwen3-235B-A22B-Instruct-2507 model card · Meituan LongCat-Flash-Chat model card (evaluation table)
GPT-J 6B EleutherAILAMBADA (OpenAI version), last-word accuracy69.068.232+0.768EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3.5 35B-A3B Alibaba CloudGPQA Diamond84.283.46+0.74Qwen/Qwen3.5-35B-A3B model card · Epoch AI — internal evaluations
DeepSeek V4 Pro (0813) DeepSeekGPQA Diamond92.491.667+0.733deepseek-ai/DeepSeek-V4.1-Flash model card · Epoch AI — internal evaluations
LLaMA 33B MetaPIQA82.381.579+0.721LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
Mistral 7B v0.1 (base) Mistral AIWinoGrande74.7574.033+0.717Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
LLaMA 13B MetaPIQA79.6578.943+0.707LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Mistral 7B v0.1 (base) Mistral AIARC Easy80.2579.545+0.705Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
DeepSeek R1 Distill Qwen 7B DeepSeekGPQA Diamond49.148.4+0.7deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card
DeepSeek R1 Distill Llama 70B DeepSeekAIME 202470.069.3+0.7deepseek-ai/DeepSeek-R1 model card · Microsoft — Phi-4-reasoning Technical Report
GLM-4.5 Z.aiIFEval prompt-level strict86.185.4+0.7GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Meituan LongCat-Flash-Thinking model card (evaluation table)
StarCoder2 15B BigCodeCRUXEval-I48.147.4+0.7StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
SmolLM2 1.7B Instruct Hugging FaceMATH (subset unspecified)21.020.3+0.7SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GPT-Neo 2.7B EleutherAIMathQA24.7224.05+0.67EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs
Gemma 3 1B GoogleMMLU-Pro14.714.04+0.66Google DeepMind Gemma 3 270M IT model card · LiquidAI/LFM2.5-1.2B-Thinking model card
StarCoder2 15B BigCodeHumanEval46.345.7+0.6StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
StarCoder2 3B BigCodeCRUXEval-I32.732.1+0.6StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Code Llama 13B Python MetaHumanEval43.342.7+0.6Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
Qwen2.5 Coder 32B Instruct Alibaba CloudHumanEval92.792.1+0.6Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs
OPT-1.3B MetaPIQA72.371.704+0.596OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
DeepSeek R1 0528 Qwen3 8B DeepSeekAIME 202576.375.8+0.5deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Qwen3.5 4B Alibaba CloudMMLU-Redux88.888.3+0.5Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
Code Llama 13B (base) MetaMBPP47.046.5+0.5Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs
GPT-4o mini (2024-07-18) OpenAIMGSM (simple-evals)87.086.5+0.5OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report
Granite 4.0 1B IBMMMLU-Pro34.0233.53+0.49ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card
GPT-J 6B EleutherAIWinoGrande64.6564.167+0.483EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs
Claude Opus 5 AnthropicTerminal-Bench 4.052.351.82+0.48Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions
GPT-NeoX-20B EleutherAIPIQA77.977.421+0.479GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3.5 397B-A17B Alibaba CloudWMT24++78.978.5+0.4Qwen/Qwen3.5-397B-A17B model card · Cohere — North Small Translate (arXiv:2609.13916v1)
GPT-4o mini (2024-07-18) OpenAIDROP F1, 3-shot (simple-evals)79.779.3+0.4OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report
Qwen3 235B-A22B Thinking 2507 Alibaba CloudOJBench32.532.1+0.4Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Pythia 1B EleutherAIWinoGrande53.753.3+0.4Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
StarCoder2 7B BigCodeCRUXEval-I34.634.2+0.4StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
StarCoder2 15B BigCodeCRUXEval-O47.146.7+0.4StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
DeepSeek V3.1 DeepSeekMMLU-Pro84.884.425+0.375deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Chat model card (evaluation table)
LLaMA 7B MetaGSM8K11.010.65+0.35LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs
DeepSeek V3 (Dec '24) DeepSeekIFEval prompt-level strict86.185.77+0.33deepseek-ai/DeepSeek-R1 model card · OpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file
LLaMA 13B MetaMMLU46.946.57+0.33LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
StarCoder2 7B BigCodeHumanEval35.435.1+0.3StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
StarCoder2 3B BigCodeHumanEval31.731.4+0.3StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
GLM-4.5-Air Z.aiMATH-50098.197.8+0.3GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Pythia 6.9B EleutherAILAMBADA (OpenAI version), last-word accuracy67.367.01+0.29Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Code Llama 13B (base) MetaCRUXEval-I42.48842.2+0.288CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Mistral 7B v0.1 (base) Mistral AIHumanEval28.3528.07+0.28Mistral 7B — authors' own runs · MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)
GPT-J 6B EleutherAIPIQA75.8575.571+0.279EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs
Phi-1.5 (1.3B) MicrosoftPIQA76.676.333+0.267Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Granite 4.0 H 1B IBMGSM8K69.8369.6+0.23ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card
OPT-30B MetaMMLU26.125.88+0.22OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy — authors' own runs
DeepSeek R1 Distill Qwen 1.5B DeepSeekGPQA Diamond33.833.586+0.214deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations
Code Llama 7B (base) MetaCRUXEval-O34.21334.0+0.212CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Granite 4.2 3B IBMBFCL V4 (December 2025 archive)52.4152.2+0.21ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card
DeepSeek R1 Distill Qwen 7B DeepSeekAIME 202455.555.3+0.2deepseek-ai/DeepSeek-R1 model card · microsoft/Phi-4-mini-reasoning model card
DeepSeek R1 0528 DeepSeekAider polyglot (225 exercises, two attempts)71.671.4+0.2deepseek-ai/DeepSeek-V3.1 model card · Aider polyglot benchmark runs
StarCoder2 3B BigCodeCRUXEval-O34.234.0+0.2StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
StarCoder2 7B BigCodeCRUXEval-O36.035.8+0.2StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Gemma 7B (base) GoogleMATH24.324.1+0.2Gemma report — original evaluations · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
GPT-4o mini (2024-07-18) OpenAIMMLU (simple-evals, zero-shot CoT)82.081.8+0.2OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report
LLaMA 65B MetaHumanEval23.723.5+0.2LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs
Qwen3.5 4B Alibaba CloudIFBench (aggregation unspecified)59.259.0+0.2Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
GLM-4.5-Air Z.aiSWE-bench Verified57.657.4+0.2GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · SERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)
LFM2.5 2.6B Liquid AIIFBench (aggregation unspecified)59.1759.0+0.17LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card
Qwen3 235B-A22B Instruct 2507 Alibaba CloudIFEval (aggregation unspecified)88.788.54+0.16Qwen/Qwen3-Next-80B-A3B-Instruct model card · Meituan LongCat-Flash-Chat model card (evaluation table)
Pythia 6.9B EleutherAIPIQA75.275.091+0.109Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Qwen3.5 4B Alibaba CloudSuperGPQA52.952.8+0.1Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
Nemotron 3 Ultra 550B-A55B NVIDIAHumanity's Last Exam26.726.6+0.1NVIDIA — Nemotron 3 Ultra technical report · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
LLaMA 65B MetaMMLU63.463.3+0.1LLaMA: Open and Efficient Foundation Language Models — authors' own runs · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runs
Qwen3 235B-A22B Thinking 2507 Alibaba CloudHMMT 2025 (edition unspecified)83.983.8+0.1Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Mistral 7B v0.1 (base) Mistral AIHellaSwag81.1581.06+0.09Mistral 7B — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Pythia 2.8B EleutherAILAMBADA (OpenAI version), last-word accuracy64.764.66+0.04Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · RedPajama — original zero-shot 3B comparison
GPT-5.6 Sol OpenAITerminal-Bench 4.037.337.27+0.03OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Terminal-Bench 4.0 official leaderboard submissions
GPT-NeoX-20B EleutherAILAMBADA (OpenAI version), last-word accuracy72.071.977+0.023GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3 235B-A22B Thinking 2507 Alibaba CloudMMLU-Pro84.484.40.0Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
DeepSeek V3 0324 DeepSeekAider polyglot (225 exercises, two attempts)55.155.10.0deepseek-ai/DeepSeek-V3.1 model card · Aider polyglot benchmark runs
Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba CloudAider polyglot (225 exercises, two attempts)59.659.60.0Qwen/Qwen3-30B-A3B-Instruct-2507 model card · Aider polyglot benchmark runs
Qwen3 32B Alibaba CloudAider polyglot (225 exercises, two attempts)40.040.00.0Qwen/Qwen3-Next-80B-A3B-Instruct model card · Aider polyglot benchmark runs
Qwen3 4B Instruct 2507 Alibaba Cloudτ²-bench Telecom13.213.20.0Qwen/Qwen3-4B-Instruct-2507 model card · Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)
Qwen3 235B-A22B Thinking 2507 Alibaba Cloudτ²-bench Airline58.058.00.0Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Qwen3.5 2B Alibaba CloudBFCL V4 (December 2025 archive)43.643.60.0Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
Qwen3.5 397B-A17B Alibaba CloudHumanity's Last Exam with tools48.348.30.0Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Qwen3.5 9B Alibaba CloudMMLU-Redux91.191.10.0Qwen/Qwen3.5-9B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card
Pythia 12B EleutherAISciQ90.290.20.0Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Magicoder-S-DS 6.7B ise-uiucHumanEval76.876.80.0Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs · EvalPlus / EvalPerf published evaluation runs
Qwen2.5 Coder 32B Instruct Alibaba CloudHumanEval+87.287.20.0Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs
CodeQwen1.5 7B Chat Alibaba CloudHumanEval83.583.50.0Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs
CodeQwen1.5 7B Chat Alibaba CloudHumanEval+78.778.70.0Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs
Command A+ (05-2026) CohereTerminal-Bench Hard25.025.00.0Cohere vendor-reported benchmark scores for its own models (cohere.com blog) · Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)
Command R Plus (08-2024) CohereRULER 128k64.664.60.0Cohere — Command A technical report · NVIDIA RULER repository README results table
Gemma 3 12B GoogleMATH (subset unspecified)83.883.80.0Google DeepMind Gemma 3 270M IT model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
CodeGemma 7B (v1.0) GoogleHumanEval44.544.50.0CodeGemma: Open Code Models Based on Gemma — authors' own runs · EvalPlus / EvalPerf published evaluation runs
Code Llama 7B (base) MetaHumanEval33.533.50.0Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs
gpt-oss-20b OpenAISimpleQA6.76.70.0OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
GPT-Neo 1.3B EleutherAIWinoGrande55.055.015-0.015EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · What Language Model to Train if You Have One Million GPU Hours? — authors' own runs
Nova Pro AmazonBFCL v360.860.82-0.02Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports) · Berkeley Function Calling Leaderboard result archive
GPT-Neo 2.7B EleutherAILAMBADA (OpenAI version), last-word accuracy62.262.235-0.035EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · RedPajama — original zero-shot 3B comparison
GPT-NeoX-20B EleutherAIWinoGrande66.166.14-0.04GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Flan-T5 XXL GoogleMMLU55.155.145-0.045Scaling Instruction-Finetuned Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
GPT-Neo 1.3B EleutherAIMathQA24.0524.1-0.05EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs · What Language Model to Train if You Have One Million GPU Hours? — authors' own runs
DeepSeek R1 (Jan '25) DeepSeekIFEval prompt-level strict83.383.36-0.06deepseek-ai/DeepSeek-R1 model card · OpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file
Pythia 2.8B EleutherAIPIQA73.973.99-0.09Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · RedPajama — original zero-shot 3B comparison
GLM-5.1 Z.aiAIME 202695.395.4-0.1zai-org/GLM-5.1 model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
LLaMA 33B MetaMMLU57.857.9-0.1LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
Code Llama 34B (base) MetaCRUXEval-O42.442.5-0.1CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Qwen3.5 35B-A3B Alibaba CloudAIME 202691.091.1-0.1Qwen/Qwen3.6-35B-A3B model card · NVIDIA — Nemotron-Cascade 2 technical report
OPT-30B MetaPIQA77.577.6-0.1OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Pythia 12B EleutherAIWinoGrande63.964.009-0.109Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
LLaMA 65B MetaWinoGrande77.077.11-0.11LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
GPT-5.6 Sol OpenAIFrontierMath tiers 1–3 v289.089.123-0.123OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Seed-OSS 36B Instruct ByteDance SeedGPQA Diamond71.471.528-0.128ByteDance-Seed/Seed-OSS-36B-Instruct model card · Epoch AI — internal evaluations
Pythia 12B EleutherAILAMBADA (OpenAI version), last-word accuracy70.570.638-0.138Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
LLaMA 7B MetaHumanEval10.510.65-0.15LLaMA: Open and Efficient Foundation Language Models — authors' own runs · InstructEval (DeCLaRe Lab) README results table
Code Llama 13B (base) MetaCRUXEval-O39.73839.9-0.163CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Qwen3 235B-A22B Thinking 2507 Alibaba CloudAIME 202592.392.5-0.2Qwen/Qwen3-235B-A22B-Thinking-2507 model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
GPT-J 6B EleutherAIOpenBookQA28.829.0-0.2GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Kimi K2 Instruct (0711) Moonshot AIτ²-bench Retail70.670.8-0.2τ²-bench leaderboard: Moonshot AI's own submission · Meituan LongCat-Flash-Chat model card (evaluation table)
OPT-1.3B MetaWinoGrande59.559.724-0.224OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Code Llama 7B (base) MetaCRUXEval-I35.9536.2-0.25CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Gemma 3 4B GoogleMMLU58.158.35-0.25Gemma Team — Gemma 3 Technical Report · LFM2 Technical Report — Liquid AI own SmolLM3 evaluation
Pythia 12B EleutherAIPIQA76.076.279-0.279Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
DeepSeek R1 0528 Qwen3 8B DeepSeekGPQA Diamond61.161.4-0.3deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
GPT-NeoX-20B EleutherAISciQ92.893.1-0.3GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Phi-3.5-vision-instruct MicrosoftChartQA81.882.1-0.3Microsoft Phi-3.5-vision-instruct model card: Benchmarks table · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Gemma 3 1B GoogleBIG-Bench Hard39.139.4-0.3Google DeepMind Gemma 3 270M IT model card · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
gpt-oss-20b OpenAIHumanity's Last Exam10.911.25-0.35OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
DeepSeek R1 (Jan '25) DeepSeekAIME 202570.070.4-0.4deepseek-ai/DeepSeek-R1-0528 model card · Microsoft — Phi-4-reasoning Technical Report
Qwen3.5 2B Alibaba CloudMMLU-Redux79.680.0-0.4Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
DeepSeek V3 (Dec '24) DeepSeekAider polyglot (225 exercises, two attempts)49.650.0-0.4deepseek-ai/DeepSeek-R1 model card · Aider polyglot benchmark runs
Phi-1.5 (1.3B) MicrosoftOpenBookQA37.237.6-0.4Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
LLaMA 33B MetaGSM8K35.636.0-0.4LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
GLM-4.5 Z.aiHumanity's Last Exam14.414.8-0.4GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Pythia 6.9B EleutherAIWinoGrande60.961.308-0.408Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
GPT-5 OpenAIGPQA Diamond85.786.174-0.474OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Granite 4.0 H 1B IBMBFCL v350.2150.69-0.48ibm-granite/granite-4.0-h-1b model card · LiquidAI/LFM2.5-1.2B-Thinking model card
Qwen3.5 397B-A17B Alibaba CloudMMLU-Pro87.888.3-0.5Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Qwen3 30B-A3B Thinking 2507 Alibaba CloudSuperGPQA56.857.3-0.5Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
GPT-J 6B EleutherAISciQ91.091.5-0.5GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Code Llama 34B (base) MetaCRUXEval-I47.21347.8-0.587CRUXEval published evaluation results (authors' generations and scores) · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
Qwen3.5 4B Alibaba CloudGPQA Diamond76.276.8-0.6Qwen/Qwen3.5-4B model card · JetBrains/Mellum2-12B-A2.5B-Instruct model card
Qwen3 30B-A3B Thinking 2507 Alibaba CloudMMMLU78.479.0-0.6Qwen/Qwen3.5-9B model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Qwen3 32B Alibaba CloudSuperGPQA48.6549.3-0.65Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
GLM-5.2 Z.aiGPQA Diamond91.291.856-0.656zai-org/GLM-5.2 model card · Epoch AI — internal evaluations
Pythia 2.8B EleutherAIWinoGrande59.460.06-0.66Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · RedPajama — original zero-shot 3B comparison
Gemma 3 1B GoogleMMLU38.839.49-0.69Gemma Team — Gemma 3 Technical Report · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GPT-4o mini (2024-07-18) OpenAIGPQA Diamond (simple-evals)40.240.9-0.7OpenAI simple-evals README benchmark results (OpenAI's own models) · Microsoft Research — Phi-4 Technical Report
Qwen3.5 4B Alibaba CloudOJBench24.124.8-0.7Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
LFM2.5 8B-A1B Liquid AIBFCL V4 (December 2025 archive)48.549.2-0.7LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
Qwen3-VL 2B Instruct Alibaba CloudRealWorldQA63.964.6-0.7Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Pythia 6.9B EleutherAISciQ89.790.45-0.75Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
GPT-Neo 2.7B EleutherAIPubMedQA57.5458.3-0.76EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs
DeepSeek V3.1 DeepSeekAIME 202493.193.9-0.8deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
GPT-NeoX-20B EleutherAIOpenBookQA29.029.8-0.8GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
StarCoder 15.5B BigCodeHumanEval33.634.45-0.85StarCoder: may the source be with you! — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
DeepSeek R1 (Jan '25) DeepSeekGPQA Diamond71.572.359-0.859deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations
o3 OpenAIGPQA Diamond81.081.859-0.859OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Phi-3.5-vision-instruct MicrosoftAI2D test78.179.0-0.9Microsoft Phi-3.5-vision-instruct model card: Benchmarks table · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Qwen3.5 4B Alibaba CloudBFCL V4 (December 2025 archive)50.351.28-0.98Qwen/Qwen3.5-9B model card · openbmb/MiniCPM5-2B model card
Qwen3 30B-A3B Thinking 2507 Alibaba CloudMMLU-Pro80.981.9-1.0Qwen/Qwen3-30B-A3B-Thinking-2507 model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Qwen3 32B Alibaba Cloudτ²-bench Telecom26.327.3-1.0Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Inkling (975B-A41B) Thinking Machines LabGPQA Diamond87.288.258-1.058thinkingmachines/Inkling-Small model card · Epoch AI — internal evaluations
GPT-5.6 Terra OpenAIFrontierMath tiers 1–3 v284.985.965-1.065OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Granite 4.2 3B IBMAIME 202578.3379.4-1.07ibm-granite/granite-4.2-30b model card · openbmb/MiniCPM5-2B model card
Qwen3.5 2B Alibaba CloudSuperGPQA37.538.6-1.1Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
DeepSeek V4 Flash (0731) DeepSeekGPQA Diamond89.991.035-1.135deepseek-ai/DeepSeek-V4.1-Flash model card · Epoch AI — internal evaluations
Jamba 1.5 Large AI21 LabsMMLU-Pro48.349.46-1.16Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1) · MMLU-Pro author-released evaluation runs
Qwen3.5 397B-A17B Alibaba CloudAIME 202691.392.5-1.2Qwen/Qwen3.5-397B-A17B model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
Qwen3 30B-A3B Thinking 2507 Alibaba CloudMMLU-ProX76.477.6-1.2Qwen/Qwen3-Next-80B-A3B-Thinking model card · NVIDIA — Nemotron 3 Nano technical report
GPT-Neo 2.7B EleutherAIWinoGrande56.557.7-1.2EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs
Code Llama 7B Python MetaHumanEval38.439.6-1.2Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
Code Llama 34B (base) MetaHumanEval48.850.0-1.2Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs
Gemma 7B (base) GoogleHumanEval32.333.55-1.25CodeGemma: Open Code Models Based on Gemma — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
LLaMA 33B MetaMATH7.18.4-1.3LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
GLM-4.5 Z.aiBFCL v377.879.1-1.3GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Meituan LongCat-Flash-Thinking model card (evaluation table)
Mistral 7B v0.1 (base) Mistral AIMMLU61.362.69-1.39Mistral 7B — authors' own runs · MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)
Code Llama 70B Instruct MetaHumanEval67.869.25-1.45Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
Gemma 2B (base) GoogleHumanEval22.023.5-1.5CodeGemma: Open Code Models Based on Gemma — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
OPT-175B MetaPIQA79.581.07-1.57OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
Qwen3 1.7B Alibaba CloudIFEval (aggregation unspecified)72.574.09-1.59Qwen/Qwen3.5-2B model card · HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
Qwen3.5 2B Alibaba CloudOCRBench84.586.1-1.6Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Qwen3.5 397B-A17B Alibaba CloudSWE-bench Multilingual69.370.9-1.6Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Granite 4.0 H Small (32B-A9B) IBMMMLU-Pro55.4757.1-1.63ibm-granite/granite-4.0-h-small model card · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
Qwen3 4B Alibaba CloudBFCL v361.7563.39-1.64Qwen/Qwen3-4B-Thinking-2507 model card · Berkeley Function Calling Leaderboard result archive
Qwen3 235B-A22B Thinking 2507 Alibaba Cloudτ²-bench Telecom45.647.3-1.7Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Qwen3.5 397B-A17B Alibaba CloudMMLU-ProX84.786.4-1.7Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Qwen3.6 27B Alibaba CloudIFBench (aggregation unspecified)69.170.8-1.7Qwen/Qwen3.8-27B model card · meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)
Nemotron Nano 9B v2 NVIDIAMMLU82.684.3-1.7NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models · Olmo 3
Kimi K2 Instruct (0711) Moonshot AIτ²-bench Telecom65.867.5-1.7τ²-bench leaderboard: Moonshot AI's own submission · Meituan LongCat-Flash-Chat model card (evaluation table)
gpt-oss-20b OpenAIMMMLU75.777.4-1.7OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
BLOOM 560M BigScienceWinoGrande49.8851.601-1.721Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
CodeQwen1.5 7B Chat Alibaba CloudMBPP+67.269.0-1.8Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs
LLaMA 33B MetaHumanEval21.723.5-1.8LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
GLM-5 Z.aiGPQA Diamond86.087.817-1.817GLM-5 Team (Zhipu AI) — GLM-5 technical report · Epoch AI — internal evaluations
Qwen3 235B-A22B Instruct 2507 Alibaba CloudMMLU-Pro83.084.83-1.83Qwen/Qwen3-235B-A22B-Instruct-2507 model card · Meituan LongCat-Flash-Chat model card (evaluation table)
Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba CloudBFCL v363.8565.72-1.87Qwen/Qwen3-4B-Thinking-2507 model card · Berkeley Function Calling Leaderboard result archive
Qwen2.5 Coder 32B Instruct Alibaba CloudMBPP+75.177.0-1.9Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · EvalPlus / EvalPerf published evaluation runs
Pythia 1B EleutherAISciQ84.086.0-2.0Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Qwen3 8B Alibaba CloudLiveCodeBench v6 (window unspecified)51.053.0-2.0Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Code Llama 34B (base) MetaMBPP55.057.0-2.0Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs
OPT-1.3B MetaCOPA77.079.0-2.0OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runs
DeepSeek R1 Distill Qwen 32B DeepSeekGPQA Diamond62.164.141-2.041deepseek-ai/DeepSeek-R1 model card · Epoch AI — internal evaluations
Claude Fable 5.1 AnthropicTerminal-Bench 4.055.857.88-2.08Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions
Claude Sonnet 5 AnthropicTerminal-Bench 4.010.312.42-2.12Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions
GLM-5.1 Z.aiGPQA Diamond86.288.349-2.149zai-org/GLM-5.1 model card · Epoch AI — internal evaluations
Qwen3.5 397B-A17B Alibaba CloudIMO-AnswerBench80.983.1-2.2Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
Gemma 2B (base) GoogleNatural Questions (closed book)12.514.76-2.26Gemma report — original evaluations · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
Qwen3.5 397B-A17B Alibaba CloudIFBench (aggregation unspecified)76.578.8-2.3Qwen/Qwen3.5-397B-A17B model card · A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
Gemma 4 E2B GoogleAIME 202637.539.8-2.3Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card
Qwen3 32B Alibaba CloudMMLU-Pro75.577.85-2.35Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
EXAONE 4.0 32B LG AI Researchτ²-bench Telecom23.726.1-2.4LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards) · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
GPT-5.6 Terra OpenAIFrontierMath tier 4 v268.370.732-2.432OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Llama 3.1 70B Instruct MetaGSM8K92.0494.5-2.46Puzzle: Distillation-Based NAS for Inference-Optimized LLMs (arXiv:2411.19146v5) · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GPT-5.6 Luna OpenAIFrontierMath tier 4 v258.560.976-2.476OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Gemma 4 E4B GoogleAIME 202642.545.0-2.5Gemma Team — Gemma 4 Technical Report · openbmb/MiniCPM5-2B model card
GPT-J 6B EleutherAIARC Challenge34.036.519-2.519GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Claude Fable 5 AnthropicTerminal-Bench 4.042.044.55-2.55Anthropic system cards and model announcements (lab-reported scores) · Terminal-Bench 4.0 official leaderboard submissions
BLOOM 176B BigScienceCOPA55.057.6-2.6Crosslingual Generalization through Multitask Finetuning — authors' own runs · From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
Qwen3 32B Alibaba CloudBFCL v366.6569.25-2.6Qwen/Qwen3-Next-80B-A3B-Thinking model card · Berkeley Function Calling Leaderboard result archive
LFM2 8B-A1B Liquid AIGPQA Diamond29.2931.9-2.61LFM2 Technical Report — Liquid AI own SmolLM3 evaluation · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
Gemma 4 E2B Googleτ²-bench Telecom19.722.37-2.67Gemma Team — Gemma 4 Technical Report · LiquidAI/LFM2.5-8B-A1B model card
GLM-4.5-Air Z.aiHumanity's Last Exam10.613.3-2.7GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical report · Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
Phi-1.5 (1.3B) MicrosoftSocial IQA52.655.374-2.774Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
GPT-NeoX-20B EleutherAIARC Challenge38.040.785-2.785GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Gemma 3 1B GoogleGPQA Diamond19.222.095-2.895Google DeepMind Gemma 3 270M IT model card · Epoch AI — internal evaluations
GPT-4.1 OpenAIτ²-bench Telecom34.036.9-2.9OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs
Mistral Small 3.2 24B Mistral AIGPQA Diamond46.1349.053-2.923mistralai/Mistral-Small-3.2-24B-Instruct-2506 model card · Epoch AI — internal evaluations
GLM-5.1 Z.aiIMO-AnswerBench83.886.8-3.0zai-org/GLM-5.2 model card · NVIDIA — Nemotron 3 Ultra technical report
Code Llama 70B Python MetaHumanEval57.360.4-3.1Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
DeepSeek R1 Distill Qwen 32B DeepSeekAIME 202472.675.7-3.1deepseek-ai/DeepSeek-R1 model card · open-thoughts/OpenThinker2-32B model card
Phi-4-reasoning-plus 14B MicrosoftMMLU-Pro76.079.2-3.2microsoft/Phi-4-reasoning model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
Pythia 12B EleutherAIARC Challenge31.835.068-3.268Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
LFM2.5 8B-A1B Liquid AIAIME 202542.5346.0-3.47LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
GPT-5.6 Luna OpenAIFrontierMath tiers 1–3 v278.682.105-3.505OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Epoch AI — internal evaluations
Llama 3.1 70B Instruct MetaMMLU81.6685.2-3.54Puzzle: Distillation-Based NAS for Inference-Optimized LLMs (arXiv:2411.19146v5) · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
DeepSeek R1 (Jan '25) DeepSeekAider polyglot (225 exercises, two attempts)53.356.9-3.6deepseek-ai/DeepSeek-R1-0528 model card · Aider polyglot benchmark runs
Phi-1.5 (1.3B) MicrosoftARC Challenge44.448.038-3.638Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
DeepSeek V3 (Dec '24) DeepSeekSWE-bench Verified42.045.8-3.8deepseek-ai/DeepSeek-R1 model card · Cohere — Command A technical report
Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba Cloudτ²-bench Airline33.036.8-3.8Qwen/Qwen3-235B-A22B-Thinking-2507 model card · HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
Qwen3 235B-A22B Thinking 2507 Alibaba CloudBFCL v371.975.7-3.8Qwen/Qwen3-Next-80B-A3B-Thinking model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
BLOOM 1.1B BigScienceWinoGrande50.9954.816-3.826Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Qwen3 4B Alibaba CloudIFEval (aggregation unspecified)81.5585.4-3.85Qwen/Qwen3-4B-Thinking-2507 model card · HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
DeepSeek V3.1 DeepSeekGPQA Diamond80.184.2-4.1deepseek-ai/DeepSeek-V3.1 model card · Meituan LongCat-Flash-Thinking model card (evaluation table)
Mistral 7B v0.1 (base) Mistral AINatural Questions (closed book)26.030.11-4.11Mixtral of Experts — base-model evaluations · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
LFM2.5 2.6B Liquid AIBFCL V4 (December 2025 archive)56.8861.1-4.22LiquidAI/LFM2.5-2.6B model card · openbmb/MiniCPM5-2B model card
Code Llama 13B (base) MetaHumanEval36.040.25-4.25Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs
CodeGemma 7B IT (v1.0) GoogleHumanEval56.160.4-4.3CodeGemma: Open Code Models Based on Gemma — authors' own runs · EvalPlus / EvalPerf published evaluation runs
Qwen3 32B Alibaba CloudIFEval (aggregation unspecified)84.188.4-4.3Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Gemma 3 12B GoogleMMLU71.976.2-4.3Gemma Team — Gemma 3 Technical Report · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
LFM2.5 8B-A1B Liquid AIMATH-50088.7693.2-4.44LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
GLM-4.5V Z.aiCharXiv (reasoning questions)58.463.0-4.6Z.ai GLM-V GitHub repository benchmark image (GLM-4.6V) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card
Mistral 7B v0.1 (base) Mistral AITriviaQA66.270.86-4.66Mistral 7B — authors' own runs · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
Qwen3.5 2B Alibaba CloudIFBench (aggregation unspecified)41.346.0-4.7Qwen/Qwen3.5-2B model card · openbmb/MiniCPM5-2B model card
DeepSeek V3.2 DeepSeekτ²-bench aggregate80.385.2-4.9deepseek-ai/DeepSeek-V3.2 model card · stepfun-ai/Step-3.5-Flash model card
Gemma 3 27B GoogleMMLU76.981.8-4.9Gemma Team — Gemma 3 Technical Report · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
GPT-4.1 mini OpenAIτ²-bench Telecom44.048.9-4.9OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs
Qwen3-VL 2B Instruct Alibaba CloudIFEval (aggregation unspecified)68.273.4-5.2Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
Gemma 3 27B GoogleGPQA Diamond42.447.727-5.327Gemma Team — Gemma 4 Technical Report · Epoch AI — internal evaluations
o4-mini OpenAIτ²-bench Telecom40.546.1-5.6OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs
Qwen3.5 397B-A17B Alibaba CloudLongBench v263.268.9-5.7Qwen/Qwen3.5-397B-A17B model card · NVIDIA — Nemotron 3 Ultra technical report
o3 OpenAIτ²-bench Telecom58.264.0-5.8OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · Meituan LongCat-Flash-Thinking model card (evaluation table)
OPT-175B MetaARC Easy67.072.97-5.97OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
BLOOM 560M BigScienceCOPA55.061.0-6.0Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
OPT-30B MetaARC Easy63.969.996-6.096OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Code Llama 13B Instruct MetaHumanEval42.68348.8-6.117Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
OPT-1.3B MetaARC Easy50.857.0-6.2OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
LLaMA 7B MetaHellaSwag66.572.99-6.49LLaMA: Open and Efficient Foundation Language Models — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
LFM2.5 8B-A1B Liquid AIAIME 202650.056.7-6.7LiquidAI/LFM2.5-8B-A1B model card · openbmb/MiniCPM5-2B model card
Gemma 7B (base) GoogleGSM8K46.453.3-6.9Gemma report — original evaluations · Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
LFM2 8B-A1B Liquid AIMMLU64.8472.1-7.26LFM2 Technical Report — Liquid AI own SmolLM3 evaluation · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
GPT-J 6B EleutherAIHellaSwag58.9566.242-7.292EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs · Stability AI — original StableLM zero-shot evaluation outputs
Mistral 7B v0.1 (base) Mistral AIARC Challenge55.262.541-7.341Mistral 7B — authors' own runs · MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)
Qwen3.5 2B Alibaba CloudHallusionBench58.065.5-7.5Qwen/Qwen3.5-2B model card · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
SmolLM2 1.7B Instruct Hugging FaceBIG-Bench Hard32.239.8-7.6SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations · 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models
BLOOM 1.7B BigScienceWinoGrande49.5757.171-7.601Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
LLaMA 13B MetaHellaSwag69.5577.64-8.09LLaMA: Open and Efficient Foundation Language Models — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Pythia 6.9B EleutherAIARC Challenge31.639.712-8.112Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Gemma 4 E4B Googleτ²-bench Telecom18.426.75-8.35Gemma Team — Gemma 4 Technical Report · LiquidAI/LFM2.5-8B-A1B model card
Code Llama 7B Instruct MetaHumanEval34.75643.3-8.544Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
Qwen3 32B Alibaba CloudLiveCodeBench v6 (Feb–May 2025)44.8553.4-8.55Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
LLaMA 13B MetaSocial IQA50.458.98-8.58LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate — authors' own runs
BLOOM 3B BigScienceWinoGrande49.9658.6-8.64Crosslingual Generalization through Multitask Finetuning — authors' own runs · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs
Pythia 1B EleutherAIARC Challenge24.433.1-8.7Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Llama 3.1 Tulu 3 8B Ai2TruthfulQA MC255.064.1-9.12 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · microsoft/Phi-4-mini-instruct model card
Pythia 6.9B EleutherAIOpenBookQA25.235.4-10.2trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
Code Llama 34B Instruct MetaHumanEval41.551.8-10.3Code Llama: Open Foundation Models for Code — authors' own runs · Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
LLaMA 33B MetaMBPP30.240.9-10.7LLaMA: Open and Efficient Foundation Language Models — authors' own runs · Mixtral of Experts — base-model evaluations
Code Llama 7B (base) MetaMBPP41.452.1-10.7Code Llama: Open Foundation Models for Code — authors' own runs · StarCoder 2 and The Stack v2: The Next Generation — authors' own runs
BLOOM 1.1B BigScienceCOPA57.068.0-11.0Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
Qwen3.6 27B Alibaba CloudOSWorld-Verified63.975.6-11.7Qwen/Qwen3.8-27B model card · meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)
BLOOM 560M BigScienceStory Cloze Test50.0861.94-11.86Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
BLOOM 3B BigScienceCOPA62.074.0-12.0Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
LFM2 8B-A1B Liquid AIMMLU-Pro37.4249.5-12.08LFM2 Technical Report — Liquid AI own SmolLM3 evaluation · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
BLOOMZ 176B BigScienceWinoGrande58.4172.5-14.09Crosslingual Generalization through Multitask Finetuning — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
Qwen3 32B Alibaba Cloudτ-bench Airline23.038.0-15.0Qwen/Qwen3-Next-80B-A3B-Thinking model card · ByteDance-Seed/Seed-OSS-36B-Instruct model card
Phi-1.5 (1.3B) MicrosoftHellaSwag47.662.617-15.017Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Pythia 6.9B EleutherAIHellaSwag48.863.841-15.041trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runs · OLMo: Accelerating the Science of Language Models — authors' own runs
BLOOM 7.1B BigScienceWinoGrande49.4164.641-15.231Crosslingual Generalization through Multitask Finetuning — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3-VL 2B Instruct Alibaba CloudHallusionBench51.467.3-15.9Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
BLOOM 1.7B BigScienceCOPA54.070.0-16.0Crosslingual Generalization through Multitask Finetuning — authors' own runs · EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)
BLOOM 1.1B BigScienceStory Cloze Test47.2563.27-16.02Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
Qwen3 4B Alibaba CloudAIME 202542.3558.8-16.45Qwen/Qwen3-4B-Thinking-2507 model card · HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
BLOOM 1.7B BigScienceStory Cloze Test48.4865.44-16.96Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
GPT-NeoX-20B EleutherAIHellaSwag53.571.43-17.93GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Stability AI — original StableLM zero-shot evaluation outputs
Qwen3 32B Alibaba CloudHMMT 2025 (edition unspecified)30.6549.8-19.15Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
BLOOM 3B BigScienceStory Cloze Test47.5167.79-20.28Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
BLOOM 176B BigScienceWinoGrande48.6270.3-21.68Crosslingual Generalization through Multitask Finetuning — authors' own runs · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs
GPT-J 6B EleutherAITriviaQA17.039.32-22.32GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs · Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
BLOOM 7.1B BigScienceStory Cloze Test49.4471.99-22.55Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs
Qwen3 32B Alibaba CloudAIME 202546.5572.15-25.6Qwen/Qwen3-Next-80B-A3B-Thinking model card · Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
BLOOM 176B BigScienceStory Cloze Test50.9976.89-25.9Crosslingual Generalization through Multitask Finetuning — authors' own runs · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs

Every result links to its source on the model's page. The data (CC BY-SA 4.0) and code (MIT) are on GitHub.