whatmodel.sh · find the best AI models you can run

Best AI models under $50 per million tokens

A leaderboard of the best AI models that cost $50 or less per million tokens. GPT-6 Astra is #1 right now, followed by Claude Opus 5.5 and GPT-6.1 Sol. Models are ranked by one score that combines 472 published benchmarks. Filter by your computer or budget to find the best one you can use.

713 models · scores combine 472 benchmarks · updated how it works · find a leaderboard · benchmark your machine
task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]
Filter by your hardware and budget · up to $50.00/M · reset
+ more

Detect your exact chip or graphics card:

curl -G whatmodel.sh/ -d max=0 --data-urlencode "mac=$(sysctl -n hw.memsize machdep.cpu.brand_string)"Mac

curl -G whatmodel.sh/ -d max=0 --data-urlencode "ram=$(awk '/MemT/{print $2}' /proc/meminfo)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null)"Linux

curl.exe -G whatmodel.sh/ -d max=0 --data-urlencode "ram=$((gcim Win32_ComputerSystem).TotalPhysicalMemory)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>$null)"Windows

$ terminal

detecting your OS…

Run curl "whatmodel.sh/?max=50"to get this list in your terminal (add &format=json for agents).

ram[8][16][24][32][48][64][96][128][192][256]GB
gpu[apple silicon][graphics card][none]
price[local][$0.02][$0.1][$0.5][$2][$10][$50][any]max per M

detecting your OS…

Score vs API price

#modelscore↓where↓speed↓RAM↓context↓get itsources
#modelscore↓where↓speed↓RAM↓context↓get itsources
1GPT-6 Astra OpenAI87±1api $20.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
2Claude Opus 5.5 Anthropic87±1api $8.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
3GPT-6.1 Sol OpenAI85±1api $4.00/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Miðeind Icelandic LLM Leaderboard
4Claude Fable 5.1 Anthropic85±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
5Claude Sonnet 5.5 Anthropic84±1api $4.00/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
6Claude Opus 5 Anthropic83±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+11
7GPT-6 Sol OpenAI83±1api $4.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard+1
10GPT-5.6 Sol OpenAI82±1api $8.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+14
11Claude Fable 5 Anthropic81±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runs+5
14GPT-5.5 OpenAI80±1api $11.25/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAgents' Last Exam — authors' own fixed-harness model comparison+15
15MiMo-V2.6-Pro Xiaomi79±1api $0.54/M +local~443GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
16Hy4 preview (770B-A49B) Tencent79±1api $1.25/M +local~248GB1Mcloud.tencent.com ↗Tencent Hy4 preview model cardLMArena leaderboard dataset
17GPT-5.6 Terra OpenAI79±1api $4.50/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)+5
18Qwen3.8 Max (2.4T-A95B) Alibaba78±1api $3.00/M +local~380GB262kdeepinfra.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runs+2
19Claude Opus 4.8 Anthropic78±0api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+11
20GPT-5.4 OpenAI78±1api $5.63/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAgents' Last Exam — authors' own fixed-harness model comparison+14
21DeepSeek V4.1 Flash DeepSeek78±1api $0.26/M +local~537GB1Mapi-docs.deepseek.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
22Claude Haiku 5.5 Anthropic77±1api $0.20/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard dataset
23Gemini 3.8 Flash Google77±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+5
24Muse Spark 1.3 Meta77±1api $2.00/M1Mdev.meta.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
25Kimi K3 Moonshot AI77±1api $6.00/M +local~884GB1Mplatform.moonshot.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodeτ²/τ³-bench leaderboard: Sierra's own evaluation runs+6
27MiMo-V2.6-Flash (309B-A15B) Xiaomi77±1api $0.17/M +local~136GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
28GPT-6 Luna OpenAI77±1api $0.20/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
29GPT-5.6 Luna OpenAI77±1api $0.45/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
30Grok 4.6 SpaceXAI (xAI)77±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
31DeepSeek V4 Pro (0813) DeepSeek76±1api $1.63/M +local~1010GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party models+2
32Gemini 3.7 Flash Google76±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+3
33GLM-5.3 Z.ai76±1api $2.15/M +local~239GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
34Qwen3.8 Max (0902) Alibaba76±1api $3.00/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
36DeepSeek V4 Flash Vision (Exp) DeepSeek76±2local~104GB1Mllama.cpp › deepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardHow Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
37Qwen3.8-Flash-Next Alibaba75±1local~59GB262kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
38Qwen3.8-Omni-Flash Alibaba75±2†api $0.23/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
39Gemini 3.1 Pro (Preview) Google75±1api $4.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)meituan-longcat/LongCat-2.0 model cardAgents' Last Exam — authors' own fixed-harness model comparison+23
40Muse Spark 1.2 Meta75±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluations
41Claude Opus 4.7 Anthropic75±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAnthropic system cards and model announcements (lab-reported scores)Agents' Last Exam — authors' own fixed-harness model comparison+9
42Grok 4.5 SpaceXAI (xAI)75±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodeτ²/τ³-bench leaderboard: Sierra's own evaluation runs+1
43Muse Spark 1.1 Meta75±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsτ²/τ³-bench leaderboard: Sierra's own evaluation runsEpoch AI — internal evaluations
46Claude Sonnet 5 Anthropic75±1api $4.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
47Grok 4.7 SpaceXAI (xAI)75±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
48Claude Opus 4.6 Anthropic75±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAgents' Last Exam — authors' own fixed-harness model comparison+21
49GPT-5.3-Codex OpenAI74±1api $4.81/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
50DeepSeek V4 Flash (0731) DeepSeek74±1api $0.11/M +local~92GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardDaekeun Kim Korean LLM benchmark runsdeepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+2
51Qwen3.7 Max Alibaba74±1api $3.75/M1Mdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
52GLM-5.3-Flash Z.ai74±1api $0.24/M +local~103GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)+2
53Gemini 3.5 Flash Google74±1api $3.38/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Legal Research Bench (Vals AI, arXiv:2610.00609v1)Miðeind Icelandic LLM Leaderboard+3
54GPT-5.2 OpenAI74±1api $4.81/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+20
55Hunyuan Hy3 (295B-A21B) Tencent74±1api $0.23/M +local~96GB256kdeepinfra.com ↗tencent/Hy3 model cardTencent Hy4 preview model cardLMArena leaderboard datasetEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
56Gemini 3.6 Flash Google74±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 Flash+2
57GLM-5.2 Z.ai74±1api $2.15/M +local~121GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetszai-org/GLM-5.2 model cardLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+6
59Qwen3.8 27B Alibaba73±1api $1.60/M +local~18GB262kconsole.groq.com ↗Qwen/Qwen3.8-27B model cardLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+3
60Grok 4.20 Multi-Agent (0309) SpaceXAI (xAI)73±2api $1.56/M1Mdocs.x.ai ↗LMArena leaderboard dataset
61Kimi K2.6 Moonshot AI73±1api $1.71/M +local~317GB262kplatform.moonshot.ai ↗Epoch AI — internal evaluationsNVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparison+11
62GPT-5 Pro OpenAI73±1api $41.25/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
63LongCat 2.0 (1.6T-A48B) Meituan73±2api $1.30/M +local~2155GB1Mlongcat.chat ↗meituan-longcat/LongCat-2.0 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
65GPT-5 OpenAI72±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+22
66Qwen3.6 Max Preview Alibaba72±1api $2.92/M256kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
67Claude Sonnet 4.6 Anthropic72±1api $6.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonKnowledge Index of Noah's Ark (arXiv:2606.05104v2)Legal Research Bench (Vals AI, arXiv:2610.00609v1)+16
68Gemini 3 Flash (Preview) Google72±1api $1.13/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGoogle DeepMind Gemini model page and model cards (lab-reported scores)+6
69DeepSeek V4 Pro (Preview, Apr 2026) DeepSeek72±1local~604GBllama.cpp › Epoch AI — internal evaluationsNVIDIA — Nemotron 3 Ultra technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetLGAI-EXAONE/K-EXAONE-2.0-750B-A37B model cardLegal Research Bench (Vals AI, arXiv:2610.00609v1)+6
70Claude Opus 4.5 Anthropic72±1api $10.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsQwen/Qwen3.6-27B model cardAnthropic system cards and model announcements (lab-reported scores)+13
71Qwen3.7 Plus Alibaba72±1api $0.70/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LMArena leaderboard datasetSciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
72Kimi K2.7 Code Moonshot AI72±1api $1.71/M +local~359GB262kplatform.kimi.ai ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
73DeepSeek V4 Flash (Preview, Apr 2026) DeepSeek72±1local~103GBllama.cpp › NVIDIA — Nemotron 3 Ultra technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsSolar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)ai-sage/GigaChat3.5-432B-A28B-Reasoning model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
74Grok 4.20 (0309 reasoning) SpaceXAI (xAI)72±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsτ²/τ³-bench leaderboard: Sierra's own evaluation runsThe Token Games: Evaluating Language Model Reasoning with Puzzle Duels — authors' own tournamentWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
75Solar Open 2 (250B-A15B) Upstage72±1†local~527GB1Mhow to run › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)
77GPT-5.4 mini OpenAI71±1api $1.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsLegal Research Bench (Vals AI, arXiv:2610.00609v1)+6
78Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)71±2†api $0.090/M +local~75GB131kdeepinfra.com ↗inclusionAI/Ling-3.0-flash model card
79Mistral Large 4 (public preview) Mistral71±2api $2.06/M1Mdocs.mistral.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluations
80GPT-5.1 OpenAI71±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsHyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+7
81GLM-5.1 Z.ai71±1api $2.15/M +local~119GB200kdocs.z.ai ↗NVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetszai-org/GLM-5.1 model cardzai-org/GLM-5.2 model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1+11
82Qwen3.5 397B-A17B Alibaba71±0api $1.09/M +local~65GB262kdeepinfra.com ↗Qwen/Qwen3.5-397B-A17B model cardNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsQwen/Qwen3.6-27B model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard dataset+15
83Ornith 1.5 35B-A3B DeepReinforce71±2†local~22GB262kLM Studio › ornith-ai/Ornith-1.5-35B-A3B model card
84MiniMax-M3 MiniMax70±1api $0.48/M +local~137GB1Mdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Miðeind Icelandic LLM Leaderboard+2
85Grok 4.3 SpaceXAI (xAI)70±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
87o3-pro OpenAI70±2api $35.00/M200kdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
88KAT-Coder-V2.5-Dev (35B-A3B) Kuaishou (Kwaipilot)70±2†local~22GB262kLM Studio › Kwaipilot/KAT-Coder-V2.5-Dev model card
89Qwen3.6 Plus Alibaba70±1api $1.13/M1Mdocs.together.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Agents' Last Exam — authors' own fixed-harness model comparisonQwen/Qwen-AgentWorld-35B-A3B model cardBEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+1
90Inkling Small (266B-A12B) Thinking Machines70±1api $0.64/M +local~86GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetMiðeind Icelandic LLM LeaderboardCohere — North Small Translate (arXiv:2609.13916v1)
92Kimi K2.5 Moonshot AI70±1api $1.20/M +local~160GB262kdocs.aws.amazon.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetstepfun-ai/Step-3.5-Flash model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsQwen/Qwen3.5-397B-A17B model card+15
93GLM-5 Z.ai70±1api $1.55/M +local~119GB205kdocs.z.ai ↗zai-org/GLM-5.1 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGLM-5 Team (Zhipu AI) — GLM-5 technical reportEpoch AI — internal evaluationszai-org/GLM-5 model cardLMArena leaderboard dataset+11
95MiMo-V2.5-Pro (1T-A42B) Xiaomi70±1api $0.65/M +local~163GB1Mnovita.ai ↗LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardXiaomi MiMo official model cards and documentationGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
96Inkling (975B-A41B) Thinking Machines70±1api $1.73/M +local~156GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsMiðeind Icelandic LLM Leaderboard
97Muse Glimmer 30B Meta70±1api $0.64/M +local~19GB131kfireworks.ai ↗meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.8-27B model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Cohere — North Small Translate (arXiv:2609.13916v1)
98Nemotron 3 Ultra 550B-A55B NVIDIA69±1api $1.35/M +local~89GB1Mdocs.together.ai ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1Epoch AI — internal evaluationsLMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)
99Qwen3.6 27B Alibaba69±1api $1.20/M +local~18GB262kconsole.groq.com ↗Qwen/Qwen3.6-27B model cardEpoch AI — internal evaluationsQwen/Qwen3.8-27B model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+4
101A.X K2 (688B-A33B) SK Telecom69±1†local~1454GB262khow to run › A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
102o3 OpenAI69±1api $3.50/M200kplatform.openai.com ↗Epoch AI — internal evaluationsMeituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard dataset+11
103MiMo-V2.5 (310B-A15B) Xiaomi69±1api $0.21/M +local~100GB1Mnovita.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5 model cardAgents' Last Exam — authors' own fixed-harness model comparisonXiaomi MiMo vendor-reported benchmark scores for its own models (mimo.xiaomi.com launch pages)SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)
104Qwen3.5 Plus Alibaba69±2api $0.90/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
105Qwen3.5 122B-A10B Alibaba69±1api $1.10/M +local~75GB262kwww.alibabacloud.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.5-27B model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)Epoch AI — internal evaluations+5
106Gemini 3.5 Flash-Lite Google69±1api $0.85/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Miðeind Icelandic LLM Leaderboard
107GLM-4.7 Z.ai69±1api $1.00/M +local~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardGLM-5 Team (Zhipu AI) — GLM-5 technical reportzai-org/GLM-5 model cardLMArena leaderboard datasetEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+5
109GPT-5.4 nano OpenAI69±1api $0.46/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeT1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)+1
110Claude Sonnet 4.5 Anthropic69±1api $6.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+18
111Qwen3.5 27B Alibaba69±0api $0.84/M +local~18GB262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.6-27B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard dataset+8
112GLM-5V-Turbo Z.ai69±2api $1.90/M200kwww.siliconflow.com ↗LMArena leaderboard dataset
115Step 3.5 Flash (196B-A11B) StepFun68±1api $0.15/M +local~118GB256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsstepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
116DeepSeek V3.2 DeepSeek68±1api $0.29/M +local~110GB128kdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsdeepseek-ai/DeepSeek-V3.2 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsstepfun-ai/Step-3.5-Flash model card+15
120MiniMax-M2.7 MiniMax68±1api $0.53/M +local~75GB205kplatform.minimax.io ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard datasetinclusionAI/Ling-3.0-flash model cardQwen/Qwen-AgentWorld-35B-A3B model cardWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+2
121Qwen3.6 35B-A3B Alibaba68±1api $0.31/M +local~23GB262kdeepinfra.com ↗Qwen/Qwen3.6-27B model cardQwen/Qwen3.6-35B-A3B model cardEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
122Ornith 1.0 35B (35B-A3B) DeepReinforce68±1local~23GB262kLM Studio › ornith-ai/Ornith-1.5-35B-A3B model cardKwaipilot/KAT-Coder-V2.5-Dev model card
123Hy3 preview (295B-A21B) Tencent68±1local~629GBhow to run › tencent/Hy3 model cardLMArena leaderboard dataset
124GPT-5 mini OpenAI67±1api $0.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardDaekeun Kim Korean LLM benchmark runsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)LMArena leaderboard dataset+16
125Gemini 2.5 Pro Google67±1api $3.44/M1Mai.google.dev ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+16
127K-EXAONE 2.0 (750B-A37B) LG AI Research67±1†local~791GB262khow to run › LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card
129Kimi K2 Thinking Moonshot AI67±1api $1.07/M +local~160GB262kdocs.aws.amazon.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsstepfun-ai/Step-3.5-Flash model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
130Laguna M.1 Poolside67±2local~135GB262kLM Studio › LMArena leaderboard datasetPoolside — Laguna M.1/XS.2 Technical Report
131Qwen3.7 Flash Alibaba67±1api $0.055/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluations
132Step 3.7 Flash (198B-A11B) StepFun67±1api $0.44/M +local~104GB256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsinclusionAI/Ling-3.0-flash model card
133Gemma 4 31B Google67±1api $0.19/M +local~21GB262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportKwaipilot/KAT-Coder-V2.5-Dev model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetEpoch AI — internal evaluationsQwen/Qwen3.6-35B-A3B model card+20
134Gemini 3.1 Flash-Lite Google67±1api $0.56/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs — authors' own model runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
135Qwen3-Coder-Next (80B-A3B) Alibaba67±2api $0.60/M +local~27GB262kwww.alibabacloud.com ↗Vector Institute Evaluation Leaderboard (Inspect Evals runs)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs
138GigaChat 3.5 Ultra Reasoning (432B-A28B) Sber67±1†local~461GB262khow to run › ai-sage/GigaChat3.5-432B-A28B-Reasoning model card
139Qwen3.5 35B-A3B Alibaba67±0api $0.35/M +local~23GB262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardNVIDIA — Nemotron-Cascade 2 technical reportLMArena leaderboard dataset+12
140DeepSeek V3.1 Terminus DeepSeek67±1local~230GB164kllama.cpp › DeepSeek-V3.1-Terminus model cardLMArena leaderboard dataset
141DeepSeek V3.2 Exp DeepSeek67±1local~726GBhow to run › Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)LMArena leaderboard datasetAider polyglot benchmark runsBerkeley Function Calling Leaderboard result archiveWebApp1K models leaderboard (onekq-ai)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
142Qwen3 235B-A22B Thinking 2507 Alibaba66±1api $0.97/M +local~76GB131knovita.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-Next-80B-A3B-Thinking model cardQwen3-VL-235B-A22B-Thinking model cardQwen/Qwen3.5-27B model cardEpoch AI — internal evaluations+6
143Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)66±1api $0.28/M128kazure.microsoft.com ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runsBerkeley Function Calling Leaderboard result archiveWebApp1K models leaderboard (onekq-ai)+4
144o4-mini OpenAI66±1api $1.93/M200kplatform.openai.com ↗Epoch AI — internal evaluationsQwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)+12
145Qwen3.6 Flash Alibaba66±1api $0.56/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsSpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
146MiMo-V2-Flash (309B-A15B) Xiaomi66±1local~108GBllama.cpp › stepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
147GLM-4.6 Z.ai66±1api $1.00/M +local~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardzai-org/GLM-4.6 model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsBerkeley Function Calling Leaderboard result archiveWebApp1K models leaderboard (onekq-ai)+1
148Kimi K2 Instruct (0711) Moonshot AI66±1api $1.00/M +local~160GB131knovita.ai ↗Meituan LongCat-Flash-Chat model card (evaluation table)Aider polyglot benchmark runsMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)LMArena leaderboard datasetEQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archive+3
149o1 (2024-12-17) OpenAI66±1api $26.25/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results+13
150MiniMax-M2.5 MiniMax66±1api $0.53/M +local~75GB205kplatform.minimax.io ↗LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+3
151Qwen3-Max (2025-09-23) Alibaba66±1api $2.40/M262kwww.alibabacloud.com ↗Epoch AI — internal evaluationsτ²-bench leaderboard: Qwen (Alibaba) own submissionsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)
152DeepSeek V3.1 DeepSeek66±1api $0.42/M +local~110GB131kdeepinfra.com ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Meituan LongCat-Flash-Chat model card (evaluation table)deepseek-ai/DeepSeek-V3.1 model cardDeepSeek-V3.1-Terminus model cardLMArena leaderboard datasetRPC-Bench leaderboard (Z.ai research paper comprehension)+3
153Spark-X2.5 4B iFLYTEK (SparkLLM)66±1†local~3.9GB1MLM Studio › XHToken/Spark-X2.5-4B model card
154MiniMax-M2.1 MiniMax66±1local~79GBllama.cpp › stepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
155Nemotron Cascade 2 30B-A3B NVIDIA66±1local~20GB256kLM Studio › NVIDIA — Nemotron-Cascade 2 technical reportEuroEval leaderboards (results collection), English generative leaderboard
156Laguna XS 2.1 (33B-A3B) Poolside66±2†local~21GB262kLM Studio › Poolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)
157LongCat-Flash-Chat (560B-A27B) Meituan66±1local~333GB128kLM Studio › Meituan LongCat-Flash-Chat model card (evaluation table)LMArena leaderboard dataset
158o3-mini OpenAI65±1api $1.93/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardMicrosoft — Phi-4-reasoning Technical ReportOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runs+13
161MiMo-V2.6-Distill-Qwen-9B Xiaomi65±2†local~7.8GBLM Studio › XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B model card
162K-EXAONE (236B-A23B) LG AI Research65±1local~152GB262kllama.cpp › LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model cardLG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)
163DeepSeek R1 0528 DeepSeek65±1api $0.91/M +local~110GB164kdeepinfra.com ↗deepseek-ai/DeepSeek-R1-0528 model cardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)deepseek-ai/DeepSeek-V3.1 model cardServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+6
164EXAONE 4.5 33B LG AI Research65±2†local~24GB262kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
165Qwen3.5 Flash Alibaba65±1api $0.17/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
167INTELLECT-3 Prime Intellect65±1local~48GBllama.cpp › Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard dataset
168Nemotron 3 Super 120B-A12B NVIDIA65±1api $0.28/M +local~73GB262kdocs.aws.amazon.com ↗NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)NVIDIA — Nemotron-Cascade 2 technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetinclusionAI/Ling-3.0-flash model card
169GLM-4.5 Z.ai65±1api $1.00/M +local~58GB131kdocs.z.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)zai-org/GLM-4.6 model cardLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard results+3
170Solar Pro 4 Upstage65±2api $0.53/M524kdevelopers.upstage.ai ↗LMArena leaderboard dataset
171MiniMax-M2 MiniMax65±1api $0.53/M +local~75GB205kdocs.aws.amazon.com ↗LMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
172Laguna XS.2 Poolside65±2local~21GB262kLM Studio › Poolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)LMArena leaderboard datasetPoolside — Laguna M.1/XS.2 Technical Report
173Qwen3 235B-A22B Instruct 2507 Alibaba65±1api $0.20/M +local~76GB262kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMeituan LongCat-Flash-Chat model card (evaluation table)Qwen3-VL-235B-A22B-Instruct model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsLMArena leaderboard dataset+3
174Mistral Medium 3.5 (128B) Mistral65±1api $3.00/M +local~42GB262kdocs.mistral.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)Miðeind Icelandic LLM LeaderboardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
175Command A+ (05-2026) Cohere64±1local~71GB128kllama.cpp › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Cohere vendor-reported benchmark scores for its own models (cohere.com blog)Cohere — North Small Translate (arXiv:2609.13916v1)
176Claude Haiku 4.5 Anthropic64±1api $2.00/M200kdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+9
177Nemotron-Labs-3 Puzzle 75B-A9B NVIDIA64±1†local~35GBllama.cpp › NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
178Qwen3-VL 235B-A22B Thinking Alibaba64±1local~111GB256kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen/Qwen3.5-397B-A17B model cardQwen3-VL-235B-A22B-Thinking model cardLMArena leaderboard dataset
179Qwen3-Coder 480B-A35B Instruct Alibaba64±1api $0.67/M +local~78GB262knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
181Gemma 4 26B-A4B Google64±1api $0.14/M +local~18GB262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical Reportgoogle/diffusiongemma-26B-A4B-it model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+12
182Kimi K2 Instruct (0905) Moonshot AI64±2local~298GBllama.cpp › LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
183Qwen3-VL 235B-A22B Instruct Alibaba64±1local~134GB256kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen3-VL-235B-A22B-Instruct model cardLMArena leaderboard datasetChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
184Nova 2 Pro (Preview) Amazon64±2api $3.78/M1Maws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsAmazon AGI — τ²-Bench-Verified
186MiniCPM5 2B OpenBMB64±1†local~3GB131kLM Studio › openbmb/MiniCPM5-2B model card
188Qwen3-Next 80B-A3B Thinking Alibaba63±1api $0.41/M +local~27GB131kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3.5-9B model cardQwen/Qwen3.5-4B model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)+3
189DeepSeek R1 (Jan '25) DeepSeek63±1api $4.00/M +local~110GB128kdocs.together.ai ↗deepseek-ai/DeepSeek-R1 model carddeepseek-ai/DeepSeek-R1-0528 model cardEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsWildEval ZeroEval published results+15
191Qwen3-Next 80B-A3B Instruct Alibaba63±1api $0.34/M +local~27GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
192Granite 4.2 30B IBM63±1api $0.28/M +local~19GB131kdeepinfra.com ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
195GLM-4.5-Air Z.ai63±1api $0.42/M +local~35GB131kdocs.z.ai ↗GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetSolar Open Technical Report (arXiv:2601.07022v1)Berkeley Function Calling Leaderboard result archive+3
196Qwen3.5 9B Alibaba63±1api $0.11/M +local~7.4GB262kdeepinfra.com ↗Qwen/Qwen3.5-9B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+17
197Seed-OSS 36B Instruct ByteDance Seed63±1local~23GB524kLM Studio › ByteDance-Seed/Seed-OSS-36B-Instruct model cardEpoch AI — internal evaluations
199Trinity-Large-Thinking Arcee AI63±2api $0.39/M +local~141GB512kdocs.arcee.ai ↗LMArena leaderboard datasetArcee AI vendor-reported benchmark scores for its own Trinity models (Hugging Face model cards)
201MiniMax-M1 (80k) MiniMax62±1api $0.96/M +local~959GB1Mnovita.ai ↗MiniMaxAI/MiniMax-M1-80k model cardLMArena leaderboard dataset
202Gemma 4 12B Google62±1local~8.5GB262kLM Studio › Gemma Team — Gemma 4 Technical ReportXHToken/Spark-X2.5-4B model cardgoogle/gemma-4-31B-it model card
203GPT-4.1 OpenAI62±1api $3.50/M1Mplatform.openai.com ↗Meituan LongCat-Flash-Chat model card (evaluation table)Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsDaekeun Kim Korean LLM benchmark runsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+17
204Nemotron 3.5 Lightning 30B-A3B NVIDIA62±2api $0.087/M +local~20GB262kfireworks.ai ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard dataset
205GLM-4.6V Z.ai62±2api $0.45/M +local~51GB128kdocs.z.ai ↗LMArena leaderboard datasetZ.ai GLM-V GitHub repository benchmark image (GLM-4.6V)SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)
206Devstral 2 (2512) Mistral62±2local~75GB256kLM Studio › LMArena leaderboard datasetmistralai/Devstral-Small-2-24B-Instruct-2512 model cardDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs
208A.X K1 (519B-A33B) SK Telecom62±1†local~1091GBhow to run › A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
209gpt-oss-120b OpenAI62±1api $0.26/M +local~67GB131kconsole.groq.com ↗OpenAI — gpt-oss-120b & gpt-oss-20b Model CardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsAider polyglot benchmark runsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+19
210GPT-5 nano OpenAI62±1api $0.14/M400kplatform.openai.com ↗Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+7
211Qwen3-VL 32B Thinking Alibaba62±1api $0.28/M +local~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)allenai/Olmo-3.1-32B-Think model card
213Mistral Large 3 (675B-A41B) Mistral62±1api $0.75/M +local~108GB262kdocs.mistral.ai ↗LMArena leaderboard datasetmistralai/Mistral-Small-4-119B-2603 model cardCohere — North Small Translate (arXiv:2609.13916v1)LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? (arXiv:2610.04118v1)Miðeind Icelandic LLM LeaderboardOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+1
214Apriel 1.6 15B Thinker ServiceNow62±2†local~10GBLM Studio › ServiceNow-AI/Apriel-1.6-15b-Thinker model card
216Gemini 2.5 Flash Google62±1api $0.85/M1Mai.google.dev ↗Meituan LongCat-Flash-Chat model card (evaluation table)LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+10
219GLM-4.5V Z.ai61±2api $0.90/M +local~69GB64kdocs.z.ai ↗Z.ai GLM-V GitHub repository benchmark image (GLM-4.6V)LMArena leaderboard datasetServiceNow-AI/Apriel-1.6-15b-Thinker model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
220Llama 3.1 Nemotron Ultra 253B v1 NVIDIA61±2local~82GB128kllama.cpp › NVIDIA — Llama-Nemotron: Efficient Reasoning ModelsLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archiveLiveCodeBench official website results (2025-08-01)
222Devstral Small 2 24B Mistral61±3†local~17GB262kllama.cpp › mistralai/Devstral-Small-2-24B-Instruct-2512 model card
225Qwen3 30B-A3B Thinking 2507 Alibaba61±1api $0.75/M +local~20GB262kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardByteDance-Seed/Seed-OSS-36B-Instruct model cardQwen/Qwen3.5-9B model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+8
226Qwen3.5 4B Alibaba61±1local~4.3GB262kllama.cpp › Qwen/Qwen3.5-9B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardQwen/Qwen3.5-4B model cardLiquidAI/LFM2.5-2.6B model card+10
227Nemotron-Cascade-14B-Thinking NVIDIA61±1†local~10GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
228Mellum2.1 12B-A2.5B Thinking JetBrains60±2†local~8.3GBLM Studio › JetBrains/Mellum2.1-12B-A2.5B-Thinking model card
229GPT-4.1 mini OpenAI60±1api $0.70/M1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runsAider polyglot benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Arena-Hard-Auto v2.0-Preview leaderboard (repository README)+13
230DeepSeek V3 0324 DeepSeek60±1local~108GB164kllama.cpp › deepseek-ai/DeepSeek-V3.1 model cardEpoch AI — internal evaluationsAider polyglot benchmark runsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)LMArena leaderboard datasetMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)+11
231Qwen3 30B-A3B Instruct 2507 Alibaba60±1api $0.15/M +local~20GB262ktokenfactory.nebius.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
232Qwen3-VL 32B Instruct Alibaba60±1†api $0.28/M +local~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
233Falcon-H1R 7B TII60±1†local~6GB262kLM Studio › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
235ERNIE 4.5 300B-A47B Baidu60±1†local~97GB131kllama.cpp › ERNIE 4.5 post-trained 300B-A47B benchmark chart
237Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba60±1api $1.23/M +local~76GB131kwww.alibabacloud.com ↗Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+14
239Granite 4.2 8B IBM60±1api $0.11/M +local~6.6GB131kwandb.ai ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
240Solar Open 100B (102B-A12B) Upstage60±1†local~41GB131kllama.cpp › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Solar Open Technical Report (arXiv:2601.07022v1)
241Qwen3-VL 30B-A3B Thinking Alibaba60±1†api $0.75/M +local~21GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
242Mistral Small 4 (119B-A6.5B) Mistral60±2api $0.26/M +local~40GB262kdocs.mistral.ai ↗mistralai/Mistral-Small-4-119B-2603 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Miðeind Icelandic LLM Leaderboard
244QwQ-32B Alibaba59±1local~21GB131kLM Studio › Epoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsAider polyglot benchmark runsArena-Hard-Auto v2.0-Preview leaderboard (repository README)LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)+7
245OpenReasoning-Nemotron-32B NVIDIA59±2†local~21GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
246DeepSeek V3 (Dec '24) DeepSeek59±1api $0.46/M +local~337GB131kdeepinfra.com ↗deepseek-ai/DeepSeek-R1 model cardCohere — Command A technical reportEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsWildEval ZeroEval published results+10
247Gemma 2 9B IT SimPO Princeton NLP59±1local~7.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)
248Gemma 2 9B IT DPO (Princeton) Princeton NLP59±1local~7.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
249Trinity-Large-Preview Arcee AI59±2local~155GBllama.cpp › Arcee AI vendor-reported benchmark scores for its own Trinity models (Hugging Face model cards)LMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
250GLM-4.7-Flash Z.ai59±1api $0.0000/M +local~19GB200kdocs.z.ai ↗Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardEQ-Bench 3 canonical leaderboard resultsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
251EXAONE 4.0 32B LG AI Research59±1local~20GB131kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesEuroEval leaderboards (results collection), English generative leaderboard
252Phi-4-reasoning-plus 14B Microsoft59±1local~10GBLM Studio › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingMicrosoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
253Nemotron-Cascade-8B NVIDIA59±1†local~6.3GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
254Qwen3 14B Alibaba59±1api $0.61/M +local~10GB33kwww.alibabacloud.com ↗Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBerkeley Function Calling Leaderboard result archiveBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+6
257gpt-oss-20b OpenAI58±1api $0.13/M +local~14GB131kconsole.groq.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardOpenAI — gpt-oss-120b & gpt-oss-20b Model CardEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)+15
258Llama 3.3 Nemotron Super 49B v1 NVIDIA58±2local~22GBllama.cpp › NVIDIA — Llama-Nemotron: Efficient Reasoning ModelsLMArena leaderboard dataset
259Olmo 3.1 32B Think Ai258±1local~21GB66kLM Studio › Olmo 3Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetallenai/Olmo-3.1-32B-Think model cardEuroEval leaderboards (results collection), English generative leaderboard
260DeepSeek R1 Distill Llama 70B DeepSeek58±1api $0.80/M +local~30GB131knovita.ai ↗Epoch AI — internal evaluationsMicrosoft — Phi-4-reasoning Technical Reportdeepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Judgemark v2 results (EQ-Bench)+1
261Phi-4-reasoning 14B Microsoft58±1api $0.22/M +local~10GB33kazure.microsoft.com ↗Microsoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
262Olmo 3 32B Think Ai258±1local~21GB66kLM Studio › Olmo 3Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetallenai/Olmo-3.1-32B-Think model card
263Magistral Small 1.2 Mistral58±1api $0.75/M +local~16GB131kdocs.aws.amazon.com ↗mistralai/Magistral-Small-2509 model cardEpoch AI — internal evaluationsApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarksmistralai/Mistral-Small-4-119B-2603 model cardInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)EuroEval leaderboards (results collection), English generative leaderboard
264Nemotron-Cascade-8B-Thinking NVIDIA58±2†local~6.3GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
265Qwen3-VL 30B-A3B Instruct Alibaba58±1†api $0.35/M +local~21GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
266Qwen3-VL 8B Thinking Alibaba58±1api $0.66/M +local~6.8GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Think model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
267Qwen3 32B Alibaba58±0api $0.13/M +local~20GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-Next-80B-A3B-Instruct model cardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+17
268Qwen3 Coder 30B-A3B Instruct Alibaba58±1api $0.12/M +local~20GB160knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
269Nova 2 Lite Amazon58±2api $0.94/M1Mdocs.aws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveAmazon AGI — τ²-Bench-VerifiedVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
270Llama 3.3 Nemotron Super 49B v1.5 NVIDIA58±1local~18GB131kllama.cpp › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsLMArena leaderboard dataset
271Nemotron 3 Nano 30B-A3B NVIDIA58±1api $0.087/M +local~20GB262kdeepinfra.com ↗NVIDIA — Nemotron-Cascade 2 technical reportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)EuroEval leaderboards (results collection), English generative leaderboardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+3
272Ministral 3 14B Reasoning 2512 Mistral58±1local~10GB262kllama.cpp › mistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsJetBrains/Mellum2-12B-A2.5B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboardSurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
273GPT-4o (2024-08-06) OpenAI57±1api $4.38/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runs+23
275DeepSeek R1 0528 Qwen3 8B DeepSeek57±1api $0.068/M +local~6.3GB128knovita.ai ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scalingdeepseek-ai/DeepSeek-R1-0528-Qwen3-8B model cardEpoch AI — internal evaluationsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
276Granite 4.2 3B IBM57±1api $0.052/M +local~3.7GB131kdeepinfra.com ↗openbmb/MiniCPM5-2B model cardibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
277Command A (03-2025) Cohere57±1api $4.38/M +local~37GB256kazure.microsoft.com ↗Cohere — Command A technical reportLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
278GPT-4o (2024-11-20) OpenAI57±1api $4.38/M128kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Qwen/Qwen3-235B-A22B-Instruct-2507 model cardAider polyglot benchmark runsQwen/Qwen3-30B-A3B-Instruct-2507 model cardMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+7
279Gemini 2.5 Flash-Lite Google57±2api $0.17/M1Mai.google.dev ↗Google DeepMind Gemini model page and model cards (lab-reported scores)Berkeley Function Calling Leaderboard result archiveQwen/Qwen3.5-4B model cardMiðeind Icelandic LLM LeaderboardGalileo Agent Leaderboard (v1 and v2 results)Open Portuguese LLM Leaderboard (archived)+1
281Qwen2.5-Omni 7B Alibaba57±2local~6.4GBllama.cpp › AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
282GPT-4o (2024-05-13) OpenAI57±0api $7.50/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)OpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runs+19
283QED-Nano LM Provers57±2local~4.1GB262kllama.cpp › MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
284Llama 4 Maverick (402B-A17B) Meta57±1api $0.42/M +local~65GB1Mdocs.aws.amazon.com ↗Epoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarks+14
286Qwen2.5 Coder 32B Instruct Alibaba57±1api $0.74/M +local~21GB131kdevelopers.cloudflare.com ↗EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
287Qwen3 4B Thinking 2507 Alibaba57±1local~3.9GB262kLM Studio › Qwen/Qwen3-4B-Thinking-2507 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Qwen/Qwen3.5-0.8B model cardKnowledge Index of Noah's Ark (arXiv:2606.05104v2)+2
290Athene-V2-Chat 72B Nexusflow56±2local~44GB33kLM Studio › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Arena-Hard-Auto v2.0-Preview leaderboard (repository README)LMArena leaderboard datasetBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)
292Sky-T1 32B Preview NovaSky (UC Berkeley)56±5local~21GBLM Studio › EuroEval leaderboards (results collection), English generative leaderboardBerkeley Function Calling Leaderboard result archive
293Qwen Plus (2025-01-25) Alibaba56±1api $0.60/M131kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
295Qwen3-VL 4B Thinking Alibaba56±1local~4.1GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)
296EXAONE Deep 32B LG AI Research56±2†local~22GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
297DeepSeek R1 Distill Qwen 32B DeepSeek56±1api $1.59/M +local~21GB131kdevelopers.cloudflare.com ↗open-thoughts/OpenThinker2-32B model carddeepseek-ai/DeepSeek-R1 model cardopen-thoughts/OpenThinker-32B model cardEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+5
298Qwen2.5-VL 72B Instruct Alibaba56±1local~46GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2.5-VL-32B-Instruct model cardLMArena leaderboard datasetChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)+3
299Granite 4.1 30B IBM56±1†local~19GBOllama › ibm-granite/granite-4.1-8b model card
300Mistral Small 3.2 24B Mistral56±1api $0.11/M +local~16GB128kdeepinfra.com ↗mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetmistralai/Mistral-Small-4-119B-2603 model cardEQ-Bench 3 canonical leaderboard resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+4
301DeepSeek-V2.5 (1210) DeepSeek56±1local~82GBllama.cpp › Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpenCompass CompassAcademic leaderboard (full version) Space: committed predictions fileDeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)
302Qwen3 8B Alibaba56±1api $0.31/M +local~6.1GB33kwww.alibabacloud.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)allenai/Olmo-3-7B-Think model cardSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+14
303DeepSeek R1 Distill Qwen 14B DeepSeek56±1local~10GB131kLM Studio › Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)EuroEval leaderboards (results collection), English generative leaderboard+2
304Ministral 3 14B Mistral56±1api $0.20/M +local~9.1GB262kdocs.mistral.ai ↗JetBrains/Mellum2-12B-A2.5B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)mistralai/Ministral-3-8B-Instruct-2512 model card+1
305Nemotron Nano 9B v2 NVIDIA56±1api $0.10/M +local~7.6GB131kdocs.aws.amazon.com ↗Olmo 3NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)allenai/Olmo-3-7B-Think model cardEuroEval leaderboards (results collection), English generative leaderboard
307GPT-4 Turbo (2024-04-09) OpenAI55±0api $15.00/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)+18
309Qwen2.5 72B Instruct Alibaba55±1api $0.37/M +local~30GB33kdeepinfra.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical Report2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)+24
310Mistral Large 2 (2411) Mistral55±1local~78GB128kllama.cpp › Epoch AI — internal evaluationsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveEQ-Bench 3 canonical leaderboard resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+8
311Ministral 3 8B Reasoning 2512 Mistral55±1local~6.5GB262kLM Studio › mistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsSurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
312Magistral Small 1.0 (2506) Mistral55±1local~16GB128kLM Studio › mistralai/Magistral-Small-2509 model cardEpoch AI — internal evaluationsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Galileo Agent Leaderboard (v1 and v2 results)
313Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba55±1api $0.35/M +local~19GB33kwww.alibabacloud.com ↗Qwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsArena-Hard-Auto v2.0-Preview leaderboard (repository README)+10
314Molmo 2 8B Ai255±2local~6.8GBllama.cpp › Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)LMArena leaderboard dataset
317Qwen3 4B Instruct 2507 Alibaba55±1local~3.9GB262kLM Studio › Qwen/Qwen3-4B-Instruct-2507 model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+4
318Qwen2.5 Coder 14B Instruct Alibaba55±1local~10GB131kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsOpen Portuguese LLM Leaderboard (archived)
319Qwen3-VL 8B Instruct Alibaba55±1api $0.31/M +local~6.8GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Instruct model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
320Spark-X2.5 1.7B iFLYTEK (SparkLLM)55±1†local~2.7GB1Mllama.cpp › XHToken/Spark-X2.5-4B model card
321GigaChat 3.5 Ultra Instruct (432B-A28B) Sber55±1†local~457GB262khow to run › ai-sage/GigaChat3.5-432B-A28B-Reasoning model card
322Llama 3.1 405B Instruct Meta55±1api $2.40/M +local~160GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+13
323Qwen2.5 32B Instruct Alibaba55±1local~21GB131kLM Studio › WildEval ZeroEval published resultsEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+11
324Command A Reasoning (08-2025) Cohere55±2local~34GBllama.cpp › Cohere vendor-reported benchmark scores for its own models (cohere.com blog)Berkeley Function Calling Leaderboard result archive
325Bielik 11B v3.0 Instruct SpeakLeash55±2local~8.1GBLM Studio › Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
326Olmo 3.1 32B Instruct Ai255±2local~21GB66kLM Studio › Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
327OpenThinker 32B Open Thoughts55±1†local~21GB33kLM Studio › open-thoughts/OpenThinker2-32B model cardopen-thoughts/OpenThinker-32B model card
328Magistral Small 1.1 (2507) Mistral55±2†local~16GB40kLM Studio › mistralai/Magistral-Small-2509 model card
329EXAONE Deep 7.8B LG AI Research55±2†local~6.5GB33kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
330Qwen2.5-Math 72B Instruct Alibaba55±2†local~44GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
331OpenThinker2 32B Open Thoughts54±1local~21GB33kLM Studio › open-thoughts/OpenThinker2-32B model cardArena-Hard-Auto v2.0-Preview leaderboard (repository README)
332Falcon-H1 34B Instruct TII54±3local~23GB262kllama.cpp › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
333Llama 3.3 70B Instruct Meta54±1api $0.72/M +local~24GB128kdocs.aws.amazon.com ↗Cohere — Command A technical reportEpoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical Report2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+21
334Llama 4 Scout (109B-A17B) Meta54±1api $0.15/M +local~36GB328kdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveEQ-Bench 3 canonical leaderboard resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+8
335Nemotron 3 Nano 4B NVIDIA54±1local~3.8GB262kLM Studio › openbmb/MiniCPM5-2B model cardNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
336Granite 4.1 8B IBM54±1api $0.11/M +local~7GB131kreplicate.com ↗ibm-granite/granite-4.1-8b model cardLMArena leaderboard datasetSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)
337Pixtral Large (2411) Mistral54±3local~51GBllama.cpp › Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)LMArena leaderboard dataset
338Bielik 11B v2.6 Instruct SpeakLeash54±3local~8.1GBLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
339Nova Premier Amazon54±2†api $5.00/M1Maws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)MASSIVE-Agents — Kulkarni et al. (2025)
340Mellum2 12B-A2.5B Thinking JetBrains54±1†local~8.6GB131kLM Studio › JetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Thinking model card
341Mistral Large 2 (2407) Mistral54±1local~45GB128kllama.cpp › WildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)Epoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsJamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+10
342Mistral Small 3.1 24B (2503) Mistral54±1local~17GB128kllama.cpp › Epoch AI — internal evaluationsmistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardLMArena leaderboard datasetMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard results+7
346Llama 3.1 Tulu 3 70B Ai254±1local~43GB131kLM Studio › Tülu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., Ai2)LMArena leaderboard dataset
347DeepSeek-V2.5 DeepSeek54±1local~151GB164kOllama › WildEval ZeroEval published resultsAider code editing benchmark runs (old edit leaderboard)MMLU-Pro author-released evaluation runsLMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)OpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file+1
348DeepSeek-V2 Chat (0628) DeepSeek54±1local~66GBllama.cpp › WildEval ZeroEval published resultsAider code editing benchmark runs (old edit leaderboard)WildBench v2 leaderboard data (Ai2)
349GLM-4 9B Chat 1M Zhipu AI (THUDM)53±5local~8.1GB1Mllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
350Mistral Small 3 24B (2501) Mistral53±1local~16GB32kLM Studio › Epoch AI — internal evaluationsInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard resultsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+6
351Qwen2-VL 72B Instruct Alibaba53±1local~34GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2.5-VL-32B-Instruct model cardPixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column
353GPT-4o mini (2024-07-18) OpenAI53±0api $0.26/M128kdevelopers.openai.com ↗EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsmicrosoft/Phi-4-mini-instruct model cardVector Institute Evaluation Leaderboard (Inspect Evals runs)Epoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+32
355Llama 3.1 Tulu 3 70B DPO Ai253±1local~22GB131kllama.cpp › Epoch AI — internal evaluationsTülu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., Ai2)
357Qwen2.5-VL 32B Instruct Alibaba53±2local~22GB33kLM Studio › Qwen2.5-VL-32B-Instruct model cardApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarksLMArena leaderboard datasetChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark (arXiv:2508.13680v5)+1
358DeepSeek-Coder-V2 Instruct (0614) DeepSeek53±1local~92GBllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LiveCodeBench official website results (2024-12-26)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+2
359s1.1 32B Simple Scaling53±1local~22GB33kLM Studio › open-thoughts/OpenThinker2-32B model cardopen-thoughts/OpenThinker-32B model cardArena-Hard-Auto v2.0-Preview leaderboard (repository README)
360DeepSeek-Coder-V2 Instruct (0724) DeepSeek53±1local~151GBllama.cpp › WildEval ZeroEval published resultsAider code editing benchmark runs (old edit leaderboard)Berkeley Function Calling Leaderboard result archive
361Olmo 3 7B Think Ai253±1local~5.8GB66kLM Studio › Olmo 3Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsallenai/Olmo-3-7B-Think model cardJetBrains/Mellum2-12B-A2.5B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
362GPT-4.1 nano OpenAI53±1api $0.17/M1Mdevelopers.openai.com ↗OpenAI simple-evals README benchmark results (OpenAI's own models)Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetAider polyglot benchmark runs+11
364Gemma 3 27B Google53±1api $0.10/M +local~18GB131kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportGoogle DeepMind Gemma 3 270M IT model cardEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard dataset+23
365Phi-4 14B Microsoft53±1api $0.087/M +local~10GB16kdeepinfra.com ↗Epoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical ReportMicrosoft — Phi-4-reasoning Technical ReportDaekeun Kim Korean LLM benchmark runsmicrosoft/phi-4 model cardmicrosoft/Phi-4-reasoning model card+11
366Gemma 4 E4B Google53±1local~6.9GB131kLM Studio › Gemma Team — Gemma 4 Technical Reportopenbmb/MiniCPM5-2B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardXHToken/Spark-X2.5-4B model cardLiquidAI/LFM2.5-2.6B model cardEuroEval leaderboards (results collection), English generative leaderboard+7
367Ministral 3 8B Mistral53±1api $0.15/M +local~6.5GB262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)mistralai/Ministral-3-8B-Instruct-2512 model card+2
368Qwen3-VL 4B Instruct Alibaba53±1local~4.1GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
370LFM2.5 2.6B Liquid AI53±1local~3.2GB131kLM Studio › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
371Bielik 11B v2.1 Instruct SpeakLeash53±3local~8.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
372Llama 3.1 70B Instruct Meta52±1api $0.72/M +local~27GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+22
374Qwen2.5-Math 7B Instruct Alibaba52±2†local~6GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
375LFM2.5 8B-A1B Liquid AI52±1local~6.5GB128kLM Studio › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-8B-A1B model card
379Bielik 11B v2.2 Instruct SpeakLeash52±3local~8.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
380Granite 4.0 H Small (32B-A9B) IBM52±1api $0.11/M +local~21GB131kreplicate.com ↗ibm-granite/granite-4.0-h-small model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)+1
381Llama 3.1 Nemotron 70B Instruct NVIDIA52±1local~43GB128kLM Studio › Tülu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., Ai2)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider code editing benchmark runs (old edit leaderboard)Arena-Hard-Auto v2.0-Preview leaderboard (repository README)LMArena leaderboard datasetLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets+4
384Aya Expanse 32B Cohere52±2api $0.75/M +local~22GB128kdocs.cohere.com ↗LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)+1
385Gemma 3 12B Google52±1api $0.075/M +local~9.1GB131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportEpoch AI — internal evaluationsMIMIC Clinical Decision Making leaderboard+10
386Qwen2.5 Coder 7B Instruct Alibaba52±1local~6GB131kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
387Qwen2.5 14B Instruct Alibaba52±1local~10GB131kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMicrosoft Research — Phi-4 Technical Reportmicrosoft/phi-4 model cardReinforcing Real-world Service Agents: InteractCS-RL (arXiv:2602.22697v1)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+7
388Cogito v1 Preview Llama 8B Deep Cogito51±5local~6.2GB128kLM Studio › EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
389Hermes 3 Llama 3.1 405B Nous Research51±1local~114GB131kllama.cpp › Nous Research vendor-reported scores for its own Hermes models (Hermes 3 Technical Report)Aider code editing benchmark runs (old edit leaderboard)
390GPT-4 (0613) OpenAI51±1api $37.50/M8kdevelopers.openai.com ↗Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsTulu 2 — base and instruction-tuned evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+9
391OpenCoder-8B-Instruct INF (infly)51±1local~6.4GBllama.cpp › EvalPlus / EvalPerf published evaluation runsEuroEval leaderboards (results collection), English generative leaderboard
393Llama 3.1 Nemotron 51B Instruct NVIDIA51±2local~34GBllama.cpp › Puzzle: Distillation-Based NAS for Inference-Optimized LLMs (arXiv:2411.19146v5)LMArena leaderboard dataset
394Ministral 3 3B Reasoning 2512 Mistral51±1local~3.5GB262kLM Studio › mistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboard
395Qwen3 4B Alibaba51±1local~3.7GB33kLM Studio › Qwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-4B-Instruct-2507 model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationHuggingFaceTB/SmolLM3-3B model card (Evaluation section)Epoch AI — internal evaluationsArena-Hard-Auto v2.0-Preview leaderboard (repository README)+8
397Llama-3-Instruct-8B-SimPO-ExPO Chujie Zheng (ExPO)51±1local~6.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
398LFM2 24B-A2B Liquid AI51±1local~16GB33kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
399AceReason-Nemotron-1.0-14B NVIDIA51±2†local~10GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
400Llama-3-Instruct-8B-SimPO Princeton NLP51±1local~6.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
401Athene-70B Nexusflow51±1local~29GBllama.cpp › WildEval ZeroEval published resultsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)LMArena leaderboard dataset
402Qwen2 72B Instruct Alibaba51±1local~33GB131kllama.cpp › WildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+14
403Bielik 11B v2.0 Instruct SpeakLeash51±3local~8.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
404OpenThinker3 7B Open Thoughts50±1local~6GB33kLM Studio › Olmo 3allenai/Olmo-3-7B-Think model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)
405CodeQwen1.5 7B Chat Alibaba50±1local~6.4GB64kllama.cpp › EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
406Jamba 1.5 Large AI21 Labs50±1local~839GBhow to run › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)MMLU-Pro author-released evaluation runsLMArena leaderboard dataset
408Llama-3-8B-ProLong-512k-Instruct Princeton NLP50±5local~35GBhow to run › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
409EXAONE Deep 2.4B LG AI Research50±2†local~3.2GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
410Mellum2 12B-A2.5B Instruct JetBrains50±2†local~8.6GB131kLM Studio › JetBrains/Mellum2-12B-A2.5B-Instruct model card
411OpenCodeInterpreter-DS-33B M-A-P50±1local~22GBLM Studio › EvalPlus / EvalPerf published evaluation runs
412Llama 3.2 90B Vision Instruct Meta50±1local~58GB128kOllama › Epoch AI — internal evaluationsPixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Open Portuguese LLM Leaderboard (archived)+1
413DeepSeek-Coder 33B Instruct DeepSeek50±1local~21GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)CRUXEval published evaluation results (authors' generations and scores)+1
414Yi 34B 200K (base) 01.AI50±5local~23GB200kllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
415Gemma 2 27B IT Google50±1local~18GB8kLM Studio › WildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)+10
416Bielik 11B v2.3 Instruct SpeakLeash50±2local~8.6GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Berkeley Function Calling Leaderboard result archiveSpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
417Qwen2.5-VL 7B Instruct Alibaba50±1local~6.4GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
418Molmo 72B (0924) Ai250±2local~309GBhow to run › Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetAi2 Molmo-72B-0924 model card: Human Preference Elo Rating column
419Codestral 22B v0.1 Mistral AI50±1local~15GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsOpen Portuguese LLM Leaderboard (archived)
421Llama 3 70B Instruct Meta50±0local~24GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluations+17
422WizardLM-2 8x22B Microsoft50±1local~91GBOllama › Epoch AI — internal evaluationsAider code editing benchmark runs (old edit leaderboard)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardJudgemark v2 results (EQ-Bench)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
423Nova Pro Amazon50±1api $1.40/M300kdocs.aws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)Aider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
424WizardCoder 33B V1.1 WizardLM50±1local~22GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsWizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)
425Mistral Small Instruct 22B (2409) Mistral AI49±2local~15GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EvalPlus / EvalPerf published evaluation runsSpeakLeash Polish EQ-Bench leaderboardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Galileo Agent Leaderboard (v1 and v2 results)
426Qwen2-Math 72B Instruct Alibaba49±1local~32GB4kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
427OLMo 2 32B Instruct (0325) Ai249±1local~21GBLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLMArena leaderboard datasetallenai/Olmo-3.1-32B-Think model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
428OpenThinker2 7B Open Thoughts49±2†local~6.4GB33kllama.cpp › open-thoughts/OpenThinker2-7B model card
429Aya 23 35B Cohere For AI49±3local~22GB8kLM Studio › SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
430Granite 4.1 3B IBM49±1†local~3.7GBOllama › ibm-granite/granite-4.1-8b model card
431OpenCodeInterpreter-DS-6.7B M-A-P49±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
432Artigenz-Coder-DS-6.7B Artigenz49±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
433LFM2.5 1.2B Thinking Liquid AI49±2local~2.2GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
434Qwen3-VL 2B Thinking Alibaba49±1local~2.7GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
435speechless-codellama-34b-v2.0 uukuguy49±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
436Pixtral 12B (2409) Mistral49±2local~9.6GBllama.cpp › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetVisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column
437Nova Lite Amazon49±2api $0.10/M300kdocs.aws.amazon.com ↗LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
438MegaBeam-Mistral-7B-512k AWS (aws-prototyping)49±4local~6.1GBllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
441AceReason-Nemotron-1.1-7B NVIDIA49±2†local~6GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
442Mixtral 8x22B Instruct v0.1 Mistral49±1local~56GB64kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+9
443Yi-Coder 9B Chat 01.AI49±1local~6.8GB128kLM Studio › LiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EvalPlus / EvalPerf published evaluation runsOpen Portuguese LLM Leaderboard (archived)
444Qwen2-Math 7B Instruct Alibaba49±2†local~6GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
445Magicoder-S-DS 6.7B UIUC (ise-uiuc)49±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
446InternLM2.5 7B Chat Shanghai AI Laboratory (InternLM)49±2local~6.1GBLM Studio › InternLM GitHub repository model cards (Shanghai AI Laboratory)Open Portuguese LLM Leaderboard (archived)
447Hermes 3 Llama 3.1 70B Nous Research49±1local~44GB131kOllama › Nous Research vendor-reported scores for its own Hermes models (Hermes 3 Technical Report)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets
448Qwen2.5 7B Instruct Alibaba48±1api $0.30/M +local~6GB131kwww.together.ai ↗microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Epoch AI — internal evaluations+12
449DeepSeek R1 Distill Qwen 7B DeepSeek48±1local~6GB131kLM Studio › open-thoughts/OpenThinker2-7B model carddeepseek-ai/DeepSeek-R1 model cardallenai/Olmo-3-7B-Think model cardMicrosoft — Phi-4-Mini Technical Reportmicrosoft/Phi-4-mini-reasoning model cardopen-thoughts/OpenThinker-7B model card+6
450XwinCoder-34B Xwin-LM48±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
451StarChat2 15B v0.1 Hugging Face48±1local~12GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runs
452Code Millenials 34B Bud Ecosystem48±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
455T5 Large Google48±9local~4.6GB1khow to run › COPEN — EMNLP 2022 original zero-shot evaluations
456WaveCoder-Ultra-6.7B Microsoft48±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
457Phi-3.5-MoE-instruct Microsoft48±1local~26GB128kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsDaekeun Kim Korean LLM benchmark runsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
460Gemma 3n E4B Google48±1local~6.2GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model cardLMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboard
461Hermes 2 Theta Llama-3 70B Nous Research48±2local~47GB8kllama.cpp › Epoch AI — internal evaluations
462Granite 4.0 H Tiny (7B-A1B) IBM48±1local~5.6GB131kLM Studio › ibm-granite/granite-4.0-h-small model cardEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
463DeepSeek-Coder 7B Instruct v1.5 DeepSeek48±1local~5.9GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
464Qwen2 57B-A14B Instruct Alibaba48±2local~35GB33kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
465Qwen2.5 Coder 3B Instruct Alibaba48±1local~3.3GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
466DeepSeek-Coder 6.7B Instruct DeepSeek48±1local~5.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)
467MiniCPM3-4B OpenBMB48±2local~4GB32kLM Studio › LiveCodeBench official website results (2024-12-26)Berkeley Function Calling Leaderboard result archive
468Bielik 4.5B v3.0 Instruct SpeakLeash48±2local~4.3GBLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
469Qwen3.5 2B Alibaba47±1local~2.9GB262kllama.cpp › Qwen/Qwen3.5-2B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)EuroEval leaderboards (results collection), English generative leaderboardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets+7
470Gemma 2 9B IT Google47±1local~7GB8kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsInternLM GitHub repository model cards (Shanghai AI Laboratory)+14
471Command R Plus (08-2024) Cohere47±1api $4.38/M +local~63GB128kdocs.cohere.com ↗Cohere — Command A technical reportLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+1
472GLM-4 9B Chat Zhipu AI (Z.ai)47±2local~7.9GB128kOllama › WildBench v2 leaderboard data (Ai2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardBerkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)+1
473Yi-1.5 34B (base) 01.AI47±3local~22GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
474Phi-4-mini-reasoning 3.8B Microsoft47±2api $0.13/M +local~3.8GB128kazure.microsoft.com ↗microsoft/Phi-4-mini-reasoning model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
475Qwen1.5 110B Chat Alibaba47±1local~45GB32kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Aider code editing benchmark runs (old edit leaderboard)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+1
476WhiteRabbitNeo-33B-v1 WhiteRabbitNeo47±1local~22GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
477DeepSeek R1 Distill Llama 8B DeepSeek47±1local~6.2GB131kLM Studio › deepseek-ai/DeepSeek-R1 model cardMicrosoft — Phi-4-Mini Technical Reportmicrosoft/Phi-4-mini-reasoning model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard+1
478EXAONE 4.0 1.2B LG AI Research47±2†local~2.2GB66kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
479InternVL2-26B Shanghai AI Laboratory (OpenGVLab)47±2local~55GBhow to run › CharXiv leaderboard (princeton-nlp/CharXiv README)OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tablesLMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
480speechless-coder-ds-6.7b uukuguy47±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
481Gemma 4 E2B Google47±1api $0.050/M +local~5.1GB131kdocs.aws.amazon.com ↗Gemma Team — Gemma 4 Technical Reportopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)LiquidAI/LFM2.5-2.6B model cardEuroEval leaderboards (results collection), English generative leaderboard+6
482Magicoder-S-CL-7B UIUC (ise-uiuc)47±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
483Aya Expanse 8B Cohere47±2local~6.2GB8kLM Studio › LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
484Phind CodeLlama 34B v2 Phind47±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
486InternLM2.5 7B Chat 1M Shanghai AI Laboratory (InternLM)47±4local~6.4GBllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
487EuroLLM 9B Instruct UTTER (EuroLLM)47±3local~6.9GB4kLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
488Granite 4.0 H Micro 3B IBM47±1local~3.4GB131kLM Studio › ibm-granite/granite-4.0-h-small model cardEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
489GPT-3.5 Turbo (0125) OpenAI46±1api $0.75/M16kdevelopers.openai.com ↗WildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+11
491Jamba 1.5 Mini AI21 Labs46±1local~34GBllama.cpp › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)LMArena leaderboard datasetNVIDIA RULER repository README results tableVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
492MedGemma 4B IT Google46±3local~4.1GBllama.cpp › Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Portuguese LLM Leaderboard (archived)SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
493OLMo 2 13B Instruct (1124) Allen Institute for AI46±1local~9.6GB4kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
494Falcon-H1 7B Instruct TII46±3local~6GB262kLM Studio › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
496Cogito v1 Preview Llama 3B Deep Cogito46±4local~3.4GB128kLM Studio › EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
497Code Llama 70B Instruct Meta46±1local~28GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+1
498Hermes 2 Pro Llama 3 8B Nous Research46±3local~6.6GB8kllama.cpp › SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)Berkeley Function Calling Leaderboard result archive
499Llama 3 70B (base) Meta46±2local~43GBLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
500Granite 4.0 Micro 3B IBM46±1local~3.5GB131kLM Studio › ibm-granite/granite-4.0-h-small model cardEpoch AI — internal evaluationsEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
501Command R (08-2024) Cohere46±1api $0.26/M +local~21GB128kdocs.cohere.com ↗LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsCohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+3
503Nova Micro Amazon46±2api $0.061/M128kdocs.aws.amazon.com ↗LMArena leaderboard datasetMASSIVE-Agents — Kulkarni et al. (2025)Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
504T5 Base Google46±9local~2.4GB1khow to run › COPEN — EMNLP 2022 original zero-shot evaluations
505OpenChat 3.5 0106 OpenChat46±1local~6.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboard+1
506Qwen2-VL 7B Instruct Alibaba46±1local~6.4GBLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating columnVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
507Qwen2.5-Math 1.5B Instruct Alibaba46±2†local~2.4GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
508Yi 1.5 34B Chat 01.AI46±1local~22GB4kLM Studio › WildEval ZeroEval published resultsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Epoch AI — internal evaluationsWildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+5
509DBRX Instruct Databricks46±1local~85GB32kOllama › EvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsWildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+3
510Granite 34B Code Instruct (8K) IBM46±1local~22GBOllama › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboard
511WizardCoder Python 34B V1.0 WizardLM46±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsWizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)CRUXEval published evaluation results (authors' generations and scores)
515Granite 3.3 8B Instruct IBM46±1local~6.3GB128kLM Studio › ibm-granite/granite-3.3-8b-instruct model cardallenai/Olmo-3-7B-Instruct model cardDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
516Tulu 2 70B Allen Institute for AI46±2†local~28GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
518OpenChat 3.5 0106 Gemma OpenChat46±3local~6.9GB8kllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
519Olmo 3 7B Instruct Ai246±1local~5.8GB66kLM Studio › Olmo 3JetBrains/Mellum2-12B-A2.5B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsallenai/Olmo-3-7B-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
520InternLM2.5 20B Chat Shanghai AI Laboratory (InternLM)46±2local~14GB33kllama.cpp › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
521StarCoder2-15B-Instruct-v0.1 BigCode46±1local~11GB16kOllama › EvalPlus / EvalPerf published evaluation runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
522Qwen1.5 72B Chat Alibaba45±1local~31GB32kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+9
523speechless-starcoder2-15b uukuguy45±1local~12GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
524Ministral 3 3B Mistral45±1api $0.10/M +local~3.5GB262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsmistralai/Ministral-3-14B-Instruct-2512 model cardmistralai/Ministral-3-8B-Instruct-2512 model cardEuroEval leaderboards (results collection), English generative leaderboard+2
526OpenThinker 7B Open Thoughts45±2†local~6GB33kLM Studio › open-thoughts/OpenThinker2-7B model cardopen-thoughts/OpenThinker-7B model card
527Phi-3 Medium 128K Instruct Microsoft45±1local~9.8GB128kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
528QwQ-32B-Preview Alibaba45±2local~21GB33kLM Studio › LMArena leaderboard datasetEvalPlus / EvalPerf published evaluation runsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)Berkeley Function Calling Leaderboard result archive+2
529DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) DeepSeek45±1local~11GB128kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)EvalPlus / EvalPerf published evaluation runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsWebApp1K models leaderboard (onekq-ai)+1
532Zephyr ORPO 141B-A35B v0.1 Hugging Face45±2local~56GB66kllama.cpp › Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetHuggingFaceH4 zephyr-orpo-141b-A35b-v0.1 model card
533Phi-3-medium-4k-instruct Microsoft45±1local~9.8GB4kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetEvalPlus / EvalPerf published evaluation runsSpeakLeash Polish EQ-Bench leaderboard+1
534Qwen2-Math 1.5B Instruct Alibaba45±2†local~2.4GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
536Llama 3.1 8B Instruct Meta45±0api $0.22/M +local~6.2GB128kdocs.aws.amazon.com ↗EvalPlus / EvalPerf published evaluation runsIBM Granite 3.0 Language Models technical report (GitHub)microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets+25
537Command R Plus (04-2024) Cohere45±1local~43GB128kllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider code editing benchmark runs (old edit leaderboard)+6
538Mistral NeMo Instruct (2407) Mistral44±1local~8.7GB128kLM Studio › WildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsWildBench v2 leaderboard data (Ai2)EvalPlus / EvalPerf published evaluation runsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+9
539LLaVA-OneVision Qwen2 72B (ov-sft) LLaVA (lmms-lab)44±3local~155GBhow to run › LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (arXiv:2505.20740v3)
540Phi-3-mini-4k-instruct (June 2024 update) Microsoft44±1local~3.7GBLM Studio › WildEval ZeroEval published resultsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
541Yi 1.5 9B Chat 01.AI44±1local~6.8GB4kLM Studio › WildEval ZeroEval published resultsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4WildBench v2 leaderboard data (Ai2)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)+1
542Command R7B (12-2024) Cohere44±1local~6.2GB128kLM Studio › Cohere — Command A technical reportInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)
543Qwen2-Audio 7B Instruct Alibaba44±3local~6.5GBllama.cpp › AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
544Gemma 3 4B Google44±1api $0.063/M +local~3.8GB131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboardLMArena leaderboard dataset+11
545Eurus-2-7B-PRIME PRIME-RL44±2local~6.4GB4kllama.cpp › PRIME-RL Eurus-2-7B-PRIME model card, Evaluation table (own models)Epoch AI — internal evaluations
546Llama 3.1 Tulu 3 8B Ai244±1local~6.2GB131kLM Studio › microsoft/Phi-4-mini-instruct model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
547Qwen2.5-VL 3B Instruct Alibaba44±1local~3.5GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
548Qwen1.5 32B Chat Alibaba44±1local~22GB32kllama.cpp › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetEpoch AI — internal evaluationsSpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
549Granite 20B Code Instruct (8K) IBM44±1local~14GBOllama › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
550Qwen2 7B Instruct Alibaba44±1local~6GB131kLM Studio › WildEval ZeroEval published resultsInternLM GitHub repository model cards (Shanghai AI Laboratory)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+4
551Falcon3 7B Instruct TII44±3local~5.9GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)
552Xwin-LM 70B v0.1 Xwin-LM44±1local~28GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)
553Qwen3-VL 2B Instruct Alibaba44±1local~2.7GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
554Dolphin 2.6 Mixtral 8x7B Cognitive Computations44±1local~29GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
555Granite 8B Code Instruct (4K) IBM44±1local~6.6GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
556Falcon3 10B Instruct TII44±3local~7.6GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)
557Code Tulu 2 34B Allen Institute for AI44±2†local~72GBhow to run › Tulu 2 — base and instruction-tuned evaluations
558Qwen3 1.7B Alibaba44±1local~2.6GB33kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen/Qwen3.5-2B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiquidAI/LFM2-1.2B model cardHuggingFaceTB/SmolLM3-3B model card (Evaluation section)LiquidAI/LFM2.5-1.2B-Thinking model card+10
559Phi-4-mini 3.8B Microsoft44±1api $0.13/M +local~3.8GB128kazure.microsoft.com ↗microsoft/Phi-4-mini-instruct model cardMicrosoft — Phi-4-Mini Technical ReportDaekeun Kim Korean LLM benchmark runsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+4
560Ministral 8B (2410) Mistral44±1local~6.2GB128kLM Studio › microsoft/Phi-4-mini-instruct model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEpoch AI — internal evaluationsMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)LMArena leaderboard datasetBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+6
561Granite 3.2 8B Instruct IBM44±1local~6.6GB131kllama.cpp › ibm-granite/granite-3.3-8b-instruct model cardBerkeley Function Calling Leaderboard result archiveVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
562Code Llama 70B Python Meta44±1local~28GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runs
563Llama 3.1 8B (base) Meta44±3local~6.2GBLM Studio › Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
564LFM2.5 1.2B Instruct Liquid AI44±2local~2.2GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
565Granite 3.0 8B Instruct IBM43±1local~6.6GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
566Phi-3.5-mini-instruct Microsoft43±1local~3.7GB128kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsDaekeun Kim Korean LLM benchmark runsMicrosoft — Phi-4-Mini Technical ReportSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+4
567Hermes 2 Theta Llama 3 8B Nous Research43±2local~6.6GB8kllama.cpp › WildBench v2 leaderboard data (Ai2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
568Yi 34B Chat 01.AI43±1local~23GB4kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Epoch AI — internal evaluationsLMArena leaderboard datasetEQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs+1
569CogVLM2 Llama3 Chat 19B Zhipu AI (THUDM)43±3local~43GBhow to run › LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Zhipu AI (THUDM) CogVLM2 model card, Benchmark table (own models)
570Code Llama 70B (base) Meta43±1local~28GB16kllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
571Gemma 3n E2B IT Google43±2local~4.7GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model cardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
573T5 Small Google43±9local~1.8GB1khow to run › COPEN — EMNLP 2022 original zero-shot evaluations
574MiniCPM-V 2.6 OpenBMB43±2local~6.4GBllama.cpp › CharXiv leaderboard (princeton-nlp/CharXiv README)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
575Qwen2.5 Coder 1.5B Instruct Alibaba43±1local~2.4GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
576LFM2 2.6B Liquid AI43±2local~3GB33kLM Studio › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationEuroEval leaderboards (results collection), English generative leaderboard
577SmolLM3 3B Hugging Face43±1local~3.3GB66kLM Studio › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationHuggingFaceTB/SmolLM3-3B model card (Evaluation section)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
578Granite 3.1 8B Instruct IBM43±1local~6.6GB128kllama.cpp › ibm-granite/granite-3.3-8b-instruct model cardibm-granite/granite-3.1-3b-a800m-instruct model cardLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
579Llama 3.2 11B Vision Instruct Meta43±2local~8.8GB128kOllama › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
580Qwen2.5 3B Instruct Alibaba43±1local~3.3GB33kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SpeakLeash Polish EQ-Bench leaderboard+5
581DeepSeek-Coder 1.3B Instruct DeepSeek43±1local~2.4GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
582Aya 23 8B Cohere For AI43±3local~6.2GB8kLM Studio › SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
583Phi-3-small-8k-instruct Microsoft43±2local~17GBhow to run › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)
584CodeGemma 7B IT (v1.0) Google43±1local~7.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CodeGemma: Open Code Models Based on Gemma — authors' own runs
585Aya Vision 32B Cohere42±3local~71GB16khow to run › LMArena leaderboard datasetKaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark (arXiv:2508.13680v5)
586Tulu 2 DPO 70B Ai242±1local~32GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+1
587Mathstral 7B v0.1 Mistral AI42±1local~5.8GB32kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsMMLU-Pro author-released evaluation runsWildEval ZeroEval published resultsMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)
588Yi-1.5 9B (base) 01.AI42±2local~6.8GB4kLM Studio › Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
590Llama 3 8B Instruct Meta42±0local~6.6GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsInternLM GitHub repository model cards (Shanghai AI Laboratory)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+19
591WizardLM 70B V1.0 Microsoft42±1local~32GBllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
592WizardCoder Python 13B V1.0 WizardLM42±2local~9.8GB16kllama.cpp › WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)CRUXEval published evaluation results (authors' generations and scores)
593Hermes 3 Llama 3.1 8B Nous Research42±1local~6.6GB128kllama.cpp › Nous Research vendor-reported scores for its own Hermes models (Hermes 3 Technical Report)Aider code editing benchmark runs (old edit leaderboard)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
594Falcon2 11B (base) TII42±3local~8.2GB8kOllama › SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
595Granite 34B Code Base (8K) IBM42±1local~24GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
596speechless-starcoder2-7b uukuguy42±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
597Command R (03-2024) Cohere42±1local~24GB128kllama.cpp › WildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)MMLU-Pro author-released evaluation runsWildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+6
599OLMo 2 7B Instruct (1124) Allen Institute for AI42±1local~5.8GB4kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsallenai/Olmo-3-7B-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
600WizardCoder-15B-V1.0 WizardLM42±1local~12GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
601Snowflake Arctic Instruct Snowflake41±2local~1007GBhow to run › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
602Phi-3-vision-128k-instruct Microsoft41±2local~10GBhow to run › CharXiv leaderboard (princeton-nlp/CharXiv README)LMArena leaderboard datasetPhi-3 Technical Report (Microsoft, arXiv 2404.14219v3)PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
603OpenChat 3.5 OpenChat41±2local~6.1GB8kllama.cpp › OpenChat 3.5 model card (openchat/openchat_3.5)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetEQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs
604InternVL2-4B Shanghai AI Laboratory (OpenGVLab)41±2local~10GBhow to run › OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tablesLMArena leaderboard datasetMicrosoft Phi-3.5-vision-instruct model card: Benchmarks tablePhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
605Qwen1.5 14B Chat Alibaba41±1local~11GB32kOllama › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)MMLU-Pro author-released evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+3
606Code Llama 34B Python Meta41±1local~23GB16kOllama › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
607speechless-coding-7b-16k-tora uukuguy41±1local~16GBhow to run › EvalPlus / EvalPerf published evaluation runs
608DeepSeek-Coder 33B Base DeepSeek41±1local~22GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
609InternLM2 Chat 20B Shanghai AI Laboratory (InternLM)41±1local~14GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
610DeepSeek LLM 67B Chat DeepSeek41±1local~31GB4kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
611Llama 2 70B (base) Meta41±1local~32GB4kllama.cpp › Mixtral of Experts — base-model evaluationsTulu 2 — base and instruction-tuned evaluationsMMLU-Pro author-released evaluation runsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
612Mixtral 8x7B v0.1 (base) Mistral AI41±1local~29GB32kllama.cpp › Mixtral of Experts — base-model evaluationsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
613North Micro Vision Instruct (2.4B) Cohere41±2†local~3.8GB128khow to run › CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
614CodeQwen1.5 7B (base) Alibaba41±1local~6.4GB66kllama.cpp › EvalPlus / EvalPerf published evaluation runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
615Granite 4.0 1B IBM41±1local~2.4GB128kLM Studio › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model cardEpoch AI — internal evaluations
616Mixtral 8x7B Instruct v0.1 Mistral41±1local~29GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+15
617Molmo 7B-D (0924) Ai241±1local~35GBhow to run › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)LMArena leaderboard datasetKaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column
618OpenHermes-2.5-Code-290k-13B ajibawa-202341±1local~9.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
619Granite 4.0 H 1B IBM40±1local~2.3GB131kLM Studio › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model card
620Llama3-OpenBioLLM-8B Saama AI Labs40±2local~6.6GBllama.cpp › Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)
621Nous Hermes 2 Mistral 7B DPO Nous Research40±2local~6.1GB33kllama.cpp › Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
622Nous Hermes 2 Mixtral 8x7B DPO Nous Research40±2local~31GB33kllama.cpp › WildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)
623Deita 7B v1.0 HKUST NLP40±2local~32GBhow to run › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
624Code Llama 34B Instruct Meta40±1local~22GB16kLM Studio › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+1
625Code Llama 34B (base) Meta40±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+1
626Starling-LM 7B beta Nexusflow40±1local~6GB8kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+3
627LFM2 1.2B Liquid AI40±2local~2.2GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
628Code Tulu 2 13B Allen Institute for AI40±2†local~29GBhow to run › Tulu 2 — base and instruction-tuned evaluations
629LLaVA-v1.6-34B LLaVA40±2local~24GB4kOllama › CharXiv leaderboard (princeton-nlp/CharXiv README)CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (arXiv:2406.18521v1)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)LLaVA repository model zoo (haotian-liu/LLaVA docs/MODEL_ZOO.md)
630Bielik 1.5B v3.0 Instruct SpeakLeash40±3local~2.4GBLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
631OpenHermes 2.5 Mistral 7B Nous Research39±1local~6.1GBllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenChat 3.5 model card (openchat/openchat_3.5)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+2
632Granite 3.3 2B Instruct IBM39±1†local~3GB128kLM Studio › ibm-granite/granite-3.3-8b-instruct model card
633Phi-3-mini-128k-instruct (Apr 2024) Microsoft39±2local~9.5GBhow to run › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard+1
634Mistral 7B Instruct v0.3 Mistral39±1local~5.8GB32kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardEpoch AI — internal evaluations+4
635Vicuna 33B v1.3 LMSYS39±1local~23GB2kOllama › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table+2
636DeepSeek-Coder 6.7B Base DeepSeek39±1local~5.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
637Starling-LM 7B alpha UC Berkeley (NEST)39±1local~6.1GB8kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+2
638Granite 8B Code Base (4K) IBM39±1local~6.6GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
639StarCoder2 15B BigCode39±1local~11GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
640speechless-code-mistral-7b-v1.0 uukuguy39±1local~17GBhow to run › EvalPlus / EvalPerf published evaluation runsOpen Portuguese LLM Leaderboard (archived)
641Falcon-H1 3B Instruct TII39±4local~3.4GB131kLM Studio › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
642Phi-2 (2.7B) Microsoft39±1local~3.2GB2kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)+2
643Granite 3.0 2B Instruct IBM39±1local~3.2GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
644Llama 2 70B Chat Meta39±1local~32GB4kllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+10
645Code Llama 13B Instruct Meta39±1local~9.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
646MistralHermes-CodePro-7B-v1 beowolx39±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
647Llama-3-Instruct-8B-SimPO-v0.2 Princeton NLP39±1local~6.6GBllama.cpp › WildEval ZeroEval published resultsWildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
648Llama 3.2 3B Instruct Meta38±1api $0.12/M +local~3.4GB131kdevelopers.cloudflare.com ↗microsoft/Phi-4-mini-instruct model cardIBM Granite 3.0 Language Models technical report (GitHub)LFM2 Technical Report — Liquid AI own SmolLM3 evaluationAider code editing benchmark runs (old edit leaderboard)microsoft/Phi-4-mini-reasoning model cardMicrosoft — Phi-4-Mini Technical Report+11
649Qwen-14B-Chat Alibaba38±1local~11GBllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetEQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs
650Falcon-180B-Chat TII38±2local~115GB2kOllama › Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset
651CodeGemma 7B (v1.0) Google38±1local~7.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
652Code Llama 13B Python Meta38±1local~9.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
653Tulu 2 13B Allen Institute for AI38±2local~9.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsOpen Portuguese LLM Leaderboard (archived)
654Phi-3-mini-4k-instruct (Apr 2024) Microsoft38±2local~4GBllama.cpp › MMLU-Pro author-released evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
655Apertus 8B Instruct (2509) Swiss AI38±4local~6.3GB66kLM Studio › allenai/Olmo-3-7B-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboardMiðeind Icelandic LLM Leaderboard
657Granite 3B Code Instruct (2K) IBM38±2local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
658DeepSeek-V2 Lite Chat (16B-A2.4B) DeepSeek38±1local~11GB32kLM Studio › DeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
659BioMistral-7B BioMistral38±3local~6.1GBllama.cpp › BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
660Qwen3.5 0.8B Alibaba38±1local~2GB262kllama.cpp › Qwen/Qwen3.5-2B model cardEuroEval leaderboards (results collection), English generative leaderboardQwen/Qwen3.5-0.8B model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+2
661Code Tulu 2 7B Allen Institute for AI38±2†local~16GBhow to run › Tulu 2 — base and instruction-tuned evaluations
662SOLAR 10.7B Instruct v1.0 Upstage37±1local~8.3GB4kllama.cpp › EvalPlus / EvalPerf published evaluation runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboard+2
663Phi-3.5-vision-instruct Microsoft37±2local~10GBhow to run › CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)LMArena leaderboard datasetMicrosoft Phi-3.5-vision-instruct model card: Benchmarks tableAi2 Molmo-72B-0924 model card: Human Preference Elo Rating columnPhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
664Tulu 2 DPO 13B Allen Institute for AI37±1local~9.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
665Yi-1.5 6B (base) 01.AI37±3local~5.1GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
666Qwen2-VL 2B Instruct Alibaba37±1local~2.8GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
667DeepSeek R1 Distill Qwen 1.5B DeepSeek37±2local~2.6GB131kLM Studio › deepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Epoch AI — internal evaluationsTHINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models (arXiv:2505.22113v1)EuroEval leaderboards (results collection), English generative leaderboard+1
668Gemma 2 2B IT Google37±1local~3.1GB8kLM Studio › IBM Granite 3.0 Language Models technical report (GitHub)WildEval ZeroEval published resultsWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+4
669Qwen2.5 Coder 0.5B Instruct Alibaba37±1local~1.8GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
670Granite 20B Code Base (8K) IBM37±1local~15GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
671Gemma 1.1 7B IT Google37±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+2
672Code Llama 7B Python Meta37±1local~5.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
673Dolphin 2.2.1 Mistral 7B Cognitive Computations37±2local~6.1GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset
674Code Llama 13B (base) Meta37±1local~9.2GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+1
675Qwen1.5 7B Chat Alibaba37±1local~6.5GB32kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)MMLU-Pro author-released evaluation runsWildBench v2 leaderboard data (Ai2)+4
676Yi 1.5 6B Chat 01.AI37±2local~5.1GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4WildBench v2 leaderboard data (Ai2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
677Falcon3 3B Instruct TII37±3local~3.4GB32kLM Studio › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
678Zephyr 7B alpha Hugging Face36±1local~6.1GBllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)
679Vicuna 13B v1.5 LMSYS36±2local~9.8GB4kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableOpen Portuguese LLM Leaderboard (archived)
680Granite 3.1 2B Instruct IBM36±1local~3.1GB128kllama.cpp › ibm-granite/granite-3.3-8b-instruct model cardibm-granite/granite-3.1-3b-a800m-instruct model cardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
681EXAONE 3.5 2.4B Instruct LG AI Research36±2local~3GB33kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
683Code Llama 7B Instruct Meta36±1local~5.5GB16kLM Studio › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
684LFM2 700M Liquid AI36±2local~1.9GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
685Granite 3.2 2B Instruct IBM36±1local~3GB131kLM Studio › ibm-granite/granite-3.3-8b-instruct model cardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
686Mistral 7B Instruct v0.2 Mistral36±1local~6GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+9
687WizardLM 13B V1.2 Microsoft36±1local~9.8GBllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
688Tulu 2 7B Allen Institute for AI36±2local~5.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsOpen Portuguese LLM Leaderboard (archived)
689xDAN-L1-Chat-RL-v1 xDAN-AI36±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
690Llama 3 8B (base) Meta36±1local~6.6GBllama.cpp › EvalPlus / EvalPerf published evaluation runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboard
691Mistral 7B v0.1 (base) Mistral AI36±0local~5.8GB8kLM Studio › EvalPlus / EvalPerf published evaluation runsStability AI — original StableLM zero-shot evaluation outputsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runsGemma report — original evaluations+4
692Qwen2.5 1.5B Instruct Alibaba36±1local~2.4GB33kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+6
694Gemma 7B (base) Google35±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGemma report — original evaluationsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)+3
695Tulu 2 DPO 7B Allen Institute for AI35±1local~5.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
696Eurus-2-7B-SFT PRIME-RL35±3†local~6.4GBllama.cpp › PRIME-RL Eurus-2-7B-PRIME model card, Evaluation table (own models)
697Falcon 40B (base) TII35±1local~27GB2kOllama › MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Open Portuguese LLM Leaderboard (archived)From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
698StarCoder2 7B BigCode35±1local~6.1GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
699Granite 3.0 3B-A800M Instruct IBM35±1local~3.7GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
700DeepSeek LLM 7B Chat DeepSeek35±2†local~5.7GBOllama › DeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)
701StarCoder 15.5B BigCode35±1local~11GBOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
702Granite 3.1 3B-A800M Instruct IBM35±2†local~3.6GB131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
703Llama 2 13B (base) Meta34±1local~9.3GB4kOllama › Stability AI — original StableLM zero-shot evaluation outputsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runsTulu 2 — base and instruction-tuned evaluationsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+3
704Code Llama 7B (base) Meta34±1local~5.5GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsMistral 7B — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluations+2
705Nous-Hermes-13b Nous Research34±2local~9.8GB2kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
706ChatGLM3 6B 128K Zhipu AI (THUDM)34±4local~5.8GB131kllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
707Zephyr 7B beta Hugging Face34±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTulu 2 — base and instruction-tuned evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)OpenChat 3.5 model card (openchat/openchat_3.5)+3
709Qwen3 0.6B Alibaba33±1local~1.8GB33kLM Studio › Olmo 3LiquidAI/LFM2-1.2B model cardBerkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard+4
710Llama 2 13B Chat Meta33±1local~9.8GB4kllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset+9
711Flan-T5 XXL Google33±1local~49GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsInstructEval (DeCLaRe Lab) README results tableTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (arXiv:2306.02561v3)
712StarCoder2 3B BigCode33±1local~3.4GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
713Granite 3B Code Base (2K) IBM33±1local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
715StarCoderBase 15.5B BigCode33±1local~12GBllama.cpp › StarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
716Vicuna 7B v1.5 LMSYS33±1local~5.8GB4kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table+1
717Guanaco 33B University of Washington33±2local~70GBhow to run › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
718Stable Code 3B Stability AI33±1local~3.3GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
719OLMo 2 1B Instruct (0425) Allen Institute for AI33±2local~2.4GB4kllama.cpp › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsOpen Portuguese LLM Leaderboard (archived)
720DeepSeek-Coder 1.3B Base DeepSeek32±1local~2.4GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
721Orca 2 13B Microsoft32±2local~9.8GB4kllama.cpp › Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
722Gemma 7B IT Google32±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+5
723Mistral 7B Instruct v0.1 Mistral32±1local~6.1GB8kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)MMLU-Pro author-released evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+2
724Qwen1.5 4B Chat Alibaba32±2local~4.1GB32kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
725Ghost 7B Alpha Ghost X32±2local~6.1GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
726CodeGemma 2B (v1.0) Google32±1local~3.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
727Qwen-7B (base) Alibaba32±1local~6.6GB33kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
728Phi-1.5 (1.3B) Microsoft32±1local~2.4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
729Falcon-H1 1.5B Instruct TII31±4local~2.4GB131kLM Studio › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
730Flan-T5 XL Google31±2local~14GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsInstructEval (DeCLaRe Lab) README results table
731Qwen2 1.5B Instruct Alibaba31±1local~2.4GB32kLM Studio › WildEval ZeroEval published resultsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsSpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+1
732Qwen-7B-Chat Alibaba31±1local~6.6GB33kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
733Gemma 3 1B Google31±1local~2.2GB32kLM Studio › Google DeepMind Gemma 3 270M IT model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2-1.2B model cardLiquidAI/LFM2.5-1.2B-Thinking model cardMIMIC Clinical Decision Making leaderboardGemma Team — Gemma 3 Technical Report+7
734StarCoderBase 7B BigCode31±1local~6GBOllama › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
735Llama 2 7B (base) Meta31±1local~5.5GB4kOllama › Stability AI — original StableLM zero-shot evaluation outputsMixtral of Experts — base-model evaluationsOLMo: Accelerating the Science of Language Models — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsMistral 7B — authors' own runsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)+5
736OPT-66B Meta31±1local~141GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
737LFM2 350M Liquid AI30±2local~1.7GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
738SmolLM2 1.7B Instruct Hugging Face30±1local~2.6GB8kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
739Llama 3.2 1B Instruct Meta30±1api $0.020/M +local~2.2GB131knovita.ai ↗IBM Granite 3.0 Language Models technical report (GitHub)2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2-1.2B model cardSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLiquidAI/LFM2.5-1.2B-Thinking model cardLMArena leaderboard dataset+13
740Falcon 7B (base) TII30±1local~5.9GB2kOllama › Stability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Open Portuguese LLM Leaderboard (archived)+1
741Llama 2 7B Chat Meta30±1local~5.8GB4kllama.cpp › Tulu 2 — base and instruction-tuned evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)WildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+11
742Baichuan2-7B-Base Baichuan30±1local~17GBhow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
745OLMo 7B Instruct Ai229±2local~16GBhow to run › OLMo: Accelerating the Science of Language Models — authors' own runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard dataset
746StableLM-3B-4E1T Stability AI29±1local~3.3GB4kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
747Gemma 2B (base) Google29±1local~3.2GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGemma report — original evaluationsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+3
748OpenLLaMA 7B v2 OpenLM Research29±1local~9.1GBllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
749Gemma 1.1 2B IT Google29±1local~3.2GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash Polish EQ-Bench leaderboardArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Open Portuguese LLM Leaderboard (archived)+1
750StableLM-Base-Alpha 7B v2 Stability AI29±1local~30GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
751Granite 3.1 1B-A400M Instruct IBM29±2†local~2.3GB131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
752OPT-30B Meta29±1local~64GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
753NexusRaven V2 13B Nexusflow29±3local~9.3GB16kOllama › Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Berkeley Function Calling Leaderboard result archive
754OLMo 7B Allen Institute for AI28±1†local~30GB2khow to run › OLMo: Accelerating the Science of Language Models — authors' own runs
755Falcon 40B Instruct TII28±2local~27GBOllama › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
756GPT-NeoX-20B EleutherAI28±1local~14GB2kLM Studio › Stability AI — original StableLM zero-shot evaluation outputsGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsOpen Portuguese LLM Leaderboard (archived)
759StarCoderBase 3B BigCode28±1local~3.4GBOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
760Granite 3.0 1B-A400M Instruct IBM27±1local~2.3GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
761RedPajama-INCITE 7B Base Together27±1local~16GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Portuguese LLM Leaderboard (archived)
762ChatGLM3-6B Zhipu AI (THUDM)27±2local~5.8GBllama.cpp › Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)Open Portuguese LLM Leaderboard (archived)
763Granite 4.0 H 350M IBM27±1†local~1.7GB33kLM Studio › ibm-granite/granite-4.0-h-1b model card
764Flan-T5 Large Google27±2local~4.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
766Gemma 2B IT Google27±1local~3.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsWildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+3
767EuroLLM 1.7B Instruct UTTER (EuroLLM)27±3local~2.4GB4kLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
768Qwen2.5 0.5B Instruct Alibaba27±2local~1.8GB33kLM Studio › HuggingFaceTB/SmolLM2-360M-Instruct model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)+1
770GPT4All-13B-Snoozy Nomic AI26±2local~56GBhow to run › Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
771OPT-13B (13 billion parameters) Meta26±1local~29GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
772Granite 4.0 350M IBM26±1local~1.7GB32kLM Studio › ibm-granite/granite-4.0-h-1b model cardBerkeley Function Calling Leaderboard result archiveEpoch AI — internal evaluations
775Falcon 7B Instruct TII25±2local~5.9GBOllama › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
776Qwen2 0.5B Instruct Alibaba25±2local~1.8GB32kLM Studio › Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
777Pythia 12B EleutherAI25±1local~9.5GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Open Portuguese LLM Leaderboard (archived)
778RedPajama-INCITE Base 3B v1 Together25±3local~3.4GB2kllama.cpp › RedPajama — original zero-shot 3B comparisonOpen Portuguese LLM Leaderboard (archived)
779ChatGLM2-6B Zhipu AI (THUDM)25±2local~15GBhow to run › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableOpen Portuguese LLM Leaderboard (archived)
780GPT-J 6B EleutherAI25±1local~14GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsEvalPlus / EvalPerf published evaluation runsGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
781OpenLLaMA 3B v2 OpenLM Research25±1local~4.2GBllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
782OPT-6.7B Meta24±1local~16GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
784Falcon3 1B Instruct TII24±4local~2.4GB8kLM Studio › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
785StableLM-Base-Alpha 3B v2 Stability AI24±1local~13GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
786Qwen1.5 1.8B Chat Alibaba24±2local~2.7GB33kOllama › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
787Pythia 6.9B EleutherAI23±1local~6.1GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Open Portuguese LLM Leaderboard (archived)
788OpenAssistant SFT-4 Pythia 12B LAION (OpenAssistant)22±1local~9.5GBllama.cpp › InstructEval (DeCLaRe Lab) README results tableAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableLLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (arXiv:2306.02561v3)
789Pythia 2.8B EleutherAI21±1local~3.4GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsRedPajama — original zero-shot 3B comparisonOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
790BLOOM 7.1B BigScience21±1local~7.1GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Open Portuguese LLM Leaderboard (archived)From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
791OLMo 1B Allen Institute for AI21±2†local~2.2GB2kllama.cpp › OLMo: Accelerating the Science of Language Models — authors' own runs
792Phi-1 (1.3B) Microsoft21±4local~2.4GB2kllama.cpp › CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)
793Qwen1.5 0.5B Chat Alibaba21±3local~1.8GB33kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
794ChatGLM-6B Zhipu AI (THUDM)20±2local~16GBhow to run › InstructEval (DeCLaRe Lab) README results tablePolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableLLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (arXiv:2306.02561v3)
795Pythia 2.8B (deduped) EleutherAI20±1local~3.4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
796Flan-T5 Base Google20±3†local~2.5GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runs
797Falcon-RW 1B TII20±2local~4.3GB2khow to run › Textbooks Are All You Need II: phi-1.5 technical report — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
798OPT-2.7B Meta19±1local~7.1GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
799SmolLM2 360M Instruct Hugging Face19±3local~1.7GB8kLM Studio › HuggingFaceTB/SmolLM2-360M-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
800GPT-Neo 2.7B EleutherAI18±1local~13GB2khow to run › EvalPlus / EvalPerf published evaluation runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsRedPajama — original zero-shot 3B comparisonHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs+1
802Pythia 1.4B EleutherAI18±2local~2.4GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
803BLOOM 3B BigScience18±1local~4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
804FastChat-T5 3B LMSYS17±2local~8.5GBhow to run › Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
806OPT-1.3B Meta16±1local~4.3GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Portuguese LLM Leaderboard (archived)
807Gemma 3 270M IT Google15±4local~1.7GB32kLM Studio › Google DeepMind Gemma 3 270M IT model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
808Pythia 1B EleutherAI14±1local~2.2GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
809StableLM-Tuned-Alpha 7B Stability AI14±2local~6.9GBllama.cpp › Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableLLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (arXiv:2306.02561v3)
810BLOOM 1.7B BigScience14±1local~3.1GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
811Flan-T5 Small Google14±3†local~1.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runs
812GPT-Neo 1.3B EleutherAI14±2local~7.1GB2khow to run › EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runsOpen Portuguese LLM Leaderboard (archived)
813GPT-2 XL (1.5B) OpenAI11±1local~2.8GB1kllama.cpp › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsEleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)+2
814BLOOM 1.1B BigScience11±1local~2.6GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
815Pythia 410M EleutherAI10±2local~1.8GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
817StableLM-Base-Alpha 7B Stability AI9±1local~35GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsEvalPlus / EvalPerf published evaluation runsInstructEval (DeCLaRe Lab) README results tableOpen Portuguese LLM Leaderboard (archived)
818Zyte-1B venkycs (aihub-app)8±2local~2.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
819GPT-Neo 125M EleutherAI5±5local~2.1GB2khow to run › Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)COPEN — EMNLP 2022 original zero-shot evaluationsOpen Portuguese LLM Leaderboard (archived)
820BLOOM 560M BigScience5±2local~2.1GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
821OPT-350M Meta5±2local~2.2GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
822StableLM-Base-Alpha 3B Stability AI4±1local~17GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
823Pythia 160M EleutherAI3±2local~1.6GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
824OPT-125M Meta3±2local~1.8GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
825GPT-2 Large (774M) OpenAI1±2local~2GB1kllama.cpp › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
826GPT-2 Medium (355M) OpenAI-1±2local~1.8GB1kllama.cpp › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
827Pythia 70M EleutherAI-7±2local~1.6GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
828GPT-2 Small (124M) OpenAI-9±3local~1.6GB1kLM Studio › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
—Grok Build 0.1 SpaceXAI (xAI)no reviewed resultsapi $1.25/M256kdocs.x.ai ↗
—K2 Horizon 7B Institute of Foundation Models (MBZUAI)not scored yetlocal~7.7GB524kllama.cpp ›

+ 117 more don't fit your machine or budget · show them

+ 228 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

830 of 1134 catalogue models listed here · sources and licences

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory