whatmodel.sh · find the best AI models you can run

Best AI models for a 8GB Mac

A leaderboard of the best AI models for a Mac with 8GB of memory. The best one that runs on it right now is Spark-X2.5 4B, followed by MiniCPM5 2B and Qwen3.5 4B. Models are ranked by one score that combines 472 published benchmarks. Models that run on it show the download that fits and the command to run it.

351 models · scores combine 472 benchmarks · updated how it works · find a leaderboard · benchmark your machine
task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]
Models for 8GB RAM · Apple Silicon → ~5GB fast · reset
+ more
chip[M1][M2][M3][M4]

Detect your exact chip or graphics card:

curl -G whatmodel.sh/ -d max=0 --data-urlencode "mac=$(sysctl -n hw.memsize machdep.cpu.brand_string)"Mac

curl -G whatmodel.sh/ -d max=0 --data-urlencode "ram=$(awk '/MemT/{print $2}' /proc/meminfo)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null)"Linux

curl.exe -G whatmodel.sh/ -d max=0 --data-urlencode "ram=$((gcim Win32_ComputerSystem).TotalPhysicalMemory)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>$null)"Windows

$ terminal

detecting your OS…

Run curl "whatmodel.sh/?ram=8&gpu=apple"to get this list in your terminal (add &format=json for agents).

ram[8][16][24][32][48][64][96][128][192][256]GB
gpu[apple silicon][graphics card][none]
price[local][$0.02][$0.1][$0.5][$2][$10][$50][any]max per M

detecting your OS…

Score vs API price

#modelscore↓where↓speed↓RAM↓context↓get itsources
#modelscore↓where↓speed↓RAM↓context↓get itsources
1GPT-6 Astra OpenAI87±1api $20.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
2Claude Opus 5.5 Anthropic87±1api $8.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
3GPT-6.1 Sol OpenAI85±1api $4.00/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Miðeind Icelandic LLM Leaderboard
4Claude Fable 5.1 Anthropic85±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
5Claude Sonnet 5.5 Anthropic84±1api $4.00/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
6Claude Opus 5 Anthropic83±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+11
7GPT-6 Sol OpenAI83±1api $4.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard+1
8GPT-5.5 Pro OpenAI82±1api $67.50/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
9Gemini 4 Argon (limited preview) Google82±1limited previewLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)
10GPT-5.6 Sol OpenAI82±1api $8.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+14
11Claude Fable 5 Anthropic81±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runs+5
12Claude Mythos Preview (limited preview) Anthropic81±1†limited preview1MAnthropic system cards and model announcements (lab-reported scores)
13GPT-5.4 Pro OpenAI81±1api $67.50/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
14GPT-5.5 OpenAI80±1api $11.25/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAgents' Last Exam — authors' own fixed-harness model comparison+15
15MiMo-V2.6-Pro Xiaomi79±1api $0.54/M~443GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
16Hy4 preview (770B-A49B) Tencent79±1api $1.25/M~248GB1Mcloud.tencent.com ↗Tencent Hy4 preview model cardLMArena leaderboard dataset
17GPT-5.6 Terra OpenAI79±1api $4.50/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)+5
18Qwen3.8 Max (2.4T-A95B) Alibaba78±1api $3.00/M~380GB262kdeepinfra.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runs+2
19Claude Opus 4.8 Anthropic78±0api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+11
20GPT-5.4 OpenAI78±1api $5.63/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAgents' Last Exam — authors' own fixed-harness model comparison+14
21DeepSeek V4.1 Flash DeepSeek78±1api $0.26/M1Mapi-docs.deepseek.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
22Claude Haiku 5.5 Anthropic77±1api $0.20/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard dataset
23Gemini 3.8 Flash Google77±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+5
24Muse Spark 1.3 Meta77±1api $2.00/M1Mdev.meta.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
25Kimi K3 Moonshot AI77±1api $6.00/M~884GB1Mplatform.moonshot.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodeτ²/τ³-bench leaderboard: Sierra's own evaluation runs+6
26GPT-5.2 Pro OpenAI77±1api $57.75/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
27MiMo-V2.6-Flash (309B-A15B) Xiaomi77±1api $0.17/M~136GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
28GPT-6 Luna OpenAI77±1api $0.20/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
29GPT-5.6 Luna OpenAI77±1api $0.45/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
30Grok 4.6 SpaceXAI (xAI)77±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
31DeepSeek V4 Pro (0813) DeepSeek76±1api $1.63/M~1010GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party models+2
32Gemini 3.7 Flash Google76±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+3
33GLM-5.3 Z.ai76±1api $2.15/M~239GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
34Qwen3.8 Max (0902) Alibaba76±1api $3.00/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
38Qwen3.8-Omni-Flash Alibaba75±2†api $0.23/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
39Gemini 3.1 Pro (Preview) Google75±1api $4.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)meituan-longcat/LongCat-2.0 model cardAgents' Last Exam — authors' own fixed-harness model comparison+23
40Muse Spark 1.2 Meta75±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluations
41Claude Opus 4.7 Anthropic75±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAnthropic system cards and model announcements (lab-reported scores)Agents' Last Exam — authors' own fixed-harness model comparison+9
42Grok 4.5 SpaceXAI (xAI)75±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodeτ²/τ³-bench leaderboard: Sierra's own evaluation runs+1
43Muse Spark 1.1 Meta75±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsτ²/τ³-bench leaderboard: Sierra's own evaluation runsEpoch AI — internal evaluations
46Claude Sonnet 5 Anthropic75±1api $4.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
47Grok 4.7 SpaceXAI (xAI)75±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
48Claude Opus 4.6 Anthropic75±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAgents' Last Exam — authors' own fixed-harness model comparison+21
49GPT-5.3-Codex OpenAI74±1api $4.81/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
50DeepSeek V4 Flash (0731) DeepSeek74±1api $0.11/M~92GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardDaekeun Kim Korean LLM benchmark runsdeepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+2
51Qwen3.7 Max Alibaba74±1api $3.75/M1Mdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
52GLM-5.3-Flash Z.ai74±1api $0.24/M~103GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)+2
53Gemini 3.5 Flash Google74±1api $3.38/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Legal Research Bench (Vals AI, arXiv:2610.00609v1)Miðeind Icelandic LLM Leaderboard+3
54GPT-5.2 OpenAI74±1api $4.81/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+20
55Hunyuan Hy3 (295B-A21B) Tencent74±1api $0.23/M~96GB256kdeepinfra.com ↗tencent/Hy3 model cardTencent Hy4 preview model cardLMArena leaderboard datasetEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
56Gemini 3.6 Flash Google74±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 Flash+2
57GLM-5.2 Z.ai74±1api $2.15/M~121GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetszai-org/GLM-5.2 model cardLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+6
59Qwen3.8 27B Alibaba73±1api $1.60/M~18GB262kconsole.groq.com ↗Qwen/Qwen3.8-27B model cardLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+3
60Grok 4.20 Multi-Agent (0309) SpaceXAI (xAI)73±2api $1.56/M1Mdocs.x.ai ↗LMArena leaderboard dataset
61Kimi K2.6 Moonshot AI73±1api $1.71/M~317GB262kplatform.moonshot.ai ↗Epoch AI — internal evaluationsNVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparison+11
62GPT-5 Pro OpenAI73±1api $41.25/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
63LongCat 2.0 (1.6T-A48B) Meituan73±2api $1.30/M1Mlongcat.chat ↗meituan-longcat/LongCat-2.0 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
65GPT-5 OpenAI72±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+22
66Qwen3.6 Max Preview Alibaba72±1api $2.92/M256kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
67Claude Sonnet 4.6 Anthropic72±1api $6.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonKnowledge Index of Noah's Ark (arXiv:2606.05104v2)Legal Research Bench (Vals AI, arXiv:2610.00609v1)+16
68Gemini 3 Flash (Preview) Google72±1api $1.13/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGoogle DeepMind Gemini model page and model cards (lab-reported scores)+6
70Claude Opus 4.5 Anthropic72±1api $10.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsQwen/Qwen3.6-27B model cardAnthropic system cards and model announcements (lab-reported scores)+13
71Qwen3.7 Plus Alibaba72±1api $0.70/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LMArena leaderboard datasetSciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
72Kimi K2.7 Code Moonshot AI72±1api $1.71/M~359GB262kplatform.kimi.ai ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
74Grok 4.20 (0309 reasoning) SpaceXAI (xAI)72±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsτ²/τ³-bench leaderboard: Sierra's own evaluation runsThe Token Games: Evaluating Language Model Reasoning with Puzzle Duels — authors' own tournamentWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
76North Small Translate 1.0 Cohere71±6†api price unknown16kdocs.cohere.com ↗Cohere — North Small Translate (arXiv:2609.13916v1)
77GPT-5.4 mini OpenAI71±1api $1.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsLegal Research Bench (Vals AI, arXiv:2610.00609v1)+6
78Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)71±2†api $0.090/M~75GB131kdeepinfra.com ↗inclusionAI/Ling-3.0-flash model card
79Mistral Large 4 (public preview) Mistral71±2api $2.06/M1Mdocs.mistral.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluations
80GPT-5.1 OpenAI71±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsHyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+7
81GLM-5.1 Z.ai71±1api $2.15/M~119GB200kdocs.z.ai ↗NVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetszai-org/GLM-5.1 model cardzai-org/GLM-5.2 model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1+11
82Qwen3.5 397B-A17B Alibaba71±0api $1.09/M~65GB262kdeepinfra.com ↗Qwen/Qwen3.5-397B-A17B model cardNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsQwen/Qwen3.6-27B model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard dataset+15
84MiniMax-M3 MiniMax70±1api $0.48/M~137GB1Mdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Miðeind Icelandic LLM Leaderboard+2
85Grok 4.3 SpaceXAI (xAI)70±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
87o3-pro OpenAI70±2api $35.00/M200kdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
89Qwen3.6 Plus Alibaba70±1api $1.13/M1Mdocs.together.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Agents' Last Exam — authors' own fixed-harness model comparisonQwen/Qwen-AgentWorld-35B-A3B model cardBEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+1
90Inkling Small (266B-A12B) Thinking Machines70±1api $0.64/M~86GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetMiðeind Icelandic LLM LeaderboardCohere — North Small Translate (arXiv:2609.13916v1)
92Kimi K2.5 Moonshot AI70±1api $1.20/M~160GB262kdocs.aws.amazon.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetstepfun-ai/Step-3.5-Flash model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsQwen/Qwen3.5-397B-A17B model card+15
93GLM-5 Z.ai70±1api $1.55/M~119GB205kdocs.z.ai ↗zai-org/GLM-5.1 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGLM-5 Team (Zhipu AI) — GLM-5 technical reportEpoch AI — internal evaluationszai-org/GLM-5 model cardLMArena leaderboard dataset+11
95MiMo-V2.5-Pro (1T-A42B) Xiaomi70±1api $0.65/M~163GB1Mnovita.ai ↗LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardXiaomi MiMo official model cards and documentationGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
96Inkling (975B-A41B) Thinking Machines70±1api $1.73/M~156GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsMiðeind Icelandic LLM Leaderboard
97Muse Glimmer 30B Meta70±1api $0.64/M~19GB131kfireworks.ai ↗meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.8-27B model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Cohere — North Small Translate (arXiv:2609.13916v1)
98Nemotron 3 Ultra 550B-A55B NVIDIA69±1api $1.35/M~89GB1Mdocs.together.ai ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1Epoch AI — internal evaluationsLMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)
99Qwen3.6 27B Alibaba69±1api $1.20/M~18GB262kconsole.groq.com ↗Qwen/Qwen3.6-27B model cardEpoch AI — internal evaluationsQwen/Qwen3.8-27B model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+4
102o3 OpenAI69±1api $3.50/M200kplatform.openai.com ↗Epoch AI — internal evaluationsMeituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard dataset+11
103MiMo-V2.5 (310B-A15B) Xiaomi69±1api $0.21/M~100GB1Mnovita.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5 model cardAgents' Last Exam — authors' own fixed-harness model comparisonXiaomi MiMo vendor-reported benchmark scores for its own models (mimo.xiaomi.com launch pages)SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)
104Qwen3.5 Plus Alibaba69±2api $0.90/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
105Qwen3.5 122B-A10B Alibaba69±1api $1.10/M~75GB262kwww.alibabacloud.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.5-27B model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)Epoch AI — internal evaluations+5
106Gemini 3.5 Flash-Lite Google69±1api $0.85/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Miðeind Icelandic LLM Leaderboard
107GLM-4.7 Z.ai69±1api $1.00/M~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardGLM-5 Team (Zhipu AI) — GLM-5 technical reportzai-org/GLM-5 model cardLMArena leaderboard datasetEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+5
109GPT-5.4 nano OpenAI69±1api $0.46/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeT1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)+1
110Claude Sonnet 4.5 Anthropic69±1api $6.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+18
111Qwen3.5 27B Alibaba69±0api $0.84/M~18GB262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.6-27B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard dataset+8
112GLM-5V-Turbo Z.ai69±2api $1.90/M200kwww.siliconflow.com ↗LMArena leaderboard dataset
115Step 3.5 Flash (196B-A11B) StepFun68±1api $0.15/M~118GB256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsstepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
116DeepSeek V3.2 DeepSeek68±1api $0.29/M~110GB128kdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsdeepseek-ai/DeepSeek-V3.2 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsstepfun-ai/Step-3.5-Flash model card+15
120MiniMax-M2.7 MiniMax68±1api $0.53/M~75GB205kplatform.minimax.io ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard datasetinclusionAI/Ling-3.0-flash model cardQwen/Qwen-AgentWorld-35B-A3B model cardWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+2
121Qwen3.6 35B-A3B Alibaba68±1api $0.31/M~23GB262kdeepinfra.com ↗Qwen/Qwen3.6-27B model cardQwen/Qwen3.6-35B-A3B model cardEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
124GPT-5 mini OpenAI67±1api $0.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardDaekeun Kim Korean LLM benchmark runsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)LMArena leaderboard dataset+16
125Gemini 2.5 Pro Google67±1api $3.44/M1Mai.google.dev ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+16
129Kimi K2 Thinking Moonshot AI67±1api $1.07/M~160GB262kdocs.aws.amazon.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsstepfun-ai/Step-3.5-Flash model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
130Laguna M.1 Poolside67±2api price unknown~135GB262khuggingface.co ↗LMArena leaderboard datasetPoolside — Laguna M.1/XS.2 Technical Report
131Qwen3.7 Flash Alibaba67±1api $0.055/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluations
132Step 3.7 Flash (198B-A11B) StepFun67±1api $0.44/M~104GB256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsinclusionAI/Ling-3.0-flash model card
133Gemma 4 31B Google67±1api $0.19/M~21GB262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportKwaipilot/KAT-Coder-V2.5-Dev model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetEpoch AI — internal evaluationsQwen/Qwen3.6-35B-A3B model card+20
134Gemini 3.1 Flash-Lite Google67±1api $0.56/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs — authors' own model runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
135Qwen3-Coder-Next (80B-A3B) Alibaba67±2api $0.60/M~27GB262kwww.alibabacloud.com ↗Vector Institute Evaluation Leaderboard (Inspect Evals runs)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs
139Qwen3.5 35B-A3B Alibaba67±0api $0.35/M~23GB262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardNVIDIA — Nemotron-Cascade 2 technical reportLMArena leaderboard dataset+12
142Qwen3 235B-A22B Thinking 2507 Alibaba66±1api $0.97/M~76GB131knovita.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-Next-80B-A3B-Thinking model cardQwen3-VL-235B-A22B-Thinking model cardQwen/Qwen3.5-27B model cardEpoch AI — internal evaluations+6
143Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)66±1api $0.28/M128kazure.microsoft.com ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runsBerkeley Function Calling Leaderboard result archiveWebApp1K models leaderboard (onekq-ai)+4
144o4-mini OpenAI66±1api $1.93/M200kplatform.openai.com ↗Epoch AI — internal evaluationsQwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)+12
145Qwen3.6 Flash Alibaba66±1api $0.56/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsSpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
147GLM-4.6 Z.ai66±1api $1.00/M~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardzai-org/GLM-4.6 model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsBerkeley Function Calling Leaderboard result archiveWebApp1K models leaderboard (onekq-ai)+1
148Kimi K2 Instruct (0711) Moonshot AI66±1api $1.00/M~160GB131knovita.ai ↗Meituan LongCat-Flash-Chat model card (evaluation table)Aider polyglot benchmark runsMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)LMArena leaderboard datasetEQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archive+3
149o1 (2024-12-17) OpenAI66±1api $26.25/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results+13
150MiniMax-M2.5 MiniMax66±1api $0.53/M~75GB205kplatform.minimax.io ↗LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+3
151Qwen3-Max (2025-09-23) Alibaba66±1api $2.40/M262kwww.alibabacloud.com ↗Epoch AI — internal evaluationsτ²-bench leaderboard: Qwen (Alibaba) own submissionsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)
152DeepSeek V3.1 DeepSeek66±1api $0.42/M~110GB131kdeepinfra.com ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Meituan LongCat-Flash-Chat model card (evaluation table)deepseek-ai/DeepSeek-V3.1 model cardDeepSeek-V3.1-Terminus model cardLMArena leaderboard datasetRPC-Bench leaderboard (Z.ai research paper comprehension)+3
153Spark-X2.5 4B · 4-bit iFLYTEK (SparkLLM)66±1†local~4.2GB1Mllama.cpp › XHToken/Spark-X2.5-4B model card
158o3-mini OpenAI65±1api $1.93/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardMicrosoft — Phi-4-reasoning Technical ReportOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runs+13
163DeepSeek R1 0528 DeepSeek65±1api $0.91/M~110GB164kdeepinfra.com ↗deepseek-ai/DeepSeek-R1-0528 model cardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)deepseek-ai/DeepSeek-V3.1 model cardServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+6
165Qwen3.5 Flash Alibaba65±1api $0.17/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
168Nemotron 3 Super 120B-A12B NVIDIA65±1api $0.28/M~73GB262kdocs.aws.amazon.com ↗NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)NVIDIA — Nemotron-Cascade 2 technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetinclusionAI/Ling-3.0-flash model card
169GLM-4.5 Z.ai65±1api $1.00/M~58GB131kdocs.z.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)zai-org/GLM-4.6 model cardLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard results+3
170Solar Pro 4 Upstage65±2api $0.53/M524kdevelopers.upstage.ai ↗LMArena leaderboard dataset
171MiniMax-M2 MiniMax65±1api $0.53/M~75GB205kdocs.aws.amazon.com ↗LMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
173Qwen3 235B-A22B Instruct 2507 Alibaba65±1api $0.20/M~76GB262kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMeituan LongCat-Flash-Chat model card (evaluation table)Qwen3-VL-235B-A22B-Instruct model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsLMArena leaderboard dataset+3
174Mistral Medium 3.5 (128B) Mistral65±1api $3.00/M~42GB262kdocs.mistral.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)Miðeind Icelandic LLM LeaderboardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
175Command A+ (05-2026) Cohere64±1api price unknown~71GB128kdocs.cohere.com ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Cohere vendor-reported benchmark scores for its own models (cohere.com blog)Cohere — North Small Translate (arXiv:2609.13916v1)
176Claude Haiku 4.5 Anthropic64±1api $2.00/M200kdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+9
179Qwen3-Coder 480B-A35B Instruct Alibaba64±1api $0.67/M~78GB262knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
181Gemma 4 26B-A4B Google64±1api $0.14/M~18GB262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical Reportgoogle/diffusiongemma-26B-A4B-it model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+12
184Nova 2 Pro (Preview) Amazon64±2api $3.78/M1Maws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsAmazon AGI — τ²-Bench-Verified
186MiniCPM5 2B · 4-bit OpenBMB64±1†local~4.3GB131kLM Studio › openbmb/MiniCPM5-2B model card
188Qwen3-Next 80B-A3B Thinking Alibaba63±1api $0.41/M~27GB131kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3.5-9B model cardQwen/Qwen3.5-4B model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)+3
189DeepSeek R1 (Jan '25) DeepSeek63±1api $4.00/M~110GB128kdocs.together.ai ↗deepseek-ai/DeepSeek-R1 model carddeepseek-ai/DeepSeek-R1-0528 model cardEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsWildEval ZeroEval published results+15
191Qwen3-Next 80B-A3B Instruct Alibaba63±1api $0.34/M~27GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
192Granite 4.2 30B IBM63±1api $0.28/M~19GB131kdeepinfra.com ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
195GLM-4.5-Air Z.ai63±1api $0.42/M~35GB131kdocs.z.ai ↗GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetSolar Open Technical Report (arXiv:2601.07022v1)Berkeley Function Calling Leaderboard result archive+3
196Qwen3.5 9B Alibaba63±1api $0.11/M~7.4GB262kdeepinfra.com ↗Qwen/Qwen3.5-9B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+17
199Trinity-Large-Thinking Arcee AI63±2api $0.39/M~141GB512kdocs.arcee.ai ↗LMArena leaderboard datasetArcee AI vendor-reported benchmark scores for its own Trinity models (Hugging Face model cards)
201MiniMax-M1 (80k) MiniMax62±1api $0.96/M~959GB1Mnovita.ai ↗MiniMaxAI/MiniMax-M1-80k model cardLMArena leaderboard dataset
203GPT-4.1 OpenAI62±1api $3.50/M1Mplatform.openai.com ↗Meituan LongCat-Flash-Chat model card (evaluation table)Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsDaekeun Kim Korean LLM benchmark runsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+17
204Nemotron 3.5 Lightning 30B-A3B NVIDIA62±2api $0.087/M~20GB262kfireworks.ai ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard dataset
205GLM-4.6V Z.ai62±2api $0.45/M~51GB128kdocs.z.ai ↗LMArena leaderboard datasetZ.ai GLM-V GitHub repository benchmark image (GLM-4.6V)SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)
209gpt-oss-120b OpenAI62±1api $0.26/M~67GB131kconsole.groq.com ↗OpenAI — gpt-oss-120b & gpt-oss-20b Model CardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsAider polyglot benchmark runsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+19
210GPT-5 nano OpenAI62±1api $0.14/M400kplatform.openai.com ↗Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+7
211Qwen3-VL 32B Thinking Alibaba62±1api $0.28/M~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)allenai/Olmo-3.1-32B-Think model card
213Mistral Large 3 (675B-A41B) Mistral62±1api $0.75/M~108GB262kdocs.mistral.ai ↗LMArena leaderboard datasetmistralai/Mistral-Small-4-119B-2603 model cardCohere — North Small Translate (arXiv:2609.13916v1)LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? (arXiv:2610.04118v1)Miðeind Icelandic LLM LeaderboardOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+1
216Gemini 2.5 Flash Google62±1api $0.85/M1Mai.google.dev ↗Meituan LongCat-Flash-Chat model card (evaluation table)LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+10
219GLM-4.5V Z.ai61±2api $0.90/M~69GB64kdocs.z.ai ↗Z.ai GLM-V GitHub repository benchmark image (GLM-4.6V)LMArena leaderboard datasetServiceNow-AI/Apriel-1.6-15b-Thinker model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
225Qwen3 30B-A3B Thinking 2507 Alibaba61±1api $0.75/M~20GB262kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardByteDance-Seed/Seed-OSS-36B-Instruct model cardQwen/Qwen3.5-9B model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+8
226Qwen3.5 4B · 4-bit Alibaba61±1local~5GB262kOllama › Qwen/Qwen3.5-9B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardQwen/Qwen3.5-4B model cardLiquidAI/LFM2.5-2.6B model card+10
229GPT-4.1 mini OpenAI60±1api $0.70/M1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runsAider polyglot benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Arena-Hard-Auto v2.0-Preview leaderboard (repository README)+13
231Qwen3 30B-A3B Instruct 2507 Alibaba60±1api $0.15/M~20GB262ktokenfactory.nebius.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
232Qwen3-VL 32B Instruct Alibaba60±1†api $0.28/M~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
237Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba60±1api $1.23/M~76GB131kwww.alibabacloud.com ↗Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+14
239Granite 4.2 8B IBM60±1api $0.11/M~6.6GB131kwandb.ai ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
241Qwen3-VL 30B-A3B Thinking Alibaba60±1†api $0.75/M~21GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
242Mistral Small 4 (119B-A6.5B) Mistral60±2api $0.26/M~40GB262kdocs.mistral.ai ↗mistralai/Mistral-Small-4-119B-2603 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Miðeind Icelandic LLM Leaderboard
246DeepSeek V3 (Dec '24) DeepSeek59±1api $0.46/M~337GB131kdeepinfra.com ↗deepseek-ai/DeepSeek-R1 model cardCohere — Command A technical reportEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsWildEval ZeroEval published results+10
250GLM-4.7-Flash Z.ai59±1api $0.0000/M~19GB200kdocs.z.ai ↗Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardEQ-Bench 3 canonical leaderboard resultsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
254Qwen3 14B Alibaba59±1api $0.61/M~10GB33kwww.alibabacloud.com ↗Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBerkeley Function Calling Leaderboard result archiveBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+6
257gpt-oss-20b OpenAI58±1api $0.13/M~14GB131kconsole.groq.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardOpenAI — gpt-oss-120b & gpt-oss-20b Model CardEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)+15
260DeepSeek R1 Distill Llama 70B DeepSeek58±1api $0.80/M~30GB131knovita.ai ↗Epoch AI — internal evaluationsMicrosoft — Phi-4-reasoning Technical Reportdeepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Judgemark v2 results (EQ-Bench)+1
261Phi-4-reasoning 14B Microsoft58±1api $0.22/M~10GB33kazure.microsoft.com ↗Microsoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
263Magistral Small 1.2 Mistral58±1api $0.75/M~16GB131kdocs.aws.amazon.com ↗mistralai/Magistral-Small-2509 model cardEpoch AI — internal evaluationsApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarksmistralai/Mistral-Small-4-119B-2603 model cardInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)EuroEval leaderboards (results collection), English generative leaderboard
265Qwen3-VL 30B-A3B Instruct Alibaba58±1†api $0.35/M~21GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
266Qwen3-VL 8B Thinking Alibaba58±1api $0.66/M~6.8GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Think model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
267Qwen3 32B Alibaba58±0api $0.13/M~20GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-Next-80B-A3B-Instruct model cardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+17
268Qwen3 Coder 30B-A3B Instruct Alibaba58±1api $0.12/M~20GB160knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
269Nova 2 Lite Amazon58±2api $0.94/M1Mdocs.aws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveAmazon AGI — τ²-Bench-VerifiedVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
271Nemotron 3 Nano 30B-A3B NVIDIA58±1api $0.087/M~20GB262kdeepinfra.com ↗NVIDIA — Nemotron-Cascade 2 technical reportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)EuroEval leaderboards (results collection), English generative leaderboardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+3
273GPT-4o (2024-08-06) OpenAI57±1api $4.38/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runs+23
275DeepSeek R1 0528 Qwen3 8B DeepSeek57±1api $0.068/M~6.3GB128knovita.ai ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scalingdeepseek-ai/DeepSeek-R1-0528-Qwen3-8B model cardEpoch AI — internal evaluationsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
276Granite 4.2 3B · 4-bit IBM57±1local~4.3GB131kOllama › openbmb/MiniCPM5-2B model cardibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
277Command A (03-2025) Cohere57±1api $4.38/M~37GB256kazure.microsoft.com ↗Cohere — Command A technical reportLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
278GPT-4o (2024-11-20) OpenAI57±1api $4.38/M128kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Qwen/Qwen3-235B-A22B-Instruct-2507 model cardAider polyglot benchmark runsQwen/Qwen3-30B-A3B-Instruct-2507 model cardMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+7
279Gemini 2.5 Flash-Lite Google57±2api $0.17/M1Mai.google.dev ↗Google DeepMind Gemini model page and model cards (lab-reported scores)Berkeley Function Calling Leaderboard result archiveQwen/Qwen3.5-4B model cardMiðeind Icelandic LLM LeaderboardGalileo Agent Leaderboard (v1 and v2 results)Open Portuguese LLM Leaderboard (archived)+1
282GPT-4o (2024-05-13) OpenAI57±0api $7.50/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)OpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runs+19
283QED-Nano · 4-bit LM Provers57±2local~4.1GB262kllama.cpp › MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
284Llama 4 Maverick (402B-A17B) Meta57±1api $0.42/M~65GB1Mdocs.aws.amazon.com ↗Epoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarks+14
286Qwen2.5 Coder 32B Instruct Alibaba57±1api $0.74/M~21GB131kdevelopers.cloudflare.com ↗EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
287Qwen3 4B Thinking 2507 · 4-bit Alibaba57±1local~4.6GB262kOllama › Qwen/Qwen3-4B-Thinking-2507 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Qwen/Qwen3.5-0.8B model cardKnowledge Index of Noah's Ark (arXiv:2606.05104v2)+2
293Qwen Plus (2025-01-25) Alibaba56±1api $0.60/M131kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
295Qwen3-VL 4B Thinking · 4-bit Alibaba56±1local~5GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)
297DeepSeek R1 Distill Qwen 32B DeepSeek56±1api $1.59/M~21GB131kdevelopers.cloudflare.com ↗open-thoughts/OpenThinker2-32B model carddeepseek-ai/DeepSeek-R1 model cardopen-thoughts/OpenThinker-32B model cardEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+5
300Mistral Small 3.2 24B Mistral56±1api $0.11/M~16GB128kdeepinfra.com ↗mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetmistralai/Mistral-Small-4-119B-2603 model cardEQ-Bench 3 canonical leaderboard resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+4
302Qwen3 8B Alibaba56±1api $0.31/M~6.1GB33kwww.alibabacloud.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)allenai/Olmo-3-7B-Think model cardSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+14
304Ministral 3 14B Mistral56±1api $0.20/M~9.1GB262kdocs.mistral.ai ↗JetBrains/Mellum2-12B-A2.5B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)mistralai/Ministral-3-8B-Instruct-2512 model card+1
305Nemotron Nano 9B v2 NVIDIA56±1api $0.10/M~7.6GB131kdocs.aws.amazon.com ↗Olmo 3NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)allenai/Olmo-3-7B-Think model cardEuroEval leaderboards (results collection), English generative leaderboard
307GPT-4 Turbo (2024-04-09) OpenAI55±0api $15.00/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)+18
308Yi-Lightning 01.AI55±1api price unknown16kplatform.lingyiwanwu.com ↗Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard dataset
309Qwen2.5 72B Instruct Alibaba55±1api $0.37/M~30GB33kdeepinfra.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical Report2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)+24
313Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba55±1api $0.35/M~19GB33kwww.alibabacloud.com ↗Qwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsArena-Hard-Auto v2.0-Preview leaderboard (repository README)+10
317Qwen3 4B Instruct 2507 · 4-bit Alibaba55±1local~4.6GB262kOllama › Qwen/Qwen3-4B-Instruct-2507 model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+4
319Qwen3-VL 8B Instruct Alibaba55±1api $0.31/M~6.8GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Instruct model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
320Spark-X2.5 1.7B · 16-bit iFLYTEK (SparkLLM)55±1†local~5.1GB1MOllama › XHToken/Spark-X2.5-4B model card
322Llama 3.1 405B Instruct Meta55±1api $2.40/M~160GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+13
333Llama 3.3 70B Instruct Meta54±1api $0.72/M~24GB128kdocs.aws.amazon.com ↗Cohere — Command A technical reportEpoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical Report2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+21
334Llama 4 Scout (109B-A17B) Meta54±1api $0.15/M~36GB328kdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveEQ-Bench 3 canonical leaderboard resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+8
335Nemotron 3 Nano 4B · 4-bit NVIDIA54±1local~4.5GB262kOllama › openbmb/MiniCPM5-2B model cardNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
336Granite 4.1 8B IBM54±1api $0.11/M~7GB131kreplicate.com ↗ibm-granite/granite-4.1-8b model cardLMArena leaderboard datasetSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)
339Nova Premier Amazon54±2†api $5.00/M1Maws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)MASSIVE-Agents — Kulkarni et al. (2025)
353GPT-4o mini (2024-07-18) OpenAI53±0api $0.26/M128kdevelopers.openai.com ↗EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsmicrosoft/Phi-4-mini-instruct model cardVector Institute Evaluation Leaderboard (Inspect Evals runs)Epoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+32
362GPT-4.1 nano OpenAI53±1api $0.17/M1Mdevelopers.openai.com ↗OpenAI simple-evals README benchmark results (OpenAI's own models)Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetAider polyglot benchmark runs+11
364Gemma 3 27B Google53±1api $0.10/M~18GB131kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportGoogle DeepMind Gemma 3 270M IT model cardEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard dataset+23
365Phi-4 14B Microsoft53±1api $0.087/M~10GB16kdeepinfra.com ↗Epoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical ReportMicrosoft — Phi-4-reasoning Technical ReportDaekeun Kim Korean LLM benchmark runsmicrosoft/phi-4 model cardmicrosoft/Phi-4-reasoning model card+11
367Ministral 3 8B Mistral53±1api $0.15/M~6.5GB262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)mistralai/Ministral-3-8B-Instruct-2512 model card+2
368Qwen3-VL 4B Instruct · 4-bit Alibaba53±1local~5GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
370LFM2.5 2.6B · 4-bit Liquid AI53±1local~4.5GB131kLM Studio › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
372Llama 3.1 70B Instruct Meta52±1api $0.72/M~27GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+22
380Granite 4.0 H Small (32B-A9B) IBM52±1api $0.11/M~21GB131kreplicate.com ↗ibm-granite/granite-4.0-h-small model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)+1
384Aya Expanse 32B Cohere52±2api $0.75/M~22GB128kdocs.cohere.com ↗LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)+1
385Gemma 3 12B Google52±1api $0.075/M~9.1GB131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportEpoch AI — internal evaluationsMIMIC Clinical Decision Making leaderboard+10
390GPT-4 (0613) OpenAI51±1api $37.50/M8kdevelopers.openai.com ↗Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsTulu 2 — base and instruction-tuned evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+9
392Command A Translate (08-2025) Cohere51±5†api price unknown8kdocs.cohere.com ↗Cohere — North Small Translate (arXiv:2609.13916v1)
394Ministral 3 3B Reasoning 2512 · 4-bit Mistral51±1local~5.3GB262kLM Studio › mistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboard
395Qwen3 4B · 4-bit Alibaba51±1local~4.6GB33kOllama › Qwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-4B-Instruct-2507 model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationHuggingFaceTB/SmolLM3-3B model card (Evaluation section)Epoch AI — internal evaluationsArena-Hard-Auto v2.0-Preview leaderboard (repository README)+8
409EXAONE Deep 2.4B · 4-bit LG AI Research50±2†local~3.2GB33kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
423Nova Pro Amazon50±1api $1.40/M300kdocs.aws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)Aider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
430Granite 4.1 3B · 4-bit IBM49±1†local~3.7GBOllama › ibm-granite/granite-4.1-8b model card
433LFM2.5 1.2B Thinking · 8-bit Liquid AI49±2local~3.5GB33kOllama › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
434Qwen3-VL 2B Thinking · 8-bit Alibaba49±1local~4.2GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
437Nova Lite Amazon49±2api $0.10/M300kdocs.aws.amazon.com ↗LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
448Qwen2.5 7B Instruct Alibaba48±1api $0.30/M~6GB131kwww.together.ai ↗microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Epoch AI — internal evaluations+12
455T5 Large · 4-bit Google48±9local~4.6GB1khow to run › COPEN — EMNLP 2022 original zero-shot evaluations
465Qwen2.5 Coder 3B Instruct · 4-bit Alibaba48±1local~5GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
467MiniCPM3-4B · 4-bit OpenBMB48±2local~4.1GB32kllama.cpp › LiveCodeBench official website results (2024-12-26)Berkeley Function Calling Leaderboard result archive
468Bielik 4.5B v3.0 Instruct · 4-bit SpeakLeash48±2local~4.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
469Qwen3.5 2B · 8-bit Alibaba47±1local~4.9GB262kOllama › Qwen/Qwen3.5-2B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)EuroEval leaderboards (results collection), English generative leaderboardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets+7
471Command R Plus (08-2024) Cohere47±1api $4.38/M~63GB128kdocs.cohere.com ↗Cohere — Command A technical reportLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+1
474Phi-4-mini-reasoning 3.8B · 4-bit Microsoft47±2local~4.8GB128kOllama › microsoft/Phi-4-mini-reasoning model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
478EXAONE 4.0 1.2B · 8-bit LG AI Research47±2†local~3.6GB66kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
481Gemma 4 E2B · 4-bit Google47±1local~5.2GB131kOllama › Gemma Team — Gemma 4 Technical Reportopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)LiquidAI/LFM2.5-2.6B model cardEuroEval leaderboards (results collection), English generative leaderboard+6
488Granite 4.0 H Micro 3B · 4-bit IBM47±1local~5.1GB131kLM Studio › ibm-granite/granite-4.0-h-small model cardEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
489GPT-3.5 Turbo (0125) OpenAI46±1api $0.75/M16kdevelopers.openai.com ↗WildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+11
492MedGemma 4B IT · 4-bit Google46±3local~5GBOllama › Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Portuguese LLM Leaderboard (archived)SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
496Cogito v1 Preview Llama 3B · 4-bit Deep Cogito46±4local~5.1GB128kLM Studio › EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
500Granite 4.0 Micro 3B · 8-bit IBM46±1local~5.3GB131kllama.cpp › ibm-granite/granite-4.0-h-small model cardEpoch AI — internal evaluationsEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
501Command R (08-2024) Cohere46±1api $0.26/M~21GB128kdocs.cohere.com ↗LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsCohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+3
503Nova Micro Amazon46±2api $0.061/M128kdocs.aws.amazon.com ↗LMArena leaderboard datasetMASSIVE-Agents — Kulkarni et al. (2025)Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
504T5 Base · 4-bit Google46±9local~2.4GB1khow to run › COPEN — EMNLP 2022 original zero-shot evaluations
507Qwen2.5-Math 1.5B Instruct · 4-bit Alibaba46±2†local~2.5GB4kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
524Ministral 3 3B · 4-bit Mistral45±1local~5.3GB262kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsmistralai/Ministral-3-14B-Instruct-2512 model cardmistralai/Ministral-3-8B-Instruct-2512 model cardEuroEval leaderboards (results collection), English generative leaderboard+2
534Qwen2-Math 1.5B Instruct · 8-bit Alibaba45±2†local~3.2GB4kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
536Llama 3.1 8B Instruct Meta45±0api $0.22/M~6.2GB128kdocs.aws.amazon.com ↗EvalPlus / EvalPerf published evaluation runsIBM Granite 3.0 Language Models technical report (GitHub)microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets+25
540Phi-3-mini-4k-instruct (June 2024 update) · 4-bit Microsoft44±1local~4GBllama.cpp › WildEval ZeroEval published resultsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
544Gemma 3 4B · 4-bit Google44±1local~4.7GB131kOllama › Google DeepMind Gemma 3 270M IT model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboardLMArena leaderboard dataset+11
547Qwen2.5-VL 3B Instruct · 8-bit Alibaba44±1local~5GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
553Qwen3-VL 2B Instruct · 8-bit Alibaba44±1local~4.2GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
558Qwen3 1.7B · 8-bit Alibaba44±1local~4.6GB33kOllama › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen/Qwen3.5-2B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiquidAI/LFM2-1.2B model cardHuggingFaceTB/SmolLM3-3B model card (Evaluation section)LiquidAI/LFM2.5-1.2B-Thinking model card+10
559Phi-4-mini 3.8B · 4-bit Microsoft44±1local~4.4GB128kOllama › microsoft/Phi-4-mini-instruct model cardMicrosoft — Phi-4-Mini Technical ReportDaekeun Kim Korean LLM benchmark runsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+4
564LFM2.5 1.2B Instruct · 4-bit Liquid AI44±2local~2.8GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
566Phi-3.5-mini-instruct · 4-bit Microsoft43±1local~4.4GB128kOllama › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsDaekeun Kim Korean LLM benchmark runsMicrosoft — Phi-4-Mini Technical ReportSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+4
571Gemma 3n E2B IT · 4-bit Google43±2local~4.7GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model cardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
573T5 Small · 4-bit Google43±9local~1.8GB1khow to run › COPEN — EMNLP 2022 original zero-shot evaluations
575Qwen2.5 Coder 1.5B Instruct · 8-bit Alibaba43±1local~3.5GB33kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
576LFM2 2.6B · 4-bit Liquid AI43±2local~4.3GB33kLM Studio › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationEuroEval leaderboards (results collection), English generative leaderboard
577SmolLM3 3B · 4-bit Hugging Face43±1local~5GB66kLM Studio › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationHuggingFaceTB/SmolLM3-3B model card (Evaluation section)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
580Qwen2.5 3B Instruct · 8-bit Alibaba43±1local~5.3GB33kllama.cpp › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SpeakLeash Polish EQ-Bench leaderboard+5
581DeepSeek-Coder 1.3B Instruct · 8-bit DeepSeek43±1local~3GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
585Aya Vision 32B Cohere42±3api price unknown~71GB16kdocs.cohere.com ↗LMArena leaderboard datasetKaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark (arXiv:2508.13680v5)
613North Micro Vision Instruct (2.4B) · 4-bit Cohere41±2†local~3.8GB128khow to run › CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
615Granite 4.0 1B · 8-bit IBM41±1local~3.3GB128kllama.cpp › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model cardEpoch AI — internal evaluations
619Granite 4.0 H 1B · 8-bit IBM40±1local~3.2GB131kOllama › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model card
627LFM2 1.2B · 4-bit Liquid AI40±2local~2.8GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
630Bielik 1.5B v3.0 Instruct · 16-bit SpeakLeash40±3local~4.8GBhow to run › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
632Granite 3.3 2B Instruct · 8-bit IBM39±1†local~4.3GB128kllama.cpp › ibm-granite/granite-3.3-8b-instruct model card
641Falcon-H1 3B Instruct · 8-bit TII39±4local~5.1GB131kllama.cpp › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
642Phi-2 (2.7B) · 8-bit Microsoft39±1local~4.7GB2kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)+2
643Granite 3.0 2B Instruct · 8-bit IBM39±1local~5.3GB4kOllama › IBM Granite 3.0 Language Models technical report (GitHub)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
648Llama 3.2 3B Instruct · 8-bit Meta38±1local~5.1GB131kllama.cpp › microsoft/Phi-4-mini-instruct model cardIBM Granite 3.0 Language Models technical report (GitHub)LFM2 Technical Report — Liquid AI own SmolLM3 evaluationAider code editing benchmark runs (old edit leaderboard)microsoft/Phi-4-mini-reasoning model cardMicrosoft — Phi-4-Mini Technical Report+11
654Phi-3-mini-4k-instruct (Apr 2024) · 4-bit Microsoft38±2local~4GBllama.cpp › MMLU-Pro author-released evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
657Granite 3B Code Instruct (2K) · 4-bit IBM38±2local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
660Qwen3.5 0.8B · 8-bit Alibaba38±1local~3.1GB262kOllama › Qwen/Qwen3.5-2B model cardEuroEval leaderboards (results collection), English generative leaderboardQwen/Qwen3.5-0.8B model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+2
665Yi-1.5 6B (base) · 4-bit 01.AI37±3local~5.1GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
666Qwen2-VL 2B Instruct · 4-bit Alibaba37±1local~3.9GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
667DeepSeek R1 Distill Qwen 1.5B · 8-bit DeepSeek37±2local~4.3GB131kOllama › deepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Epoch AI — internal evaluationsTHINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models (arXiv:2505.22113v1)EuroEval leaderboards (results collection), English generative leaderboard+1
668Gemma 2 2B IT · 8-bit Google37±1local~5.3GB8kOllama › IBM Granite 3.0 Language Models technical report (GitHub)WildEval ZeroEval published resultsWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+4
669Qwen2.5 Coder 0.5B Instruct · 4-bit Alibaba37±1local~2GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
676Yi 1.5 6B Chat · 4-bit 01.AI37±2local~5.1GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4WildBench v2 leaderboard data (Ai2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
677Falcon3 3B Instruct · 8-bit TII37±3local~5.1GB32kllama.cpp › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
680Granite 3.1 2B Instruct · 8-bit IBM36±1local~5.2GB128kOllama › ibm-granite/granite-3.3-8b-instruct model cardibm-granite/granite-3.1-3b-a800m-instruct model cardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
681EXAONE 3.5 2.4B Instruct · 8-bit LG AI Research36±2local~5.1GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
684LFM2 700M · 4-bit Liquid AI36±2local~2.3GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
685Granite 3.2 2B Instruct · 8-bit IBM36±1local~5.2GB131kOllama › ibm-granite/granite-3.3-8b-instruct model cardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
692Qwen2.5 1.5B Instruct · 8-bit Alibaba36±1local~4GB33kOllama › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+6
699Granite 3.0 3B-A800M Instruct · 8-bit IBM35±1local~5.3GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
702Granite 3.1 3B-A800M Instruct · 8-bit IBM35±2†local~5.2GB131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
709Qwen3 0.6B · 8-bit Alibaba33±1local~3GB33kOllama › Olmo 3LiquidAI/LFM2-1.2B model cardBerkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard+4
712StarCoder2 3B · 4-bit BigCode33±1local~5.1GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
713Granite 3B Code Base (2K) · 4-bit IBM33±1local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
718Stable Code 3B · 8-bit Stability AI33±1local~4.7GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
719OLMo 2 1B Instruct (0425) · 8-bit Allen Institute for AI33±2local~3.2GB4kllama.cpp › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsOpen Portuguese LLM Leaderboard (archived)
720DeepSeek-Coder 1.3B Base · 8-bit DeepSeek32±1local~3GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
724Qwen1.5 4B Chat · 4-bit Alibaba32±2local~4.5GB32kOllama › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
726CodeGemma 2B (v1.0) · 8-bit Google32±1local~4.3GBOllama › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
728Phi-1.5 (1.3B) · 4-bit Microsoft32±1local~4.5GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
729Falcon-H1 1.5B Instruct · 16-bit TII31±4local~4.8GB131khow to run › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
731Qwen2 1.5B Instruct · 8-bit Alibaba31±1local~4GB32kOllama › WildEval ZeroEval published resultsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsSpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+1
733Gemma 3 1B · 4-bit Google31±1local~3GB32kLM Studio › Google DeepMind Gemma 3 270M IT model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2-1.2B model cardLiquidAI/LFM2.5-1.2B-Thinking model cardMIMIC Clinical Decision Making leaderboardGemma Team — Gemma 3 Technical Report+7
737LFM2 350M · 4-bit Liquid AI30±2local~1.9GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
738SmolLM2 1.7B Instruct · 8-bit Hugging Face30±1local~4.2GB8kOllama › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
739Llama 3.2 1B Instruct · 4-bit Meta30±1local~5.2GB131kOllama › IBM Granite 3.0 Language Models technical report (GitHub)2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2-1.2B model cardSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLiquidAI/LFM2.5-1.2B-Thinking model cardLMArena leaderboard dataset+13
746StableLM-3B-4E1T · 8-bit Stability AI29±1local~4.7GB4kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
747Gemma 2B (base) · 8-bit Google29±1local~4.3GB8kOllama › EvalPlus / EvalPerf published evaluation runsGemma report — original evaluationsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+3
749Gemma 1.1 2B IT · 8-bit Google29±1local~5.2GB8kOllama › EvalPlus / EvalPerf published evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetSpeakLeash Polish EQ-Bench leaderboardArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Open Portuguese LLM Leaderboard (archived)+1
751Granite 3.1 1B-A400M Instruct · 8-bit IBM29±2†local~3GB131kOllama › ibm-granite/granite-3.1-3b-a800m-instruct model card
759StarCoderBase 3B · 4-bit BigCode28±1local~3.4GBOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
760Granite 3.0 1B-A400M Instruct · 8-bit IBM27±1local~3GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
763Granite 4.0 H 350M · 4-bit IBM27±1†local~1.9GB33kLM Studio › ibm-granite/granite-4.0-h-1b model card
764Flan-T5 Large · 4-bit Google27±2local~4.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
766Gemma 2B IT · 8-bit Google27±1local~5.2GB8kOllama › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsWildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMArena leaderboard datasetAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+3
767EuroLLM 1.7B Instruct · 4-bit UTTER (EuroLLM)27±3local~2.6GB4kllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
768Qwen2.5 0.5B Instruct · 8-bit Alibaba27±2local~2.6GB33kOllama › HuggingFaceTB/SmolLM2-360M-Instruct model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)+1
772Granite 4.0 350M · 16-bit IBM26±1local~2.9GB32kOllama › ibm-granite/granite-4.0-h-1b model cardBerkeley Function Calling Leaderboard result archiveEpoch AI — internal evaluations
776Qwen2 0.5B Instruct · 8-bit Alibaba25±2local~2.6GB32kOllama › Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
778RedPajama-INCITE Base 3B v1 · 4-bit Together25±3local~3.4GB2kllama.cpp › RedPajama — original zero-shot 3B comparisonOpen Portuguese LLM Leaderboard (archived)
781OpenLLaMA 3B v2 · 4-bit OpenLM Research25±1local~4.2GBllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
784Falcon3 1B Instruct · 8-bit TII24±4local~4.1GB8kOllama › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
786Qwen1.5 1.8B Chat · 4-bit Alibaba24±2local~3.1GB33kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
789Pythia 2.8B · 8-bit EleutherAI21±1local~4.7GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsRedPajama — original zero-shot 3B comparisonOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
791OLMo 1B · 8-bit Allen Institute for AI21±2†local~2.9GB2kllama.cpp › OLMo: Accelerating the Science of Language Models — authors' own runs
792Phi-1 (1.3B) · 4-bit Microsoft21±4local~4.5GB2khow to run › CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)
793Qwen1.5 0.5B Chat · 4-bit Alibaba21±3local~1.9GB33kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
795Pythia 2.8B (deduped) · 8-bit EleutherAI20±1local~4.7GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
796Flan-T5 Base · 4-bit Google20±3†local~2.5GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runs
797Falcon-RW 1B · 16-bit TII20±2local~4.3GB2khow to run › Textbooks Are All You Need II: phi-1.5 technical report — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
799SmolLM2 360M Instruct · 4-bit Hugging Face19±3local~2.3GB8kOllama › HuggingFaceTB/SmolLM2-360M-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
802Pythia 1.4B · 4-bit EleutherAI18±2local~4.6GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
803BLOOM 3B · 4-bit BigScience18±1local~4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
806OPT-1.3B · 4-bit Meta16±1local~4.3GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Portuguese LLM Leaderboard (archived)
807Gemma 3 270M IT · 8-bit Google15±4local~2.4GB32kOllama › Google DeepMind Gemma 3 270M IT model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
808Pythia 1B · 4-bit EleutherAI14±1local~3.7GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
810BLOOM 1.7B · 4-bit BigScience14±1local~5.1GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
811Flan-T5 Small · 4-bit Google14±3†local~1.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runs
813GPT-2 XL (1.5B) · 16-bit OpenAI11±1local~4.8GB1kLM Studio › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsEleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)+2
814BLOOM 1.1B · 4-bit BigScience11±1local~3.7GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
815Pythia 410M · 4-bit EleutherAI10±2local~2.5GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
818Zyte-1B · 4-bit venkycs (aihub-app)8±2local~3.8GBhow to run › EvalPlus / EvalPerf published evaluation runs
819GPT-Neo 125M · 4-bit EleutherAI5±5local~2.1GB2khow to run › Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)COPEN — EMNLP 2022 original zero-shot evaluationsOpen Portuguese LLM Leaderboard (archived)
820BLOOM 560M · 4-bit BigScience5±2local~2.7GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)
821OPT-350M · 4-bit Meta5±2local~2.2GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
823Pythia 160M · 4-bit EleutherAI3±2local~1.9GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
824OPT-125M · 4-bit Meta3±2local~1.8GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
825GPT-2 Large (774M) · 4-bit OpenAI1±2local~4.9GB1khow to run › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
826GPT-2 Medium (355M) · 4-bit OpenAI-1±2local~3.1GB1khow to run › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
827Pythia 70M · 4-bit EleutherAI-7±2local~1.7GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
828GPT-2 Small (124M) · 4-bit OpenAI-9±3local~2.1GB1khow to run › Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
—Grok Build 0.1 SpaceXAI (xAI)no reviewed resultsapi $1.25/M256kdocs.x.ai ↗

+ 479 more don't fit your machine or budget · show them

+ 41 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

830 of 1134 catalogue models listed here · sources and licences

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory