whatmodel.sh · find the best AI models you can run

Best AI models for a 4GB Mac

A leaderboard of the best AI models for a Mac with 4GB of memory. The best one that runs on it right now is MiniCPM5 2B, followed by Spark-X2.5 1.7B and LFM2.5 2.6B. Models are ranked by one score that combines 575 published benchmarks. Models that run on it show the download that fits and the command to run it.

291 models · 575 benchmarks · updated
Filter by your hardware and budget → ~3GB fast · reset
ram[2][4][8][16][24][32][48][64][96][128][192][256]
[]
GB
gpu[apple silicon][graphics card][unified memory][none]
price[local][$0.02][$0.1][$0.5][$2][$10][$50][any]max per M
Filter by task
task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]
New #1s for your hardware:
orRSS

detecting your OS…

$ terminal

detecting your OS…

Run curl "whatmodel.sh/?ram=4&gpu=apple&max=any"to get this list in your terminal (add &format=json for agents).

Score vs release date

#modelscore↓where↓RAM↓speed↓context↓get itsources
#modelscore↓where↓RAM↓speed↓context↓get itsources
1GPT-6 Astra OpenAI85±1api $20.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
2Claude Opus 5.5 Anthropic85±1api $8.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
3GPT-6.1 Sol OpenAI84±1api $4.00/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
4Claude Fable 5.1 Anthropic83±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
5Muse Wonder Full Meta82±2Muse Code only1Mresearch.meta.ai ↗LMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)
6Claude Sonnet 5.5 Anthropic82±1api $4.00/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
7Gemini 4 Argon Google81±1api $4.00/M1Mai.google.dev ↗LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)
8Claude Opus 5 Anthropic81±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+11
9GPT-6 Sol OpenAI81±1api $4.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard+1
10GPT-5.5 Pro OpenAI80±1api $67.50/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
11GPT-5.6 Sol OpenAI80±1api $8.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+14
12Claude Fable 5 Anthropic80±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runs+5
13Claude Mythos Preview (limited preview) Anthropic80±1†limited preview1MAnthropic system cards and model announcements (lab-reported scores)
14GPT-5.4 Pro OpenAI79±1api $67.50/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
15DeepSeek V4.1 Flash 1017 DeepSeek79±1api $0.26/M1Mapi-docs.deepseek.com ↗DeepSeek V4.1 Flash 1017 model cardLMArena leaderboard dataset
16Muse Wonder Meta79±1api $8.00/M1Mdev.meta.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)
17GPT-5.5 OpenAI78±0api $11.25/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAgents' Last Exam — authors' own fixed-harness model comparison+16
18MiMo-V2.6-Pro Xiaomi78±1api $0.54/M~443GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
19Hy4 preview (770B-A49B) Tencent77±1api $1.25/M~248GB1Mcloud.tencent.com ↗Tencent Hy4 preview model cardLMArena leaderboard dataset
20GPT-5.6 Terra OpenAI77±1api $4.50/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
21Qwen3.8 Max (2.4T-A95B) Alibaba77±1api $3.00/M~380GB262kdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)+2
22Claude Opus 4.8 Anthropic77±0api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+11
23GPT-5.4 OpenAI76±1api $5.63/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAgents' Last Exam — authors' own fixed-harness model comparison+14
25MiMo-V2.6-Flash (309B-A15B) Xiaomi76±1api $0.17/M~136GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
26Muse Spark 1.3 Meta76±1api $2.00/M1Mdev.meta.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)Terminal-Bench 4.0 official leaderboard submissionsE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
27Gemini 3.8 Flash Google76±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+5
28Claude Haiku 5.5 Anthropic76±1api $0.20/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard dataset
29GPT-5.2 Pro OpenAI76±1api $57.75/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
30Kimi K3 Moonshot AI76±1api $6.00/M~884GB1Mplatform.moonshot.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runs+7
31GPT-6 Luna OpenAI75±1api $0.20/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
32GPT-5.6 Luna OpenAI75±1api $0.45/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
33Grok 4.6 SpaceXAI (xAI)75±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
34Gemini 3.7 Flash Google75±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+3
35DeepSeek V4 Pro (0813) DeepSeek75±1api $1.63/M~1010GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party models+2
36GLM-5.3 Z.ai75±1api $2.15/M~239GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
37Qwen3.8 Max (0902) Alibaba75±1api $3.00/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
38Muse Spark Open Meta75±1api $2.00/M~190GB1Mfireworks.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)
40Gemini 3.1 Pro (Preview) Google74±0api $4.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)meituan-longcat/LongCat-2.0 model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+23
43Claude Opus 4.7 Anthropic74±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAnthropic system cards and model announcements (lab-reported scores)Agents' Last Exam — authors' own fixed-harness model comparison+9
44Grok 4.5 SpaceXAI (xAI)74±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runs+1
45Qwen3.8-Omni-Flash Alibaba74±1†api $0.23/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
46Claude Sonnet 5 Anthropic74±1api $4.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissions+5
48Claude Opus 4.6 Anthropic74±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsmeituan-longcat/LongCat-2.0 model card+22
49Muse Spark 1.1 Meta74±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsτ²/τ³-bench leaderboard: Sierra's own evaluation runsEpoch AI — internal evaluations
50Grok 4.7 SpaceXAI (xAI)73±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
51Muse Spark 1.2 Meta73±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)Epoch AI — internal evaluations
53GPT-5.3-Codex OpenAI73±1api $4.81/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
54DeepSeek V4 Flash (0731) DeepSeek73±1api $0.11/M~92GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardDaekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetdeepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+2
55Gemini 3.5 Flash Google73±1api $3.38/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Legal Research Bench (Vals AI, arXiv:2610.00609v1)Miðeind Icelandic LLM Leaderboard+3
56Qwen3.7 Max Alibaba73±1api $3.75/M1Mdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
57GLM-5.3-Flash Z.ai73±1api $0.24/M~103GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)+2
58GPT-5.2 OpenAI73±1api $4.81/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+21
59Gemini 3.6 Flash Google73±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 Flash+2
60GLM-5.2 Z.ai72±1api $2.15/M~121GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetzai-org/GLM-5.2 model cardzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+6
61Grok 4.20 Multi-Agent (0309) SpaceXAI (xAI)72±2api $1.56/M1Mdocs.x.ai ↗LMArena leaderboard dataset
63Hunyuan Hy3 (295B-A21B) Tencent72±1api $0.23/M~94GB256kdeepinfra.com ↗tencent/Hy3 model cardLMArena leaderboard datasetTencent Hy4 preview model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
64Qwen3.8 27B Alibaba72±1api $1.60/M~18GB~8.6 tok/s262kconsole.groq.com ↗LMArena leaderboard datasetQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+3
65LongCat 2.0 (1.6T-A48B) Meituan72±2api $1.30/M~2155GB1Mlongcat.chat ↗meituan-longcat/LongCat-2.0 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
66Kimi K2.6 Moonshot AI72±1api $1.71/M~317GB262kplatform.moonshot.ai ↗Epoch AI — internal evaluationsNVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsAgents' Last Exam — authors' own fixed-harness model comparison+11
67GPT-5 Pro OpenAI72±1api $41.25/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
69GPT-5 OpenAI72±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+22
70Mistral Large 4 (public preview) Mistral72±1api $2.06/M1Mdocs.mistral.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluations
71Qwen3.6 Max Preview Alibaba71±1api $2.92/M256kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
72Claude Sonnet 4.6 Anthropic71±1api $6.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetMulti-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe (arXiv:2609.05395v1)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs+18
73Gemini 3 Flash (Preview) Google71±1api $1.13/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGoogle DeepMind Gemini model page and model cards (lab-reported scores)+8
75Claude Opus 4.5 Anthropic71±1api $10.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAnthropic system cards and model announcements (lab-reported scores)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)+14
76Qwen3.7 Plus Alibaba71±1api $0.70/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LMArena leaderboard datasetSciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
77Kimi K2.7 Code Moonshot AI71±1api $1.71/M~359GB262kplatform.kimi.ai ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
79North Small Translate 1.0 Cohere71±5†api price unknown16kdocs.cohere.com ↗Cohere — North Small Translate (arXiv:2609.13916v1)
81Grok 4.20 (0309 reasoning) SpaceXAI (xAI)71±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsτ²/τ³-bench leaderboard: Sierra's own evaluation runsThe Token Games: Evaluating Language Model Reasoning with Puzzle Duels — authors' own tournamentWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
82GPT-5.4 mini OpenAI70±1api $1.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsLegal Research Bench (Vals AI, arXiv:2610.00609v1)+6
83Beam 1.0 501B A23B Reflection70±1api price unknown1Mdevelopers.reflection.ai ↗LMArena leaderboard dataset
84Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)70±2†api $0.090/M~71GB131kdeepinfra.com ↗inclusionAI/Ling-3.0-flash model card
85GPT-5.1 OpenAI70±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsHyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+8
86GLM-5.1 Z.ai70±0api $2.15/M~119GB200kdocs.z.ai ↗NVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetszai-org/GLM-5.1 model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1zai-org/GLM-5.2 model card+11
87o3-pro OpenAI70±2api $35.00/M200kdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
88Qwen3.5 397B-A17B Alibaba70±0api $1.09/M~65GB262kdeepinfra.com ↗Qwen/Qwen3.5-397B-A17B model cardNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsQwen/Qwen3.6-27B model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard dataset+15
92Grok 4.3 SpaceXAI (xAI)69±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
93MiniMax-M3 MiniMax69±1api $0.48/M~131GB1Mdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationszai-org/GLM-5.2 model cardLegal Research Bench (Vals AI, arXiv:2610.00609v1)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Miðeind Icelandic LLM Leaderboard+2
94Qwen3.6 Plus Alibaba69±1api $1.13/M1Mdocs.together.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Agents' Last Exam — authors' own fixed-harness model comparisonQwen/Qwen-AgentWorld-35B-A3B model cardBEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+1
95Kimi K2.5 Moonshot AI69±1api $1.20/M~160GB262kdocs.aws.amazon.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsstepfun-ai/Step-3.5-Flash model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsQwen/Qwen3.5-397B-A17B model card+15
98GLM-5 Z.ai69±1api $1.55/M~119GB205kdocs.z.ai ↗zai-org/GLM-5.1 model cardEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGLM-5 Team (Zhipu AI) — GLM-5 technical reportzai-org/GLM-5 model cardLMArena leaderboard dataset+11
99MiMo-V2.5-Pro (1T-A42B) Xiaomi69±1api $0.65/M~163GB1Mnovita.ai ↗LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardXiaomi MiMo official model cards and documentationGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
100o3 OpenAI69±1api $3.50/M200kplatform.openai.com ↗Epoch AI — internal evaluationsMeituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard dataset+11
101Muse Glimmer 30B Meta69±1api $0.64/M~18GB~27 tok/s131kfireworks.ai ↗meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.8-27B model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Cohere — North Small Translate (arXiv:2609.13916v1)
102Nemotron 3 Ultra 550B-A55B NVIDIA69±1api $1.35/M~89GB1Mdocs.together.ai ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1Epoch AI — internal evaluationsLMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)
103Inkling Small (266B-A12B) Thinking Machines69±1api $0.64/M~86GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetMiðeind Icelandic LLM LeaderboardCohere — North Small Translate (arXiv:2609.13916v1)
104Qwen3.6 27B Alibaba68±1api $1.20/M~18GB~16 tok/s262kconsole.groq.com ↗Qwen/Qwen3.6-27B model cardEpoch AI — internal evaluationsQwen/Qwen3.8-27B model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+4
109MiMo-V2.5 (310B-A15B) Xiaomi68±1api $0.21/M~100GB1Mnovita.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5 model cardAgents' Last Exam — authors' own fixed-harness model comparisonSciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)Xiaomi MiMo vendor-reported benchmark scores for its own models (mimo.xiaomi.com launch pages)
110Inkling (975B-A41B) Thinking Machines68±1api $1.73/M~156GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)τ²/τ³-bench leaderboard: Sierra's own evaluation runsMiðeind Icelandic LLM Leaderboard
111Qwen3.5 Plus Alibaba68±2api $0.90/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
112Qwen3.5 122B-A10B Alibaba68±1api $1.10/M~71GB262kwww.alibabacloud.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.5-27B model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)Epoch AI — internal evaluations+5
114Gemini 3.5 Flash-Lite Google68±1api $0.85/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Miðeind Icelandic LLM Leaderboard
116GPT-5.4 nano OpenAI68±1api $0.46/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeT1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)+1
117Claude Sonnet 4.5 Anthropic68±1api $6.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+19
118GLM-4.7 Z.ai68±1api $1.00/M~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardGLM-5 Team (Zhipu AI) — GLM-5 technical reportzai-org/GLM-5 model cardLMArena leaderboard datasetEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+5
119GLM-5V-Turbo Z.ai68±2api $1.90/M200kwww.siliconflow.com ↗LMArena leaderboard dataset
120Qwen3.5 27B Alibaba68±0api $0.84/M~18GB~29 tok/s262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.6-27B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard dataset+9
125DeepSeek V3.2 DeepSeek67±1api $0.29/M~110GB128kdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsdeepseek-ai/DeepSeek-V3.2 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsstepfun-ai/Step-3.5-Flash model card+15
126Step 3.5 Flash (196B-A11B) StepFun67±1api $0.15/M~118GB~18 tok/s256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsstepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
127Qwen3.6 35B-A3B Alibaba67±1api $0.31/M~21GB~63 tok/s262kdeepinfra.com ↗Qwen/Qwen3.6-27B model cardQwen/Qwen3.6-35B-A3B model cardEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
128MiniMax-M2.7 MiniMax67±1api $0.53/M~75GB205kplatform.minimax.io ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard datasetinclusionAI/Ling-3.0-flash model cardQwen/Qwen-AgentWorld-35B-A3B model cardWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+2
129Qwen3-Coder-Next (80B-A3B) Alibaba67±2api $0.60/M~25GB~62 tok/s262kwww.alibabacloud.com ↗Vector Institute Evaluation Leaderboard (Inspect Evals runs)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs
131GPT-5 mini OpenAI67±1api $0.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardDaekeun Kim Korean LLM benchmark runsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)LMArena leaderboard dataset+16
133Gemini 2.5 Pro Google67±1api $3.44/M1Mai.google.dev ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+16
136Gemma 4 31B Google66±1api $0.19/M~24GB~11 tok/s262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportKwaipilot/KAT-Coder-V2.5-Dev model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+21
139Gemini 3.1 Flash-Lite Google66±1api $0.56/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs — authors' own model runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
140Kimi K2 Thinking Moonshot AI66±1api $1.07/M~160GB262kdocs.aws.amazon.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsstepfun-ai/Step-3.5-Flash model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
141Qwen3.7 Flash Alibaba66±1api $0.055/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluations
143Laguna M.1 Poolside66±2api price unknown~135GB262khuggingface.co ↗LMArena leaderboard datasetPoolside — Laguna M.1/XS.2 Technical Report
147Qwen3 235B-A22B Thinking 2507 Alibaba66±1api $0.97/M~76GB131knovita.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-Next-80B-A3B-Thinking model cardQwen3-VL-235B-A22B-Thinking model cardQwen/Qwen3.5-27B model cardEpoch AI — internal evaluations+6
148Kimi K2 Instruct (0711) Moonshot AI66±1api $1.00/M~152GB131knovita.ai ↗Meituan LongCat-Flash-Chat model card (evaluation table)Aider polyglot benchmark runsLMArena leaderboard datasetMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)EQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archive+3
149Qwen3.5 35B-A3B Alibaba66±0api $0.35/M~21GB~15 tok/s262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardNVIDIA — Nemotron-Cascade 2 technical reportLMArena leaderboard dataset+13
150Step 3.7 Flash (198B-A11B) StepFun66±1api $0.44/M~104GB256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsinclusionAI/Ling-3.0-flash model card
151o1 (2024-12-17) OpenAI66±1api $26.25/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results+13
152o4-mini OpenAI66±1api $1.93/M200kplatform.openai.com ↗Epoch AI — internal evaluationsQwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runs+12
153Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)66±1api $0.28/M128kazure.microsoft.com ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archiveτ²/τ³-bench leaderboard: Sierra's own evaluation runs+4
156DeepSeek V3.1 DeepSeek66±1api $0.42/M~110GB131kdeepinfra.com ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Meituan LongCat-Flash-Chat model card (evaluation table)deepseek-ai/DeepSeek-V3.1 model cardDeepSeek-V3.1-Terminus model cardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+3
157Qwen3.6 Flash Alibaba66±1api $0.56/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsSpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
158GLM-4.6 Z.ai65±1api $1.00/M~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardzai-org/GLM-4.6 model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archive+1
160Qwen3-Max (2025-09-23) Alibaba65±1api $2.40/M262kwww.alibabacloud.com ↗Epoch AI — internal evaluationsτ²-bench leaderboard: Qwen (Alibaba) own submissionsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)
161MiniMax-M2.5 MiniMax65±1api $0.53/M~75GB205kplatform.minimax.io ↗LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+3
162o3-mini OpenAI65±1api $1.93/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardMicrosoft — Phi-4-reasoning Technical ReportOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runs+13
165DeepSeek R1 0528 DeepSeek65±1api $0.91/M~104GB164kdeepinfra.com ↗deepseek-ai/DeepSeek-R1-0528 model cardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)deepseek-ai/DeepSeek-V3.1 model cardServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+6
171Qwen3 235B-A22B Instruct 2507 Alibaba65±1api $0.20/M~73GB~15 tok/s262kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMeituan LongCat-Flash-Chat model card (evaluation table)Qwen3-VL-235B-A22B-Instruct model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsLMArena leaderboard dataset+3
174GLM-4.5 Z.ai64±1api $1.00/M~58GB131kdocs.z.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)zai-org/GLM-4.6 model cardLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard results+3
175Mistral Medium 3.5 (128B) Mistral64±1api $3.00/M~43GB262kdocs.mistral.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)Miðeind Icelandic LLM LeaderboardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
176Qwen3.5 Flash Alibaba64±1api $0.17/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
177MiniMax-M2 MiniMax64±1api $0.53/M~75GB205kdocs.aws.amazon.com ↗LMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
179Nemotron 3 Super 120B-A12B NVIDIA64±1api $0.28/M~73GB~20 tok/s262kdocs.aws.amazon.com ↗NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)NVIDIA — Nemotron-Cascade 2 technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetinclusionAI/Ling-3.0-flash model card
180Command A+ (05-2026) Cohere64±1api price unknown~71GB128kdocs.cohere.com ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Cohere vendor-reported benchmark scores for its own models (cohere.com blog)Cohere — North Small Translate (arXiv:2609.13916v1)
182Claude Haiku 4.5 Anthropic64±1api $2.00/M200kdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+9
183Gemma 4 26B-A4B Google64±1api $0.14/M~18GB~55 tok/s262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical Reportgoogle/diffusiongemma-26B-A4B-it model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+13
185Solar Pro 4 Upstage64±1api $0.53/M524kdevelopers.upstage.ai ↗LMArena leaderboard dataset
188Qwen3-Coder 480B-A35B Instruct Alibaba63±1api $0.67/M~78GB262knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
192MiniCPM5 2B · 4-bit OpenBMB63±1†local~2.5GB131kllama.cpp › openbmb/MiniCPM5-2B model card
194DeepSeek R1 (Jan '25) DeepSeek63±1api $4.00/M~104GB128kdocs.together.ai ↗deepseek-ai/DeepSeek-R1 model carddeepseek-ai/DeepSeek-R1-0528 model cardEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsWildEval ZeroEval published results+15
196Qwen3-Next 80B-A3B Instruct Alibaba63±1api $0.34/M~26GB~55 tok/s131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
197Nova 2 Pro (Preview) Amazon63±2api $3.78/M1Maws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsAmazon AGI — τ²-Bench-Verified
199Qwen3-Next 80B-A3B Thinking Alibaba63±1api $0.41/M~25GB131kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3.5-9B model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsQwen/Qwen3.5-4B model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)+3
200Granite 4.2 30B IBM63±1api $0.28/M~19GB131kdeepinfra.com ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
203GPT-4.1 OpenAI62±1api $3.50/M1Mplatform.openai.com ↗Meituan LongCat-Flash-Chat model card (evaluation table)Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runsAider polyglot benchmark runsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+17
204GLM-4.5-Air Z.ai62±1api $0.42/M~35GB~23 tok/s131kdocs.z.ai ↗Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetSERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)Solar Open Technical Report (arXiv:2601.07022v1)+4
207Trinity-Large-Thinking Arcee AI62±2api $0.39/M~141GB512kdocs.arcee.ai ↗LMArena leaderboard datasetArcee AI vendor-reported benchmark scores for its own Trinity models (Hugging Face model cards)
208Qwen3.5 9B Alibaba62±0api $0.11/M~6.7GB~22 tok/s262kdeepinfra.com ↗Qwen/Qwen3.5-9B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+19
209Nemotron 3.5 Lightning 30B-A3B NVIDIA62±2api $0.087/M~20GB~41 tok/s262kfireworks.ai ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard dataset
211MiniMax-M1 (80k) MiniMax62±1api $0.96/M~959GB1Mnovita.ai ↗MiniMaxAI/MiniMax-M1-80k model cardLMArena leaderboard dataset
212GLM-4.6V Z.ai62±2api $0.45/M~50GB128kdocs.z.ai ↗LMArena leaderboard datasetZ.ai GLM-V GitHub repository benchmark image (GLM-4.6V)SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)
214Gemini 2.5 Flash Google62±1api $0.85/M1Mai.google.dev ↗Meituan LongCat-Flash-Chat model card (evaluation table)LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+10
217Qwen3-VL 32B Thinking Alibaba62±1api $0.28/M~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)allenai/Olmo-3.1-32B-Think model card
218gpt-oss-120b OpenAI62±1api $0.26/M~64GB~44 tok/s131kconsole.groq.com ↗OpenAI — gpt-oss-120b & gpt-oss-20b Model CardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsAider polyglot benchmark runs+19
220GPT-5 nano OpenAI62±1api $0.14/M400kplatform.openai.com ↗Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsBerkeley Function Calling Leaderboard result archive+7
221Mistral Large 3 (675B-A41B) Mistral62±1api $0.75/M~108GB262kdocs.mistral.ai ↗LMArena leaderboard datasetmistralai/Mistral-Small-4-119B-2603 model cardCohere — North Small Translate (arXiv:2609.13916v1)LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? (arXiv:2610.04118v1)Miðeind Icelandic LLM LeaderboardOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+1
226GLM-4.5V Z.ai61±2api $0.90/M~69GB64kdocs.z.ai ↗LMArena leaderboard datasetZ.ai GLM-V GitHub repository benchmark image (GLM-4.6V)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
234GPT-4.1 mini OpenAI60±1api $0.70/M1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runsAider polyglot benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard dataset+13
236Qwen3 30B-A3B Instruct 2507 Alibaba60±1api $0.15/M~19GB~100 tok/s262ktokenfactory.nebius.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
237Qwen3 30B-A3B Thinking 2507 Alibaba60±1api $0.75/M~19GB262kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardByteDance-Seed/Seed-OSS-36B-Instruct model cardQwen/Qwen3.5-9B model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+8
246Qwen3-VL 32B Instruct Alibaba60±1api $0.28/M~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
247DeepSeek V3 (Dec '24) DeepSeek60±1api $0.46/M~337GB131kdeepinfra.com ↗deepseek-ai/DeepSeek-R1 model cardCohere — Command A technical reportEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsWildEval ZeroEval published results+10
248Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba60±0api $1.23/M~73GB131kwww.alibabacloud.com ↗Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+14
250Granite 4.2 8B IBM60±1api $0.11/M~6.8GB131kwandb.ai ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
251Mistral Small 4 (119B-A6.5B) Mistral60±2api $0.26/M~37GB262kdocs.mistral.ai ↗mistralai/Mistral-Small-4-119B-2603 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Miðeind Icelandic LLM Leaderboard
254Qwen3-VL 30B-A3B Thinking Alibaba59±1api $0.75/M~20GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
259GLM-4.7-Flash Z.ai59±1api $0.0000/M~18GB~44 tok/s200kdocs.z.ai ↗Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardSERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)EQ-Bench 3 canonical leaderboard results+1
262gpt-oss-20b OpenAI58±1api $0.13/M~13GB~23 tok/s131kconsole.groq.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardOpenAI — gpt-oss-120b & gpt-oss-20b Model CardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEpoch AI — internal evaluationsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)+16
264Qwen3 14B Alibaba58±1api $0.61/M~10GB33kwww.alibabacloud.com ↗Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBerkeley Function Calling Leaderboard result archiveBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+7
266DeepSeek R1 Distill Llama 70B DeepSeek58±1api $0.80/M~30GB131knovita.ai ↗Epoch AI — internal evaluationsMicrosoft — Phi-4-reasoning Technical Reportdeepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Judgemark v2 results (EQ-Bench)+1
267GPT-4o (2024-08-06) OpenAI58±1api $4.38/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsVector Institute Evaluation Leaderboard (Inspect Evals runs)OpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runs+23
269Command A (03-2025) Cohere58±1api $4.38/M~37GB256kazure.microsoft.com ↗Cohere — Command A technical reportLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
271GPT-4o (2024-05-13) OpenAI58±0api $7.50/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runs+19
272Phi-4-reasoning 14B Microsoft58±1api $0.22/M~10GB33kazure.microsoft.com ↗Microsoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
273GPT-4o (2024-11-20) OpenAI58±1api $4.38/M128kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Qwen/Qwen3-235B-A22B-Instruct-2507 model cardAider polyglot benchmark runsQwen/Qwen3-30B-A3B-Instruct-2507 model cardMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+7
275Nova 2 Lite Amazon58±1api $0.94/M1Mdocs.aws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveAmazon AGI — τ²-Bench-VerifiedVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
276Magistral Small 1.2 Mistral58±1api $0.75/M~16GB131kdocs.aws.amazon.com ↗mistralai/Magistral-Small-2509 model cardEpoch AI — internal evaluationsApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarksInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)mistralai/Mistral-Small-4-119B-2603 model cardEuroEval leaderboards (results collection), English generative leaderboard
277Qwen3-VL 30B-A3B Instruct Alibaba58±1api $0.35/M~20GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
278Qwen3-VL 8B Thinking Alibaba58±1api $0.66/M~6.9GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Think model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
279Qwen3 32B Alibaba58±0api $0.13/M~20GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-Next-80B-A3B-Instruct model cardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+17
282Qwen3 Coder 30B-A3B Instruct Alibaba58±1api $0.12/M~19GB~93 tok/s160knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
283Qwen2.5 Coder 32B Instruct Alibaba57±1api $0.74/M~21GB131kdevelopers.cloudflare.com ↗EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
284Gemini 2.5 Flash-Lite Google57±2api $0.17/M1Mai.google.dev ↗Google DeepMind Gemini model page and model cards (lab-reported scores)Berkeley Function Calling Leaderboard result archiveQwen/Qwen3.5-4B model cardMiðeind Icelandic LLM LeaderboardGalileo Agent Leaderboard (v1 and v2 results)Open Portuguese LLM Leaderboard (archived)+1
285Llama 4 Maverick (402B-A17B) Meta57±1api $0.42/M~65GB1Mdocs.aws.amazon.com ↗Epoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarks+14
286DeepSeek R1 0528 Qwen3 8B DeepSeek57±1api $0.068/M~6.5GB~21 tok/s128knovita.ai ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scalingdeepseek-ai/DeepSeek-R1-0528-Qwen3-8B model cardEpoch AI — internal evaluationsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
288Granite 4.2 3B IBM57±1api $0.052/M~3.3GB131kdeepinfra.com ↗openbmb/MiniCPM5-2B model cardibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
291Nemotron 3 Nano 30B-A3B NVIDIA57±1api $0.087/M~20GB~105 tok/s262kdeepinfra.com ↗NVIDIA — Nemotron-Cascade 2 technical reportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)LMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+3
300Qwen Plus (2025-01-25) Alibaba57±1api $0.60/M131kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
301Mistral Small 3.2 24B Mistral57±1api $0.11/M~15GB128kdeepinfra.com ↗mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard resultsmistralai/Mistral-Small-4-119B-2603 model card+4
305GPT-4 Turbo (2024-04-09) OpenAI56±0api $15.00/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)+18
306Yi-Lightning 01.AI56±1api price unknown16kplatform.lingyiwanwu.com ↗Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard dataset
309Qwen2.5 72B Instruct Alibaba56±1api $0.37/M~31GB33kdeepinfra.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMicrosoft Research — Phi-4 Technical ReportLiveCodeBench official website results (2024-12-26)+24
311Llama 3.1 405B Instruct Meta56±1api $2.40/M~160GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+13
314DeepSeek R1 Distill Qwen 32B DeepSeek56±1api $1.59/M~21GB~7.3 tok/s131kdevelopers.cloudflare.com ↗open-thoughts/OpenThinker2-32B model cardopen-thoughts/OpenThinker-32B model cardEpoch AI — internal evaluationsdeepseek-ai/DeepSeek-R1 model cardWebApp1K models leaderboard (onekq-ai)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+5
316Qwen3 8B Alibaba56±1api $0.31/M~6.3GB~20 tok/s33kwww.alibabacloud.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)allenai/Olmo-3-7B-Think model cardSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+15
318Nemotron Nano 9B v2 NVIDIA56±1api $0.10/M~7.6GB131kdocs.aws.amazon.com ↗Olmo 3NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)allenai/Olmo-3-7B-Think model cardEuroEval leaderboards (results collection), English generative leaderboard
321Ministral 3 14B Mistral56±1api $0.20/M~9.1GB~46 tok/s262kdocs.mistral.ai ↗JetBrains/Mellum2-12B-A2.5B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)mistralai/Ministral-3-8B-Instruct-2512 model card+2
324Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba55±0api $0.35/M~18GB33kwww.alibabacloud.com ↗Qwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)+10
326Nova Premier Amazon55±2†api $5.00/M1Maws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)MASSIVE-Agents — Kulkarni et al. (2025)
327Granite 4.1 8B IBM55±1api $0.11/M~7.1GB~70 tok/s131kreplicate.com ↗ibm-granite/granite-4.1-8b model cardLMArena leaderboard datasetSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)
329Spark-X2.5 1.7B · 4-bit iFLYTEK (SparkLLM)55±1†local~1.9GB~166 tok/s1Mllama.cpp › XHToken/Spark-X2.5-4B model card
330Llama 3.3 70B Instruct Meta55±1api $0.72/M~25GB~13 tok/s128kdocs.aws.amazon.com ↗Cohere — Command A technical reportEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMicrosoft Research — Phi-4 Technical ReportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLiveCodeBench official website results (2024-12-26)+21
332Qwen3-VL 8B Instruct Alibaba55±1api $0.31/M~6.9GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Instruct model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
336Llama 4 Scout (109B-A17B) Meta55±1api $0.15/M~36GB~19 tok/s328kdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard resultsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+8
357GPT-4o mini (2024-07-18) OpenAI54±0api $0.26/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsmicrosoft/Phi-4-mini-instruct model cardEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+32
371Llama 3.1 70B Instruct Meta54±0api $0.72/M~27GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+22
372Gemma 3 27B Google54±1api $0.10/M~19GB131kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportGoogle DeepMind Gemma 3 270M IT model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard dataset+23
373GPT-4.1 nano OpenAI53±1api $0.17/M1Mdevelopers.openai.com ↗OpenAI simple-evals README benchmark results (OpenAI's own models)Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetAider polyglot benchmark runs+11
375Phi-4 14B Microsoft53±1api $0.087/M~10GB16kdeepinfra.com ↗Epoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical ReportDaekeun Kim Korean LLM benchmark runsmicrosoft/phi-4 model cardMicrosoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model card+11
382Ministral 3 8B Mistral53±1api $0.15/M~6.5GB~68 tok/s262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardMulti-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe (arXiv:2609.05395v1)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)+3
385GPT-4 (0613) OpenAI53±1api $37.50/M8kdevelopers.openai.com ↗Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsTulu 2 — base and instruction-tuned evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+11
386LFM2.5 2.6B · 4-bit Liquid AI53±1local~2.4GB131kllama.cpp › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
392Granite 4.0 H Small (32B-A9B) IBM53±1api $0.11/M~21GB131kreplicate.com ↗ibm-granite/granite-4.0-h-small model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)+1
394Aya Expanse 32B Cohere53±2api $0.75/M~22GB128kdocs.cohere.com ↗LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)+1
397Gemma 3 12B Google52±1api $0.075/M~10GB~16 tok/s131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportEpoch AI — internal evaluationsLMArena leaderboard dataset+10
407Command A Translate (08-2025) Cohere52±4†api price unknown8kdocs.cohere.com ↗Cohere — North Small Translate (arXiv:2609.13916v1)
424Nova Pro Amazon51±1api $1.40/M300kdocs.aws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)Aider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
447Nova Lite Amazon50±2api $0.10/M300kdocs.aws.amazon.com ↗LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
449Qwen2.5 7B Instruct Alibaba50±1api $0.30/M~5.4GB~49 tok/s131kwww.together.ai ↗microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Epoch AI — internal evaluations+12
450LFM2.5 1.2B Thinking · 8-bit Liquid AI50±2local~2.5GB33kOllama › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
475Qwen2.5 Coder 3B Instruct · 4-bit Alibaba49±1local~2.6GB~9.5 tok/s33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
480Command R Plus (08-2024) Cohere49±1api $4.38/M~63GB128kdocs.cohere.com ↗Cohere — Command A technical reportLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+1
489GPT-3.5 Turbo (0125) OpenAI48±1api $0.75/M16kdevelopers.openai.com ↗WildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+11
497Command R (08-2024) Cohere48±1api $0.26/M~21GB128kdocs.cohere.com ↗LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsCohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+3
504Qwen3.5 2B · 4-bit Alibaba47±1local~2.6GB~174 tok/s262kLM Studio › Qwen/Qwen3.5-2B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)EuroEval leaderboards (results collection), English generative leaderboardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets+7
509Nova Micro Amazon47±2api $0.061/M128kdocs.aws.amazon.com ↗LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveMASSIVE-Agents — Kulkarni et al. (2025)Galileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
515Gemma 4 E2B Google47±1api $0.050/M~4.5GB131kdocs.aws.amazon.com ↗Gemma Team — Gemma 4 Technical Reportopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)LiquidAI/LFM2.5-2.6B model cardEuroEval leaderboards (results collection), English generative leaderboard+6
523EXAONE 4.0 1.2B · 4-bit LG AI Research47±1†local~2.2GB66kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
526Qwen2.5-Math 1.5B Instruct · 4-bit Alibaba47±2†local~1.9GB4kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
544Phi-4-mini-reasoning 3.8B Microsoft46±2api $0.13/M~4GB128kazure.microsoft.com ↗microsoft/Phi-4-mini-reasoning model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
546Qwen2-Math 1.5B Instruct · 4-bit Alibaba46±2†local~2.6GB4kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
547Llama 3.1 8B Instruct Meta46±0api $0.22/M~6.2GB~22 tok/s128kdocs.aws.amazon.com ↗EvalPlus / EvalPerf published evaluation runsmicrosoft/Phi-4-mini-instruct model cardIBM Granite 3.0 Language Models technical report (GitHub)WildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets+25
548Ministral 3 3B Mistral46±1api $0.10/M~3.4GB~128 tok/s262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsmistralai/Ministral-3-14B-Instruct-2512 model cardmistralai/Ministral-3-8B-Instruct-2512 model cardEuroEval leaderboards (results collection), English generative leaderboard+2
564Gemma 3 4B Google45±1api $0.063/M~3.8GB~74 tok/s131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboardLMArena leaderboard dataset+11
567Phi-4-mini 3.8B Microsoft45±1api $0.13/M~4GB128kazure.microsoft.com ↗microsoft/Phi-4-mini-instruct model cardMicrosoft — Phi-4-Mini Technical ReportDaekeun Kim Korean LLM benchmark runsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
583LFM2.5 1.2B Instruct · 8-bit Liquid AI45±2local~1.8GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
584Qwen3 1.7B · 4-bit Alibaba45±1local~2.6GB~96 tok/s33kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen/Qwen3.5-2B model cardLiquidAI/LFM2-1.2B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HuggingFaceTB/SmolLM3-3B model card (Evaluation section)LiquidAI/LFM2.5-1.2B-Thinking model card+11
590Qwen2.5 3B Instruct · 4-bit Alibaba44±1local~2.6GB~61 tok/s33kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)SpeakLeash Polish EQ-Bench leaderboard+5
594LFM2 2.6B · 4-bit Liquid AI44±1local~2.2GB33kllama.cpp › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationEuroEval leaderboards (results collection), English generative leaderboard
604Qwen2.5 Coder 1.5B Instruct · 8-bit Alibaba44±1local~2.5GB33kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
611Aya Vision 32B Cohere44±3api price unknown~71GB16kdocs.cohere.com ↗LMArena leaderboard datasetKaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark (arXiv:2508.13680v5)
635Granite 4.0 1B · 4-bit IBM42±1local~2.6GB128kllama.cpp › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model cardEpoch AI — internal evaluations
638Granite 4.0 H 1B · 4-bit IBM42±1local~2.4GB131kllama.cpp › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model card
650Bielik 1.5B v3.0 Instruct · 4-bit SpeakLeash41±3local~2.6GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
657LFM2 1.2B · 8-bit Liquid AI41±1local~1.8GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
663Llama 3.2 3B Instruct Meta41±1api $0.12/M~3.3GB~162 tok/s131kdevelopers.cloudflare.com ↗microsoft/Phi-4-mini-instruct model cardIBM Granite 3.0 Language Models technical report (GitHub)LFM2 Technical Report — Liquid AI own SmolLM3 evaluationAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetmicrosoft/Phi-4-mini-reasoning model card+11
697Qwen2.5 Coder 0.5B Instruct · 8-bit Alibaba39±1local~1.2GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
702Qwen2-VL 2B Instruct · 4-bit Alibaba38±1local~2.6GB33kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
704Qwen2.5 1.5B Instruct · 8-bit Alibaba38±1local~2.5GB~98 tok/s33kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+6
707LFM2 700M · 8-bit Liquid AI38±1local~1.4GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
708Qwen3.5 0.8B · 8-bit Alibaba38±1local~2.5GB~319 tok/s262kOllama › Qwen/Qwen3.5-2B model cardEuroEval leaderboards (results collection), English generative leaderboardQwen/Qwen3.5-0.8B model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+2
723DeepSeek R1 Distill Qwen 1.5B · 4-bit DeepSeek37±2local~2.4GB131kOllama › deepseek-ai/DeepSeek-R1 model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardTHINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models (arXiv:2505.22113v1)Epoch AI — internal evaluations+1
731Qwen3 0.6B · 4-bit Alibaba36±1local~2.5GB~191 tok/s33kOllama › Olmo 3LiquidAI/LFM2-1.2B model cardUnsupervised Confidence Calibration for Reasoning LLMs from a Single Generation (arXiv:2604.19444v1)Berkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+5
742OLMo 2 1B Instruct (0425) · 4-bit Allen Institute for AI35±1local~2.4GB4kllama.cpp › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsOpen Portuguese LLM Leaderboard (archived)
750DeepSeek-Coder 1.3B Base · 4-bit DeepSeek34±1local~2.4GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
756Qwen2 1.5B Instruct · 8-bit Alibaba34±1local~2.5GB32kllama.cpp › WildEval ZeroEval published resultsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsSpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+1
758Falcon-H1 1.5B Instruct · 4-bit TII33±4local~2.4GB131kllama.cpp › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
759Gemma 3 1B · 8-bit Google33±1local~2.4GB~180 tok/s32kLM Studio › Google DeepMind Gemma 3 270M IT model cardLiquidAI/LFM2-1.2B model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2.5-1.2B-Thinking model cardGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboard+7
762LFM2 350M · 8-bit Liquid AI33±2local~1.9GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
767Llama 3.2 1B Instruct · 8-bit Meta33±1local~2.2GB~251 tok/s131kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)LiquidAI/LFM2-1.2B model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLiquidAI/LFM2.5-1.2B-Thinking model cardLMArena leaderboard dataset+13
768Phi-1.5 (1.3B) · 4-bit Microsoft33±1local~2.4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
777Granite 3.1 1B-A400M Instruct · 4-bit IBM31±2†local~2.3GB~20 tok/s131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
788Qwen2.5 0.5B Instruct · 8-bit Alibaba30±2local~1.8GB33kOllama › HuggingFaceTB/SmolLM2-360M-Instruct model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)+1
793Granite 3.0 1B-A400M Instruct · 4-bit IBM30±1local~2.4GB4kOllama › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
799EuroLLM 1.7B Instruct · 4-bit UTTER (EuroLLM)29±3local~2.6GB4kllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
800Granite 4.0 H 350M · 8-bit IBM29±1†local~1.9GB33kLM Studio › ibm-granite/granite-4.0-h-1b model card
804Granite 4.0 350M · 8-bit IBM28±1local~1.9GB32kllama.cpp › ibm-granite/granite-4.0-h-1b model cardBerkeley Function Calling Leaderboard result archiveEpoch AI — internal evaluations
808Qwen2 0.5B Instruct · 8-bit Alibaba28±2local~1.8GB32kOllama › Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
814Falcon3 1B Instruct · 4-bit TII26±4local~2.7GB8kllama.cpp › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
815Qwen1.5 1.8B Chat · 4-bit Alibaba26±2local~2.7GB33kOllama › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
821Flan-T5 Base · 4-bit Google25±1local~2.5GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsModel-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsAugmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runsSpecializing Smaller Language Models towards Multi-Step Reasoning — authors' own runs+3
831Qwen1.5 0.5B Chat · 4-bit Alibaba23±3local~1.9GB33kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
835Phi-1 (1.3B) · 4-bit Microsoft22±4local~2.4GB2kllama.cpp › CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)
837SmolLM2 360M Instruct · 4-bit Hugging Face22±3local~1.5GB8kOllama › HuggingFaceTB/SmolLM2-360M-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
839OLMo 1B · 4-bit Allen Institute for AI21±2†local~2.2GB2kllama.cpp › OLMo: Accelerating the Science of Language Models — authors' own runs
849Gemma 3 270M IT · 8-bit Google17±4local~1.8GB32kOllama › Google DeepMind Gemma 3 270M IT model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
850Flan-T5 Small · 4-bit Google16±3local~1.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsPOSQA: Probe the World Models of LLMs with Size Comparisons — authors' own runsAn Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs
855T5 Base · 4-bit Google14±3local~2.4GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
856Pythia 1.4B · 4-bit EleutherAI14±1local~2.4GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
858Pythia 1B · 8-bit EleutherAI12±1local~2.7GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
861T5 Small · 4-bit Google12±3local~1.8GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
863Zyte-1B · 4-bit venkycs (aihub-app)11±3local~2.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
866Pythia 410M · 4-bit EleutherAI8±2local~2.5GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)
868BLOOM 1.1B · 4-bit BigScience7±1local~2.6GB2kllama.cpp › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Crosslingual Generalization through Multitask Finetuning — authors' own runs
872OPT-350M · 4-bit Meta1±1local~2.2GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsRevisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? — authors' own runsDo Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models — authors' own runs+1
875BLOOM 560M · 4-bit BigScience0±1local~2.7GB2khow to run › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsCrosslingual Generalization through Multitask Finetuning — authors' own runs+1
876Pythia 160M · 4-bit EleutherAI-2±2local~1.9GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)
877GPT-2 Large (774M) · 8-bit OpenAI-3±1local~2.4GB~137 tok/s1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsGenerative Calibration for In-context Learning — authors' own runs+3
878OPT-125M · 4-bit Meta-3±1local~1.8GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsPreserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsCan Language Models Understand Physical Concepts? — authors' own runs+1
879GPT-Neo 125M · 4-bit EleutherAI-3±2local~2.1GB2khow to run › Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsPrompting Language Models for Linguistic Structure — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluationsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)TruthfulQA: Measuring How Models Mimic Human Falsehoods — authors' own runs
880GPT-2 Medium (355M) · 8-bit OpenAI-5±1local~1.9GB1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsRankGen: Improving Text Generation with Large Ranking Models — authors' own runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluations+3
881Pythia 70M · 4-bit EleutherAI-13±2local~1.7GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
882GPT-2 Small (124M) · 4-bit OpenAI-13±2local~2.1GB1khow to run › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluationsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsLarge Language Models Are Reasoning Teachers — authors' own runs+3
—Grok Build 0.1 SpaceXAI (xAI)no reviewed resultsapi $1.25/M256kdocs.x.ai ↗

+ 622 more don't fit your machine or budget · show them

+ 25 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

913 of 1219 catalogue models listed here · sources and licences

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory

291 models · 575 benchmarks · updated