whatmodel.sh

· find the best AI models you can run
891 models · 575 benchmarks · updated
Filter by your hardware and budget
ram[2][4][8][16][24][32][48][64][96][128][192][256]
[]
GB
gpu[apple silicon][graphics card][unified memory][none]
price[local][$0.02][$0.1][$0.5][$2][$10][$50][any]max per M
Filter by task
task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]
New #1s:
orRSS

detecting your OS…

$ terminal

detecting your OS…

Run curl whatmodel.shto get this list in your terminal (add &format=json for agents).

Score vs release date

Release dates available for 881 of 886 models. Undated models are omitted; month-only dates are plotted at the start of the month.

#modelscore↓where↓RAM↓speed↓context↓get itsources
#modelscore↓where↓RAM↓speed↓context↓get itsources
1GPT-6 Astra OpenAI85±1api $20.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
2Claude Opus 5.5 Anthropic85±1api $8.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
3GPT-6.1 Terra OpenAI84±1api $2.00/Mopenai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
4Gemini 4 Carbon Google83±1limited previewLMArena leaderboard dataset
5GPT-6.1 Sol OpenAI83±1api $4.00/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
6Claude Fable 5.1 Anthropic82±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+1
7Muse Wonder Full Meta82±1Muse Code only1Mresearch.meta.ai ↗LMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)
8Claude Sonnet 5.5 Anthropic82±1api $4.00/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
9Gemini 4 Argon Google81±1api $4.00/M1Mai.google.dev ↗LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)
10Claude Opus 5 Anthropic81±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+11
11GPT-6 Sol OpenAI80±1api $4.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard+1
12GPT-5.5 Pro OpenAI80±1api $67.50/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
13GPT-5.6 Sol OpenAI80±1api $8.00/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissions+14
14Claude Fable 5 Anthropic79±1api $20.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runs+5
15Claude Mythos Preview (limited preview) Anthropic79±1†limited preview1MAnthropic system cards and model announcements (lab-reported scores)
16GPT-5.4 Pro OpenAI79±1api $67.50/M1.1Mdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
17Muse Wonder Meta78±1api $8.00/M1Mdev.meta.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)
18GPT-5.5 OpenAI78±0api $11.25/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAgents' Last Exam — authors' own fixed-harness model comparison+16
19MiMo-V2.6-Pro Xiaomi77±1api $0.54/M +local~443GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
20DeepSeek V4.1 Flash 1017 DeepSeek77±1api $0.26/M1Mapi-docs.deepseek.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetDeepSeek V4.1 Flash 1017 model card
21Hy4 preview (770B-A49B) Tencent77±1api $1.25/M +local~248GB1Mcloud.tencent.com ↗LMArena leaderboard datasetTencent Hy4 preview model card
22GPT-5.6 Terra OpenAI77±1api $4.50/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
23Qwen3.8 Max (2.4T-A95B) Alibaba76±1api $3.00/M +local~380GB262kdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)+2
24Claude Opus 4.8 Anthropic76±0api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+11
25GPT-5.4 OpenAI76±1api $5.63/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAgents' Last Exam — authors' own fixed-harness model comparison+14
26Muse Spark 1.3 Meta76±1api $2.00/M1Mdev.meta.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)Terminal-Bench 4.0 official leaderboard submissionsE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
27MiMo-V2.6-Flash (309B-A15B) Xiaomi76±1api $0.17/M +local~136GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
28Gemini 3.8 Flash Google76±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+5
29DeepSeek V4.1 Flash DeepSeek76±1local~512GB~12 tok/s1Mhow to run › Epoch AI — internal evaluationsLMArena leaderboard datasetdeepseek-ai/DeepSeek-V4.1-Flash model cardDeepSeek V4.1 Flash 1017 model cardVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
30Kimi K3 Moonshot AI75±1api $6.00/M +local~884GB1Mplatform.moonshot.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runs+7
31GPT-5.2 Pro OpenAI75±1api $57.75/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
32Claude Haiku 5.5 Anthropic75±1api $0.20/M1Mplatform.claude.com ↗Epoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)LMArena leaderboard dataset
33GPT-6 Luna OpenAI75±1api $0.20/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMiðeind Icelandic LLM Leaderboard
34GPT-5.6 Luna OpenAI75±1api $0.45/M1.1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
35Grok 4.6 SpaceXAI (xAI)75±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
36Gemini 3.7 Flash Google75±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashTerminal-Bench 4.0 official leaderboard submissions+3
37DeepSeek V4 Pro (0813) DeepSeek75±1api $1.63/M +local~1010GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party models+2
38GLM-5.3 Z.ai75±1api $2.15/M +local~239GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
39Qwen3.8 Max (0902) Alibaba74±1api $3.00/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
40Muse Spark Open Meta74±1api $2.00/M +local~190GB1Mfireworks.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)
41DeepSeek V3.2 Speciale DeepSeek74±3†comparison onlydeepseek-ai/DeepSeek-V3.2 model card
42Muse Spark Meta74±2comparison onlyLMArena leaderboard datasetEpoch AI — internal evaluations
43Gemini 3.1 Pro (Preview) Google74±0api $4.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)meituan-longcat/LongCat-2.0 model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+23
44Claude Opus 4.7 Anthropic74±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetmeituan-longcat/LongCat-2.0 model cardAnthropic system cards and model announcements (lab-reported scores)Agents' Last Exam — authors' own fixed-harness model comparison+9
45Grok 4.5 SpaceXAI (xAI)74±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runs+1
46Qwen3.8-Omni-Flash Alibaba74±1†api $0.23/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
47DeepSeek V4 Flash Vision (Exp) DeepSeek74±2local~104GB1Mllama.cpp › deepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardHow Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
48Claude Sonnet 5 Anthropic73±1api $4.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissions+5
49Qwen3.8-Flash-Next Alibaba73±1local~59GB~27 tok/s262kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
50Claude Opus 4.6 Anthropic73±1api $10.00/M1Mdocs.anthropic.com ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsmeituan-longcat/LongCat-2.0 model card+22
51Muse Spark 1.1 Meta73±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsτ²/τ³-bench leaderboard: Sierra's own evaluation runsEpoch AI — internal evaluations
52Grok 4.7 SpaceXAI (xAI)73±1api $3.00/M500kdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
53Muse Spark 1.2 Meta73±1api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)Epoch AI — internal evaluations
54Step 5 Preview StepFun73±1comparison onlyLMArena leaderboard dataset
55GPT-5.3-Codex OpenAI73±1api $4.81/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
56Gemini 3.5 Flash Google73±1api $3.38/M1Mai.google.dev ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Legal Research Bench (Vals AI, arXiv:2610.00609v1)Miðeind Icelandic LLM Leaderboard+3
57Qwen3.7 Max Alibaba73±1api $3.75/M1Mdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
58DeepSeek V4 Flash (0731) DeepSeek72±1api $0.11/M +local~92GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-V4.1-Flash model cardDaekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetdeepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+2
59GLM-5.3-Flash Z.ai72±1api $0.24/M +local~103GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeTerminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)+2
60GPT-5.2 OpenAI72±1api $4.81/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsDaekeun Kim Korean LLM benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+21
61Gemini 3.6 Flash Google72±1api $1.50/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGoogle DeepMind Gemini model page and model cards (lab-reported scores)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 Flash+2
62GLM-5.2 Z.ai72±1api $2.15/M +local~121GB1Mdocs.z.ai ↗Epoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetzai-org/GLM-5.2 model cardzai-org/GLM-5.3-Flash model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+6
63Grok 4.20 Multi-Agent (0309) SpaceXAI (xAI)72±2api $1.56/M1Mdocs.x.ai ↗LMArena leaderboard dataset
64Dola Seed 2.0 Pro ByteDance72±2comparison onlyLMArena leaderboard datasetByteDance Seed - Seed2.0 Model Card (arXiv:2607.00248v1)
65Hunyuan Hy3 (295B-A21B) Tencent72±1api $0.23/M +local~94GB256kdeepinfra.com ↗tencent/Hy3 model cardLMArena leaderboard datasetTencent Hy4 preview model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
66Qwen3.8 27B Alibaba72±1api $1.60/M +local~18GB~6.7 tok/s262kconsole.groq.com ↗LMArena leaderboard datasetQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+3
67LongCat 2.0 (1.6T-A48B) Meituan72±2api $1.30/M +local~2155GB1Mlongcat.chat ↗meituan-longcat/LongCat-2.0 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
68Kimi K2.6 Moonshot AI72±1api $1.71/M +local~317GB262kplatform.moonshot.ai ↗Epoch AI — internal evaluationsNVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsAgents' Last Exam — authors' own fixed-harness model comparison+11
69GPT-5 Pro OpenAI72±1api $41.25/M400kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets
70Gemini 3 Pro (Preview) Google71±1comparison only1MLMArena leaderboard datasetEpoch AI — internal evaluationsGoogle DeepMind Gemini model page and model cards (lab-reported scores)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsτ²/τ³-bench leaderboard: Sierra's own evaluation runsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets+15
71GPT-5 OpenAI71±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsτ²/τ³-bench leaderboard: Sierra's own evaluation runsKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+22
72Qwen3.6 Max Preview Alibaba71±1api $2.92/M256kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
73Mistral Large 4 (public preview) Mistral71±1comparison only1MLMArena leaderboard datasetEpoch AI — internal evaluations
74Claude Sonnet 4.6 Anthropic71±1api $6.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetMulti-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe (arXiv:2609.05395v1)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs+18
75Gemini 3 Flash (Preview) Google71±1api $1.13/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGoogle DeepMind Gemini model page and model cards (lab-reported scores)+8
76DeepSeek V4 Pro (Preview, Apr 2026) DeepSeek71±1local~604GBllama.cpp › Epoch AI — internal evaluationsNVIDIA — Nemotron 3 Ultra technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetLGAI-EXAONE/K-EXAONE-2.0-750B-A37B model cardLegal Research Bench (Vals AI, arXiv:2610.00609v1)+6
77Claude Opus 4.5 Anthropic71±1api $10.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsAnthropic system cards and model announcements (lab-reported scores)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)+14
78Qwen3.7 Plus Alibaba71±1api $0.70/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LMArena leaderboard datasetSciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
79Kimi K2.7 Code Moonshot AI71±1api $1.71/M +local~359GB262kplatform.kimi.ai ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
80North Small Translate 1.0 Cohere71±5†api price unknown16kdocs.cohere.com ↗Cohere — North Small Translate (arXiv:2609.13916v1)
81Solar Open 2 (250B-A15B) Upstage71±1†local~527GB1Mhow to run › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)
82DeepSeek V4 Flash (Preview, Apr 2026) DeepSeek70±1local~103GBllama.cpp › NVIDIA — Nemotron 3 Ultra technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsSolar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)ai-sage/GigaChat3.5-432B-A28B-Reasoning model cardLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+4
83Grok 4.20 (0309 reasoning) SpaceXAI (xAI)70±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsτ²/τ³-bench leaderboard: Sierra's own evaluation runsThe Token Games: Evaluating Language Model Reasoning with Puzzle Duels — authors' own tournamentWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
84Beam 1.0 501B A23B Reflection70±1api price unknown1Mdevelopers.reflection.ai ↗LMArena leaderboard dataset
85GPT-5.4 mini OpenAI70±1api $1.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsLegal Research Bench (Vals AI, arXiv:2610.00609v1)+6
86Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)70±2†api $0.090/M +local~71GB131kdeepinfra.com ↗inclusionAI/Ling-3.0-flash model card
87GPT-5.1 OpenAI70±1api $3.44/M400kplatform.openai.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetDaekeun Kim Korean LLM benchmark runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsHyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+8
88GLM-5.1 Z.ai70±0api $2.15/M +local~119GB200kdocs.z.ai ↗NVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetszai-org/GLM-5.1 model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1zai-org/GLM-5.2 model card+11
89o3-pro OpenAI70±2api $35.00/M200kdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Aider polyglot benchmark runsOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
90Qwen3.5 397B-A17B Alibaba69±0api $1.09/M +local~65GB262kdeepinfra.com ↗Qwen/Qwen3.5-397B-A17B model cardNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsQwen/Qwen3.6-27B model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard dataset+15
91Gemini 2.5 Pro Preview (06-05) Google69±2comparison onlyAider polyglot benchmark runsEpoch AI — internal evaluationsEQ-Bench 3 canonical leaderboard resultsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
92KAT-Coder-V2.5-Dev (35B-A3B) Kuaishou (Kwaipilot)69±2†local~21GB262kLM Studio › Kwaipilot/KAT-Coder-V2.5-Dev model card
93Ornith 1.5 35B-A3B DeepReinforce69±1†local~21GB~80 tok/s262kLM Studio › ornith-ai/Ornith-1.5-35B-A3B model card
94Grok 4.3 SpaceXAI (xAI)69±1api $1.56/M1Mdocs.x.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetAgents' Last Exam — authors' own fixed-harness model comparisonLegal Research Bench (Vals AI, arXiv:2610.00609v1)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
95MiniMax-M3 MiniMax69±1api $0.48/M +local~131GB1Mdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationszai-org/GLM-5.2 model cardLegal Research Bench (Vals AI, arXiv:2610.00609v1)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Miðeind Icelandic LLM Leaderboard+2
96Qwen3.6 Plus Alibaba69±1api $1.13/M1Mdocs.together.ai ↗Epoch AI — internal evaluationsLMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Agents' Last Exam — authors' own fixed-harness model comparisonQwen/Qwen-AgentWorld-35B-A3B model cardBEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+1
97Kimi K2.5 Moonshot AI69±1api $1.20/M +local~160GB262kdocs.aws.amazon.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsstepfun-ai/Step-3.5-Flash model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsQwen/Qwen3.5-397B-A17B model card+15
98MiMo-V2-Pro Xiaomi69±1comparison onlyLMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardXiaomiMiMo/MiMo-V2.5 model cardWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
99Qwen3-Max (2026-01-23) Alibaba69±1†comparison onlyQwen/Qwen3.5-397B-A17B model cardτ²-bench leaderboard: Qwen (Alibaba) own submissions
100GLM-5 Z.ai69±1api $1.55/M +local~119GB205kdocs.z.ai ↗zai-org/GLM-5.1 model cardEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsGLM-5 Team (Zhipu AI) — GLM-5 technical reportzai-org/GLM-5 model cardLMArena leaderboard dataset+11
101MiMo-V2.5-Pro (1T-A42B) Xiaomi69±1api $0.65/M +local~163GB1Mnovita.ai ↗LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardXiaomi MiMo official model cards and documentationGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
102Muse Glimmer 30B Meta69±1api $0.64/M +local~18GB~27 tok/s131kfireworks.ai ↗meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.8-27B model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Cohere — North Small Translate (arXiv:2609.13916v1)
103o3 OpenAI68±1api $3.50/M200kplatform.openai.com ↗Epoch AI — internal evaluationsMeituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard dataset+11
104Nemotron 3 Ultra 550B-A55B NVIDIA68±1api $1.35/M +local~89GB1Mdocs.together.ai ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1Epoch AI — internal evaluationsLMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)
105Inkling Small (266B-A12B) Thinking Machines68±1api $0.64/M +local~86GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetMiðeind Icelandic LLM LeaderboardCohere — North Small Translate (arXiv:2609.13916v1)
106Qwen3.6 27B Alibaba68±1api $1.20/M +local~18GB~16 tok/s262kconsole.groq.com ↗Qwen/Qwen3.6-27B model cardEpoch AI — internal evaluationsQwen/Qwen3.8-27B model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+4
107Ring-2.6-1T inclusionAI (Ant Group)68±2†comparison onlyinclusionAI/Ling-3.0-flash model card
108A.X K2 (688B-A33B) SK Telecom68±1†local~1454GB262khow to run › A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
109MiMo-V2.5 (310B-A15B) Xiaomi68±1api $0.21/M +local~100GB1Mnovita.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5 model cardAgents' Last Exam — authors' own fixed-harness model comparisonSciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)Xiaomi MiMo vendor-reported benchmark scores for its own models (mimo.xiaomi.com launch pages)
110GPT-5.5 Instant OpenAI68±1comparison onlyEpoch AI — internal evaluationsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
111Qwen3.5 Plus Alibaba68±2api $0.90/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
112Gemini 3.5 Flash-Lite Google68±1api $0.85/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Miðeind Icelandic LLM Leaderboard
113Qwen3.5 122B-A10B Alibaba68±1api $1.10/M +local~71GB262kwww.alibabacloud.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.5-27B model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)Epoch AI — internal evaluationsReassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)+5
114Inkling (975B-A41B) Thinking Machines68±1api $1.73/M +local~156GB1Mdeepinfra.com ↗Epoch AI — internal evaluationsthinkingmachines/Inkling-Small model cardLMArena leaderboard datasetMeta vendor-reported benchmark scores for its own Muse models (Muse Wonder model card, Muse Spark 1.3 evaluation methodology, Muse Spark Open model card)τ²/τ³-bench leaderboard: Sierra's own evaluation runsMiðeind Icelandic LLM Leaderboard
115Nemotron 3.5 Bolt 74B A9B NVIDIA68±2†local~157GB262khow to run › NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
116GPT-5.4 nano OpenAI68±1api $0.46/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeT1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)+1
117Claude Sonnet 4.5 Anthropic68±1api $6.00/M200kdocs.anthropic.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsτ²/τ³-bench leaderboard: Sierra's own evaluation runsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets+19
118GLM-5V-Turbo Z.ai68±2api $1.90/M200kwww.siliconflow.com ↗LMArena leaderboard dataset
119LongCat-Flash-Thinking (560B-A27B) Meituan68±1†comparison only128kMeituan LongCat-Flash-Thinking model card (evaluation table)
120GLM-4.7 Z.ai68±1api $1.00/M +local~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardGLM-5 Team (Zhipu AI) — GLM-5 technical reportzai-org/GLM-5 model cardLMArena leaderboard datasetEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models+5
121Qwen3.5 27B Alibaba68±0api $0.84/M +local~18GB~29 tok/s262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.6-27B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard dataset+9
122Gemini 2.5 Pro Experimental (03-25) Google67±2comparison onlyEpoch AI — internal evaluationsBerkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)LiveCodeBench official website results (2025-08-01)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
123Grok 4 (0709) SpaceXAI (xAI)67±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)+1
124MiMo-V2-Omni Xiaomi67±2comparison only262kLMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5 model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Xiaomi MiMo vendor-reported benchmark scores for its own models (mimo.xiaomi.com launch pages)
125Nemotron 3.5 Super VL 120B A12B NVIDIA67±1local~254GB1Mhow to run › LMArena leaderboard datasetNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
126Spark-X2.6 4B iFLYTEK (SparkLLM)67±1†local~5.2GB1Mllama.cpp › Spark-X2.6 model cards (thinking high)
127DeepSeek V3.2 DeepSeek67±1api $0.29/M +local~110GB128kdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsdeepseek-ai/DeepSeek-V3.2 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsstepfun-ai/Step-3.5-Flash model card+15
128Step 3.5 Flash (196B-A11B) StepFun67±1api $0.15/M +local~118GB~18 tok/s256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsstepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
129GPT-5.2-Codex OpenAI67±2comparison onlyLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)
130Qwen3.6 35B-A3B Alibaba67±1api $0.31/M +local~21GB~63 tok/s262kdeepinfra.com ↗Qwen/Qwen3.6-27B model cardQwen/Qwen3.6-35B-A3B model cardEpoch AI — internal evaluationsMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
131Qwen3-Coder-Next (80B-A3B) Alibaba67±2api $0.60/M +local~25GB~62 tok/s262kwww.alibabacloud.com ↗Vector Institute Evaluation Leaderboard (Inspect Evals runs)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs
132GPT-5 mini OpenAI67±1api $0.69/M400kplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardDaekeun Kim Korean LLM benchmark runsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)LMArena leaderboard dataset+16
133MiniMax-M2.7 MiniMax67±1api $0.53/M +local~75GB205kplatform.minimax.io ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1LMArena leaderboard datasetinclusionAI/Ling-3.0-flash model cardQwen/Qwen-AgentWorld-35B-A3B model cardWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+2
134Ornith 1.0 35B (35B-A3B) DeepReinforce67±1local~23GB~103 tok/s262kLM Studio › ornith-ai/Ornith-1.5-35B-A3B model cardKwaipilot/KAT-Coder-V2.5-Dev model card
135Claude Opus 4 Anthropic67±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)τ-bench leaderboard: Anthropic's own submissionsInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+4
136Gemini 2.5 Pro Google66±1api $3.44/M1Mai.google.dev ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities+16
137Claude Opus 4.1 Anthropic66±1comparison onlyEpoch AI — internal evaluationsLMArena leaderboard datasetKorean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilitiesτ-bench leaderboard: Anthropic's own submissionsWebApp1K models leaderboard (onekq-ai)Anthropic system cards and model announcements (lab-reported scores)+2
138Hy3 preview (295B-A21B) Tencent66±1local~629GBhow to run › tencent/Hy3 model cardLMArena leaderboard dataset
139Gemma 4 31B Google66±1api $0.19/M +local~24GB~11 tok/s262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportKwaipilot/KAT-Coder-V2.5-Dev model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)LMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+21
140Kimi K2 Thinking Moonshot AI66±1api $1.07/M +local~160GB262kdocs.aws.amazon.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsstepfun-ai/Step-3.5-Flash model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
141Gemini 2.5 Pro Preview (05-06) Google66±1comparison onlyAider polyglot benchmark runsEpoch AI — internal evaluationsAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)LiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)+2
142Gemini 2.5 Pro Preview (03-25) Google66±2comparison onlyAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)Galileo Agent Leaderboard (v1 and v2 results)
143K-EXAONE 2.0 (750B-A37B) LG AI Research66±1†local~791GB262khow to run › LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card
144Gemini 3.1 Flash-Lite Google66±1api $0.56/M1Mai.google.dev ↗Epoch AI — internal evaluationsLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs — authors' own model runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
145Qwen3.7 Flash Alibaba66±1api $0.055/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluations
146Laguna M.1 Poolside66±2api price unknown +local~135GB262khuggingface.co ↗LMArena leaderboard datasetPoolside — Laguna M.1/XS.2 Technical Report
147DeepSeek V3.1 Terminus DeepSeek66±1local~230GB164kllama.cpp › DeepSeek-V3.1-Terminus model cardLMArena leaderboard dataset
148DeepSeek V3.2 Exp DeepSeek66±1local~726GBhow to run › Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)LMArena leaderboard datasetAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archiveVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
149Kimi K2 Instruct (0711) Moonshot AI66±1api $1.00/M +local~152GB131knovita.ai ↗Meituan LongCat-Flash-Chat model card (evaluation table)Aider polyglot benchmark runsLMArena leaderboard datasetMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)EQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archive+3
150o1 (2024-12-17) OpenAI66±1api $26.25/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results+13
151GigaChat 3.5 Ultra Reasoning (432B-A28B) Sber66±1†local~461GB262khow to run › ai-sage/GigaChat3.5-432B-A28B-Reasoning model card
152o4-mini OpenAI66±1api $1.93/M200kplatform.openai.com ↗Epoch AI — internal evaluationsQwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runs+12
153Qwen3 235B-A22B Thinking 2507 Alibaba66±1api $0.97/M +local~76GB131knovita.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-Next-80B-A3B-Thinking model cardQwen3-VL-235B-A22B-Thinking model cardQwen/Qwen3.5-27B model cardEpoch AI — internal evaluations+6
154Qwen3.5 35B-A3B Alibaba66±0api $0.35/M +local~21GB~50 tok/s262kdeepinfra.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.5-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardNVIDIA — Nemotron-Cascade 2 technical reportLMArena leaderboard dataset+13
155Step 3.7 Flash (198B-A11B) StepFun66±1api $0.44/M +local~104GB256kplatform.stepfun.ai ↗MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsinclusionAI/Ling-3.0-flash model card
156Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)66±1api $0.28/M128kazure.microsoft.com ↗LMArena leaderboard datasetMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archiveτ²/τ³-bench leaderboard: Sierra's own evaluation runs+4
157LongCat-Flash-Chat (560B-A27B) Meituan65±1local~333GB128kLM Studio › Meituan LongCat-Flash-Chat model card (evaluation table)LMArena leaderboard dataset
158DeepSeek V3.1 DeepSeek65±1api $0.42/M +local~110GB131kdeepinfra.com ↗Meituan LongCat-Flash-Thinking model card (evaluation table)Meituan LongCat-Flash-Chat model card (evaluation table)deepseek-ai/DeepSeek-V3.1 model cardDeepSeek-V3.1-Terminus model cardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+3
159GLM-4.6 Z.ai65±1api $1.00/M +local~58GB205kdocs.z.ai ↗zai-org/GLM-4.7 model cardzai-org/GLM-4.6 model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archive+1
160Qwen3.6 Flash Alibaba65±1api $0.56/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsSpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models (arXiv:2610.12402v1)
161Qwen3-Max (2025-09-23) Alibaba65±1api $2.40/M262kwww.alibabacloud.com ↗Epoch AI — internal evaluationsτ²-bench leaderboard: Qwen (Alibaba) own submissionsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)
162MiniMax-M2.5 MiniMax65±1api $0.53/M +local~75GB205kplatform.minimax.io ↗LMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsKnowledge Index of Noah's Ark (arXiv:2606.05104v2)WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)+3
163o3-mini OpenAI65±1api $1.93/M200kdevelopers.openai.com ↗Epoch AI — internal evaluationsServiceNow-AI/Apriel-1.6-15b-Thinker model cardMicrosoft — Phi-4-reasoning Technical ReportOpenAI simple-evals README benchmark results (OpenAI's own models)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runs+13
164Claude Sonnet 4 Anthropic65±1comparison onlyMeituan LongCat-Flash-Chat model card (evaluation table)Epoch AI — internal evaluationsAider polyglot benchmark runsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+12
165MiMo-V2-Flash (309B-A15B) Xiaomi65±1local~103GBllama.cpp › stepfun-ai/Step-3.5-Flash model cardLMArena leaderboard datasetWildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)
166DeepSeek R1 0528 DeepSeek65±1api $0.91/M +local~104GB164kdeepinfra.com ↗deepseek-ai/DeepSeek-R1-0528 model cardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)deepseek-ai/DeepSeek-V3.1 model cardServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+6
167Claude 3.7 Sonnet Anthropic65±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsMicrosoft — Phi-4-reasoning Technical ReportLMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runsArena-Hard-Auto v2.0-Preview leaderboard (repository README)+16
168MiniMax-M2.1 MiniMax65±1local~79GBllama.cpp › LMArena leaderboard datasetstepfun-ai/Step-3.5-Flash model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
169Spark-X2.5 4B iFLYTEK (SparkLLM)65±1†local~3.5GB~114 tok/s1MLM Studio › Spark-X2.6 model cards (thinking high)XHToken/Spark-X2.5-4B model card
170Nemotron Cascade 2 30B-A3B NVIDIA65±1local~20GB~90 tok/s256kLM Studio › NVIDIA — Nemotron-Cascade 2 technical reportEuroEval leaderboards (results collection), English generative leaderboard
171ChatGPT-4o (2025-03-26) OpenAI65±2comparison onlyAider polyglot benchmark runsLMArena leaderboard dataset
172MiMo-V2.6-Distill-Qwen-9B Xiaomi64±2†local~7.8GBLM Studio › XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B model card
173Laguna XS 2.1 (33B-A3B) Poolside64±2†local~21GB262kLM Studio › Poolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)
174K-EXAONE (236B-A23B) LG AI Research64±1local~152GB262kllama.cpp › LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model cardLG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)
175Qwen3 235B-A22B Instruct 2507 Alibaba64±1api $0.20/M +local~73GB~15 tok/s262kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMeituan LongCat-Flash-Chat model card (evaluation table)Qwen3-VL-235B-A22B-Instruct model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsLMArena leaderboard dataset+3
176Mistral Medium 3.5 (128B) Mistral64±1api $3.00/M +local~43GB262kdocs.mistral.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)LMArena leaderboard datasetCohere — North Small Translate (arXiv:2609.13916v1)Miðeind Icelandic LLM LeaderboardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
177INTELLECT-3 Prime Intellect64±1local~48GBllama.cpp › Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard dataset
178GLM-4.5 Z.ai64±1api $1.00/M +local~58GB131kdocs.z.ai ↗Meituan LongCat-Flash-Thinking model card (evaluation table)GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)zai-org/GLM-4.6 model cardLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard results+3
179Qwen3.5 Flash Alibaba64±1api $0.17/M1Mwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
180Mistral Medium 3.1 (2508) Mistral64±2comparison onlyLMArena leaderboard datasetGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
181MiniMax-M2 MiniMax64±1api $0.53/M +local~75GB205kdocs.aws.amazon.com ↗LMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
182Command A+ (05-2026) Cohere64±1api price unknown +local~71GB128kdocs.cohere.com ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Cohere vendor-reported benchmark scores for its own models (cohere.com blog)Cohere — North Small Translate (arXiv:2609.13916v1)
183Claude Haiku 4.5 Anthropic64±1api $2.00/M200kdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsLMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+9
184Nemotron 3 Super 120B-A12B NVIDIA64±1api $0.28/M +local~73GB~20 tok/s262kdocs.aws.amazon.com ↗NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)NVIDIA — Nemotron-Cascade 2 technical reportMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsLMArena leaderboard datasetinclusionAI/Ling-3.0-flash model card
185Laguna XS.2 Poolside64±1local~21GB262kLM Studio › LMArena leaderboard datasetPoolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)Poolside — Laguna M.1/XS.2 Technical Report
186Gemma 4 26B-A4B Google64±1api $0.14/M +local~18GB~55 tok/s262kdeepinfra.com ↗Gemma Team — Gemma 4 Technical Reportgoogle/diffusiongemma-26B-A4B-it model cardKwaipilot/KAT-Coder-V2.5-Dev model cardLMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+13
187EXAONE 4.5 33B LG AI Research64±2†local~24GB262kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe (arXiv:2609.05395v1)
188Solar Pro 4 Upstage63±1api $0.53/M524kdevelopers.upstage.ai ↗LMArena leaderboard dataset
189Kimi K2 Instruct (0905) Moonshot AI63±2local~298GBllama.cpp › LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
190Qwen3-Coder 480B-A35B Instruct Alibaba63±1api $0.67/M +local~78GB262knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardLMArena leaderboard datasetTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsWebApp1K models leaderboard (onekq-ai)
191Qwen3-VL 235B-A22B Thinking Alibaba63±1local~106GB256kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen/Qwen3.5-397B-A17B model cardQwen3-VL-235B-A22B-Thinking model cardLMArena leaderboard datasetWebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
192DeepSeek R1 (Jan '25) DeepSeek63±1api $4.00/M +local~104GB128kdocs.together.ai ↗deepseek-ai/DeepSeek-R1 model carddeepseek-ai/DeepSeek-R1-0528 model cardEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsWildEval ZeroEval published results+15
193o1-preview OpenAI63±1comparison onlyWildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsOpenAI simple-evals README benchmark results (OpenAI's own models)Epoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)+7
194Grok 3 Beta SpaceXAI (xAI)63±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)EQ-Bench 3 canonical leaderboard resultsWebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archive
195Grok 3 Mini Beta SpaceXAI (xAI)63±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)LiveCodeBench official website results (2025-08-01)EQ-Bench 3 canonical leaderboard results+2
196Qwen3-VL 235B-A22B Instruct Alibaba63±1local~128GB256kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen3-VL-235B-A22B-Instruct model cardLMArena leaderboard datasetWebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
197MiniCPM5 2B OpenBMB63±1†local~2.4GB131kLM Studio › openbmb/MiniCPM5-2B model card
198Nemotron-Labs-3 Puzzle 75B-A9B NVIDIA63±1†local~35GBllama.cpp › NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
199Qwen3-Next 80B-A3B Instruct Alibaba63±1api $0.34/M +local~26GB~55 tok/s131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
200Nova 2 Pro (Preview) Amazon63±2api $3.78/M1Maws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsAmazon AGI — τ²-Bench-Verified
201Gemini 2.5 Flash Preview (09-2025) Google63±1comparison onlyServiceNow-AI/Apriel-1.6-15b-Thinker model cardLMArena leaderboard dataset
202Qwen3-Next 80B-A3B Thinking Alibaba63±1api $0.41/M +local~25GB131kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3.5-9B model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsQwen/Qwen3.5-4B model cardLMArena leaderboard datasetKnowledge Index of Noah's Ark (arXiv:2606.05104v2)+3
203Granite 4.2 30B IBM63±1api $0.28/M +local~19GB131kdeepinfra.com ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
204GPT-4.5 Preview OpenAI63±1comparison onlyOpenAI simple-evals README benchmark results (OpenAI's own models)Epoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)+6
205Gemini 2.5 Flash Preview (04-17) Google63±1comparison onlyAider polyglot benchmark runsEpoch AI — internal evaluationsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (arXiv:2505.20740v3)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
206GPT-4.1 OpenAI62±1api $3.50/M1Mplatform.openai.com ↗Meituan LongCat-Flash-Chat model card (evaluation table)Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runsAider polyglot benchmark runsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+17
207GLM-4.5-Air Z.ai62±1api $0.42/M +local~35GB~23 tok/s131kdocs.z.ai ↗Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetSERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)Solar Open Technical Report (arXiv:2601.07022v1)+4
208Trinity-Large-Thinking Arcee AI62±2api $0.39/M +local~141GB512kdocs.arcee.ai ↗LMArena leaderboard datasetArcee AI vendor-reported benchmark scores for its own Trinity models (Hugging Face model cards)
209Nemotron 3.5 Lightning 30B-A3B NVIDIA62±2api $0.087/M +local~20GB~64 tok/s262kfireworks.ai ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard dataset
210Seed-OSS 36B Instruct ByteDance Seed62±1local~23GB524kLM Studio › ByteDance-Seed/Seed-OSS-36B-Instruct model cardEpoch AI — internal evaluations
211Qwen3.5 9B Alibaba62±0api $0.11/M +local~6.7GB~22 tok/s262kdeepinfra.com ↗Qwen/Qwen3.5-9B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+19
212MiniMax-M1 (80k) MiniMax62±1api $0.96/M +local~959GB1Mnovita.ai ↗MiniMaxAI/MiniMax-M1-80k model cardLMArena leaderboard dataset
213Gemma 4 12B Google62±1local~9.2GB~29 tok/s262kLM Studio › Gemma Team — Gemma 4 Technical ReportXHToken/Spark-X2.5-4B model cardgoogle/gemma-4-31B-it model card
214GLM-4.6V Z.ai62±2api $0.45/M +local~50GB128kdocs.z.ai ↗LMArena leaderboard datasetZ.ai GLM-V GitHub repository benchmark image (GLM-4.6V)SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)
215Magistral Medium 1.2 (2509) Mistral62±2†comparison only128kmistralai/Magistral-Small-2509 model card
216HyperCLOVA X SEED 32B Think NAVER Cloud62±2†comparison only128kHyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)
217Gemini 2.5 Flash Google62±1api $0.85/M1Mai.google.dev ↗Meituan LongCat-Flash-Chat model card (evaluation table)LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+10
218Gemini 2.5 Flash Preview (05-20) Google62±1comparison onlyAider polyglot benchmark runsAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)Epoch AI — internal evaluationsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)+1
219Gemini 2.0 Flash-Lite (001) Google62±2comparison onlyAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)Galileo Agent Leaderboard (v1 and v2 results)
220Qwen3-VL 32B Thinking Alibaba62±1api $0.28/M +local~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)allenai/Olmo-3.1-32B-Think model card
221gpt-oss-120b OpenAI62±1api $0.26/M +local~64GB~56 tok/s131kconsole.groq.com ↗OpenAI — gpt-oss-120b & gpt-oss-20b Model CardPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsAider polyglot benchmark runs+19
222GPT-5 nano OpenAI62±1api $0.14/M400kplatform.openai.com ↗Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsBerkeley Function Calling Leaderboard result archive+7
223Mistral Large 3 (675B-A41B) Mistral62±1api $0.75/M +local~108GB262kdocs.mistral.ai ↗LMArena leaderboard datasetmistralai/Mistral-Small-4-119B-2603 model cardCohere — North Small Translate (arXiv:2609.13916v1)LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? (arXiv:2610.04118v1)Miðeind Icelandic LLM LeaderboardOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+1
224Apriel 1.6 15B Thinker ServiceNow62±1†local~10GBLM Studio › ServiceNow-AI/Apriel-1.6-15b-Thinker model card
225Devstral 2 (2512) Mistral62±2local~74GB256kLM Studio › LMArena leaderboard datasetmistralai/Devstral-Small-2-24B-Instruct-2512 model cardDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runs
226Gemini 2.5 Flash-Lite Preview (06-17) Google62±2comparison onlyLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archive
227A.X K1 (519B-A33B) SK Telecom61±1†local~1091GBhow to run › A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
228o1-mini OpenAI61±1comparison onlyWildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)+12
229GLM-4.5V Z.ai61±2api $0.90/M +local~69GB64kdocs.z.ai ↗LMArena leaderboard datasetZ.ai GLM-V GitHub repository benchmark image (GLM-4.6V)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
230Devstral Small 2 24B Mistral61±2local~16GB262kllama.cpp › mistralai/Devstral-Small-2-24B-Instruct-2512 model cardSERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)Agentic coding without the cloud: evaluating open-weight LLMs on longitudinal data preparation tasks (arXiv:2607.21482v2)Improving Natural-Language Combinatorial-Optimization Accuracy in Resource-Constrained Language Models via Formal Abstractions (arXiv:2608.18409v2)
231MiniMax-M1 (40k) MiniMax61±1†comparison onlyMiniMaxAI/MiniMax-M1-80k model card
232Llama 3.1 Nemotron Ultra 253B v1 NVIDIA61±1local~82GB128kllama.cpp › NVIDIA — Llama-Nemotron: Efficient Reasoning ModelsLMArena leaderboard datasetEQ-Bench 3 canonical leaderboard resultsBerkeley Function Calling Leaderboard result archiveLiveCodeBench official website results (2025-08-01)
233Gemini 2.0 Pro Experimental (02-05) Google61±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
234Grok 4 Fast SpaceXAI (xAI)61±1comparison onlyLMArena leaderboard datasetARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation setsServiceNow-AI/Apriel-1.6-15b-Thinker model cardτ²/τ³-bench leaderboard: Sierra's own evaluation runsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
235DeepSeek V3 0324 DeepSeek60±1local~102GB164kllama.cpp › deepseek-ai/DeepSeek-V3.1 model cardEpoch AI — internal evaluationsAider polyglot benchmark runsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)LMArena leaderboard datasetMoonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)+11
236Claude 3.5 Sonnet (2024-10-22) Anthropic60±1comparison onlyWildEval ZeroEval published resultsEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)Aider polyglot benchmark runsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)LMArena leaderboard dataset+14
237GPT-4.1 mini OpenAI60±1api $0.70/M1Mplatform.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runsAider polyglot benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard dataset+13
238Qwen3 30B-A3B Thinking 2507 Alibaba60±1api $0.75/M +local~19GB262kwww.alibabacloud.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardByteDance-Seed/Seed-OSS-36B-Instruct model cardQwen/Qwen3.5-9B model cardEpoch AI — internal evaluationsQwen/Qwen3.5-4B model card+8
239Gemini 2.0 Flash (001) Google60±1comparison onlyEpoch AI — internal evaluationsAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)LMArena leaderboard datasetBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EQ-Bench 3 canonical leaderboard resultsFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)+7
240Qwen3 30B-A3B Instruct 2507 Alibaba60±1api $0.15/M +local~19GB~100 tok/s262ktokenfactory.nebius.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
241ERNIE 4.5 300B-A47B Baidu60±1†local~97GB131kllama.cpp › ERNIE 4.5 post-trained 300B-A47B benchmark chart
242Qwen3.5 4B Alibaba60±1local~3.8GB262kllama.cpp › Qwen/Qwen3.5-9B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardQwen/Qwen3.5-4B model cardLiquidAI/LFM2.5-2.6B model card+11
243Mistral Medium 3 (2505) Mistral60±1comparison onlyEpoch AI — internal evaluationsLMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)Berkeley Function Calling Leaderboard result archiveDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)
244Mellum2.1 12B-A2.5B Thinking JetBrains60±1†local~8.3GBLM Studio › JetBrains/Mellum2.1-12B-A2.5B-Thinking model card
245Gemma 2 9B IT SimPO Princeton NLP60±1local~7.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetOpen Portuguese LLM Leaderboard (archived)
246Gemma 2 9B IT DPO (Princeton) Princeton NLP60±1local~7.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
247Falcon-H1R 7B TII60±1†local~5.3GB262kLM Studio › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
248Nemotron-Cascade-14B-Thinking NVIDIA60±1†local~10GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
249Qwen3-VL 32B Instruct Alibaba60±1api $0.28/M +local~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
250DeepSeek V3 (Dec '24) DeepSeek60±1api $0.46/M +local~337GB131kdeepinfra.com ↗deepseek-ai/DeepSeek-R1 model cardCohere — Command A technical reportEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsWildEval ZeroEval published results+10
251Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba60±0api $1.23/M +local~73GB131kwww.alibabacloud.com ↗Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+14
252Gemini 2.0 Flash-Lite Preview (02-05) Google60±2comparison onlyLMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
253Granite 4.2 8B IBM59±1api $0.11/M +local~6.8GB131kwandb.ai ↗ibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
254Mistral Small 4 (119B-A6.5B) Mistral59±2api $0.26/M +local~37GB262kdocs.mistral.ai ↗mistralai/Mistral-Small-4-119B-2603 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Miðeind Icelandic LLM Leaderboard
255QwQ-32B Alibaba59±1local~21GB131kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEpoch AI — internal evaluationsAider polyglot benchmark runsArena-Hard-Auto v2.0-Preview leaderboard (repository README)LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)+7
256Solar Open 100B (102B-A12B) Upstage59±1†local~41GB131kllama.cpp › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Solar Open Technical Report (arXiv:2601.07022v1)
257Qwen3-VL 30B-A3B Thinking Alibaba59±1api $0.75/M +local~20GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
258Trinity-Large-Preview Arcee AI59±2local~155GBllama.cpp › Arcee AI vendor-reported benchmark scores for its own Trinity models (Hugging Face model cards)LMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
259OpenReasoning-Nemotron-32B NVIDIA59±2†local~21GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
260Spark-X2.6 1.7B iFLYTEK (SparkLLM)59±1†local~3.4GB1Mllama.cpp › Spark-X2.6 model cards (thinking high)
261EXAONE 4.0 32B LG AI Research59±1local~20GB131kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesEuroEval leaderboards (results collection), English generative leaderboard
262Phi-4-reasoning-plus 14B Microsoft59±1local~10GBLM Studio › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingMicrosoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
263GLM-4.7-Flash Z.ai58±1api $0.0000/M +local~18GB~44 tok/s200kdocs.z.ai ↗Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardSERA: Soft-Verified Efficient Repository Agents (arXiv:2601.20789v3)EQ-Bench 3 canonical leaderboard results+1
264Mercury 2 Inception58±2comparison onlyLMArena leaderboard datasetVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
265gpt-oss-20b OpenAI58±1api $0.13/M +local~13GB~79 tok/s131kconsole.groq.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardOpenAI — gpt-oss-120b & gpt-oss-20b Model CardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+16
266Olmo 3.1 32B Think Ai258±1local~21GB66kLM Studio › Olmo 3Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetallenai/Olmo-3.1-32B-Think model cardEuroEval leaderboards (results collection), English generative leaderboard
267DiffusionGemma 26B-A4B Google58±1†comparison only262kgoogle/diffusiongemma-26B-A4B-it model card
268Qwen3 14B Alibaba58±1api $0.61/M +local~10GB33kwww.alibabacloud.com ↗Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBerkeley Function Calling Leaderboard result archiveBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+7
269Llama 3.3 Nemotron Super 49B v1 NVIDIA58±2local~22GBllama.cpp › NVIDIA — Llama-Nemotron: Efficient Reasoning ModelsLMArena leaderboard dataset
270DeepSeek R1 Distill Llama 70B DeepSeek58±1api $0.80/M +local~30GB131knovita.ai ↗Epoch AI — internal evaluationsMicrosoft — Phi-4-reasoning Technical Reportdeepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Judgemark v2 results (EQ-Bench)+1
271GPT-4o (2024-08-06) OpenAI58±1api $4.38/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsVector Institute Evaluation Leaderboard (Inspect Evals runs)OpenAI simple-evals README benchmark results (OpenAI's own models)Aider polyglot benchmark runs+23
272Nemotron-Cascade-8B NVIDIA58±1†local~6.3GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
273Command A (03-2025) Cohere58±1api $4.38/M +local~37GB256kazure.microsoft.com ↗Cohere — Command A technical reportLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
274Olmo 3 32B Think Ai258±1local~21GB66kLM Studio › Olmo 3Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetallenai/Olmo-3.1-32B-Think model card
275GPT-4o (2024-05-13) OpenAI58±0api $7.50/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)Daekeun Kim Korean LLM benchmark runs+19
276Phi-4-reasoning 14B Microsoft58±1api $0.22/M +local~10GB33kazure.microsoft.com ↗Microsoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
277Claude 3.5 Sonnet (2024-06-20) Anthropic58±0comparison onlyWildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)+16
278Nova 2 Lite Amazon58±1api $0.94/M1Mdocs.aws.amazon.com ↗Daekeun Kim Korean LLM benchmark runsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveAmazon AGI — τ²-Bench-VerifiedVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
279Magistral Small 1.2 Mistral58±1api $0.75/M +local~16GB131kdocs.aws.amazon.com ↗mistralai/Magistral-Small-2509 model cardEpoch AI — internal evaluationsApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarksInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)mistralai/Mistral-Small-4-119B-2603 model cardEuroEval leaderboards (results collection), English generative leaderboard
280GPT-4o (2024-11-20) OpenAI58±1api $4.38/M128kdevelopers.openai.com ↗Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Qwen/Qwen3-235B-A22B-Instruct-2507 model cardAider polyglot benchmark runsQwen/Qwen3-30B-A3B-Instruct-2507 model cardMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+7
281Qwen3-VL 30B-A3B Instruct Alibaba58±1api $0.35/M +local~20GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (arXiv:2604.18224v1)
282Qwen3-VL 8B Thinking Alibaba58±1api $0.66/M +local~6.9GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Think model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
283Qwen3 32B Alibaba57±0api $0.13/M +local~20GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-Next-80B-A3B-Instruct model cardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingByteDance-Seed/Seed-OSS-36B-Instruct model cardEpoch AI — internal evaluationsAider polyglot benchmark runs+17
284Qwen3 Coder 30B-A3B Instruct Alibaba57±1api $0.12/M +local~19GB~93 tok/s160knovita.ai ↗Kwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3-Coder-30B-A3B-Instruct model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
285Gemini 2.0 Flash Thinking Experimental (01-21) Google57±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsBerkeley Function Calling Leaderboard result archive
286Qwen2.5-Max (2025-01-25) Alibaba57±1comparison onlyEpoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)
287Qwen2.5 Coder 32B Instruct Alibaba57±1api $0.74/M +local~21GB131kdevelopers.cloudflare.com ↗EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+5
288Gemini 2.5 Flash-Lite Google57±2api $0.17/M1Mai.google.dev ↗Google DeepMind Gemini model page and model cards (lab-reported scores)Berkeley Function Calling Leaderboard result archiveQwen/Qwen3.5-4B model cardMiðeind Icelandic LLM LeaderboardGalileo Agent Leaderboard (v1 and v2 results)Open Portuguese LLM Leaderboard (archived)+1
289Llama 4 Maverick (402B-A17B) Meta57±1api $0.42/M +local~65GB1Mdocs.aws.amazon.com ↗Epoch AI — internal evaluationsAider polyglot benchmark runsLMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarks+14
290DeepSeek R1 0528 Qwen3 8B DeepSeek57±1api $0.068/M +local~6.5GB~21 tok/s128knovita.ai ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scalingdeepseek-ai/DeepSeek-R1-0528-Qwen3-8B model cardEpoch AI — internal evaluationsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
291Gemini 1.5 Pro Experimental (0827) Google57±1comparison onlyWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAider code editing benchmark runs (old edit leaderboard)
292Granite 4.2 3B IBM57±1api $0.052/M +local~3.3GB131kdeepinfra.com ↗openbmb/MiniCPM5-2B model cardibm-granite/granite-4.2-30b model cardLMArena leaderboard dataset
293Gemini 1.5 Pro (002) Google57±1comparison onlyEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)LMArena leaderboard datasetAider code editing benchmark runs (old edit leaderboard)+5
294Llama 3.3 Nemotron Super 49B v1.5 NVIDIA57±1local~18GB131kllama.cpp › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsLMArena leaderboard dataset
295Nemotron 3 Nano 30B-A3B NVIDIA57±1api $0.087/M +local~20GB~90 tok/s262kdeepinfra.com ↗NVIDIA — Nemotron-Cascade 2 technical reportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)LMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+3
296Athene-V2-Chat 72B Nexusflow57±1local~44GB33kLM Studio › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Arena-Hard-Auto v2.0-Preview leaderboard (repository README)LMArena leaderboard datasetBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)
297Grok 2 (1212) SpaceXAI (xAI)57±1comparison onlyWildEval ZeroEval published resultsEpoch AI — internal evaluations
298Granite 4.1 30B IBM57±1†local~19GB~25 tok/sOllama › ibm-granite/granite-4.1-8b model card
299Nemotron-Cascade-8B-Thinking NVIDIA57±2local~6.3GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsUnsupervised Confidence Calibration for Reasoning LLMs from a Single Generation (arXiv:2604.19444v1)
300Gemini 2.0 Flash Experimental Google57±1comparison onlyAider polyglot benchmark runsAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)Aider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2025-08-01)
301Qwen2.5-Omni 7B Alibaba57±2local~5.8GBllama.cpp › AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
302Sky-T1 32B Preview NovaSky (UC Berkeley)57±4local~21GBLM Studio › EuroEval leaderboards (results collection), English generative leaderboardBerkeley Function Calling Leaderboard result archive
303Ministral 3 14B Reasoning 2512 Mistral57±1local~10GB262kllama.cpp › mistralai/Ministral-3-3B-Reasoning-2512 model cardmistralai/Ministral-3-8B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsJetBrains/Mellum2-12B-A2.5B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboardSurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
304Qwen Plus (2025-01-25) Alibaba57±1api $0.60/M131kwww.alibabacloud.com ↗Epoch AI — internal evaluationsLMArena leaderboard dataset
305DeepSeek-V2.5 (1210) DeepSeek56±1local~82GBllama.cpp › Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpenCompass CompassAcademic leaderboard (full version) Space: committed predictions fileDeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)
306QED-Nano LM Provers56±2local~4.1GB262kllama.cpp › MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets
307Mistral Small 3.2 24B Mistral56±1api $0.11/M +local~15GB128kdeepinfra.com ↗mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardEpoch AI — internal evaluationsLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard resultsmistralai/Mistral-Small-4-119B-2603 model card+4
308Qwen3 4B Thinking 2507 Alibaba56±1local~4.2GB~67 tok/s262kLM Studio › Qwen/Qwen3-4B-Thinking-2507 model cardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasetsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)Qwen/Qwen3.5-0.8B model card+3
309GPT-4 Turbo (2024-04-09) OpenAI56±0api $15.00/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenAI simple-evals README benchmark results (OpenAI's own models)+18
310Yi-Lightning 01.AI56±1api price unknown16kplatform.lingyiwanwu.com ↗Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard dataset
311Gemini 1.5 Pro Experimental (0801) Google56±1comparison onlyWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets
312Qwen3-VL 4B Thinking Alibaba56±1local~4.4GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)
313Qwen2.5 72B Instruct Alibaba56±1api $0.37/M +local~31GB33kdeepinfra.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMicrosoft Research — Phi-4 Technical ReportLiveCodeBench official website results (2024-12-26)+24
314Qwen2.5-VL 72B Instruct Alibaba56±1local~46GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2.5-VL-32B-Instruct model cardLMArena leaderboard datasetChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)+3
315Llama 3.1 405B Instruct Meta56±1api $2.40/M +local~160GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsCohere — Command A technical reportEpoch AI — internal evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+13
316Mistral Large 2 (2411) Mistral56±1local~77GB128kllama.cpp › Epoch AI — internal evaluationsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard results+8
317Qwen2.5 Coder 14B Instruct Alibaba56±1local~11GB~18 tok/s131kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsOpen Portuguese LLM Leaderboard (archived)
318DeepSeek R1 Distill Qwen 32B DeepSeek56±1api $1.59/M +local~21GB~7.3 tok/s131kdevelopers.cloudflare.com ↗open-thoughts/OpenThinker2-32B model cardopen-thoughts/OpenThinker-32B model cardEpoch AI — internal evaluationsdeepseek-ai/DeepSeek-R1 model cardWebApp1K models leaderboard (onekq-ai)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+5
319Qwen2.5 32B Instruct Alibaba56±1local~21GB131kLM Studio › WildEval ZeroEval published resultsEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+11
320Qwen3 8B Alibaba56±1api $0.31/M +local~6.3GB~25 tok/s33kwww.alibabacloud.com ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)allenai/Olmo-3-7B-Think model cardSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+15
321DeepSeek R1 Distill Qwen 14B DeepSeek56±1local~11GB~15 tok/s131kLM Studio › Epoch AI — internal evaluationsdeepseek-ai/DeepSeek-R1 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboardSurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)+2
322Magistral Medium 1.0 (2506) Mistral56±1comparison onlymistralai/Magistral-Small-2509 model cardLMArena leaderboard datasetGalileo Agent Leaderboard (v1 and v2 results)
323Molmo 2 8B Ai256±2local~6.8GBllama.cpp › LMArena leaderboard datasetMolmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)
324Nemotron Nano 9B v2 NVIDIA55±1api $0.10/M +local~7.6GB131kdocs.aws.amazon.com ↗Olmo 3NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)allenai/Olmo-3-7B-Think model cardEuroEval leaderboards (results collection), English generative leaderboard
325EXAONE Deep 32B LG AI Research55±2†local~22GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
326Qwen3 4B Instruct 2507 Alibaba55±1local~4.2GB~107 tok/s262kLM Studio › Qwen/Qwen3-4B-Instruct-2507 model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)+4
327Ministral 3 14B Mistral55±1api $0.20/M +local~9.1GB~46 tok/s262kdocs.mistral.ai ↗JetBrains/Mellum2-12B-A2.5B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)mistralai/Ministral-3-8B-Instruct-2512 model card+2
328Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba55±0api $0.35/M +local~18GB33kwww.alibabacloud.com ↗Qwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardEpoch AI — internal evaluationsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)+10
329Nova Premier Amazon55±2†api $5.00/M1Maws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)MASSIVE-Agents — Kulkarni et al. (2025)
330Granite 4.1 8B IBM55±1api $0.11/M +local~7.1GB~70 tok/s131kreplicate.com ↗ibm-granite/granite-4.1-8b model cardLMArena leaderboard datasetSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)
331Magistral Small 1.0 (2506) Mistral55±1local~16GB128kLM Studio › mistralai/Magistral-Small-2509 model cardEpoch AI — internal evaluationsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Galileo Agent Leaderboard (v1 and v2 results)
332Bielik 11B v3.0 Instruct SpeakLeash55±2local~8.1GBLM Studio › Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
333Qwen2.5-Math 72B Instruct Alibaba55±2†local~44GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
334Llama 3.3 70B Instruct Meta55±1api $0.72/M +local~25GB~13 tok/s128kdocs.aws.amazon.com ↗Cohere — Command A technical reportEpoch AI — internal evaluations2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMicrosoft Research — Phi-4 Technical ReportMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLiveCodeBench official website results (2024-12-26)+21
335Magistral Medium 1.1 (2507) Mistral55±1†comparison only40kmistralai/Magistral-Small-2509 model card
336Qwen3-VL 8B Instruct Alibaba55±1api $0.31/M +local~6.9GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Instruct model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
337Mistral Large 2 (2407) Mistral55±1local~45GB128kllama.cpp › WildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsJamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+10
338Falcon-H1 34B Instruct TII55±3local~23GB262kllama.cpp › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
339Llama 4 Scout (109B-A17B) Meta55±1api $0.15/M +local~36GB~19 tok/s328kdeepinfra.com ↗Epoch AI — internal evaluationsLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard resultsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+8
340GPT-4 Turbo Preview (1106) OpenAI55±1comparison onlyEvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)LiveCodeBench official website results (2024-12-26)+5
341Llama 3.1 Tulu 3 70B Ai255±1local~43GB131kLM Studio › Tülu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., Ai2)LMArena leaderboard dataset
342Bielik 11B v2.6 Instruct SpeakLeash55±3local~8.1GBLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
343GigaChat 3.5 Ultra Instruct (432B-A28B) Sber55±1†local~457GB262khow to run › ai-sage/GigaChat3.5-432B-A28B-Reasoning model card
344Mistral Small 3.1 24B (2503) Mistral55±1local~16GB128kllama.cpp › Epoch AI — internal evaluationsmistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)EQ-Bench 3 canonical leaderboard results+7
345Pixtral Large (2411) Mistral55±3local~51GBllama.cpp › LMArena leaderboard datasetMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)
346Olmo 3.1 32B Instruct Ai255±2local~21GB66kLM Studio › Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
347Spark-X2.5 1.7B iFLYTEK (SparkLLM)55±1†local~1.9GB~166 tok/s1Mllama.cpp › Spark-X2.6 model cards (thinking high)XHToken/Spark-X2.5-4B model card
348GPT-4 Turbo Preview (0125) OpenAI55±1comparison onlyTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider code editing benchmark runs (old edit leaderboard)+3
349DeepSeek-V2.5 DeepSeek55±1local~151GB164kOllama › WildEval ZeroEval published resultsAider code editing benchmark runs (old edit leaderboard)MMLU-Pro author-released evaluation runsWebApp1K models leaderboard (onekq-ai)LMArena leaderboard datasetOpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file+1
350OpenThinker2 32B Open Thoughts55±1local~21GB33kLM Studio › open-thoughts/OpenThinker2-32B model cardArena-Hard-Auto v2.0-Preview leaderboard (repository README)
351Command A Reasoning (08-2025) Cohere55±2local~34GBllama.cpp › Cohere vendor-reported benchmark scores for its own models (cohere.com blog)Berkeley Function Calling Leaderboard result archive
352DeepSeek-V2 Chat (0628) DeepSeek55±1local~66GBllama.cpp › WildEval ZeroEval published resultsAider code editing benchmark runs (old edit leaderboard)WildBench v2 leaderboard data (Ai2)
353OpenThinker 32B Open Thoughts55±1†local~21GB33kLM Studio › open-thoughts/OpenThinker2-32B model cardopen-thoughts/OpenThinker-32B model card
354Claude 3.5 Haiku Anthropic54±1comparison onlyWildEval ZeroEval published resultsEpoch AI — internal evaluationsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetMMLU-Pro author-released evaluation runsAider polyglot benchmark runs+9
355Magistral Small 1.1 (2507) Mistral54±1†local~16GB40kLM Studio › mistralai/Magistral-Small-2509 model card
356Ministral 3 8B Reasoning 2512 Mistral54±1local~6.5GB262kLM Studio › mistralai/Ministral-3-3B-Reasoning-2512 model cardmistralai/Ministral-3-8B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsSurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
357Llama 3.1 Tulu 3 70B DPO Ai254±1local~22GB131kllama.cpp › Epoch AI — internal evaluationsTülu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., Ai2)
358Gemini 1.5 Flash Experimental (0827) Google54±1comparison onlyWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAider code editing benchmark runs (old edit leaderboard)
359Mellum2 12B-A2.5B Thinking JetBrains54±1†local~8.6GB131kLM Studio › JetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Thinking model card
360Nemotron 3 Nano 4B NVIDIA54±1local~2.9GB~52 tok/s262kLM Studio › openbmb/MiniCPM5-2B model cardNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
361GPT-4o mini (2024-07-18) OpenAI54±0api $0.26/M128kdevelopers.openai.com ↗WildEval ZeroEval published resultsmicrosoft/Phi-4-mini-instruct model cardEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsVector Institute Evaluation Leaderboard (Inspect Evals runs)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+32
362Gemini 1.5 Pro (001) Google54±1comparison onlyWildEval ZeroEval published resultsEpoch AI — internal evaluationsAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)LMArena leaderboard datasetCharXiv leaderboard (princeton-nlp/CharXiv README)VHELM: A Holistic Evaluation of Vision Language Models (Stanford CRFM), arXiv 2410.07112v2+3
363DeepSeek-Coder-V2 Instruct (0614) DeepSeek54±1local~92GBllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LiveCodeBench official website results (2024-12-26)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+2
364DeepSeek-Coder-V2 Instruct (0724) DeepSeek54±1local~151GBllama.cpp › WildEval ZeroEval published resultsAider code editing benchmark runs (old edit leaderboard)Berkeley Function Calling Leaderboard result archive
365Mistral Small 3 24B (2501) Mistral54±1local~15GB32kLM Studio › Epoch AI — internal evaluationsInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EQ-Bench 3 canonical leaderboard resultsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+6
366GPT-4 (0314) OpenAI54±1comparison onlyWildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Aider code editing benchmark runs (old edit leaderboard)Epoch AI — internal evaluationsIt Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset — authors' own runs+21
367Gemini 1.5 Flash (002) Google54±1comparison onlyEvalPlus / EvalPerf published evaluation runsEpoch AI — internal evaluationsLiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)LMArena leaderboard dataset+5
368Qwen Turbo (2024-11-01) Alibaba54±2comparison onlyEpoch AI — internal evaluationsOpen Portuguese LLM Leaderboard (archived)
369GLM-4 9B Chat 1M Zhipu AI (THUDM)54±4local~8.1GB1Mllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
370EXAONE Deep 7.8B LG AI Research54±2†local~6.5GB33kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
371Claude 3 Opus Anthropic54±0comparison onlyWildEval ZeroEval published resultsEvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluations+15
372Qwen2-VL 72B Instruct Alibaba54±1local~34GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2.5-VL-32B-Instruct model cardPixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column
373Hermes 3 Llama 3.1 405B Nous Research54±1local~114GB131kllama.cpp › Nous Research vendor-reported scores for its own Hermes models (Hermes 3 Technical Report)Aider code editing benchmark runs (old edit leaderboard)
374Yi-Large Preview 01.AI54±1comparison onlyWildEval ZeroEval published resultsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)
375Llama 3.1 70B Instruct Meta53±0api $0.72/M +local~27GB128kdocs.aws.amazon.com ↗WildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+22
376Gemma 3 27B Google53±1api $0.10/M +local~19GB131kdeepinfra.com ↗Gemma Team — Gemma 4 Technical ReportGoogle DeepMind Gemma 3 270M IT model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEpoch AI — internal evaluationsMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard dataset+23
377Qwen2.5-VL 32B Instruct Alibaba53±2local~23GB33kLM Studio › Qwen2.5-VL-32B-Instruct model cardApriel-1.5-15b-Thinker (arXiv:2510.01141v1), Table 4 multimodal benchmarksLMArena leaderboard datasetChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)Open Portuguese LLM Leaderboard (archived)+1
378Phi-4 14B Microsoft53±1api $0.087/M +local~10GB16kdeepinfra.com ↗Epoch AI — internal evaluationsMicrosoft Research — Phi-4 Technical ReportDaekeun Kim Korean LLM benchmark runsmicrosoft/phi-4 model cardMicrosoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model card+11
379GPT-4.1 nano OpenAI53±1api $0.17/M1Mdevelopers.openai.com ↗OpenAI simple-evals README benchmark results (OpenAI's own models)Epoch AI — internal evaluationsDaekeun Kim Korean LLM benchmark runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetAider polyglot benchmark runs+11
380GLM-4 (0520) Z.ai53±1comparison onlyChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)
381Bielik 11B v2.1 Instruct SpeakLeash53±3local~8.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
382Qwen2.5-Math 7B Instruct Alibaba53±2†local~5.4GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
383s1.1 32B Simple Scaling53±1local~22GB33kLM Studio › open-thoughts/OpenThinker2-32B model cardopen-thoughts/OpenThinker-32B model cardArena-Hard-Auto v2.0-Preview leaderboard (repository README)
384Ministral 3 8B Mistral53±1api $0.15/M +local~6.5GB~68 tok/s262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboardmistralai/Ministral-3-14B-Instruct-2512 model cardMulti-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe (arXiv:2609.05395v1)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)+3
385Olmo 3 7B Think Ai253±1local~5.8GB66kLM Studio › Olmo 3Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsallenai/Olmo-3-7B-Think model cardJetBrains/Mellum2-12B-A2.5B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
386Gemma 4 E4B Google53±1local~6.4GB~21 tok/s131kLM Studio › openbmb/MiniCPM5-2B model cardGemma Team — Gemma 4 Technical ReportJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardXHToken/Spark-X2.5-4B model cardLiquidAI/LFM2.5-2.6B model cardEuroEval leaderboards (results collection), English generative leaderboard+8
387Llama 3.1 Nemotron 70B Instruct NVIDIA53±1local~43GB128kLM Studio › Tülu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., Ai2)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetArena-Hard-Auto v2.0-Preview leaderboard (repository README)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
388Qwen3-VL 4B Instruct Alibaba53±1local~4.4GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
389GPT-4 (0613) OpenAI53±1api $37.50/M8kdevelopers.openai.com ↗Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsTulu 2 — base and instruction-tuned evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+11
390Bielik 11B v2.2 Instruct SpeakLeash53±3local~8.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
391Llama 3.1 Nemotron 51B Instruct NVIDIA53±2local~34GBllama.cpp › Puzzle: Distillation-Based NAS for Inference-Optimized LLMs (arXiv:2411.19146v5)LMArena leaderboard dataset
392Nemotron-4 340B Instruct NVIDIA53±2comparison onlyWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Portuguese LLM Leaderboard (archived)
393ERNIE 4.5 21B-A3B Baidu53±1comparison onlyERNIE 4.5 post-trained 300B-A47B benchmark chartEuroEval leaderboards (results collection), English generative leaderboardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
394Palmyra X4 (palmyra-x-004) Writer53±3comparison onlyBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
395LFM2.5 2.6B Liquid AI53±1local~2.3GB131kLM Studio › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
396Granite 4.0 H Small (32B-A9B) IBM53±1api $0.11/M +local~21GB131kreplicate.com ↗ibm-granite/granite-4.0-h-small model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboardDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)+1
397Qwen2.5 Coder 7B Instruct Alibaba53±1local~5.4GB~3.8 tok/s131kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
398Aya Expanse 32B Cohere53±2api $0.75/M +local~22GB128kdocs.cohere.com ↗LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning CapabilitiesOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)+1
399OpenCoder-8B-Instruct INF (infly)53±1local~6.4GBllama.cpp › EvalPlus / EvalPerf published evaluation runsEuroEval leaderboards (results collection), English generative leaderboard
400Gemma 3 12B Google52±1api $0.075/M +local~10GB~13 tok/s131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportEpoch AI — internal evaluationsLMArena leaderboard dataset+10
401Qwen2.5 14B Instruct Alibaba52±1local~11GB131kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsMicrosoft Research — Phi-4 Technical Reportmicrosoft/phi-4 model cardCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Reinforcing Real-world Service Agents: InteractCS-RL (arXiv:2602.22697v1)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+7
402LFM2.5 8B-A1B Liquid AI52±1local~5.5GB~174 tok/s128kLM Studio › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-8B-A1B model card
403Yi-Large 01.AI52±1comparison onlyWildEval ZeroEval published resultsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)
404Llama-3-Instruct-8B-SimPO-ExPO Chujie Zheng (ExPO)52±1local~6.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
405Cogito v1 Preview Llama 8B Deep Cogito52±4local~6.2GB128kLM Studio › EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
406Llama-3-Instruct-8B-SimPO Princeton NLP52±1local~6.6GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
407Athene-70B Nexusflow52±1local~29GBllama.cpp › WildEval ZeroEval published resultsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)LMArena leaderboard dataset
408Jamba 1.5 Large AI21 Labs52±1local~839GBhow to run › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)MMLU-Pro author-released evaluation runsLMArena leaderboard dataset
409Qwen2 72B Instruct Alibaba52±1local~33GB131kllama.cpp › WildEval ZeroEval published resultsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)+14
410CodeQwen1.5 7B Chat Alibaba52±1local~6.4GB64kllama.cpp › EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
411Command A Translate (08-2025) Cohere52±4†api price unknown8kdocs.cohere.com ↗Cohere — North Small Translate (arXiv:2609.13916v1)
412OpenCodeInterpreter-DS-33B M-A-P51±1local~22GBLM Studio › EvalPlus / EvalPerf published evaluation runs
413Flan-U-PaLM 540B Google51±1†never releasedScaling Instruction-Finetuned Language Models — authors' own runs
414Gemini 1.5 Flash (001) Google51±1comparison onlyWildEval ZeroEval published resultsEpoch AI — internal evaluationsLMArena leaderboard datasetAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)VHELM: A Holistic Evaluation of Vision Language Models (Stanford CRFM), arXiv 2410.07112v2CharXiv leaderboard (princeton-nlp/CharXiv README)+3
415Bielik 11B v2.0 Instruct SpeakLeash51±3local~8.5GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
416LFM2 24B-A2B Liquid AI51±1local~14GB33kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
417Qwen3 4B Alibaba51±1local~4GB33kLM Studio › Qwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-4B-Instruct-2507 model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationHuggingFaceTB/SmolLM3-3B model card (Evaluation section)Epoch AI — internal evaluationsArena-Hard-Auto v2.0-Preview leaderboard (repository README)+8
418DeepSeek-Coder 33B Instruct DeepSeek51±1local~21GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)CRUXEval published evaluation results (authors' generations and scores)+1
419GLM-4 (0116) Z.ai51±1comparison onlyChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)
420Codestral 22B v0.1 Mistral AI51±1local~15GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsOpen Portuguese LLM Leaderboard (archived)
421Gemma 2 27B IT Google51±1local~19GB8kLM Studio › WildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)+10
422Llama 3 70B Instruct Meta51±0local~24GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsEpoch AI — internal evaluations+17
423Llama 3.2 90B Vision Instruct Meta51±1local~58GB128kOllama › Epoch AI — internal evaluationsPixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Open Portuguese LLM Leaderboard (archived)+1
424WizardCoder 33B V1.1 WizardLM51±1local~22GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsWizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)
425Bielik 11B v2.3 Instruct SpeakLeash51±2local~8.6GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Berkeley Function Calling Leaderboard result archiveSpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
426Flan-PaLM 540B Google51±1†never releasedScaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Encode Clinical Knowledge — authors' own runs
427Llama-3-8B-ProLong-512k-Instruct Princeton NLP51±4local~35GBhow to run › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
428Nova Pro Amazon51±1api $1.40/M300kdocs.aws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)Aider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
429OpenCodeInterpreter-DS-6.7B M-A-P51±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
430Qwen2-Math 72B Instruct Alibaba51±1local~32GB4kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
431WizardLM-2 8x22B Microsoft51±1local~91GBOllama › Epoch AI — internal evaluationsAider code editing benchmark runs (old edit leaderboard)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardJudgemark v2 results (EQ-Bench)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
432Mistral Small Instruct 22B (2409) Mistral AI51±2local~15GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SpeakLeash Polish EQ-Bench leaderboardEvalPlus / EvalPerf published evaluation runsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Galileo Agent Leaderboard (v1 and v2 results)
433OpenThinker3 7B Open Thoughts51±1local~6GB33kLM Studio › Olmo 3allenai/Olmo-3-7B-Think model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)
434Granite 4.1 3B IBM51±1†local~3.7GB~131 tok/sOllama › ibm-granite/granite-4.1-8b model card
435Artigenz-Coder-DS-6.7B Artigenz51±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
436Mellum2 12B-A2.5B Instruct JetBrains51±2†local~7.6GB131kLM Studio › JetBrains/Mellum2-12B-A2.5B-Instruct model card
437Molmo 72B (0924) Ai250±2local~309GBhow to run › Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetAi2 Molmo-72B-0924 model card: Human Preference Elo Rating column
438Yi 34B 200K (base) 01.AI50±4local~23GB200kllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
439Hermes 3 Llama 3.1 70B Nous Research50±1local~43GB131kOllama › Nous Research vendor-reported scores for its own Hermes models (Hermes 3 Technical Report)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets
440speechless-codellama-34b-v2.0 uukuguy50±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
441OLMo 2 32B Instruct (0325) Ai250±1local~21GBLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLMArena leaderboard datasetallenai/Olmo-3.1-32B-Think model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
442Ministral 3 3B Reasoning 2512 Mistral50±1local~3.4GB262kLM Studio › mistralai/Ministral-3-3B-Reasoning-2512 model cardmistralai/Ministral-3-8B-Reasoning-2512 model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsEuroEval leaderboards (results collection), English generative leaderboard
443Aya 23 35B Cohere For AI50±2local~22GB8kLM Studio › SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
444AceReason-Nemotron-1.0-14B NVIDIA50±2†local~10GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
445Claude 3 Sonnet Anthropic50±0comparison onlyEvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+12
446Qwen2.5-VL 7B Instruct Alibaba50±1local~5.8GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
447Mixtral 8x22B Instruct v0.1 Mistral50±1local~56GB64kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LMArena leaderboard dataset+9
448Magicoder-S-DS 6.7B UIUC (ise-uiuc)50±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
449EXAONE Deep 2.4B LG AI Research50±2†local~3.2GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
450Pixtral 12B (2409) Mistral50±2local~9.6GBllama.cpp › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetVisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column
451Nova Lite Amazon50±2api $0.10/M300kdocs.aws.amazon.com ↗LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
452XwinCoder-34B Xwin-LM50±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
453Qwen2.5 7B Instruct Alibaba50±1api $0.30/M +local~5.4GB~54 tok/s131kwww.together.ai ↗microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveCodeBench official website results (2024-12-26)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Epoch AI — internal evaluations+12
454LFM2.5 1.2B Thinking Liquid AI50±2local~1.3GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
455Yi-Coder 9B Chat 01.AI50±1local~6.3GB128kLM Studio › Aider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EvalPlus / EvalPerf published evaluation runsOpen Portuguese LLM Leaderboard (archived)
456InternLM2.5 7B Chat Shanghai AI Laboratory (InternLM)50±2local~6.1GBLM Studio › InternLM GitHub repository model cards (Shanghai AI Laboratory)Open Portuguese LLM Leaderboard (archived)
457StarChat2 15B v0.1 Hugging Face50±1local~12GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runs
458Code Millenials 34B Bud Ecosystem50±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
459Qwen2-Math 7B Instruct Alibaba50±2†local~6GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
460WaveCoder-Ultra-6.7B Microsoft50±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
461Reka Core (2024-05-01) Reka AI50±1comparison onlyWildEval ZeroEval published resultsWildBench v2 leaderboard data (Ai2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)
462GPT-4o Audio Preview (2024-12-17) OpenAI49±2comparison onlyAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
463MegaBeam-Mistral-7B-512k AWS (aws-prototyping)49±4local~5.9GBllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
464Gemini 1.5 Flash-8B (001) Google49±2comparison onlyLMArena leaderboard datasetEpoch AI — internal evaluationsOpen Portuguese LLM Leaderboard (archived)
465Phi-3.5-MoE-instruct Microsoft49±1local~25GB128kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsDaekeun Kim Korean LLM benchmark runsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
466Qwen3-VL 2B Thinking Alibaba49±1local~2.7GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
467Gemma 3n E4B Google49±1local~5.6GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model cardLMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboard
468Claude 3 Haiku Anthropic49±0comparison onlyEvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+16
469DeepSeek-Coder 7B Instruct v1.5 DeepSeek49±1local~5.9GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
470OpenThinker2 7B Open Thoughts49±2†local~6.4GB33kllama.cpp › open-thoughts/OpenThinker2-7B model card
471DeepSeek-Coder 6.7B Instruct DeepSeek49±1local~8.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)
472Qwen2 57B-A14B Instruct Alibaba49±2local~35GB33kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
473Hermes 2 Theta Llama-3 70B Nous Research49±2local~47GB8kllama.cpp › Epoch AI — internal evaluations
474MiniCPM3-4B OpenBMB49±2local~4GB32kLM Studio › LiveCodeBench official website results (2024-12-26)Berkeley Function Calling Leaderboard result archive
475Granite 4.0 H Tiny (7B-A1B) IBM49±1local~4.5GB~12 tok/s131kLM Studio › ibm-granite/granite-4.0-h-small model cardEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
476Claude 1.3 Anthropic49±1comparison onlyAnthropic system cards and model announcements (lab-reported scores)Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance — authors' own runsLLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsclembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents — authors' own runsSummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization — authors' own runs+1
477Yi-1.5 34B (base) 01.AI49±2local~22GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
478Bielik 4.5B v3.0 Instruct SpeakLeash49±2local~4.3GBLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SpeakLeash Polish EQ-Bench leaderboardEuroEval leaderboards (results collection), English generative leaderboard
479Qwen2.5 Coder 3B Instruct Alibaba49±1local~2.6GB~9.5 tok/s33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
480GPT-4o mini Audio Preview (2024-12-17) OpenAI49±2comparison onlyAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
481Gemma 2 9B IT Google49±1local~8.9GB8kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsInternLM GitHub repository model cards (Shanghai AI Laboratory)Epoch AI — internal evaluations+14
482WhiteRabbitNeo-33B-v1 WhiteRabbitNeo49±1local~22GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
483Qwen1.5 110B Chat Alibaba48±1local~45GB32kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Aider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+1
484Command R Plus (08-2024) Cohere48±1api $4.38/M +local~63GB128kdocs.cohere.com ↗Cohere — Command A technical reportLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+1
485GLM-4 9B Chat Zhipu AI (Z.ai)48±2local~7.1GB128kOllama › WildBench v2 leaderboard data (Ai2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardBerkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)+1
486Magicoder-S-CL-7B UIUC (ise-uiuc)48±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
487speechless-coder-ds-6.7b uukuguy48±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
488Phind CodeLlama 34B v2 Phind48±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
489AceReason-Nemotron-1.1-7B NVIDIA48±2†local~6GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
490Jamba 1.5 Mini AI21 Labs48±1local~34GBllama.cpp › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)LMArena leaderboard datasetNVIDIA RULER repository README results tableVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
491DeepSeek R1 Distill Qwen 7B DeepSeek48±1local~5.4GB~35 tok/s131kLM Studio › open-thoughts/OpenThinker2-7B model carddeepseek-ai/DeepSeek-R1 model cardallenai/Olmo-3-7B-Think model cardmicrosoft/Phi-4-mini-reasoning model cardMicrosoft — Phi-4-Mini Technical ReportCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+7
492U-PaLM 540B Google48±1†never releasedTranscending Scaling Laws with 0.1% Extra Compute — maker's own resultsScaling Instruction-Finetuned Language Models — authors' own runs
493GPT-3.5 Turbo (0125) OpenAI48±1api $0.75/M16kdevelopers.openai.com ↗WildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+11
494Mistral Large (2402) Mistral48±1comparison onlyEvalPlus / EvalPerf published evaluation runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+3
495InternVL2-26B Shanghai AI Laboratory (OpenGVLab)48±2local~55GBhow to run › CharXiv leaderboard (princeton-nlp/CharXiv README)OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tablesLMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
496Aya Expanse 8B Cohere48±2local~6.4GB8kLM Studio › LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
497EuroLLM 9B Instruct UTTER (EuroLLM)48±2local~6.9GB4kLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
498Flan-UL2 20B Google48±4comparison onlyZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsFANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions — authors' own runs
499GPT-3.5 Turbo (1106) OpenAI48±1comparison onlyEvalPlus / EvalPerf published evaluation runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Epoch AI — internal evaluationsLMArena leaderboard dataset+2
500Granite 4.0 H Micro 3B IBM48±1local~3.4GB131kLM Studio › ibm-granite/granite-4.0-h-small model cardEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
501Minerva 540B Google48±1†never releasedSolving Quantitative Reasoning Problems with Language Models — authors' own runsTranscending Scaling Laws with 0.1% Extra Compute — maker's own results
502Code Llama 70B Instruct Meta48±1local~28GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LMArena leaderboard dataset+1
503Command R (08-2024) Cohere48±1api $0.26/M +local~21GB128kdocs.cohere.com ↗LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsCohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)NVIDIA RULER repository README results table+3
504OLMo 2 13B Instruct (1124) Allen Institute for AI48±1local~9.6GB4kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
505InternLM2.5 7B Chat 1M Shanghai AI Laboratory (InternLM)48±4local~6.4GBllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
506Falcon-H1 7B Instruct TII48±3local~6GB262kLM Studio › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
507GPT-3.5 Turbo (0301) OpenAI47±1comparison onlyEvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Tulu 2 — base and instruction-tuned evaluationsIs ChatGPT a General-Purpose Natural Language Processing Task Solver? — authors' own runsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)It Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset — authors' own runs+24
508Llama 3 70B (base) Meta47±2local~43GBLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
509Tulu 2 70B Allen Institute for AI47±2†local~28GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
510CogVLM2 Llama3 Chinese Chat 19B Zhipu AI (THUDM)47±7†comparison onlyZhipu AI (THUDM) CogVLM2 model card, Benchmark table (own models)
511Hermes 2 Pro Llama 3 8B Nous Research47±3local~6.6GB8kllama.cpp › SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)Berkeley Function Calling Leaderboard result archive
512MedGemma 4B IT Google47±2local~4.1GBllama.cpp › Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Portuguese LLM Leaderboard (archived)SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
513Nova Micro Amazon47±2api $0.061/M128kdocs.aws.amazon.com ↗LMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveMASSIVE-Agents — Kulkarni et al. (2025)Galileo Agent Leaderboard (v1 and v2 results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
514Qwen3.5 2B Alibaba47±1local~2.2GB~137 tok/s262kllama.cpp › Qwen/Qwen3.5-2B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)EuroEval leaderboards (results collection), English generative leaderboardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets+7
515Claude 2.0 Anthropic47±1comparison onlyEvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Anthropic system cards and model announcements (lab-reported scores)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Epoch AI — internal evaluationsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+2
516Yi 1.5 34B Chat 01.AI47±1local~22GB4kLM Studio › WildEval ZeroEval published resultsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Epoch AI — internal evaluationsWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+5
517OpenChat 3.5 0106 OpenChat47±1local~6.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboard+1
518GPT-3.5 Turbo (0613) OpenAI47±1comparison onlyAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Tulu 2 — base and instruction-tuned evaluationsDaekeun Kim Korean LLM benchmark runsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider code editing benchmark runs (old edit leaderboard)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+5
519WizardCoder Python 34B V1.0 WizardLM47±1local~22GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsWizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)CRUXEval published evaluation results (authors' generations and scores)
520Codex code-davinci-002 (GPT-3.5 base) OpenAI47±0comparison onlyChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsSelf-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runsFaithful Chain-of-Thought Reasoning — authors' own runsLarge Language Models are Better Reasoners with Self-Verification — authors' own runsBatch Prompting: Efficient Inference with Large Language Model APIs — authors' own runs+20
521Granite 4.0 Micro 3B IBM47±1local~3.5GB~6.2 tok/s131kLM Studio › ibm-granite/granite-4.0-h-small model cardEpoch AI — internal evaluationsEuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
522Gemma 4 E2B Google47±1api $0.050/M +local~4.5GB131kdocs.aws.amazon.com ↗openbmb/MiniCPM5-2B model cardGemma Team — Gemma 4 Technical ReportXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)LiquidAI/LFM2.5-2.6B model cardEuroEval leaderboards (results collection), English generative leaderboard+6
523Gemini 1.0 Pro (001) Google47±2comparison onlyEpoch AI — internal evaluationsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Portuguese LLM Leaderboard (archived)
524DBRX Instruct Databricks47±1local~85GB32kOllama › EvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+3
525Mistral Medium (2312) Mistral47±1comparison onlyAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Berkeley Function Calling Leaderboard result archive
526Granite 34B Code Instruct (8K) IBM47±1local~22GBOllama › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboard
527Cogito v1 Preview Llama 3B Deep Cogito47±4local~3.4GB128kLM Studio › EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
528Qwen2.5-Math 1.5B Instruct Alibaba47±2†local~1.8GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
529EXAONE 4.0 1.2B LG AI Research47±1†local~2.2GB66kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
530InternLM2.5 20B Chat Shanghai AI Laboratory (InternLM)47±2local~14GB33kllama.cpp › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
531StarCoder2-15B-Instruct-v0.1 BigCode47±1local~11GB16kOllama › EvalPlus / EvalPerf published evaluation runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
532Flan-cont-PaLM 62B Google47±1†never releasedScaling Instruction-Finetuned Language Models — authors' own runs
533Qwen1.5 72B Chat Alibaba47±1local~31GB32kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+9
534speechless-starcoder2-15b uukuguy47±1local~12GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
535OpenChat 3.5 0106 Gemma OpenChat47±2local~6.9GB8kllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
536Granite 3.3 8B Instruct IBM47±1local~6.3GB128kLM Studio › ibm-granite/granite-3.3-8b-instruct model cardallenai/Olmo-3-7B-Instruct model cardDiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
537Phi-3 Medium 128K Instruct Microsoft47±1local~10GB128kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
538Qwen2-VL 7B Instruct Alibaba47±1local~5.8GBLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating columnVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
539PaLM 540B Google47±0†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsPaLM: Scaling Language Modeling with Pathways — authors' own runsTranscending Scaling Laws with 0.1% Extra Compute — maker's own resultsSelf-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runsSolving Quantitative Reasoning Problems with Language Models — authors' own runs+5
540Claude 2.1 Anthropic47±1comparison onlyTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Epoch AI — internal evaluationsLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
541Zephyr ORPO 141B-A35B v0.1 Hugging Face47±2local~56GB66kllama.cpp › LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)HuggingFaceH4 zephyr-orpo-141b-A35b-v0.1 model card
542LFM2 8B-A1B Liquid AI47±1comparison onlyMarco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingLFM2 Technical Report — Liquid AI own SmolLM3 evaluationLiquidAI/LFM2.5-8B-A1B model card
543DeepSeek R1 Distill Llama 8B DeepSeek46±1local~6.2GB131kLM Studio › deepseek-ai/DeepSeek-R1 model cardmicrosoft/Phi-4-mini-reasoning model cardMicrosoft — Phi-4-Mini Technical ReportCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation (arXiv:2604.19444v1)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+2
544Phi-3-medium-4k-instruct Microsoft46±1local~10GB4kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)EvalPlus / EvalPerf published evaluation runsSpeakLeash Polish EQ-Bench leaderboard+1
545QwQ-32B-Preview Alibaba46±2local~21GB33kLM Studio › LMArena leaderboard datasetEvalPlus / EvalPerf published evaluation runsBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)Berkeley Function Calling Leaderboard result archive+2
546Mistral Small (2402) Mistral46±1comparison onlyLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Berkeley Function Calling Leaderboard result archive
547Minerva 62B Google46±2†never releasedSolving Quantitative Reasoning Problems with Language Models — authors' own runs
548Phi-4-mini-reasoning 3.8B Microsoft46±2api $0.13/M +local~4GB128kazure.microsoft.com ↗microsoft/Phi-4-mini-reasoning model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)EuroEval leaderboards (results collection), English generative leaderboard
549DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) DeepSeek46±1local~9.6GB128kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EvalPlus / EvalPerf published evaluation runsDSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation — authors' own model runsWebApp1K models leaderboard (onekq-ai)+1
550Qwen2-Math 1.5B Instruct Alibaba46±2†local~2.4GB4kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
551Llama 3.1 8B Instruct Meta46±0api $0.22/M +local~6.2GB~47 tok/s128kdocs.aws.amazon.com ↗EvalPlus / EvalPerf published evaluation runsmicrosoft/Phi-4-mini-instruct model cardIBM Granite 3.0 Language Models technical report (GitHub)WildEval ZeroEval published results2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets+25
552Ministral 3 3B Mistral46±1api $0.10/M +local~3.4GB~128 tok/s262kdocs.mistral.ai ↗Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsmistralai/Ministral-3-14B-Instruct-2512 model cardmistralai/Ministral-3-8B-Instruct-2512 model cardEuroEval leaderboards (results collection), English generative leaderboard+2
553Olmo 3 7B Instruct Ai246±1local~5.8GB66kLM Studio › Olmo 3JetBrains/Mellum2-12B-A2.5B-Instruct model cardMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsallenai/Olmo-3-7B-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
554GPT-3.5 text-davinci-003 OpenAI46±1comparison onlyScaling Instruction-Finetuned Language Models — authors' own runsFaithful Chain-of-Thought Reasoning — authors' own runsIs ChatGPT a General-Purpose Natural Language Processing Task Solver? — authors' own runsINFORM : Information eNtropy based multi-step reasoning FOR large language Models — authors' own runsBatch Prompting: Efficient Inference with Large Language Model APIs — authors' own runsUnnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text — authors' own runs+34
555Command R Plus (04-2024) Cohere46±1local~43GB128kllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsVector Institute Evaluation Leaderboard (Inspect Evals runs)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Aider code editing benchmark runs (old edit leaderboard)+6
556Mistral NeMo Instruct (2407) Mistral46±1local~8.8GB128kLM Studio › WildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEpoch AI — internal evaluationsWildBench v2 leaderboard data (Ai2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+9
557GPT-4o Audio Preview (2024-10-01) OpenAI46±2comparison onlyAHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
558Llama 3.1 Tulu 3 8B Ai246±1local~6.2GB131kLM Studio › microsoft/Phi-4-mini-instruct model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
559Phi-3-mini-4k-instruct (June 2024 update) Microsoft46±1local~5.7GBLM Studio › WildEval ZeroEval published resultsLMArena leaderboard datasetEuroEval leaderboards (results collection), English generative leaderboard
560Yi 1.5 9B Chat 01.AI46±1local~6.8GB4kLM Studio › WildEval ZeroEval published resultsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4WildBench v2 leaderboard data (Ai2)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)+1
561Qwen1.5 32B Chat Alibaba46±1local~22GB32kllama.cpp › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Epoch AI — internal evaluationsSpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
562Xwin-LM 70B v0.1 Xwin-LM46±1local~28GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)
563Qwen2 7B Instruct Alibaba46±1local~5.4GB131kLM Studio › WildEval ZeroEval published resultsInternLM GitHub repository model cards (Shanghai AI Laboratory)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+4
564Command R7B (12-2024) Cohere46±1local~6.2GB128kLM Studio › Cohere — Command A technical reportInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)
565Eurus-2-7B-PRIME PRIME-RL45±2local~6.4GB4kllama.cpp › PRIME-RL Eurus-2-7B-PRIME model card, Evaluation table (own models)Epoch AI — internal evaluations
566Granite 20B Code Instruct (8K) IBM45±1local~14GBOllama › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
567Dolphin 2.6 Mixtral 8x7B Cognitive Computations45±1local~29GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
568Gemma 3 4B Google45±1api $0.063/M +local~3.8GB~74 tok/s131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardLFM2 Technical Report — Liquid AI own SmolLM3 evaluationMedmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboardLMArena leaderboard dataset+11
569Granite 3.0 8B Instruct IBM45±1local~6.6GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
570LLaVA-OneVision Qwen2 72B (ov-sft) LLaVA (lmms-lab)45±3local~155GBhow to run › LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (arXiv:2505.20740v3)
571Granite 8B Code Instruct (4K) IBM45±1local~6.6GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
572Falcon3 7B Instruct TII45±2local~5.9GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)
573Phi-4-mini 3.8B Microsoft45±1api $0.13/M +local~4GB128kazure.microsoft.com ↗microsoft/Phi-4-mini-instruct model cardDaekeun Kim Korean LLM benchmark runsMicrosoft — Phi-4-Mini Technical ReportSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+4
574Ministral 8B (2410) Mistral45±1local~6.2GB128kLM Studio › microsoft/Phi-4-mini-instruct model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsEpoch AI — internal evaluationsMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)LMArena leaderboard datasetBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+6
575Code Llama 70B Python Meta45±1local~28GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runs
576Code Tulu 2 34B Allen Institute for AI45±2†local~72GBhow to run › Tulu 2 — base and instruction-tuned evaluations
577Granite 3.2 8B Instruct IBM45±1local~6.6GB131kllama.cpp › ibm-granite/granite-3.3-8b-instruct model cardBerkeley Function Calling Leaderboard result archiveVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
578OpenThinker 7B Open Thoughts45±2†local~6GB33kLM Studio › open-thoughts/OpenThinker2-7B model cardopen-thoughts/OpenThinker-7B model card
579Falcon3 10B Instruct TII45±2local~7.6GB~13 tok/s32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)
580Yi 34B Chat 01.AI45±1local~23GB4kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetEpoch AI — internal evaluationsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs+1
581Hermes 2 Theta Llama 3 8B Nous Research45±2local~6.6GB8kllama.cpp › WildBench v2 leaderboard data (Ai2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
582Llama 3.1 8B (base) Meta45±3local~6.2GBLM Studio › Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
583Gemma 3n E2B IT Google45±2local~4.1GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model cardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
584Code Llama 70B (base) Meta45±1local~28GB16kllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
585Phi-3.5-mini-instruct Microsoft45±1local~5.7GB128kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsDaekeun Kim Korean LLM benchmark runsMicrosoft — Phi-4-Mini Technical ReportSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+4
586Qwen2.5-VL 3B Instruct Alibaba45±1local~2.8GBllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)
587Qwen3 1.7B Alibaba45±1local~2.6GB~96 tok/s33kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen/Qwen3.5-2B model cardLiquidAI/LFM2-1.2B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HuggingFaceTB/SmolLM3-3B model card (Evaluation section)LiquidAI/LFM2.5-1.2B-Thinking model card+11
588Qwen3-VL 2B Instruct Alibaba45±1local~2.7GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)
589LFM2.5 1.2B Instruct Liquid AI45±2local~1.3GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
590Claude Instant 1.1 Anthropic45±2comparison onlyAnthropic system cards and model announcements (lab-reported scores)An Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs
591Granite 3.1 8B Instruct IBM44±1local~6.7GB128kllama.cpp › ibm-granite/granite-3.3-8b-instruct model cardibm-granite/granite-3.1-3b-a800m-instruct model cardLMArena leaderboard datasetBerkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
592Reka Flash 21B (2024-02-26) Reka AI44±1comparison onlyWildEval ZeroEval published resultsCharXiv leaderboard (princeton-nlp/CharXiv README)WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetCharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (arXiv:2406.18521v1)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)
593GPT-3.5 text-davinci-002 OpenAI44±0comparison onlyChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runsFaithful Chain-of-Thought Reasoning — authors' own runsINFORM : Information eNtropy based multi-step reasoning FOR large language Models — authors' own runsDo Androids Laugh at Electric Sheep? Humor ``Understanding'' Benchmarks from The New Yorker Caption Contest — authors' own runs+22
594Flan-PaLM 62B Google44±1†never releasedScaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Encode Clinical Knowledge — authors' own runs
595Qwen2.5 3B Instruct Alibaba44±1local~2.6GB~61 tok/s33kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)SpeakLeash Polish EQ-Bench leaderboard+5
596CogVLM2 Llama3 Chat 19B Zhipu AI (THUDM)44±3local~43GBhow to run › LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)Zhipu AI (THUDM) CogVLM2 model card, Benchmark table (own models)
597Llama 3.2 11B Vision Instruct Meta44±2local~8.9GB128kOllama › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)LMArena leaderboard datasetInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
598LFM2 2.6B Liquid AI44±1local~2GB33kLM Studio › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationEuroEval leaderboards (results collection), English generative leaderboard
599Tulu 2 DPO 70B Ai244±1local~32GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+1
600Phi-3-small-8k-instruct Microsoft44±2local~17GBhow to run › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Portuguese LLM Leaderboard (archived)
601MiniCPM-V 2.6 OpenBMB44±2local~5.8GBllama.cpp › CharXiv leaderboard (princeton-nlp/CharXiv README)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
602CodeGemma 7B IT (v1.0) Google44±1local~7.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CodeGemma: Open Code Models Based on Gemma — authors' own runs
603Yi-1.5 9B (base) 01.AI44±2local~6.8GB4kLM Studio › Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
604Claude Instant 1.2 Anthropic44±1comparison onlyEvalPlus / EvalPerf published evaluation runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Anthropic system cards and model announcements (lab-reported scores)
605Qwen2-Audio 7B Instruct Alibaba44±3local~6.5GBllama.cpp › AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2)
606DeepSeek-Coder 1.3B Instruct DeepSeek44±1local~2.9GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
607Aya 23 8B Cohere For AI44±3local~6.2GB8kLM Studio › SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
608Qwen2.5 Coder 1.5B Instruct Alibaba44±1local~1.8GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
609WizardLM 70B V1.0 Microsoft44±1local~32GBllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
610Mathstral 7B v0.1 Mistral AI44±1local~5.6GB32kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsMMLU-Pro author-released evaluation runsWildEval ZeroEval published resultsMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)
611SmolLM3 3B Hugging Face44±1local~2.9GB66kLM Studio › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationHuggingFaceTB/SmolLM3-3B model card (Evaluation section)Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 modelsInception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)EuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
612Hermes 3 Llama 3.1 8B Nous Research44±1local~6.6GB128kllama.cpp › Nous Research vendor-reported scores for its own Hermes models (Hermes 3 Technical Report)Aider code editing benchmark runs (old edit leaderboard)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
613Llama 3 8B Instruct Meta44±0local~6.6GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsInternLM GitHub repository model cards (Shanghai AI Laboratory)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+19
614Command R (03-2024) Cohere44±1local~33GB128kllama.cpp › WildEval ZeroEval published resultsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)MMLU-Pro author-released evaluation runsWildBench v2 leaderboard data (Ai2)LMArena leaderboard dataset+6
615Aya Vision 32B Cohere44±3api price unknown +local~71GB16kdocs.cohere.com ↗LMArena leaderboard datasetKaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark (arXiv:2508.13680v5)
616WizardCoder Python 13B V1.0 WizardLM43±2local~15GB16kllama.cpp › WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)CRUXEval published evaluation results (authors' generations and scores)
617OpenChat 3.5 OpenChat43±1local~6.1GB8kllama.cpp › OpenChat 3.5 model card (openchat/openchat_3.5)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs
618Granite 34B Code Base (8K) IBM43±1local~24GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
619speechless-starcoder2-7b uukuguy43±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
620Falcon2 11B (base) TII43±3local~8.2GB8kOllama › SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
621Snowflake Arctic Instruct Snowflake43±2local~1007GBhow to run › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
622OLMo 2 7B Instruct (1124) Allen Institute for AI43±1local~5.8GB4kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsallenai/Olmo-3-7B-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
623WizardCoder-15B-V1.0 WizardLM43±1local~12GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
624Ministral 3B (2410) Mistral43±1comparison onlymicrosoft/Phi-4-mini-instruct model cardMistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)Epoch AI — internal evaluationsVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
625Phi-3-vision-128k-instruct Microsoft43±2local~10GBhow to run › CharXiv leaderboard (princeton-nlp/CharXiv README)LMArena leaderboard datasetPhi-3 Technical Report (Microsoft, arXiv 2404.14219v3)PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
626cont-PaLM 62B (PaLM 62B trained to 1.3T tokens) Google43±1†never releasedPaLM: Scaling Language Modeling with Pathways — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runs
627DeepSeek LLM 67B Chat DeepSeek43±1local~31GB4kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Epoch AI — internal evaluationsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
628Qwen1.5 14B Chat Alibaba43±1local~11GB32kOllama › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)MMLU-Pro author-released evaluation runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)+3
629Llama 2 70B (base) Meta43±1local~32GB4kllama.cpp › Mixtral of Experts — base-model evaluationsTulu 2 — base and instruction-tuned evaluationsMMLU-Pro author-released evaluation runsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
630Mixtral 8x7B v0.1 (base) Mistral AI43±1local~29GB32kllama.cpp › Mixtral of Experts — base-model evaluationsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
631Code Llama 34B Python Meta43±1local~23GB16kOllama › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
632DeepSeek-Coder 33B Base DeepSeek43±1local~22GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
633speechless-coding-7b-16k-tora uukuguy43±1local~16GBhow to run › EvalPlus / EvalPerf published evaluation runs
634InternVL2-4B Shanghai AI Laboratory (OpenGVLab)43±2local~10GBhow to run › OpenGVLab InternVL2 model cards (InternVL2-4B and InternVL2-26B), Evaluation tablesLMArena leaderboard datasetMicrosoft Phi-3.5-vision-instruct model card: Benchmarks tablePhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
635InternLM2 Chat 20B Shanghai AI Laboratory (InternLM)43±1local~14GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
636CodeQwen1.5 7B (base) Alibaba42±1local~6.4GB66kllama.cpp › EvalPlus / EvalPerf published evaluation runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
637Nous Hermes 2 Mistral 7B DPO Nous Research42±3local~6.1GB33kllama.cpp › Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
638Mixtral 8x7B Instruct v0.1 Mistral42±0local~28GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsWildEval ZeroEval published resultsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+15
639U-PaLM 62B Google42±2†never releasedTranscending Scaling Laws with 0.1% Extra Compute — maker's own results
640Granite 4.0 1B IBM42±1local~2.4GB128kLM Studio › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model cardEpoch AI — internal evaluations
641OpenHermes-2.5-Code-290k-13B ajibawa-202342±1local~9.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
642Granite 4.0 H 1B IBM42±1local~2.3GB131kLM Studio › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model card
643Nous Hermes 2 Mixtral 8x7B DPO Nous Research42±2local~30GB33kllama.cpp › WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Portuguese LLM Leaderboard (archived)
644Deita 7B v1.0 HKUST NLP42±2local~32GBhow to run › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
645Vicuna 13B v1.1 LMSYS42±2comparison onlySearch Augmented Instruction Learning — authors' own runsThis is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models — authors' own runsSummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization — authors' own runs
646Molmo 7B-D (0924) Ai242±1local~35GBhow to run › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)LMArena leaderboard datasetKaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arXiv:2504.07072v2)Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column
647OpenHermes 2.5 Mistral 7B Nous Research42±1local~5.9GBllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpenChat 3.5 model card (openchat/openchat_3.5)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+2
648Code Llama 34B Instruct Meta42±1local~22GB16kLM Studio › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4LMArena leaderboard dataset+1
649North Micro Vision Instruct (2.4B) Cohere42±2†local~3.8GB128khow to run › CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)
650Starling-LM 7B beta Nexusflow42±1local~6GB8kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+3
651Code Llama 34B (base) Meta42±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+1
652Vicuna 33B v1.3 LMSYS41±1local~23GB2kOllama › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table+2
653Phi-3-mini-128k-instruct (Apr 2024) Microsoft41±2local~9.5GBhow to run › Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard+1
654Bielik 1.5B v3.0 Instruct SpeakLeash41±3local~2.4GBLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
655Starling-LM 7B alpha UC Berkeley (NEST)41±1local~6.1GB8kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)+2
656Code Tulu 2 13B Allen Institute for AI41±2†local~29GBhow to run › Tulu 2 — base and instruction-tuned evaluations
657Mistral 7B Instruct v0.3 Mistral41±1local~5.6GB32kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardEpoch AI — internal evaluations+4
658LLaVA-v1.6-34B LLaVA41±2local~24GB4kOllama › CharXiv leaderboard (princeton-nlp/CharXiv README)CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (arXiv:2406.18521v1)LMArena leaderboard datasetCapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)LLaVA repository model zoo (haotian-liu/LLaVA docs/MODEL_ZOO.md)
659Granite 3.3 2B Instruct IBM41±1†local~3GB128kLM Studio › ibm-granite/granite-3.3-8b-instruct model card
660Llama3-OpenBioLLM-8B Saama AI Labs41±2local~6.6GBllama.cpp › Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)
661DeepSeek-Coder 6.7B Base DeepSeek41±1local~8.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
662LFM2 1.2B Liquid AI41±1local~1.3GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
663Granite 3.0 2B Instruct IBM41±1local~3.2GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)LMArena leaderboard datasetCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
664Llama 2 70B Chat Meta41±1local~32GB4kllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)WildBench v2 leaderboard data (Ai2)LMArena leaderboard dataset+13
665Granite 8B Code Base (4K) IBM41±1local~6.6GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
666StarCoder2 15B BigCode41±1local~11GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
667Llama 3.2 3B Instruct Meta40±1api $0.12/M +local~3.3GB~139 tok/s131kdevelopers.cloudflare.com ↗microsoft/Phi-4-mini-instruct model cardIBM Granite 3.0 Language Models technical report (GitHub)LFM2 Technical Report — Liquid AI own SmolLM3 evaluationAider code editing benchmark runs (old edit leaderboard)LMArena leaderboard datasetmicrosoft/Phi-4-mini-reasoning model card+11
668Falcon-H1 3B Instruct TII40±3local~3.4GB131kLM Studio › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
669Phi-2 (2.7B) Microsoft40±1local~4.7GB2kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)+2
670speechless-code-mistral-7b-v1.0 uukuguy40±1local~17GBhow to run › EvalPlus / EvalPerf published evaluation runsOpen Portuguese LLM Leaderboard (archived)
671Llama-3-Instruct-8B-SimPO-v0.2 Princeton NLP40±1local~6.6GBllama.cpp › WildEval ZeroEval published resultsWildBench v2 leaderboard data (Ai2)Open Portuguese LLM Leaderboard (archived)
672Qwen-14B-Chat Alibaba40±1local~11GBllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs
673Code Llama 13B Instruct Meta40±1local~15GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
674Tulu 2 13B Allen Institute for AI40±2local~9.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsOpen Portuguese LLM Leaderboard (archived)
675Falcon-180B-Chat TII40±2local~115GB2kOllama › LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)
676MistralHermes-CodePro-7B-v1 beowolx40±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
677LLaMA 65B Meta40±1comparison onlyLLaMA: Open and Efficient Foundation Language Models — authors' own runsHow Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runsEnhancing Chat Language Models by Scaling High-quality Instructional Conversations — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios — authors' own runs+7
678Phi-3-mini-4k-instruct (Apr 2024) Microsoft40±2local~6GBllama.cpp › MMLU-Pro author-released evaluation runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)
679CodeGemma 7B (v1.0) Google40±1local~7.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
680Code Llama 13B Python Meta40±1local~9.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
681Apertus 8B Instruct (2509) Swiss AI40±3local~6.3GB66kLM Studio › allenai/Olmo-3-7B-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboardMiðeind Icelandic LLM Leaderboard
682DeepSeek-V2 Lite Chat (16B-A2.4B) DeepSeek40±1local~11GB32kLM Studio › DeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLiveCodeBench official website results (2024-12-26)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
683Granite 3B Code Instruct (2K) IBM40±1local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
684Tulu 2 DPO 13B Allen Institute for AI39±1local~9.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
685BioMistral-7B BioMistral39±3local~6.1GBllama.cpp › BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)
686PaLM 62B Google39±1†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsPaLM: Scaling Language Modeling with Pathways — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runsSolving Quantitative Reasoning Problems with Language Models — authors' own runsTranscending Scaling Laws with 0.1% Extra Compute — maker's own resultsUniversal Self-Adaptive Prompting — authors' own runs+4
687SOLAR 10.7B Instruct v1.0 Upstage39±1local~8.3GB4kllama.cpp › EvalPlus / EvalPerf published evaluation runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboard+2
688Code Tulu 2 7B Allen Institute for AI39±2†local~16GBhow to run › Tulu 2 — base and instruction-tuned evaluations
689Gemma 2 2B IT Google39±1local~3.2GB8kLM Studio › IBM Granite 3.0 Language Models technical report (GitHub)WildEval ZeroEval published resultsWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+4
690Yi-1.5 6B (base) 01.AI39±3local~5.1GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
691Flan-T5 XXL Google39±1local~49GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsInstructEval (DeCLaRe Lab) README results tableSpecializing Smaller Language Models towards Multi-Step Reasoning — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsUnnatural Instructions: Tuning Language Models with (Almost) No Human Labor — authors' own runs+12
692Dolphin 2.2.1 Mistral 7B Cognitive Computations39±2local~5.9GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)
693Gemma 1.1 7B IT Google39±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)+2
694Phi-3.5-vision-instruct Microsoft39±2local~10GBhow to run › CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)LMArena leaderboard datasetMicrosoft Phi-3.5-vision-instruct model card: Benchmarks tableAi2 Molmo-72B-0924 model card: Human Preference Elo Rating columnPhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)
695Granite 20B Code Base (8K) IBM39±1local~15GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
696Granite 3.1 2B Instruct IBM39±1local~3.1GB128kllama.cpp › ibm-granite/granite-3.3-8b-instruct model cardibm-granite/granite-3.1-3b-a800m-instruct model cardLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
697Code Llama 7B Python Meta39±1local~5.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
698Qwen1.5 7B Chat Alibaba39±1local~9.7GB32kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)MMLU-Pro author-released evaluation runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)WildBench v2 leaderboard data (Ai2)+4
699Granite 3.2 2B Instruct IBM39±1local~3GB131kLM Studio › ibm-granite/granite-3.3-8b-instruct model cardVectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
700Code Llama 13B (base) Meta38±1local~9.2GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4+1
701Falcon3 3B Instruct TII38±3local~3.4GB32kLM Studio › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
702Qwen2.5 Coder 0.5B Instruct Alibaba38±1local~1GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
703Vicuna 13B v1.5 LMSYS38±1local~9.8GB4kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableOpen Portuguese LLM Leaderboard (archived)
704Zephyr 7B alpha Hugging Face38±1local~6.1GBllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Portuguese LLM Leaderboard (archived)
705Yi 1.5 6B Chat 01.AI38±2local~4.4GB4kLM Studio › BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4WildBench v2 leaderboard data (Ai2)SpeakLeash Polish EQ-Bench leaderboardOpen Portuguese LLM Leaderboard (archived)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
706WizardLM 13B V1.2 Microsoft38±1local~9.8GBllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
707Qwen2-VL 2B Instruct Alibaba38±1local~2.1GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
708Qwen2.5 1.5B Instruct Alibaba38±1local~1.8GB~98 tok/s33kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+6
709Mistral 7B Instruct v0.2 Mistral38±1local~5.8GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)MMLU-Pro author-released evaluation runs+9
710Code Llama 7B Instruct Meta38±1local~5.5GB16kLM Studio › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
711Tulu 2 7B Allen Institute for AI38±2local~5.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsOpen Portuguese LLM Leaderboard (archived)
712Qwen3.5 0.8B Alibaba38±1local~1.4GB~94 tok/s262kllama.cpp › Qwen/Qwen3.5-2B model cardEuroEval leaderboards (results collection), English generative leaderboardQwen/Qwen3.5-0.8B model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+2
713MPT-30B-Chat MosaicML38±2comparison onlyLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
714LFM2 700M Liquid AI38±1local~1GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
715LLaMA 33B Meta38±1comparison onlyEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Mixtral of Experts — base-model evaluationsLLaMA: Open and Efficient Foundation Language Models — authors' own runsHow Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsA Frustratingly Easy Post-Training Quantization Scheme for LLMs — authors' own runs+9
716EXAONE 3.5 2.4B Instruct LG AI Research38±2local~3GB33kLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
717xDAN-L1-Chat-RL-v1 xDAN-AI37±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
718Flan-PaLM 8B Google37±1†never releasedScaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Encode Clinical Knowledge — authors' own runs
719Mistral 7B v0.1 (base) Mistral AI37±1local~5.8GB8kLM Studio › EvalPlus / EvalPerf published evaluation runsStability AI — original StableLM zero-shot evaluation outputsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runsGemma report — original evaluations+4
720Llama 3 8B (base) Meta37±1local~6.6GBllama.cpp › EvalPlus / EvalPerf published evaluation runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)EuroEval leaderboards (results collection), English generative leaderboard
721Flan-T5 XL Google37±1local~14GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsInstructEval (DeCLaRe Lab) README results tableSpecializing Smaller Language Models towards Multi-Step Reasoning — authors' own runsAugmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsChain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance — authors' own runs+8
722Gemma 7B (base) Google37±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGemma report — original evaluationsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)+3
723Tulu 2 DPO 7B Allen Institute for AI37±1local~5.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
724Vicuna 13B v1.3 LMSYS37±2local~29GB2khow to run › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Can Language Models Understand Physical Concepts? — authors' own runs
725Granite 3.0 3B-A800M Instruct IBM37±1local~3.7GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
726Granite 3.1 3B-A800M Instruct IBM37±2†local~3.6GB~23 tok/s131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
727DeepSeek R1 Distill Qwen 1.5B DeepSeek37±2local~1.9GB131kLM Studio › deepseek-ai/DeepSeek-R1 model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardTHINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models (arXiv:2505.22113v1)Epoch AI — internal evaluations+1
728StarCoder2 7B BigCode37±1local~5.4GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
729Vicuna 7B v1.1 LMSYS37±2comparison onlySearch Augmented Instruction Learning — authors' own runsThis is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models — authors' own runs
730DeepSeek LLM 7B Chat DeepSeek37±2†local~5.7GBOllama › DeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)
731Falcon 40B (base) TII36±1local~27GB2kOllama › MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models — authors' own runs+2
732StarCoder 15.5B BigCode36±1local~11GBOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder: may the source be with you! — authors' own runsExecution-Based Evaluation for Open-Domain Code Generation — authors' own runs
733Eurus-2-7B-SFT PRIME-RL36±3†local~6.4GBllama.cpp › PRIME-RL Eurus-2-7B-PRIME model card, Evaluation table (own models)
734Nous-Hermes-13b Nous Research36±2local~9.8GB2kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)
735Qwen3 0.6B Alibaba36±1local~1.9GB~182 tok/s33kLM Studio › Olmo 3LiquidAI/LFM2-1.2B model cardUnsupervised Confidence Calibration for Reasoning LLMs from a Single Generation (arXiv:2604.19444v1)Berkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+5
736PaLM 2 Chat (chat-bison@001) Google36±2comparison onlyLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
737Code Llama 7B (base) Meta36±1local~5.5GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsMistral 7B — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluations+2
738Llama 2 13B (base) Meta36±1local~9.3GB4kOllama › Stability AI — original StableLM zero-shot evaluation outputsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runsTulu 2 — base and instruction-tuned evaluationsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+4
739Llama 2 13B Chat Meta36±1local~15GB4kllama.cpp › Orca 2: Teaching Small Language Models How to Reason (Mitra et al., Microsoft)Tulu 2 — base and instruction-tuned evaluationsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)+10
740Zephyr 7B beta Hugging Face35±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTulu 2 — base and instruction-tuned evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)OpenChat 3.5 model card (openchat/openchat_3.5)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+3
741Guanaco 33B University of Washington35±1local~70GBhow to run › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableBattle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT - A Text-to-SQL Parsing Comparison — authors' own runs
742StarCoder2 3B BigCode35±1local~3.4GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
743StarCoderBase 15.5B BigCode35±1local~12GBllama.cpp › StarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCRUXEval published evaluation results (authors' generations and scores)StarCoder: may the source be with you! — authors' own runsExecution-Based Evaluation for Open-Domain Code Generation — authors' own runs
744Vicuna 7B v1.5 LMSYS35±1local~5.8GB4kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table+1
745Granite 3B Code Base (2K) IBM35±1local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4EuroEval leaderboards (results collection), English generative leaderboard
746ChatGLM3 6B 128K Zhipu AI (THUDM)35±4local~5.8GB131kllama.cpp › NVIDIA RULER repository README results tableOpen Portuguese LLM Leaderboard (archived)
747Minerva 8B Google35±2†never releasedSolving Quantitative Reasoning Problems with Language Models — authors' own runs
748OLMo 2 1B Instruct (0425) Allen Institute for AI35±1local~2.4GB4kllama.cpp › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsOpen Portuguese LLM Leaderboard (archived)
749Vicuna 7B v1.3 LMSYS35±2local~5.8GB2kllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Can Language Models Understand Physical Concepts? — authors' own runs
750Stable Code 3B Stability AI35±1local~4.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
751Orca 2 13B Microsoft35±3local~9.8GB4kllama.cpp › Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
752T0 11B BigScience35±3comparison onlyKnowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering — authors' own runsCrosslingual Generalization through Multitask Finetuning — authors' own runsSuper-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks — authors' own runsThe CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning — authors' own runsSelf-Instruct: Aligning Language Models with Self-Generated Instructions — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runs
753DeepSeek-Coder 1.3B Base DeepSeek34±1local~2.4GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
754Qwen1.5 4B Chat Alibaba34±2local~4.1GB32kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
755Gemma 7B IT Google34±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)LMArena leaderboard dataset+5
756Mistral 7B Instruct v0.1 Mistral34±1local~5.9GB8kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)LMArena leaderboard datasetMMLU-Pro author-released evaluation runsPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+3
757GPT-3 text-davinci-001 OpenAI34±1comparison onlyUPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation — authors' own runsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Do Androids Laugh at Electric Sheep? Humor ``Understanding'' Benchmarks from The New Yorker Caption Contest — authors' own runsPAL: Program-aided Language Models — authors' own runsEQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runsSay What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge — authors' own runs+5
758LaMDA 137B (pre-trained, LaMDA-PT) Google34±1†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsSelf-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runsPaLM: Scaling Language Modeling with Pathways — authors' own runs
759CodeGemma 2B (v1.0) Google34±1local~3.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4
760Qwen2 1.5B Instruct Alibaba34±1local~1.8GB32kLM Studio › WildEval ZeroEval published resultsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsSpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+1
761Falcon-H1 1.5B Instruct TII33±4local~2.4GB131kLM Studio › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
762Ghost 7B Alpha Ghost X33±2local~6.1GBllama.cpp › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
763Gemma 3 1B Google33±1local~1.7GB~98 tok/s32kLM Studio › Google DeepMind Gemma 3 270M IT model cardLiquidAI/LFM2-1.2B model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2.5-1.2B-Thinking model cardGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboard+7
764SmolLM2 1.7B Instruct Hugging Face33±1local~3GB8kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLMArena leaderboard datasetSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
765Qwen-7B (base) Alibaba33±1local~6.6GB33kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
766LFM2 350M Liquid AI33±2local~1.7GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
767OPT-175B Meta33±1comparison onlyGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsOPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsCan Language Models Understand Physical Concepts? — authors' own runs
768StarCoderBase 7B BigCode33±1local~6GBOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
769BLOOMZ 176B BigScience33±2comparison onlyCrosslingual Generalization through Multitask Finetuning — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsTowards Effective Disambiguation for Machine Translation with Large Language Models — authors' own runs
770Phi-1.5 (1.3B) Microsoft33±1local~2.4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
771Codex code-davinci-001 OpenAI33±1comparison onlyFaithful Chain-of-Thought Reasoning — authors' own runsLarge Language Models are Better Reasoners with Self-Verification — authors' own runsCodeIE: Large Code Generation Models are Better Few-Shot Information Extractors — authors' own runsExecution-Based Evaluation for Open-Domain Code Generation — authors' own runs
772Llama 3.2 1B Instruct Meta33±1api $0.020/M +local~1.6GB~28 tok/s131knovita.ai ↗IBM Granite 3.0 Language Models technical report (GitHub)LiquidAI/LFM2-1.2B model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLiquidAI/LFM2.5-1.2B-Thinking model cardLMArena leaderboard dataset+13
773LLaMA 13B Meta32±1comparison onlyEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)LLaMA: Open and Efficient Foundation Language Models — authors' own runsInstructEval (DeCLaRe Lab) README results tableHow Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runsEnhancing Chat Language Models by Scaling High-quality Instructional Conversations — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs+19
774Qwen-7B-Chat Alibaba32±1local~6.6GB33kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
775Llama 2 7B Chat Meta32±1local~8.8GB4kllama.cpp › Tulu 2 — base and instruction-tuned evaluationsLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)WildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+13
776OLMo 7B Instruct Ai232±2local~16GBhow to run › OLMo: Accelerating the Science of Language Models — authors' own runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)
777Llama 2 7B (base) Meta32±1local~8.5GB4kOllama › Stability AI — original StableLM zero-shot evaluation outputsMixtral of Experts — base-model evaluationsOLMo: Accelerating the Science of Language Models — authors' own runsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runs+6
778Koala 13B UC Berkeley (BAIR)32±1comparison onlyInstructEval (DeCLaRe Lab) README results tableEnhancing Chat Language Models by Scaling High-quality Instructional Conversations — authors' own runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableEQ-Bench: An Emotional Intelligence Benchmark for Large Language Models — authors' own runs+1
779Flan-T5 Large Google32±1local~4.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsSpecializing Smaller Language Models towards Multi-Step Reasoning — authors' own runsModel-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers — authors' own runsAugmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runs+5
780GPT-3 davinci (API base model) OpenAI32±1comparison onlyGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsUPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsMaking Language Models Better Reasoners with Step-Aware Verifier — authors' own runsGLM-130B: An Open Bilingual Pre-trained Model — authors' own runs+13
781Falcon 40B Instruct TII31±2local~27GBOllama › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableclembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents — authors' own runsThis is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions — authors' own runs
782Granite 3.1 1B-A400M Instruct IBM31±2†local~2.3GB~20 tok/s131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
783Gemma 2B (base) Google31±1local~3.2GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGemma report — original evaluationsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+3
784Gemma 1.1 2B IT Google31±1local~3.2GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)SpeakLeash Polish EQ-Bench leaderboardArena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)Open Portuguese LLM Leaderboard (archived)+1
785Falcon 7B (base) TII31±1local~5.9GB2kOllama › Stability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)+4
786Alpaca 7B Stanford CRFM31±1comparison onlyEnhancing Chat Language Models by Scaling High-quality Instructional Conversations — authors' own runsINFORM : Information eNtropy based multi-step reasoning FOR large language Models — authors' own runsAligning Large Language Models through Synthetic Feedback — authors' own runsSCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables — authors' own runsAPI-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs — authors' own runs
787Baichuan2-7B-Base Baichuan30±1local~17GBhow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
788LLaMA 7B Meta30±1comparison onlyStability AI — original StableLM zero-shot evaluation outputsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)LLaMA: Open and Efficient Foundation Language Models — authors' own runsOLMo: Accelerating the Science of Language Models — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsInstructEval (DeCLaRe Lab) README results table+17
789NexusRaven V2 13B Nexusflow30±3local~9.3GB16kOllama › Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Berkeley Function Calling Leaderboard result archive
790MPT-7B (base) MosaicML30±1comparison onlyStability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsEnhancing Chat Language Models by Scaling High-quality Instructional Conversations — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)
791ChatGLM3-6B Zhipu AI (THUDM)30±2local~5.8GBllama.cpp › LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)Open Portuguese LLM Leaderboard (archived)
792Qwen2.5 0.5B Instruct Alibaba30±2local~1GB33kLM Studio › HuggingFaceTB/SmolLM2-360M-Instruct model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)+1
793OLMo 7B Allen Institute for AI30±1†local~30GB2khow to run › OLMo: Accelerating the Science of Language Models — authors' own runs
794StableLM-3B-4E1T Stability AI30±1local~3.3GB4kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
795StarCoderBase 3B BigCode30±1local~3.4GBOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
796OpenLLaMA 7B v2 OpenLM Research30±1local~9.1GBllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
797StableLM-Base-Alpha 7B v2 Stability AI30±1local~30GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
798Granite 3.0 1B-A400M Instruct IBM30±1local~2.3GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
799OPT-66B Meta29±1local~141GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runs+2
800MPT-7B-Chat MosaicML29±2comparison onlyLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableOLMo: Accelerating the Science of Language Models — authors' own runs
801GPT4All-13B-Snoozy Nomic AI29±2local~56GBhow to run › LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
802Alpaca 13B Stanford CRFM29±2comparison onlyINFORM : Information eNtropy based multi-step reasoning FOR large language Models — authors' own runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
803EuroLLM 1.7B Instruct UTTER (EuroLLM)29±3local~2.4GB4kLM Studio › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
804Granite 4.0 H 350M IBM29±1†local~1.7GB33kLM Studio › ibm-granite/granite-4.0-h-1b model card
805BLOOMZ 7.1B BigScience29±2comparison onlyCrosslingual Generalization through Multitask Finetuning — authors' own runsTowards Effective Disambiguation for Machine Translation with Large Language Models — authors' own runsConic10K: A Challenging Math Problem Understanding and Reasoning Dataset — authors' own runsLarge Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs
806BLOOM 176B BigScience29±1local~372GB2khow to run › GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsGLM-130B: An Open Bilingual Pre-trained Model — authors' own runsCrosslingual Generalization through Multitask Finetuning — authors' own runsTowards Effective Disambiguation for Machine Translation with Large Language Models — authors' own runs+1
807Gemma 2B IT Google29±1local~3.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsWildBench v2 leaderboard data (Ai2)LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)+3
808Granite 4.0 350M IBM28±1local~1.7GB32kLM Studio › ibm-granite/granite-4.0-h-1b model cardBerkeley Function Calling Leaderboard result archiveEpoch AI — internal evaluations
809OPT-30B Meta28±1local~64GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runsOutlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runs+5
810UL2 20B Google28±3†comparison onlySelf-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runsRecitation-Augmented Language Models — authors' own runs
811Qwen2 0.5B Instruct Alibaba28±2local~1GB32kLM Studio › Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
812GPT-NeoX-20B EleutherAI28±1local~14GB2kLM Studio › Stability AI — original StableLM zero-shot evaluation outputsGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsPrompting Language Models for Linguistic Structure — authors' own runsGenerative Calibration for In-context Learning — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsOpen Portuguese LLM Leaderboard (archived)+1
813RedPajama-INCITE 7B Base Together28±1local~16GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Portuguese LLM Leaderboard (archived)This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models — authors' own runs
814ChatGLM2-6B Zhipu AI (THUDM)28±2local~15GBhow to run › LMArena leaderboard datasetAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableOpen Portuguese LLM Leaderboard (archived)
815RWKV-4-Raven 14B RWKV (BlinkDL)27±2comparison onlyLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
816LaMDA 68B (pre-trained) Google27±1†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs
817BLOOMZ 3B BigScience27±2comparison onlyCrosslingual Generalization through Multitask Finetuning — authors' own runsLarge Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs
818Falcon3 1B Instruct TII26±4local~2.4GB8kLM Studio › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
819Qwen1.5 1.8B Chat Alibaba26±2local~2.7GB33kOllama › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
820Tk-Instruct 11B (def) AllenAI / UW26±4comparison onlyUnnatural Instructions: Tuning Language Models with (Almost) No Human Labor — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runs
821OPT-13B (13 billion parameters) Meta26±1local~29GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering — authors' own runs+4
822PaLM 8B Google26±1†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsPaLM: Scaling Language Modeling with Pathways — authors' own runsScaling Instruction-Finetuned Language Models — authors' own runsSolving Quantitative Reasoning Problems with Language Models — authors' own runsLarge Language Models Encode Clinical Knowledge — authors' own runsWinoDict: Probing language models for in-context word acquisition — authors' own runs
823Falcon 7B Instruct TII26±2local~5.9GBOllama › AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset — authors' own runsThis is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models — authors' own runsHave LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models — authors' own runs+1
824OpenAssistant SFT-4 Pythia 12B LAION (OpenAssistant)25±1local~9.5GBllama.cpp › InstructEval (DeCLaRe Lab) README results tableLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableAligning Large Language Models through Synthetic Feedback — authors' own runs+1
825Flan-T5 Base Google25±1local~2.5GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsModel-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsAugmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runsSpecializing Smaller Language Models towards Multi-Step Reasoning — authors' own runs+3
826RedPajama-INCITE Base 3B v1 Together25±3local~3.4GB2kllama.cpp › RedPajama — original zero-shot 3B comparisonOpen Portuguese LLM Leaderboard (archived)
827GPT-3 curie (API base model) OpenAI25±1comparison onlyGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsIt Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset — authors' own runsIncreasing Probability Mass on Answer Choices Does Not Always Improve Accuracy — authors' own runsWinoDict: Probing language models for in-context word acquisition — authors' own runs+1
828Pythia 12B EleutherAI25±1local~9.5GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsTowards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios — authors' own runsOpen Portuguese LLM Leaderboard (archived)+1
829GPT-J 6B EleutherAI25±1local~14GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsEvalPlus / EvalPerf published evaluation runsGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runs+6
830OpenLLaMA 3B v2 OpenLM Research25±1local~4.2GBllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
831ChatGLM-6B Zhipu AI (THUDM)25±1local~16GBhow to run › InstructEval (DeCLaRe Lab) README results tableLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableAPI-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs — authors' own runsConic10K: A Challenging Math Problem Understanding and Reasoning Dataset — authors' own runs+3
832StableLM-Base-Alpha 3B v2 Stability AI24±1local~13GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
833OPT-6.7B Meta24±1local~16GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runs+5
834T0 3B BigScience23±4comparison onlyKnowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering — authors' own runsThe CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning — authors' own runs
835Qwen1.5 0.5B Chat Alibaba23±3local~1.7GB33kLM Studio › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
836BLOOMZ 1.7B BigScience23±3comparison onlyCrosslingual Generalization through Multitask Finetuning — authors' own runsLarge Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs
837Pythia 6.9B EleutherAI23±1local~6.1GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOLMo: Accelerating the Science of Language Models — authors' own runsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsHow Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runs+2
838T0++ (T0pp) 11B BigScience23±3comparison onlyZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsModel-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers — authors' own runsUnnatural Instructions: Tuning Language Models with (Almost) No Human Labor — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runsCausal Reasoning of Entities and Events in Procedural Texts — authors' own runs
839Phi-1 (1.3B) Microsoft22±4local~2.4GB2kllama.cpp › CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)
840Dolly v2 12B Databricks22±1comparison onlyEnhancing Chat Language Models by Scaling High-quality Instructional Conversations — authors' own runsLMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)InstructEval (DeCLaRe Lab) README results tableLMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableAligning Large Language Models through Synthetic Feedback — authors' own runs+3
841SmolLM2 360M Instruct Hugging Face22±3local~1GB8kLM Studio › HuggingFaceTB/SmolLM2-360M-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
842FastChat-T5 3B LMSYS21±2local~8.5GBhow to run › LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table
843OLMo 1B Allen Institute for AI21±2†local~2.2GB2kllama.cpp › OLMo: Accelerating the Science of Language Models — authors' own runs
844BLOOM 7.1B BigScience21±1local~7.1GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsUPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+7
845Pythia 2.8B EleutherAI20±1local~3.4GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsRedPajama — original zero-shot 3B comparisonImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
846Pythia 2.8B (deduped) EleutherAI20±1local~3.4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
847Falcon-RW 1B TII19±2local~4.3GB2khow to run › Textbooks Are All You Need II: phi-1.5 technical report — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)
848OPT-2.7B Meta19±1local~7.1GB2khow to run › Stability AI — original StableLM zero-shot evaluation outputsPythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runs+3
849GPT-Neo 2.7B EleutherAI18±1local~13GB2khow to run › EvalPlus / EvalPerf published evaluation runsUPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsPreserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsRedPajama — original zero-shot 3B comparison+8
850StableLM-Tuned-Alpha 7B Stability AI18±2local~6.9GBllama.cpp › LMArena leaderboard datasetPolyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard tableLLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (arXiv:2306.02561v3)
851BLOOMZ 1.1B BigScience18±3comparison onlyCrosslingual Generalization through Multitask Finetuning — authors' own runsLarge Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs
852T5 Large Google17±2local~4.6GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
853Gemma 3 270M IT Google16±4local~1.1GB32kLM Studio › Google DeepMind Gemma 3 270M IT model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
854Flan-T5 Small Google16±3local~1.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsPOSQA: Probe the World Models of LLMs with Size Comparisons — authors' own runsAn Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs
855BLOOM 3B BigScience16±1local~4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)+2
856GPT-3 babbage (API base model) OpenAI15±1comparison onlyGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsLarge Language Models Are Reasoning Teachers — authors' own runsIt Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset — authors' own runsWinoDict: Probing language models for in-context word acquisition — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runs
857BLOOMZ 560M BigScience14±4comparison onlyCrosslingual Generalization through Multitask Finetuning — authors' own runsDo Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models — authors' own runsLarge Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs
858OPT-1.3B Meta14±1local~4.3GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsOPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runs+5
859T5 Base Google14±3local~2.4GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
860Pythia 1.4B EleutherAI14±1local~2.4GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
861Pythia 1B EleutherAI12±1local~2.2GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
862LaMDA 8B (pre-trained) Google12±2†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs
863GPT-3 text-curie-001 OpenAI12±1comparison onlyChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsIn-Context Demonstration Selection with Cross Entropy Difference — authors' own runsSay What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge — authors' own runsDo Androids Laugh at Electric Sheep? Humor ``Understanding'' Benchmarks from The New Yorker Caption Contest — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runsAn Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs+3
864GPT-Neo 1.3B EleutherAI12±2local~7.1GB2khow to run › Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsPrompting Language Models for Linguistic Structure — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs+3
865T5 Small Google12±3local~1.8GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
866BLOOM 1.7B BigScience11±1local~3.1GB2kllama.cpp › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Crosslingual Generalization through Multitask Finetuning — authors' own runs+1
867Zyte-1B venkycs (aihub-app)11±2local~2.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
868LaMDA 2B (pre-trained) Google10±2†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs
869StableLM-Base-Alpha 7B Stability AI8±1local~35GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsEvalPlus / EvalPerf published evaluation runsInstructEval (DeCLaRe Lab) README results tableOpen Portuguese LLM Leaderboard (archived)
870Pythia 410M EleutherAI8±2local~1.8GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)
871GPT-3 ada (API base model) OpenAI8±1comparison onlyGPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsLarge Language Models Are Reasoning Teachers — authors' own runsEleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runsWinoDict: Probing language models for in-context word acquisition — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runs
872BLOOM 1.1B BigScience7±1local~2.6GB2kllama.cpp › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Crosslingual Generalization through Multitask Finetuning — authors' own runs
873GPT-2 XL (1.5B) OpenAI7±1local~2.8GB~75 tok/s1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsEleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runsTunable Soft Prompts are Messengers in Federated Learning — authors' own runs+9
874GPT-3 text-babbage-001 OpenAI4±2comparison onlyChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsDo Androids Laugh at Electric Sheep? Humor ``Understanding'' Benchmarks from The New Yorker Caption Contest — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runsAn Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs
875GPT-3 text-ada-001 OpenAI1±3comparison onlyChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runsDo Androids Laugh at Electric Sheep? Humor ``Understanding'' Benchmarks from The New Yorker Caption Contest — authors' own runsLMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runsAn Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs
876OPT-350M Meta1±1local~2.2GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsRevisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? — authors' own runsDo Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models — authors' own runs+1
877LaMDA 422M (pre-trained) Google1±3†never releasedChain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs
878StableLM-Base-Alpha 3B Stability AI1±2local~17GB4khow to run › Stability AI — original StableLM zero-shot evaluation outputsOpen Portuguese LLM Leaderboard (archived)
879BLOOM 560M BigScience0±1local~2.1GB2kllama.cpp › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsCrosslingual Generalization through Multitask Finetuning — authors' own runs+1
880Pythia 160M EleutherAI-2±2local~1.6GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)
881GPT-2 Large (774M) OpenAI-3±1local~2GB~137 tok/s1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsGenerative Calibration for In-context Learning — authors' own runs+3
882OPT-125M Meta-3±1local~1.8GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsPreserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsCan Language Models Understand Physical Concepts? — authors' own runs+1
883GPT-Neo 125M EleutherAI-3±2local~2.1GB2khow to run › Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsPrompting Language Models for Linguistic Structure — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluationsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)TruthfulQA: Measuring How Models Mimic Human Falsehoods — authors' own runs
884GPT-2 Medium (355M) OpenAI-5±1local~1.8GB1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsRankGen: Improving Text Generation with Large Ranking Models — authors' own runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluations+3
885GPT-2 Small (124M) OpenAI-13±2local~1.6GB1kLM Studio › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluationsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsLarge Language Models Are Reasoning Teachers — authors' own runs+3
886Pythia 70M EleutherAI-13±2local~1.6GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
—Grok Build 0.1 SpaceXAI (xAI)no reviewed resultsapi $1.25/M256kdocs.x.ai ↗
—K2 Horizon 7B Institute of Foundation Models (MBZUAI)not scored yetlocal~8GB524kllama.cpp ›
—LFM2.5 24B-A2B Liquid AInot yet calibratedlocal~17GB131kllama.cpp › Liquid AI own-model benchmark facts
—LFM3 1.7B Liquid AInot yet calibratedlocal~2.6GB262kllama.cpp › Liquid AI own-model benchmark facts
—Mistral Large 4 (October 29 weights) Mistralno reviewed resultsapi price unknown +local~2102GBdocs.mistral.ai ↗

+ 308 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

Includes 232 comparison-only models with no verified API or download · hide them

891 of 1230 catalogue models listed here · sources and licences

Models outside this leaderboard (31 models)

No current run option verified: GPT-3 Small (125M, Brown et al. 2020 paper model) · GPT-3 Medium (350M, Brown et al. 2020 paper model) · GPT-3 Large (760M, Brown et al. 2020 paper model) · GPT-3 XL (1.3B, Brown et al. 2020 paper model) · GPT-3 2.7B (Brown et al. 2020 paper model) · GPT-3 6.7B (Brown et al. 2020 paper model) · GPT-3 13B (Brown et al. 2020 paper model) · GPT-3 175B (Brown et al. 2020 paper model) · Jurassic-1 Jumbo (J1-Jumbo, 178B) · Jurassic-1 Large (J1-Large, 7.5B) · Gopher (280B) · Chinchilla (70B) · GLaM 0.1B/64E (MoE) · GLaM 1.7B/64E (MoE) · GLaM 8B/64E (MoE) · GLaM (64B/64E, 1.2T MoE) · GLaM dense 0.1B · GLaM dense 1.7B · GLaM dense 8B · GLaM dense 137B · Luminous Base (13B) · Luminous Extended (30B) · Luminous Supreme (70B) · Gopher family 417M · Gopher family 1.4B · Gopher family 7.1B · Gopher family 44M · Gopher family 117M

Raw results, insufficient overlap for an overall score: Recursive Flint · LFM3 VL 2B

No reviewed benchmark results yet: Ling 3.1 Flash 1019

* new or thinly tested: ranked on too few benchmark results · † no independently run results verified · ~RAM is estimated; full context may need more memory

891 models · 575 benchmarks · updated