whatmodel.sh · find the best AI models you can run

Best AI models for coding

A leaderboard of the best AI models for coding. Claude Opus 5.5 is #1 right now, followed by GPT-6 Astra and Claude Sonnet 5.5. Models are ranked on coding tests like HumanEval, SWE-bench and Aider. Filter by your computer or budget to find the best one you can use.

563 models · scores combine 59 benchmarks · updated how it works · find a leaderboard · benchmark your machine
task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]

Scores use reviewed coding evaluations only. Compare scores within this task. Models without task measurements are excluded.

Filter by your hardware and budget · reset
+ more
set[all sets][Arena WebDev (Code Arena) · 549][LiveCodeBench v5 (window unspecified) · 7][HumanEval · 6][BigCodeBench · 4][BigCodeBench · 4][HumanEval · 3][LiveCodeBench (version unspecified) · 3][LiveCodeBench 2024-08–2025-05 (version unstated) · 2][HumanEval · 2][HumanEval+ · 2][LiveCodeBench 2024-08–2025-05 (version unstated) · 2][HumanEval · 2][HumanEval · 2][Tulu 2: CodexEval pass@10 · 2][HumanEval · 2]

Filter the model list while keeping task scores. Models without a shared score scale are not yet calibrated; select a set to see its own scores.

Detect your exact chip or graphics card:

curl -G whatmodel.sh/ -d max=0 --data-urlencode "mac=$(sysctl -n hw.memsize machdep.cpu.brand_string)"Mac

curl -G whatmodel.sh/ -d max=0 --data-urlencode "ram=$(awk '/MemT/{print $2}' /proc/meminfo)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null)"Linux

curl.exe -G whatmodel.sh/ -d max=0 --data-urlencode "ram=$((gcim Win32_ComputerSystem).TotalPhysicalMemory)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>$null)"Windows

$ terminal

detecting your OS…

Run curl whatmodel.shto get this list in your terminal (add &format=json for agents).

ram[8][16][24][32][48][64][96][128][192][256]GB
gpu[apple silicon][graphics card][none]
price[local][$0.02][$0.1][$0.5][$2][$10][any]max per M

detecting your OS…

Score vs API price

#modeltask score↓where↓speed↓RAM↓context↓get itsources
#modeltask score↓where↓speed↓RAM↓context↓get itsources
1Claude Opus 5.5 Anthropic88±1api $8.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
2GPT-6 Astra OpenAI86±1api $20.00/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
3Claude Sonnet 5.5 Anthropic86±1api $4.00/M1Mplatform.claude.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
4GPT-6.1 Sol OpenAI84±1*api $4.00/M1.1Mdevelopers.openai.com ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
5Claude Fable 5.1 Anthropic84±1api $20.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
6Claude Opus 5 Anthropic81±1api $10.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
7GPT-6 Sol OpenAI81±1api $4.00/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
8Gemini 4 Argon (limited preview) Google80±1*limited previewLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)
9Claude Mythos Preview (limited preview) Anthropic80±2†limited preview1MAnthropic system cards and model announcements (lab-reported scores)
10Qwen3.8 Max (2.4T-A95B) Alibaba80±1api $3.00/M +local~380GB262kdeepinfra.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
11Qwen3.8 Max (0902) Alibaba80±1*api $3.00/M1Mwww.alibabacloud.com ↗LMArena leaderboard dataset
12Muse Spark 1.3 Meta79±1*api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
13Kimi K3 Moonshot AI79±1api $6.00/M +local~884GB1Mplatform.moonshot.ai ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
14Claude Fable 5 Anthropic78±1api $20.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
15Grok 4.7 SpaceXAI (xAI)78±1*api $3.00/M500kdocs.x.ai ↗LMArena leaderboard dataset
16MiMo-V2.6-Flash (309B-A15B) Xiaomi78±1*api $0.17/M +local~136GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
17Qwen3.8-Flash-Next Alibaba78±1local~59GB262kllama.cpp › LMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
18Hy4 preview (770B-A49B) Tencent77±1api $1.25/M +local~248GB1Mcloud.tencent.com ↗LMArena leaderboard datasetTencent Hy4 preview model card
19GPT-5.6 Sol OpenAI77±1api $8.00/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
20Grok 4.6 SpaceXAI (xAI)77±1*api $3.00/M500kdocs.x.ai ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
21GLM-5.3 Z.ai77±1*api $2.15/M +local~239GB1Mdocs.z.ai ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
22DeepSeek V4.1 Flash DeepSeek77±1*api $0.26/M +local~537GB1Mapi-docs.deepseek.com ↗LMArena leaderboard datasetdeepseek-ai/DeepSeek-V4.1-Flash model card
23MiMo-V2.6-Pro Xiaomi77±1*api $0.54/M +local~443GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
24GLM-5.3-Flash Z.ai76±1api $0.24/M +local~103GB1Mdocs.z.ai ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodezai-org/GLM-5.3-Flash model card
25Qwen3.8-Omni-Flash Alibaba76±3†api $0.23/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
26GLM-5.2 Z.ai75±1api $2.15/M +local~121GB1Mdocs.z.ai ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Epoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)zai-org/GLM-5.2 model card+1
27Gemini 3.7 Flash Google75±1api $1.50/M1Mai.google.dev ↗LMArena leaderboard datasetGoogle Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeGoogle DeepMind Gemini model page and model cards (lab-reported scores)
28DeepSeek V4 Pro (0813) DeepSeek75±1api $1.63/M +local~1010GB1Mdeepinfra.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodedeepseek-ai/DeepSeek-V4.1-Flash model card
29Gemini 3.8 Flash Google75±1api $1.50/M1Mai.google.dev ↗LMArena leaderboard datasetGoogle Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google DeepMind Gemini model page and model cards (lab-reported scores)
30Claude Haiku 5.5 Anthropic75±1api $0.20/M1Mplatform.claude.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)
31Qwen3.8 27B Alibaba75±1api $1.60/M +local~18GB262kconsole.groq.com ↗LMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen/Qwen3.8-27B model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
32GPT-6 Luna OpenAI75±1api $0.20/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
33DeepSeek V4 Flash (0731) DeepSeek75±1api $0.11/M +local~92GB1Mdeepinfra.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodedeepseek-ai/DeepSeek-V4.1-Flash model carddeepseek-ai/DeepSeek-V4-Flash-Vision-Exp model card
34Step 5 Preview StepFun74±1*comparison onlyLMArena leaderboard dataset
35Claude Opus 4.8 Anthropic74±1api $10.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
36Claude Opus 4.7 Anthropic73±1api $10.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsSWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)
37Grok 4.5 SpaceXAI (xAI)73±1*api $3.00/M500kdocs.x.ai ↗LMArena leaderboard datasetRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
38Claude Opus 4.6 Anthropic73±1api $10.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
39Muse Spark 1.1 Meta72±1*api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard dataset
40Mistral Large 4 (public preview) Mistral72±1*api $2.06/M1Mdocs.mistral.ai ↗LMArena leaderboard dataset
41Claude Sonnet 5 Anthropic72±1api $4.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
42Gemini 3.6 Flash Google72±1*api $1.50/M1Mai.google.dev ↗LMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
43Muse Spark 1.2 Meta72±1*api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard dataset
44GPT-5.6 Terra OpenAI72±1api $4.50/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
45GPT-5.6 Luna OpenAI71±1api $0.45/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
46Claude Sonnet 4.6 Anthropic71±1api $6.00/M1Mdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)
47GPT-5.5 OpenAI71±1api $11.25/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
48Qwen3.7 Max Alibaba71±1*api $3.75/M1Mdeepinfra.com ↗LMArena leaderboard datasetEpoch AI — internal evaluations
49Qwen3.7 Plus Alibaba71±2†api $0.70/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen/Qwen3.8-27B model card
50Kimi K2.6 Moonshot AI71±1api $1.71/M +local~317GB262kplatform.moonshot.ai ↗LMArena leaderboard datasetNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
51Hunyuan Hy3 (295B-A21B) Tencent71±1api $0.23/M +local~96GB256kdeepinfra.com ↗LMArena leaderboard datasettencent/Hy3 model cardTencent Hy4 preview model card
52GPT-5.3-Codex OpenAI70±2api $4.81/M400kplatform.openai.com ↗Epoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
53Gemini 3.5 Flash Google70±1*api $3.38/M1Mai.google.dev ↗LMArena leaderboard datasetEpoch AI — internal evaluations
54GLM-5.1 Z.ai70±1api $2.15/M +local~119GB200kdocs.z.ai ↗LMArena leaderboard datasetNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1zai-org/GLM-5.1 model cardLGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card+1
55Claude Opus 4.5 Anthropic70±1api $10.00/M200kdocs.anthropic.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)Qwen/Qwen3.6-27B model card
56Gemini 2.5 Pro Experimental (03-25) Google70±5*comparison onlyLiveCodeBench official website results (2025-08-01)
57LongCat-Flash-Thinking (560B-A27B) Meituan70±7*†comparison only128kMeituan LongCat-Flash-Thinking model card (evaluation table)
58Qwen3.6 Max Preview Alibaba69±1*api $2.92/M256kwww.alibabacloud.com ↗LMArena leaderboard datasetEpoch AI — internal evaluations
59MiniMax-M3 MiniMax69±1*api $0.48/M +local~137GB1Mdeepinfra.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)
60Kimi K2.7 Code Moonshot AI69±1*api $1.71/M +local~359GB262kplatform.kimi.ai ↗LMArena leaderboard dataset
61MiMo-V2.5-Pro (1T-A42B) Xiaomi69±1api $0.65/M +local~163GB1Mnovita.ai ↗LMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Xiaomi MiMo official model cards and documentation
62GPT-5.4 OpenAI69±1api $5.63/M1.1Mplatform.openai.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsSWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
63DeepSeek V4 Pro (Preview, Apr 2026) DeepSeek69±1local~604GBllama.cpp › LMArena leaderboard datasetNVIDIA — Nemotron 3 Ultra technical reportEpoch AI — internal evaluationsGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)
64Ornith 1.5 35B-A3B DeepReinforce68±3†local~22GB262kLM Studio › ornith-ai/Ornith-1.5-35B-A3B model card
65Solar Open 2 (250B-A15B) Upstage68±4*†local~527GB1Mhow to run › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)
66Gemini 3.1 Pro (Preview) Google67±1api $4.50/M1Mai.google.dev ↗LMArena leaderboard datasetVibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
67Qwen3-Max (2026-01-23) Alibaba67±3†comparison onlyQwen/Qwen3.5-397B-A17B model card
68Qwen3.6 Plus Alibaba67±1*api $1.13/M1Mdocs.together.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluations
69KAT-Coder-V2.5-Dev (35B-A3B) Kuaishou (Kwaipilot)67±3†local~22GB262kLM Studio › Kwaipilot/KAT-Coder-V2.5-Dev model card
70Gemini 3.5 Flash-Lite Google67±1*api $0.85/M1Mai.google.dev ↗LMArena leaderboard dataset
71MiMo-V2.5 (310B-A15B) Xiaomi67±1api $0.21/M +local~100GB1Mnovita.ai ↗LMArena leaderboard datasetSolar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)XiaomiMiMo/MiMo-V2.5 model card
72Gemini 3 Pro (Preview) Google67±1comparison only1MLMArena leaderboard datasetEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)
73Gemini 3 Flash (Preview) Google67±1api $1.13/M1Mai.google.dev ↗LMArena leaderboard datasetEpoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)
74o3-pro OpenAI67±2*api $35.00/M200kdevelopers.openai.com ↗Aider polyglot benchmark runs
75GLM-5 Z.ai67±1api $1.55/M +local~119GB205kdocs.z.ai ↗LMArena leaderboard datasetEpoch AI — internal evaluationszai-org/GLM-5 model cardzai-org/GLM-5.1 model cardVibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)
76MiMo-V2-Pro Xiaomi67±1comparison onlyLMArena leaderboard datasetXiaomiMiMo/MiMo-V2.5-Pro model cardXiaomiMiMo/MiMo-V2.5 model card
77DeepSeek V4 Flash (Preview, Apr 2026) DeepSeek67±1local~103GBllama.cpp › LMArena leaderboard datasetNVIDIA — Nemotron 3 Ultra technical reportai-sage/GigaChat3.5-432B-A28B-Reasoning model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
78DeepSeek V4 Flash Vision (Exp) DeepSeek67±8*†local~104GB1Mllama.cpp › deepseek-ai/DeepSeek-V4-Flash-Vision-Exp model card
79Kimi K2.5 Moonshot AI67±1api $1.20/M +local~160GB262kdocs.aws.amazon.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)
80GLM-4.7 Z.ai67±1api $1.00/M +local~58GB205kdocs.z.ai ↗LMArena leaderboard datasetzai-org/GLM-4.7 model cardzai-org/GLM-5 model card
81GPT-5 OpenAI66±1api $3.44/M400kplatform.openai.com ↗LMArena leaderboard datasetAider polyglot benchmark runsEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
82A.X K2 (688B-A33B) SK Telecom66±4*†local~1454GB262khow to run › A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
83Llama 3.1 Nemotron Ultra 253B v1 NVIDIA66±5*local~82GB128kllama.cpp › LiveCodeBench official website results (2025-08-01)
84Qwen3.6 27B Alibaba66±2api $1.20/M +local~18GB262kconsole.groq.com ↗Qwen/Qwen3.6-27B model cardQwen/Qwen3.8-27B model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
85GPT-5.2 OpenAI66±1api $4.81/M400kplatform.openai.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsVibe Code Bench (Vals AI, arXiv:2603.04601v3)
86Gemini 2.5 Pro Preview (06-05) Google66±2comparison onlyAider polyglot benchmark runsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)
87Inkling (975B-A41B) Thinking Machines65±1api $1.73/M +local~156GB1Mdeepinfra.com ↗LMArena leaderboard datasetthinkingmachines/Inkling-Small model card
88Inkling Small (266B-A12B) Thinking Machines65±1api $0.64/M +local~86GB1Mdeepinfra.com ↗LMArena leaderboard datasetthinkingmachines/Inkling-Small model card
89Qwen3.5 397B-A17B Alibaba65±1api $1.09/M +local~65GB262kdeepinfra.com ↗LMArena leaderboard datasetNVIDIA — Nemotron 3 Ultra technical reportQwen/Qwen3.6-27B model cardQwen/Qwen3.5-397B-A17B model cardA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
90GPT-5.4 nano OpenAI65±4*api $0.46/M400kplatform.openai.com ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
91Grok 4 (0709) SpaceXAI (xAI)65±2*comparison onlyAider polyglot benchmark runs
92Nemotron 3 Ultra 550B-A55B NVIDIA65±2api $1.35/M +local~89GB1Mdocs.together.ai ↗NVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
93GLM-5V-Turbo Z.ai65±1*api $1.90/M200kwww.siliconflow.com ↗LMArena leaderboard dataset
94GPT-5.4 mini OpenAI65±1*api $1.69/M400kplatform.openai.com ↗LMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
95Claude Sonnet 4.5 Anthropic65±1api $6.00/M200kdocs.anthropic.com ↗LMArena leaderboard datasetServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)
96Qwen3 235B-A22B Thinking 2507 Alibaba65±3api $0.97/M +local~76GB131knovita.ai ↗Qwen/Qwen3.5-27B model cardQwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen3-VL-235B-A22B-Thinking model cardMeituan LongCat-Flash-Thinking model card (evaluation table)
97MiniMax-M2.7 MiniMax65±1api $0.53/M +local~75GB205kplatform.minimax.io ↗LMArena leaderboard datasetNVIDIA — Nemotron 3 Ultra technical reportA.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
98Qwen3.6 35B-A3B Alibaba64±1api $0.31/M +local~23GB262kdeepinfra.com ↗Qwen/Qwen3.6-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3.6-35B-A3B model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
99GPT-5.1 OpenAI64±1api $3.44/M400kplatform.openai.com ↗LMArena leaderboard datasetEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)
100Claude Opus 4.1 Anthropic64±1comparison onlyLMArena leaderboard datasetEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)
101K-EXAONE 2.0 (750B-A37B) LG AI Research64±4*†local~791GB262khow to run › LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card
102o4-mini OpenAI64±1api $1.93/M200kplatform.openai.com ↗Aider polyglot benchmark runsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI simple-evals README benchmark results (OpenAI's own models)
103o3 OpenAI64±1api $3.50/M200kplatform.openai.com ↗Aider polyglot benchmark runsEpoch AI — internal evaluationsLiveCodeBench official website results (2025-08-01)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardMeituan LongCat-Flash-Thinking model card (evaluation table)OpenAI simple-evals README benchmark results (OpenAI's own models)
104Gemini 2.5 Pro Preview (05-06) Google64±2comparison onlyAider polyglot benchmark runsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)
105Step 3.5 Flash (196B-A11B) StepFun64±5*†api $0.15/M +local~118GB256kplatform.stepfun.ai ↗stepfun-ai/Step-3.5-Flash model card
106MiniMax-M2.1 MiniMax64±1*local~79GBllama.cpp › LMArena leaderboard dataset
107MiniMax-M2.5 MiniMax64±1api $0.53/M +local~75GB205kplatform.minimax.io ↗LMArena leaderboard datasetVibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)
108Nemotron Cascade 2 30B-A3B NVIDIA63±2†local~20GB256kLM Studio › NVIDIA — Nemotron-Cascade 2 technical report
109Grok 4.20 (0309 reasoning) SpaceXAI (xAI)63±1*api $1.56/M1Mdocs.x.ai ↗LMArena leaderboard dataset
110Qwen3.5 27B Alibaba63±1api $0.84/M +local~18GB262kdeepinfra.com ↗LMArena leaderboard datasetQwen/Qwen3.6-27B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.5-27B model cardQwen/Qwen3.5-122B-A10B model card
111Claude Opus 4 Anthropic63±1comparison onlyAider polyglot benchmark runsEpoch AI — internal evaluationsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)
112Solar Pro 4 Upstage63±1*api $0.53/M524kdevelopers.upstage.ai ↗LMArena leaderboard dataset
113Ornith 1.0 35B (35B-A3B) DeepReinforce63±2local~23GB262kLM Studio › Kwaipilot/KAT-Coder-V2.5-Dev model cardornith-ai/Ornith-1.5-35B-A3B model card
114DeepSeek V3.1 Terminus DeepSeek63±2†local~230GB164kllama.cpp › DeepSeek-V3.1-Terminus model card
115Qwen3.5 122B-A10B Alibaba63±1api $1.10/M +local~75GB262kwww.alibabacloud.com ↗LMArena leaderboard datasetQwen/Qwen3.5-27B model cardQwen/Qwen3.5-122B-A10B model card
116Muse Glimmer 30B Meta63±1api $0.64/M +local~19GB131kfireworks.ai ↗LMArena leaderboard datasetmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.8-27B model card
117Hy3 preview (295B-A21B) Tencent63±1local~629GBhow to run › LMArena leaderboard datasettencent/Hy3 model card
118Laguna XS 2.1 (33B-A3B) Poolside63±3†local~21GB262kLM Studio › Poolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)
119Gemma 4 31B Google62±1api $0.19/M +local~21GB262kdeepinfra.com ↗LMArena leaderboard datasetKwaipilot/KAT-Coder-V2.5-Dev model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Gemma Team — Gemma 4 Technical ReportRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.6-27B model card+1
120DeepSeek V3.1 DeepSeek62±1api $0.42/M +local~110GB131kdeepinfra.com ↗deepseek-ai/DeepSeek-V3.1 model cardDeepSeek-V3.1-Terminus model cardMeituan LongCat-Flash-Chat model card (evaluation table)Meituan LongCat-Flash-Thinking model card (evaluation table)
121Gemini 2.5 Pro Preview (03-25) Google62±2*comparison onlyAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)
122GPT-5 mini OpenAI62±1api $0.69/M400kplatform.openai.com ↗ServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluationsVisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
123Grok 4.3 SpaceXAI (xAI)62±1*api $1.56/M1Mdocs.x.ai ↗LMArena leaderboard dataset
124DeepSeek V3.2 DeepSeek62±1api $0.29/M +local~110GB128kdeepinfra.com ↗LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)
125Laguna M.1 Poolside62±1api price unknown +local~135GB262khuggingface.co ↗LMArena leaderboard datasetPoolside — Laguna M.1/XS.2 Technical Report
126GigaChat 3.5 Ultra Reasoning (432B-A28B) Sber62±3*†local~461GB262khow to run › ai-sage/GigaChat3.5-432B-A28B-Reasoning model card
127DeepSeek R1 0528 DeepSeek62±1api $0.91/M +local~110GB164kdeepinfra.com ↗Aider polyglot benchmark runsServiceNow-AI/Apriel-1.6-15b-Thinker model carddeepseek-ai/DeepSeek-V3.1 model carddeepseek-ai/DeepSeek-R1-0528 model cardWebApp1K models leaderboard (onekq-ai)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode+2
128GPT-5.2-Codex OpenAI62±1comparison onlyLMArena leaderboard datasetVibe Code Bench (Vals AI, arXiv:2603.04601v3)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
129o1 (2024-12-17) OpenAI62±1api $26.25/M200kdevelopers.openai.com ↗Aider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
130Nemotron 3 Super 120B-A12B NVIDIA61±2†api $0.28/M +local~73GB262kdocs.aws.amazon.com ↗NVIDIA — Nemotron-Cascade 2 technical reportNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
131GLM-4.6 Z.ai61±1api $1.00/M +local~58GB205kdocs.z.ai ↗LMArena leaderboard datasetzai-org/GLM-4.7 model cardzai-org/GLM-4.6 model cardWebApp1K models leaderboard (onekq-ai)
132Claude Haiku 4.5 Anthropic61±1api $2.00/M200kdocs.anthropic.com ↗LMArena leaderboard datasetAnthropic system cards and model announcements (lab-reported scores)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeVibe Code Bench (Vals AI, arXiv:2603.04601v3)
133Gemma 4 26B-A4B Google61±1api $0.14/M +local~18GB262kdeepinfra.com ↗LMArena leaderboard datasetKwaipilot/KAT-Coder-V2.5-Dev model cardGemma Team — Gemma 4 Technical ReportRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcodegoogle/diffusiongemma-26B-A4B-it model cardQwen/Qwen3.6-35B-A3B model card
134Claude 3.7 Sonnet Anthropic61±1comparison onlyAider polyglot benchmark runsEpoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
135MiMo-V2.6-Distill-Qwen-9B Xiaomi61±3*†local~7.8GBLM Studio › XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B model card
136Seed-OSS 36B Instruct ByteDance Seed61±3†local~23GB524kLM Studio › ByteDance-Seed/Seed-OSS-36B-Instruct model card
137o3-mini OpenAI61±1api $1.93/M200kdevelopers.openai.com ↗Aider polyglot benchmark runsServiceNow-AI/Apriel-1.6-15b-Thinker model cardSciCode leaderboard (repository README)WebApp1K models leaderboard (onekq-ai)LiveCodeBench official website results (2025-08-01)Vector Institute Evaluation Leaderboard (Inspect Evals runs)+3
138Kimi K2 Thinking Moonshot AI60±1*api $1.07/M +local~160GB262kdocs.aws.amazon.com ↗LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)
139K-EXAONE (236B-A23B) LG AI Research60±3†local~152GB262kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card
140Claude Sonnet 4 Anthropic60±1comparison onlyAider polyglot benchmark runsMeituan LongCat-Flash-Chat model card (evaluation table)LiveCodeBench official website results (2025-08-01)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
141Nemotron 3.5 Lightning 30B-A3B NVIDIA60±5*api $0.087/M +local~20GB262kfireworks.ai ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
142Gemma 4 12B Google60±3local~8.5GB262kLM Studio › Gemma Team — Gemma 4 Technical ReportXHToken/Spark-X2.5-4B model card
143Laguna XS.2 Poolside60±1local~21GB262kLM Studio › LMArena leaderboard datasetPoolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)Poolside — Laguna M.1/XS.2 Technical Report
144Devstral Small 2 24B Mistral59±4*†local~17GB262kllama.cpp › mistralai/Devstral-Small-2-24B-Instruct-2512 model card
145Qwen3-VL 235B-A22B Thinking Alibaba59±6*†local~111GB256kLM Studio › Qwen3-VL-235B-A22B-Thinking model card
146Nemotron-Labs-3 Puzzle 75B-A9B NVIDIA59±4*†local~35GBllama.cpp › NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
147Gemini 2.5 Flash Preview (05-20) Google59±1comparison onlyAider polyglot benchmark runsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)
148Kimi K2 Instruct (0711) Moonshot AI59±1api $1.00/M +local~160GB131knovita.ai ↗Aider polyglot benchmark runsMeituan LongCat-Flash-Chat model card (evaluation table)
149MiniMax-M2 MiniMax59±1*api $0.53/M +local~75GB205kdocs.aws.amazon.com ↗LMArena leaderboard dataset
150DeepSeek V3.2 Exp DeepSeek59±1local~726GBhow to run › LMArena leaderboard datasetAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
151o1-preview OpenAI59±1comparison onlyEvalPlus / EvalPerf published evaluation runsLiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)SciCode leaderboard (repository README)WebApp1K models leaderboard (onekq-ai)WildEval ZeroEval published results+1
152MiMo-V2-Flash (309B-A15B) Xiaomi59±1*local~108GBllama.cpp › LMArena leaderboard datasetstepfun-ai/Step-3.5-Flash model card
153Spark-X2.5 4B iFLYTEK (SparkLLM)59±2†local~3.9GB1MLM Studio › XHToken/Spark-X2.5-4B model card
154DeepSeek R1 (Jan '25) DeepSeek59±1api $4.00/M +local~110GB128kdocs.together.ai ↗Aider polyglot benchmark runsdeepseek-ai/DeepSeek-R1-0528 model carddeepseek-ai/DeepSeek-R1 model cardSciCode leaderboard (repository README)WebApp1K models leaderboard (onekq-ai)Vector Institute Evaluation Leaderboard (Inspect Evals runs)+2
155Grok 3 Beta SpaceXAI (xAI)59±1*comparison onlyAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)
156Claude 3.5 Sonnet (2024-10-22) Anthropic58±1comparison onlyAider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)SciCode leaderboard (repository README)Vector Institute Evaluation Leaderboard (Inspect Evals runs)LiveCodeBench official website results (2025-08-01)WildEval ZeroEval published results+2
157Qwen3.5 Plus Alibaba58±6*api $0.90/M1Mwww.alibabacloud.com ↗Vibe Code Bench (Vals AI, arXiv:2603.04601v3)
158Grok 3 Mini Beta SpaceXAI (xAI)58±1*comparison onlyAider polyglot benchmark runsLiveCodeBench official website results (2025-08-01)
159Qwen3-Next 80B-A3B Thinking Alibaba58±4*†api $0.41/M +local~27GB131kwww.alibabacloud.com ↗Qwen/Qwen3.5-4B model cardQwen/Qwen3-Next-80B-A3B-Thinking model card
160Qwen3.5 35B-A3B Alibaba58±1api $0.35/M +local~23GB262kdeepinfra.com ↗LMArena leaderboard datasetNVIDIA — Nemotron-Cascade 2 technical reportKwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.5-27B model cardQwen/Qwen3.5-122B-A10B model card
161Qwen3-Coder 480B-A35B Instruct Alibaba58±1api $0.67/M +local~78GB262knovita.ai ↗LMArena leaderboard datasetQwen/Qwen3-Coder-30B-A3B-Instruct model cardWebApp1K models leaderboard (onekq-ai)
162GPT-4.1 OpenAI58±1api $3.50/M1Mplatform.openai.com ↗Aider polyglot benchmark runsMeituan LongCat-Flash-Chat model card (evaluation table)Epoch AI — internal evaluationsWebApp1K models leaderboard (onekq-ai)OpenAI simple-evals README benchmark results (OpenAI's own models)
163Phi-4-reasoning 14B Microsoft58±3*†api $0.22/M +local~10GB33kazure.microsoft.com ↗Microsoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model card
164DeepSeek V3 0324 DeepSeek58±1local~108GB164kllama.cpp › Aider polyglot benchmark runsdeepseek-ai/DeepSeek-V3.1 model cardWebApp1K models leaderboard (onekq-ai)deepseek-ai/DeepSeek-V3-0324 model card
165GLM-4.5 Z.ai58±2api $1.00/M +local~58GB131kdocs.z.ai ↗zai-org/GLM-4.6 model cardGLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)Meituan LongCat-Flash-Thinking model card (evaluation table)
166Gemini 2.5 Flash Preview (04-17) Google58±1comparison onlyAider polyglot benchmark runsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)
167Granite 4.2 30B IBM58±2†api $0.28/M +local~19GB131kdeepinfra.com ↗ibm-granite/granite-4.2-30b model card
168Ministral 3 8B Reasoning 2512 Mistral58±6*†local~6.5GB262kLM Studio › mistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model card
169Mistral Medium 3.5 (128B) Mistral57±1api $3.00/M +local~42GB262kdocs.mistral.ai ↗LMArena leaderboard datasetSolar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)
170Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba57±1api $1.23/M +local~76GB131kwww.alibabacloud.com ↗Aider polyglot benchmark runsQwen/Qwen3-30B-A3B-Instruct-2507 model cardLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)Qwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model card+1
171o1-mini OpenAI57±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)Aider code editing benchmark runs (old edit leaderboard)WebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)+3
172DeepSeek V3 (Dec '24) DeepSeek57±1api $0.46/M +local~337GB131kdeepinfra.com ↗Cohere — Command A technical reportAider polyglot benchmark runsLiveCodeBench official website results (2024-12-26)deepseek-ai/DeepSeek-R1 model cardWebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)+1
173A.X K1 (519B-A33B) SK Telecom57±4*†local~1091GBhow to run › A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v1
174Command A+ (05-2026) Cohere57±3api price unknown +local~71GB128kdocs.cohere.com ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Cohere vendor-reported benchmark scores for its own models (cohere.com blog)
175INTELLECT-3 Prime Intellect57±6*†local~48GBllama.cpp › Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
176MiniCPM5 2B OpenBMB57±2†local~3GB131kLM Studio › openbmb/MiniCPM5-2B model card
177GPT-4.5 Preview OpenAI57±1comparison onlyAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)OpenAI simple-evals README benchmark results (OpenAI's own models)
178QwQ-32B-Preview Alibaba57±4*local~21GB33kLM Studio › LiveCodeBench official website results (2025-08-01)
179ChatGPT-4o (2025-03-26) OpenAI57±1*comparison onlyAider polyglot benchmark runs
180Gemini 3.1 Flash-Lite Google57±1*api $0.56/M1Mai.google.dev ↗LMArena leaderboard dataset
181gpt-oss-120b OpenAI57±1api $0.26/M +local~67GB131kconsole.groq.com ↗Aider polyglot benchmark runsServiceNow-AI/Apriel-1.6-15b-Thinker model cardOpenAI — gpt-oss-120b & gpt-oss-20b Model CardWebApp1K models leaderboard (onekq-ai)Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
182Qwen3 235B-A22B Instruct 2507 Alibaba57±1api $0.20/M +local~76GB262kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardMeituan LongCat-Flash-Chat model card (evaluation table)Qwen3-VL-235B-A22B-Instruct model cardQwen/Qwen3-235B-A22B-Instruct-2507 model card
183LongCat-Flash-Chat (560B-A27B) Meituan56±2†local~333GB128kLM Studio › Meituan LongCat-Flash-Chat model card (evaluation table)
184Kimi K2 Instruct (0905) Moonshot AI56±4*local~298GBllama.cpp › WebApp1K models leaderboard (onekq-ai)
185GLM-4.5-Air Z.ai56±3api $0.42/M +local~35GB131kdocs.z.ai ↗GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportPrime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)
186Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)56±1api $0.28/M128kazure.microsoft.com ↗LMArena leaderboard datasetWebApp1K models leaderboard (onekq-ai)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)
187Qwen3.5 Flash Alibaba56±1*api $0.17/M1Mwww.alibabacloud.com ↗LMArena leaderboard dataset
188Gemini 2.5 Pro Google56±1api $3.44/M1Mai.google.dev ↗LMArena leaderboard datasetEpoch AI — internal evaluationsRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeMeituan LongCat-Flash-Thinking model card (evaluation table)
189Nemotron-Cascade-14B-Thinking NVIDIA56±1†local~10GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
190Mellum2.1 12B-A2.5B Thinking JetBrains56±3†local~8.3GBLM Studio › JetBrains/Mellum2.1-12B-A2.5B-Thinking model card
191gpt-oss-20b OpenAI56±2api $0.13/M +local~14GB131kconsole.groq.com ↗OpenAI — gpt-oss-120b & gpt-oss-20b Model CardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeByteDance-Seed/Seed-OSS-36B-Instruct model card
192Qwen3-Next 80B-A3B Instruct Alibaba56±2†api $0.34/M +local~27GB131kdeepinfra.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model card
193Trinity-Large-Thinking Arcee AI56±1*api $0.39/M +local~141GB512kdocs.arcee.ai ↗LMArena leaderboard dataset
194Mistral Large 3 (675B-A41B) Mistral55±1*api $0.75/M +local~108GB262kdocs.mistral.ai ↗LMArena leaderboard dataset
195Gemini 2.5 Flash Preview (09-2025) Google55±2*comparison onlyServiceNow-AI/Apriel-1.6-15b-Thinker model card
196Qwen3-VL 235B-A22B Instruct Alibaba55±4*†local~134GB256kLM Studio › Qwen3-VL-235B-A22B-Instruct model card
197Phi-4-reasoning-plus 14B Microsoft55±2local~10GBLM Studio › Microsoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
198Qwen3-Coder-Next (80B-A3B) Alibaba55±3*api $0.60/M +local~27GB262kwww.alibabacloud.com ↗Vector Institute Evaluation Leaderboard (Inspect Evals runs)
199Gemini 2.0 Pro Experimental (02-05) Google55±2*comparison onlyAider polyglot benchmark runs
200Mistral Medium 3 (2505) Mistral55±4*comparison onlyWebApp1K models leaderboard (onekq-ai)
201Qwen3-VL 32B Thinking Alibaba55±3*†api $0.28/M +local~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
202GPT-4.1 mini OpenAI55±2*api $0.70/M1Mplatform.openai.com ↗Aider polyglot benchmark runsOpenAI simple-evals README benchmark results (OpenAI's own models)
203Apriel 1.6 15B Thinker ServiceNow55±2*†local~10GBLM Studio › ServiceNow-AI/Apriel-1.6-15b-Thinker model card
204OpenReasoning-Nemotron-32B NVIDIA54±2†local~21GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
205OpenThinker 32B Open Thoughts54±9*†local~21GB33kLM Studio › open-thoughts/OpenThinker-32B model card
206Devstral 2 (2512) Mistral54±1local~75GB256kLM Studio › LMArena leaderboard datasetmistralai/Devstral-Small-2-24B-Instruct-2512 model card
207Qwen3-VL 30B-A3B Thinking Alibaba54±3*†api $0.75/M +local~21GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
208Qwen3.5 9B Alibaba54±1api $0.11/M +local~7.4GB262kdeepinfra.com ↗JetBrains/Mellum2.1-12B-A2.5B-Thinking model cardXiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardXHToken/Spark-X2.5-4B model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau LeetcodeQwen/Qwen3.5-4B model card+3
209Qwen3 30B-A3B Thinking 2507 Alibaba54±3api $0.75/M +local~20GB262kwww.alibabacloud.com ↗ByteDance-Seed/Seed-OSS-36B-Instruct model cardQwen/Qwen3.5-4B model cardQwen/Qwen3-30B-A3B-Thinking-2507 model card
210Command A Reasoning (08-2025) Cohere54±7*†local~34GBllama.cpp › Cohere vendor-reported benchmark scores for its own models (cohere.com blog)
211DiffusionGemma 26B-A4B Google54±5*†comparison only262kgoogle/diffusiongemma-26B-A4B-it model card
212Qwen2.5 Coder 32B Instruct Alibaba53±1api $0.74/M +local~21GB131kdevelopers.cloudflare.com ↗EvalPlus / EvalPerf published evaluation runsAider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiveCodeBench official website results (2024-12-26)+2
213GPT-4o (2024-05-13) OpenAI53±1api $7.50/M128kdevelopers.openai.com ↗Aider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published resultsMicrosoft — Phi-4-reasoning Technical ReportWebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)+2
214GPT-4o (2024-08-06) OpenAI53±1api $4.38/M128kdevelopers.openai.com ↗Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Vector Institute Evaluation Leaderboard (Inspect Evals runs)WildEval ZeroEval published resultsLiveCodeBench official website results (2025-08-01)WebApp1K models leaderboard (onekq-ai)+1
215Claude 3.5 Haiku Anthropic53±1comparison onlyAider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
216GPT-4 Turbo (2024-04-09) OpenAI53±1api $15.00/M128kdevelopers.openai.com ↗EvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published resultsLiveCodeBench official website results (2025-08-01)SciCode leaderboard (repository README)OpenAI simple-evals README benchmark results (OpenAI's own models)
217DeepSeek-V2.5 DeepSeek53±1local~151GB164kOllama › Aider code editing benchmark runs (old edit leaderboard)WebApp1K models leaderboard (onekq-ai)WildEval ZeroEval published results
218Qwen3 32B Alibaba53±1api $0.13/M +local~20GB131kdeepinfra.com ↗Aider polyglot benchmark runsQwen/Qwen3-Next-80B-A3B-Instruct model cardByteDance-Seed/Seed-OSS-36B-Instruct model cardFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingWebApp1K models leaderboard (onekq-ai)Qwen/Qwen3-Next-80B-A3B-Thinking model card
219Granite 4.2 8B IBM53±2†api $0.11/M +local~6.6GB131kwandb.ai ↗ibm-granite/granite-4.2-30b model card
220Claude 3.5 Sonnet (2024-06-20) Anthropic53±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)WebApp1K models leaderboard (onekq-ai)WildEval ZeroEval published resultsSciCode leaderboard (repository README)
221DeepSeek-Coder-V2 Instruct (0724) DeepSeek53±2*local~151GBllama.cpp › Aider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
222Nemotron-Cascade-8B NVIDIA53±2†local~6.3GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
223Granite 4.1 8B IBM53±1api $0.11/M +local~7GB131kreplicate.com ↗LMArena leaderboard datasetibm-granite/granite-4.1-8b model card
224Nemotron-Cascade-8B-Thinking NVIDIA53±2†local~6.3GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
225Ministral 3 3B Reasoning 2512 Mistral52±6*†local~3.5GB262kLM Studio › mistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model card
226Grok 2 (1212) SpaceXAI (xAI)52±3*comparison onlyWildEval ZeroEval published results
227Qwen2.5 32B Instruct Alibaba52±1local~21GB131kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results
228Qwen3 30B-A3B Instruct 2507 Alibaba52±1†api $0.15/M +local~20GB262ktokenfactory.nebius.com ↗Qwen/Qwen3-Next-80B-A3B-Instruct model cardQwen/Qwen3-30B-A3B-Instruct-2507 model card
229Gemini 2.5 Flash Google52±2api $0.85/M1Mai.google.dev ↗Meituan LongCat-Flash-Chat model card (evaluation table)
230Gemini 1.5 Pro Experimental (0801) Google52±3*comparison onlyWildEval ZeroEval published results
231Gemini 1.5 Pro Experimental (0827) Google52±2*comparison onlyAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
232Mercury 2 Inception52±1*comparison onlyLMArena leaderboard dataset
233DeepSeek-V2 Chat (0628) DeepSeek52±2*local~66GBllama.cpp › Aider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
234Llama 3.3 Nemotron Super 49B v1.5 NVIDIA52±2†local~18GB131kllama.cpp › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
235DeepSeek-V2.5 (1210) DeepSeek52±1*local~82GBllama.cpp › Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)
236Olmo 3.1 32B Think Ai252±3†local~21GB66kLM Studio › Olmo 3
237QwQ-32B Alibaba52±1local~21GB131kLM Studio › Aider polyglot benchmark runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WebApp1K models leaderboard (onekq-ai)
238Gemini 2.0 Flash Experimental Google52±1comparison onlyAider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2025-08-01)
239Falcon-H1R 7B TII52±4*†local~6GB262kLM Studio › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
240Grok 4 Fast SpaceXAI (xAI)51±1*comparison onlyLMArena leaderboard dataset
241Olmo 3 32B Think Ai251±3†local~21GB66kLM Studio › Olmo 3
242Granite 4.1 30B IBM51±2†local~19GBOllama › ibm-granite/granite-4.1-8b model card
243s1.1 32B Simple Scaling51±9*local~22GB33kLM Studio › open-thoughts/OpenThinker-32B model card
244Qwen3 Coder 30B-A3B Instruct Alibaba51±2api $0.12/M +local~20GB160knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardKwaipilot/KAT-Coder-V2.5-Dev model cardRoblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
245GPT-4o (2024-11-20) OpenAI51±1api $4.38/M128kdevelopers.openai.com ↗Aider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Epoch AI — internal evaluationsOpenAI simple-evals README benchmark results (OpenAI's own models)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
246Gemini 1.5 Pro (002) Google51±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)
247Qwen3-VL 8B Thinking Alibaba51±2*†api $0.66/M +local~6.8GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
248GPT-4 (0314) OpenAI51±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
249Qwen2.5-Max (2025-01-25) Alibaba51±2*comparison onlyAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)
250Qwen2.5 Coder 14B Instruct Alibaba51±1local~10GB131kLM Studio › Aider code editing benchmark runs (old edit leaderboard)Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
251DeepSeek-Coder-V2 Instruct (0614) DeepSeek51±1local~92GBllama.cpp › EvalPlus / EvalPerf published evaluation runsLiveCodeBench official website results (2024-12-26)WebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)
252Mistral Large 2 (2411) Mistral51±2*local~78GB128kllama.cpp › Aider code editing benchmark runs (old edit leaderboard)
253Llama 3.1 405B Instruct Meta51±1api $2.40/M +local~160GB128kdocs.aws.amazon.com ↗Cohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)
254Llama 4 Maverick (402B-A17B) Meta51±2api $0.42/M +local~65GB1Mdocs.aws.amazon.com ↗Aider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
255Qwen2.5 72B Instruct Alibaba51±1api $0.37/M +local~30GB33kdeepinfra.com ↗Cohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)Microsoft Research — Phi-4 Technical ReportWildEval ZeroEval published results+2
256GPT-4 Turbo Preview (1106) OpenAI51±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)OpenAI simple-evals README benchmark results (OpenAI's own models)
257Nemotron Nano 9B v2 NVIDIA50±1api $0.10/M +local~7.6GB131kdocs.aws.amazon.com ↗NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsOlmo 3NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
258GPT-4 Turbo Preview (0125) OpenAI50±1comparison onlyTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAider code editing benchmark runs (old edit leaderboard)OpenAI simple-evals README benchmark results (OpenAI's own models)
259GPT-4o mini (2024-07-18) OpenAI50±1api $0.26/M128kdevelopers.openai.com ↗EvalPlus / EvalPerf published evaluation runsAider polyglot benchmark runsAider code editing benchmark runs (old edit leaderboard)Vector Institute Evaluation Leaderboard (Inspect Evals runs)Microsoft Research — Phi-4 Technical ReportWildEval ZeroEval published results+3
260Mistral Large 2 (2407) Mistral50±1local~45GB128kllama.cpp › Aider code editing benchmark runs (old edit leaderboard)Vector Institute Evaluation Leaderboard (Inspect Evals runs)WildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)
261Command A (03-2025) Cohere50±2†api $4.38/M +local~37GB256kazure.microsoft.com ↗Cohere — Command A technical report
262GPT-4 (0613) OpenAI50±1api $37.50/M8kdevelopers.openai.com ↗Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)Tulu 2 — base and instruction-tuned evaluationsCRUXEval published evaluation results (authors' generations and scores)
263Gemini 2.0 Flash Thinking Experimental (01-21) Google50±2*comparison onlyAider polyglot benchmark runs
264Hermes 3 Llama 3.1 405B Nous Research50±2*local~114GB131kllama.cpp › Aider code editing benchmark runs (old edit leaderboard)
265Qwen3 4B Thinking 2507 Alibaba50±6*†local~3.9GB262kLM Studio › Qwen/Qwen3-4B-Thinking-2507 model card
266Gemini 1.5 Pro (001) Google50±2comparison onlyWildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)
267Ministral 3 14B Reasoning 2512 Mistral50±4local~10GB262kllama.cpp › JetBrains/Mellum2-12B-A2.5B-Thinking model cardmistralai/Ministral-3-8B-Reasoning-2512 model cardmistralai/Ministral-3-3B-Reasoning-2512 model card
268Phi-4 14B Microsoft50±1api $0.087/M +local~10GB16kdeepinfra.com ↗CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Microsoft Research — Phi-4 Technical ReportMicrosoft — Phi-4-reasoning Technical Reportmicrosoft/Phi-4-reasoning model cardmicrosoft/phi-4 model card
269Granite 4.2 3B IBM50±2api $0.052/M +local~3.7GB131kdeepinfra.com ↗openbmb/MiniCPM5-2B model cardibm-granite/granite-4.2-30b model card
270Mellum2 12B-A2.5B Thinking JetBrains50±2†local~8.6GB131kLM Studio › JetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Thinking model card
271Yi-Lightning 01.AI50±1*api price unknown16kplatform.lingyiwanwu.com ↗Aider code editing benchmark runs (old edit leaderboard)Aider polyglot benchmark runs
272EXAONE 4.0 32B LG AI Research49±3*†local~20GB131kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
273Olmo 3 7B Think Ai249±2local~5.8GB66kLM Studio › Olmo 3JetBrains/Mellum2-12B-A2.5B-Thinking model card
274Gemma 2 27B IT Google49±2local~18GB8kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WildEval ZeroEval published results
275Claude 3 Opus Anthropic49±1comparison onlyEvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published resultsSciCode leaderboard (repository README)
276InternLM2.5 20B Chat Shanghai AI Laboratory (InternLM)49±2local~14GB33kllama.cpp › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
277Qwen3.5 4B Alibaba49±1local~4.3GB262kllama.cpp › openbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardLiquidAI/LFM2.5-2.6B model cardQwen/Qwen3.5-4B model cardJetBrains/Mellum2-12B-A2.5B-Thinking model card+1
278Llama 3.3 70B Instruct Meta49±1api $0.72/M +local~24GB128kdocs.aws.amazon.com ↗Cohere — Command A technical reportAider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)Microsoft Research — Phi-4 Technical ReportWebApp1K models leaderboard (onekq-ai)
279Mistral Small 3 24B (2501) Mistral49±4*local~16GB32kLM Studio › WebApp1K models leaderboard (onekq-ai)
280GigaChat 3.5 Ultra Instruct (432B-A28B) Sber49±3*†local~457GB262khow to run › ai-sage/GigaChat3.5-432B-A28B-Reasoning model card
281DeepSeek R1 Distill Qwen 14B DeepSeek49±2local~10GB131kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)deepseek-ai/DeepSeek-R1 model card
282Qwen2.5 14B Instruct Alibaba49±1local~10GB131kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Microsoft Research — Phi-4 Technical Reportmicrosoft/phi-4 model card
283Mistral Small Instruct 22B (2409) Mistral AI48±2local~15GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
284Gemini 1.5 Flash (001) Google48±2*comparison onlyWildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)
285Qwen3 30B-A3B (Apr 2025, hybrid) Alibaba48±1†api $0.35/M +local~19GB33kwww.alibabacloud.com ↗Qwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model card
286CodeQwen1.5 7B Chat Alibaba48±1local~6.4GB64kllama.cpp › EvalPlus / EvalPerf published evaluation runsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
287OpenCoder-8B-Instruct INF (infly)48±1local~6.4GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
288DeepSeek R1 Distill Llama 70B DeepSeek48±2api $0.80/M +local~30GB131knovita.ai ↗CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)deepseek-ai/DeepSeek-R1 model card
289Llama 3.1 70B Instruct Meta48±1api $0.72/M +local~27GB128kdocs.aws.amazon.com ↗Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)Vector Institute Evaluation Leaderboard (Inspect Evals runs)WildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)+1
290Qwen3 8B Alibaba48±2api $0.31/M +local~6.1GB33kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
291Gemini 1.5 Flash (002) Google48±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)
292Qwen2.5 Coder 7B Instruct Alibaba48±1local~6GB131kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)
293DeepSeek R1 Distill Qwen 32B DeepSeek48±2api $1.59/M +local~21GB131kdevelopers.cloudflare.com ↗CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WebApp1K models leaderboard (onekq-ai)deepseek-ai/DeepSeek-R1 model cardopen-thoughts/OpenThinker-32B model card
294Gemini 1.5 Flash Experimental (0827) Google48±2*comparison onlyAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
295DeepSeek R1 0528 Qwen3 8B DeepSeek47±4api $0.068/M +local~6.3GB128knovita.ai ↗Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scalingdeepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card
296Qwen3-VL 4B Thinking Alibaba47±2*†local~4.1GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
297Nemotron 3 Nano 30B-A3B NVIDIA47±3†api $0.087/M +local~20GB262kdeepinfra.com ↗NVIDIA — Nemotron-Cascade 2 technical report
298Nova Premier Amazon47±7*†api $5.00/M1Maws.amazon.com ↗Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)
299DeepSeek LLM 67B Chat DeepSeek47±2local~31GB4kllama.cpp › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
300OpenThinker3 7B Open Thoughts47±2local~6GB33kLM Studio › Olmo 3
301GPT-4.1 nano OpenAI47±2*api $0.17/M1Mdevelopers.openai.com ↗Aider polyglot benchmark runsOpenAI simple-evals README benchmark results (OpenAI's own models)
302Qwen2 72B Instruct Alibaba47±1local~33GB131kllama.cpp › Aider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published resultsSciCode leaderboard (repository README)
303Yi-Large Preview 01.AI47±3*comparison onlyWildEval ZeroEval published results
304Phi-3 Medium 128K Instruct Microsoft47±2local~9.8GB128kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
305OpenCodeInterpreter-DS-33B M-A-P47±1local~22GBLM Studio › EvalPlus / EvalPerf published evaluation runs
306Yi-Large 01.AI47±3*comparison onlyWildEval ZeroEval published results
307Gemma 4 E4B Google47±1local~6.9GB131kLM Studio › openbmb/MiniCPM5-2B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardGemma Team — Gemma 4 Technical ReportLiquidAI/LFM2.5-2.6B model cardXHToken/Spark-X2.5-4B model card
308DeepSeek-Coder 33B Instruct DeepSeek46±1local~21GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)
309Codestral 22B v0.1 Mistral AI46±1local~15GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)
310WizardCoder 33B V1.1 WizardLM46±1local~22GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsWizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)
311Gemma 2 9B IT Google46±2local~7GB8kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WildEval ZeroEval published results
312Nova Pro Amazon46±2*api $1.40/M300kdocs.aws.amazon.com ↗Aider code editing benchmark runs (old edit leaderboard)Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)
313Qwen2.5 7B Instruct Alibaba46±1api $0.30/M +local~6GB131kwww.together.ai ↗CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results
314Mistral NeMo Instruct (2407) Mistral46±2local~8.7GB128kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
315Llama 3.1 Nemotron 70B Instruct NVIDIA46±2*local~43GB128kLM Studio › Aider code editing benchmark runs (old edit leaderboard)WebApp1K models leaderboard (onekq-ai)
316Llama 3 70B Instruct Meta46±1local~24GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsLiveCodeBench official website results (2024-12-26)WildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)
317OpenCodeInterpreter-DS-6.7B M-A-P46±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
318GPT-3.5 Turbo (0125) OpenAI46±1api $0.75/M16kdevelopers.openai.com ↗Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published results
319Mixtral 8x22B Instruct v0.1 Mistral46±1local~56GB64kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsWebApp1K models leaderboard (onekq-ai)SciCode leaderboard (repository README)
320speechless-codellama-34b-v2.0 uukuguy46±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
321Magicoder-S-DS 6.7B UIUC (ise-uiuc)46±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
322Artigenz-Coder-DS-6.7B Artigenz46±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
323GPT-3.5 Turbo (1106) OpenAI45±1comparison onlyEvalPlus / EvalPerf published evaluation runs
324GPT-3.5 Turbo (0613) OpenAI45±1comparison onlyAider code editing benchmark runs (old edit leaderboard)Tulu 2 — base and instruction-tuned evaluationsCRUXEval published evaluation results (authors' generations and scores)
325Yi 1.5 9B Chat 01.AI45±2local~6.8GB4kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WildEval ZeroEval published results
326Nemotron 3 Nano 4B NVIDIA45±2local~3.8GB262kLM Studio › openbmb/MiniCPM5-2B model cardNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
327Pixtral 12B (2409) Mistral45±3*local~9.6GBllama.cpp › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)
328Claude 3 Haiku Anthropic45±1comparison onlyEvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsAider code editing benchmark runs (old edit leaderboard)WildEval ZeroEval published resultsLiveCodeBench official website results (2025-08-01)
329Granite 4.1 3B IBM45±2†local~3.7GBOllama › ibm-granite/granite-4.1-8b model card
330XwinCoder-34B Xwin-LM45±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
331StarChat2 15B v0.1 Hugging Face45±1local~12GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runs
332Code Millenials 34B Bud Ecosystem45±1local~23GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
333WizardLM 70B V1.0 Microsoft45±2local~32GBllama.cpp › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
334DeepSeek-Coder 6.7B Instruct DeepSeek45±1local~5.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsLiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)
335AceReason-Nemotron-1.0-14B NVIDIA45±2†local~10GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
336WaveCoder-Ultra-6.7B Microsoft45±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
337DBRX Instruct Databricks45±1local~85GB32kOllama › EvalPlus / EvalPerf published evaluation runs
338Yi-Coder 9B Chat 01.AI45±1local~6.8GB128kLM Studio › Aider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)
339MiniCPM3-4B OpenBMB45±2*local~4GB32kLM Studio › LiveCodeBench official website results (2024-12-26)
340Gemma 3 27B Google44±1api $0.10/M +local~18GB131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardAider polyglot benchmark runsWebApp1K models leaderboard (onekq-ai)Gemma Team — Gemma 4 Technical ReportVisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
341Ministral 3 14B Mistral44±4*api $0.20/M +local~9.1GB262kdocs.mistral.ai ↗JetBrains/Mellum2-12B-A2.5B-Instruct model card
342OpenThinker2 7B Open Thoughts44±9*†local~6.4GB33kllama.cpp › open-thoughts/OpenThinker2-7B model card
343DeepSeek-Coder 7B Instruct v1.5 DeepSeek44±1local~5.9GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
344GPT-3.5 Turbo (0301) OpenAI44±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)Tulu 2 — base and instruction-tuned evaluations
345Qwen3 4B Alibaba44±2local~3.7GB33kLM Studio › Qwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardHuggingFaceTB/SmolLM3-3B model card (Evaluation section)
346Gemma 3 12B Google44±2api $0.075/M +local~9.1GB131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardVisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)
347Yi 1.5 34B Chat 01.AI44±2local~22GB4kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WildEval ZeroEval published results
348Claude 3 Sonnet Anthropic44±1comparison onlyEvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published resultsSciCode leaderboard (repository README)
349Qwen3-VL 32B Instruct Alibaba44±2*†api $0.28/M +local~22GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
350OpenChat 3.5 0106 OpenChat44±1local~6.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
351Athene-70B Nexusflow43±3*local~29GBllama.cpp › WildEval ZeroEval published results
352Granite 34B Code Instruct (8K) IBM43±1local~22GBOllama › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
353Phind CodeLlama 34B v2 Phind43±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
354WhiteRabbitNeo-33B-v1 WhiteRabbitNeo43±1local~22GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
355Magicoder-S-CL-7B UIUC (ise-uiuc)43±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
356Qwen2.5 Coder 3B Instruct Alibaba43±1local~3.3GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
357Code Llama 70B Instruct Meta43±1local~28GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runs
358Qwen3-VL 30B-A3B Instruct Alibaba43±2*†api $0.35/M +local~21GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
359speechless-coder-ds-6.7b uukuguy43±1local~5.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
360Spark-X2.5 1.7B iFLYTEK (SparkLLM)43±3†local~2.7GB1Mllama.cpp › XHToken/Spark-X2.5-4B model card
361WizardLM-2 8x22B Microsoft43±2*local~91GBOllama › Aider code editing benchmark runs (old edit leaderboard)
362WizardCoder Python 34B V1.0 WizardLM43±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsWizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)CRUXEval published evaluation results (authors' generations and scores)
363Phi-4-mini 3.8B Microsoft43±2†api $0.13/M +local~3.8GB128kazure.microsoft.com ↗Microsoft — Phi-4-Mini Technical Report
364Code Tulu 2 34B Allen Institute for AI43±3*†local~72GBhow to run › Tulu 2 — base and instruction-tuned evaluations
365AceReason-Nemotron-1.1-7B NVIDIA42±2†local~6GBLM Studio › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
366Claude 2.1 Anthropic42±1*comparison onlyTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
367DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) DeepSeek42±2local~11GB128kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)WebApp1K models leaderboard (onekq-ai)
368Solar Open 100B (102B-A12B) Upstage42±3*†local~41GB131kllama.cpp › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)
369Qwen3 4B Instruct 2507 Alibaba42±2local~3.9GB262kLM Studio › Qwen/Qwen3-4B-Instruct-2507 model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
370Mistral Large (2402) Mistral42±1comparison onlyEvalPlus / EvalPerf published evaluation runs
371Claude 2.0 Anthropic42±1*comparison onlyEvalPlus / EvalPerf published evaluation runs
372Reka Core (2024-05-01) Reka AI42±3*comparison onlyWildEval ZeroEval published results
373Mellum2 12B-A2.5B Instruct JetBrains42±4*†local~8.6GB131kLM Studio › JetBrains/Mellum2-12B-A2.5B-Instruct model card
374Llama 3.1 8B Instruct Meta42±1api $0.22/M +local~6.2GB128kdocs.aws.amazon.com ↗EvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)IBM Granite 3.0 Language Models technical report (GitHub)LiveCodeBench official website results (2024-12-26)WildEval ZeroEval published results+2
375Llama 3.2 11B Vision Instruct Meta42±3*local~8.8GB128kOllama › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
376speechless-starcoder2-15b uukuguy42±1local~12GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
377Granite 3.0 8B Instruct IBM42±2local~6.6GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
378Qwen2-VL 7B Instruct Alibaba42±3*local~6.4GBLM Studio › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
379Phi-3-mini-128k-instruct (Apr 2024) Microsoft42±2local~9.5GBhow to run › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
380Qwen2.5 3B Instruct Alibaba42±2local~3.3GB33kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)WildEval ZeroEval published results
381Qwen1.5 72B Chat Alibaba41±1local~31GB32kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
382Phi-3-mini-4k-instruct (June 2024 update) Microsoft41±3*local~3.7GBLM Studio › WildEval ZeroEval published results
383Qwen3-VL 8B Instruct Alibaba41±2*†api $0.31/M +local~6.8GB256kwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
384StarCoder2-15B-Instruct-v0.1 BigCode41±1local~11GB16kOllama › EvalPlus / EvalPerf published evaluation runs
385Command R Plus (08-2024) Cohere41±1api $4.38/M +local~63GB128kdocs.cohere.com ↗Aider code editing benchmark runs (old edit leaderboard)Cohere — Command A technical report
386Granite 8B Code Instruct (4K) IBM41±2†local~6.6GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
387Granite 20B Code Instruct (8K) IBM41±2†local~14GBOllama › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
388Phi-3.5-mini-instruct Microsoft41±2local~3.7GB128kLM Studio › Microsoft — Phi-4-Mini Technical ReportWildEval ZeroEval published results
389Falcon3 7B Instruct TII41±2local~5.9GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
390Qwen3-VL 4B Instruct Alibaba41±2*†local~4.1GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
391Qwen1.5 110B Chat Alibaba40±2*local~45GB32kllama.cpp › Aider code editing benchmark runs (old edit leaderboard)
392Code Llama 70B Python Meta40±1local~28GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runs
393Dolphin 2.6 Mixtral 8x7B Cognitive Computations40±1local~29GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
394Command R (08-2024) Cohere40±1api $0.26/M +local~21GB128kdocs.cohere.com ↗Aider code editing benchmark runs (old edit leaderboard)Cohere — Command A technical report
395DeepSeek-Coder 1.3B Instruct DeepSeek40±1local~2.4GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)LiveCodeBench official website results (2024-12-26)CRUXEval published evaluation results (authors' generations and scores)
396Command R Plus (04-2024) Cohere40±1local~43GB128kllama.cpp › EvalPlus / EvalPerf published evaluation runsAider code editing benchmark runs (old edit leaderboard)Vector Institute Evaluation Leaderboard (Inspect Evals runs)
397GLM-4.7-Flash Z.ai40±6*api $0.0000/M +local~19GB200kdocs.z.ai ↗Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode
398Olmo 3 7B Instruct Ai239±2local~5.8GB66kLM Studio › Olmo 3JetBrains/Mellum2-12B-A2.5B-Instruct model card
399OpenChat 3.5 OpenChat39±3*†local~6.1GB8kllama.cpp › OpenChat 3.5 model card (openchat/openchat_3.5)
400Llama 3 8B Instruct Meta39±1local~6.6GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsWildEval ZeroEval published resultsLiveCodeBench official website results (2024-12-26)WebApp1K models leaderboard (onekq-ai)
401Code Llama 70B (base) Meta39±1local~28GB16kllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runs
402Qwen2 7B Instruct Alibaba39±3*local~6GB131kLM Studio › WildEval ZeroEval published results
403Falcon3 10B Instruct TII39±2local~7.6GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
404Code Tulu 2 13B Allen Institute for AI39±2*†local~29GBhow to run › Tulu 2 — base and instruction-tuned evaluations
405WizardCoder Python 13B V1.0 WizardLM39±3local~9.8GB16kllama.cpp › WizardLM team vendor-reported scores for its own WizardCoder models (Hugging Face card)CRUXEval published evaluation results (authors' generations and scores)
406Qwen2.5 Coder 1.5B Instruct Alibaba39±1local~2.4GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
407Gemma 4 E2B Google39±2api $0.050/M +local~5.1GB131kdocs.aws.amazon.com ↗openbmb/MiniCPM5-2B model cardGemma Team — Gemma 4 Technical ReportLiquidAI/LFM2.5-2.6B model cardXHToken/Spark-X2.5-4B model card
408CodeGemma 7B IT (v1.0) Google39±1local~7.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runs
409LFM2.5 2.6B Liquid AI38±2local~3.2GB131kLM Studio › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
410Granite 34B Code Base (8K) IBM38±1†local~24GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
411Reka Flash 21B (2024-02-26) Reka AI38±3*comparison onlyWildEval ZeroEval published results
412speechless-starcoder2-7b uukuguy38±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
413Claude Instant 1.2 Anthropic38±1*comparison onlyEvalPlus / EvalPerf published evaluation runs
414Code Llama 34B Python Meta38±1local~23GB16kOllama › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
415WizardCoder-15B-V1.0 WizardLM38±1local~12GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
416Qwen2.5 1.5B Instruct Alibaba37±2local~2.4GB33kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations
417Hermes 3 Llama 3.1 8B Nous Research37±2*local~6.6GB128kllama.cpp › Aider code editing benchmark runs (old edit leaderboard)
418speechless-coding-7b-16k-tora uukuguy37±1local~16GBhow to run › EvalPlus / EvalPerf published evaluation runs
419Qwen3 1.7B Alibaba37±2*local~2.6GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
420OpenHermes 2.5 Mistral 7B Nous Research37±3*local~6.1GBllama.cpp › OpenChat 3.5 model card (openchat/openchat_3.5)
421Gemma 3 4B Google37±2api $0.063/M +local~3.8GB131kdeepinfra.com ↗Google DeepMind Gemma 3 270M IT model cardVisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)
422Code Llama 34B Instruct Meta37±1local~22GB16kLM Studio › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsTulu 2 — base and instruction-tuned evaluations
423Command R7B (12-2024) Cohere37±2†local~6.2GB128kLM Studio › Cohere — Command A technical report
424OpenHermes-2.5-Code-290k-13B ajibawa-202337±1local~9.8GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
425WizardLM 13B V1.2 Microsoft36±3local~9.8GBllama.cpp › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
426Qwen1.5 14B Chat Alibaba36±1*local~11GB32kOllama › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
427Granite 3.0 2B Instruct IBM36±2local~3.2GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
428DeepSeek-Coder 33B Base DeepSeek36±1local~22GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
429Qwen3-VL 2B Thinking Alibaba36±3*†local~2.7GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
430CodeQwen1.5 7B (base) Alibaba36±1local~6.4GB66kllama.cpp › EvalPlus / EvalPerf published evaluation runs
431Code Llama 34B (base) Meta36±1local~23GB16kOllama › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsCRUXEval published evaluation results (authors' generations and scores)
432Llama 3.2 3B Instruct Meta36±1api $0.12/M +local~3.4GB131kdevelopers.cloudflare.com ↗Aider code editing benchmark runs (old edit leaderboard)IBM Granite 3.0 Language Models technical report (GitHub)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
433Granite 3B Code Instruct (2K) IBM36±2†local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
434Phi-2 (2.7B) Microsoft35±1local~3.2GB2kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)CRUXEval published evaluation results (authors' generations and scores)
435Falcon3 3B Instruct TII35±3local~3.4GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
436SmolLM3 3B Hugging Face35±6*†local~3.3GB66kLM Studio › HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
437Tulu 2 DPO 70B Ai235±2*†local~32GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
438Qwen-14B-Chat Alibaba35±1*local~11GBllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
439Code Llama 13B Instruct Meta35±1local~9.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsTulu 2 — base and instruction-tuned evaluations
440Code Tulu 2 7B Allen Institute for AI35±2*†local~16GBhow to run › Tulu 2 — base and instruction-tuned evaluations
441LFM2.5 8B-A1B Liquid AI35±2local~6.5GB128kLM Studio › openbmb/MiniCPM5-2B model card
442DeepSeek-V2 Lite Chat (16B-A2.4B) DeepSeek35±2local~11GB32kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)DeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)LiveCodeBench official website results (2024-12-26)
443Tulu 2 70B Allen Institute for AI35±2*†local~28GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
444Granite 8B Code Base (4K) IBM35±1†local~6.6GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
445DeepSeek-Coder 6.7B Base DeepSeek35±1local~5.8GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
446Mixtral 8x7B Instruct v0.1 Mistral34±1local~29GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsWildEval ZeroEval published resultsWebApp1K models leaderboard (onekq-ai)
447MistralHermes-CodePro-7B-v1 beowolx34±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
448speechless-code-mistral-7b-v1.0 uukuguy34±1local~17GBhow to run › EvalPlus / EvalPerf published evaluation runs
449Code Llama 13B Python Meta34±1local~9.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
450StarCoder2 15B BigCode34±1local~11GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runs
451Mixtral 8x7B v0.1 (base) Mistral AI34±1local~29GB32kllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsMixtral of Experts — base-model evaluationsCRUXEval published evaluation results (authors' generations and scores)
452Xwin-LM 70B v0.1 Xwin-LM34±2*local~28GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
453CodeGemma 7B (v1.0) Google34±1local~7.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runs
454Qwen1.5 7B Chat Alibaba34±1*local~6.5GB32kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
455EXAONE 3.5 2.4B Instruct LG AI Research34±3local~3GB33kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
456Granite 20B Code Base (8K) IBM33±2†local~15GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
457Code Llama 7B Instruct Meta33±1local~5.5GB16kLM Studio › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsTulu 2 — base and instruction-tuned evaluations
458Code Llama 7B Python Meta33±1local~5.8GB16kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
459Code Llama 13B (base) Meta33±1local~9.2GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsCRUXEval published evaluation results (authors' generations and scores)
460SOLAR 10.7B Instruct v1.0 Upstage33±1local~8.3GB4kllama.cpp › EvalPlus / EvalPerf published evaluation runs
461Qwen2.5 Coder 0.5B Instruct Alibaba33±1local~1.8GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
462Gemma 1.1 7B IT Google32±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runs
463Gemma 2 2B IT Google32±2local~3.1GB8kLM Studio › IBM Granite 3.0 Language Models technical report (GitHub)WildEval ZeroEval published results
464DeepSeek LLM 7B Chat DeepSeek32±3*†local~5.7GBOllama › DeepSeek vendor-reported benchmark scores for its own models (Hugging Face model cards)
465SmolLM2 1.7B Instruct Hugging Face32±4*†local~2.6GB8kLM Studio › SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations
466Mistral 7B Instruct v0.2 Mistral32±1local~6GB32kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
467Llama 3.2 1B Instruct Meta31±2api $0.020/M +local~2.2GB131knovita.ai ↗IBM Granite 3.0 Language Models technical report (GitHub)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)SmolLM2 paper (Hugging Face) — authors' own instruct-model evaluations
468xDAN-L1-Chat-RL-v1 xDAN-AI31±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
469Granite 3.0 3B-A800M Instruct IBM31±2*†local~3.7GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)
470Code Llama 7B (base) Meta31±1local~5.5GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCode Llama: Open Foundation Models for Code — authors' own runsTulu 2 — base and instruction-tuned evaluationsMistral 7B — authors' own runs+1
471Llama 2 70B (base) Meta30±1local~32GB4kllama.cpp › Tulu 2 — base and instruction-tuned evaluationsMixtral of Experts — base-model evaluations
472Granite 3B Code Base (2K) IBM30±2†local~3.7GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
473StarCoder2 7B BigCode30±1local~6.1GB16kLM Studio › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
474Qwen3-VL 2B Instruct Alibaba30±3*†local~2.7GB256kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
475StarCoder 15.5B BigCode30±1local~11GBOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
476Gemma 7B (base) Google30±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsGemma report — original evaluations
477Llama 3 8B (base) Meta30±1local~6.6GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
478StarCoder2 3B BigCode29±1local~3.4GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
479StarCoderBase 15.5B BigCode29±1local~12GBllama.cpp › Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
480Mistral 7B Instruct v0.1 Mistral29±1*local~6.1GB8kllama.cpp › Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs
481Llama 2 70B Chat Meta28±2*local~32GB4kllama.cpp › Tulu 2 — base and instruction-tuned evaluations
482LLaMA 65B Meta28±2*†comparison onlyLLaMA: Open and Efficient Foundation Language Models — authors' own runs
483DeepSeek-Coder 1.3B Base DeepSeek28±1local~2.4GB16kOllama › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
484Stable Code 3B Stability AI28±1local~3.3GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runs
485OpenThinker 7B Open Thoughts28±8*†local~6GB33kLM Studio › open-thoughts/OpenThinker2-7B model cardopen-thoughts/OpenThinker-7B model card
486Mistral 7B v0.1 (base) Mistral AI28±1local~5.8GB8kLM Studio › EvalPlus / EvalPerf published evaluation runsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runsGemma report — original evaluations
487Gemma 7B IT Google28±1local~7.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
488Zephyr 7B beta Hugging Face28±1local~6.1GBllama.cpp › EvalPlus / EvalPerf published evaluation runsTulu 2 — base and instruction-tuned evaluationsOpenChat 3.5 model card (openchat/openchat_3.5)
489DeepSeek R1 Distill Qwen 7B DeepSeek27±3local~6GB131kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)deepseek-ai/DeepSeek-R1 model cardopen-thoughts/OpenThinker2-7B model cardopen-thoughts/OpenThinker-7B model card
490DeepSeek R1 Distill Llama 8B DeepSeek27±4local~6.2GB131kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)deepseek-ai/DeepSeek-R1 model card
491Tulu 2 13B Allen Institute for AI27±2*†local~9.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
492Tulu 2 DPO 13B Allen Institute for AI27±2*†local~9.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
493Granite 3.0 1B-A400M Instruct IBM27±2*†local~2.3GB4kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)
494CodeGemma 2B (v1.0) Google27±1local~3.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runs
495LLaMA 33B Meta27±2comparison onlyMixtral of Experts — base-model evaluationsLLaMA: Open and Efficient Foundation Language Models — authors' own runs
496Falcon 40B (base) TII26±2*local~27GB2kOllama › MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)
497StarCoderBase 7B BigCode26±1local~6GBOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCRUXEval published evaluation results (authors' generations and scores)
498Falcon3 1B Instruct TII26±4local~2.4GB8kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)
499Gemma 3 1B Google25±2†local~2.2GB32kLM Studio › Google DeepMind Gemma 3 270M IT model card
500Gemma 2B (base) Google25±1local~3.2GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runsCodeGemma: Open Code Models Based on Gemma — authors' own runsGemma report — original evaluations
501Qwen3.5 2B Alibaba24±3local~2.9GB262kllama.cpp › openbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model card
502Gemma 1.1 2B IT Google24±1local~3.2GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runs
503Tulu 2 DPO 7B Allen Institute for AI23±2*†local~5.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
504StarCoderBase 3B BigCode23±1local~3.4GBOllama › EvalPlus / EvalPerf published evaluation runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
505LLaMA 13B Meta23±2comparison onlyInstructEval (DeCLaRe Lab) README results tableLLaMA: Open and Efficient Foundation Language Models — authors' own runs
506Llama 2 13B (base) Meta22±1local~9.3GB4kOllama › Tulu 2 — base and instruction-tuned evaluationsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runs
507Tulu 2 7B Allen Institute for AI22±2*†local~5.8GBllama.cpp › Tulu 2 — base and instruction-tuned evaluations
508Gemma 2B IT Google22±1local~3.1GB8kllama.cpp › EvalPlus / EvalPerf published evaluation runsGranite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs
509Koala 13B UC Berkeley (BAIR)21±2*comparison onlyInstructEval (DeCLaRe Lab) README results table
510Llama 2 13B Chat Meta21±2*local~9.8GB4kllama.cpp › Tulu 2 — base and instruction-tuned evaluations
511LLaMA 7B Meta20±2comparison onlyInstructEval (DeCLaRe Lab) README results tableLLaMA: Open and Efficient Foundation Language Models — authors' own runs
512OpenAssistant SFT-4 Pythia 12B LAION (OpenAssistant)20±2*local~9.5GBllama.cpp › InstructEval (DeCLaRe Lab) README results table
513Phi-1 (1.3B) Microsoft19±5*local~2.4GB2kllama.cpp › CRUXEval published evaluation results (authors' generations and scores)
514GPT-J 6B EleutherAI19±1*local~14GB2khow to run › EvalPlus / EvalPerf published evaluation runs
515Llama 2 7B (base) Meta19±1local~5.5GB4kOllama › Tulu 2 — base and instruction-tuned evaluationsMixtral of Experts — base-model evaluationsMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)Mistral 7B — authors' own runs
516Dolly v2 12B Databricks19±3*comparison onlyInstructEval (DeCLaRe Lab) README results table
517Qwen3 0.6B Alibaba17±3*local~1.8GB33kLM Studio › Olmo 3
518Llama 2 7B Chat Meta16±2*local~5.8GB4kllama.cpp › Tulu 2 — base and instruction-tuned evaluations
519Molmo 7B-D (0924) Ai214±5*local~35GBhow to run › Pixtral 12B (Mistral AI, arXiv 2410.07073v2)
520GPT-Neo 2.7B EleutherAI14±1*local~13GB2khow to run › EvalPlus / EvalPerf published evaluation runs
521DeepSeek R1 Distill Qwen 1.5B DeepSeek14±5local~2.6GB131kLM Studio › CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)deepseek-ai/DeepSeek-R1 model card
522ChatGLM-6B Zhipu AI (THUDM)12±3*local~16GBhow to run › InstructEval (DeCLaRe Lab) README results table
523StableLM-Base-Alpha 7B Stability AI5±2local~35GB4khow to run › EvalPlus / EvalPerf published evaluation runsInstructEval (DeCLaRe Lab) README results table
524Zyte-1B venkycs (aihub-app)3±2local~2.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
—BLOOM 1.1B BigSciencenot yet calibratedlocal~2.6GB2kllama.cpp › BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runs
—BLOOM 1.7B BigSciencenot yet calibratedlocal~3.1GB2kllama.cpp › BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runs
—BLOOM 3B BigSciencenot yet calibratedlocal~4GB2kllama.cpp › BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runs
—BLOOM 560M BigSciencenot yet calibratedlocal~2.1GB2kllama.cpp › BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runs
—BLOOM 7.1B BigSciencenot yet calibratedlocal~7.1GB2kllama.cpp › BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runs
—ERNIE 4.5 300B-A47B Baidunot yet calibratedlocal~97GB131kllama.cpp › ERNIE 4.5 post-trained 300B-A47B benchmark chart
—EXAONE Deep 2.4B LG AI Researchnot yet calibratedlocal~3.2GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
—EXAONE Deep 32B LG AI Researchnot yet calibratedlocal~22GB33kOllama › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
—EXAONE Deep 7.8B LG AI Researchnot yet calibratedlocal~6.5GB33kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
—Gemma 3n E2B IT Googlenot yet calibratedlocal~4.7GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model card
—Gemma 3n E4B Googlenot yet calibratedlocal~6.2GB32kllama.cpp › Google DeepMind Gemma 3n E2B IT model card
—Granite 4.0 1B IBMnot yet calibratedlocal~2.4GB128kLM Studio › ibm-granite/granite-4.0-h-1b model card
—Granite 4.0 350M IBMnot yet calibratedlocal~1.7GB32kLM Studio › ibm-granite/granite-4.0-h-1b model card
—Granite 4.0 H 1B IBMnot yet calibratedlocal~2.3GB131kLM Studio › ibm-granite/granite-4.0-h-1b model card
—Granite 4.0 H 350M IBMnot yet calibratedlocal~1.7GB33kLM Studio › ibm-granite/granite-4.0-h-1b model card
—Granite 4.0 H Micro 3B IBMnot yet calibratedlocal~3.4GB131kLM Studio › ibm-granite/granite-4.0-h-small model card
—Granite 4.0 H Small (32B-A9B) IBMnot yet calibratedapi $0.11/M +local~21GB131kreplicate.com ↗ibm-granite/granite-4.0-h-small model card
—Granite 4.0 H Tiny (7B-A1B) IBMnot yet calibratedlocal~5.6GB131kLM Studio › ibm-granite/granite-4.0-h-small model card
—Granite 4.0 Micro 3B IBMnot yet calibratedlocal~3.5GB131kLM Studio › ibm-granite/granite-4.0-h-small model card
—Jamba 1.5 Large AI21 Labsnot yet calibratedlocal~839GBhow to run › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)
—Jamba 1.5 Mini AI21 Labsnot yet calibratedlocal~34GBllama.cpp › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)
—K2 Horizon 7B Institute of Foundation Models (MBZUAI)not scored yetlocal~7.7GB524kllama.cpp ›
—Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)not yet calibratedapi $0.090/M +local~75GB131kdeepinfra.com ↗inclusionAI/Ling-3.0-flash model card
—Llama 3.3 Nemotron Super 49B v1 NVIDIAnot scored yetlocal~22GBllama.cpp ›
—LongCat 2.0 (1.6T-A48B) Meituannot scored yetapi $1.30/M +local~2155GB1Mlongcat.chat ↗
—Magistral Small 1.0 (2506) Mistralnot yet calibratedlocal~16GB128kLM Studio › mistralai/Magistral-Small-2509 model card
—Magistral Small 1.1 (2507) Mistralnot yet calibratedlocal~16GB40kLM Studio › mistralai/Magistral-Small-2509 model card
—Magistral Small 1.2 Mistralnot yet calibratedapi $0.75/M +local~16GB131kdocs.aws.amazon.com ↗mistralai/Mistral-Small-4-119B-2603 model cardmistralai/Magistral-Small-2509 model card
—MiniMax-M1 (80k) MiniMaxnot yet calibratedapi $0.96/M +local~959GB1Mnovita.ai ↗MiniMaxAI/MiniMax-M1-80k model card
—Ministral 8B (2410) Mistralnot yet calibratedlocal~6.2GB128kLM Studio › Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)
—Mistral 7B Instruct v0.3 Mistralnot yet calibratedlocal~5.8GB32kLM Studio › Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards)
—Mistral Small 3.1 24B (2503) Mistralnot scored yetlocal~17GB128kllama.cpp ›
—Mistral Small 3.2 24B Mistralnot scored yetapi $0.11/M +local~16GB128kdeepinfra.com ↗
—Mistral Small 4 (119B-A6.5B) Mistralnot yet calibratedapi $0.26/M +local~40GB262kdocs.mistral.ai ↗mistralai/Mistral-Small-4-119B-2603 model card
—OLMo 7B Allen Institute for AInot yet calibratedlocal~30GB2khow to run › OLMo: Accelerating the Science of Language Models — authors' own runs
—OLMo 7B Instruct Ai2not yet calibratedlocal~16GBhow to run › OLMo: Accelerating the Science of Language Models — authors' own runs
—Qwen2 0.5B Instruct Alibabanot yet calibratedlocal~1.8GB32kLM Studio › Qwen2-0.5B-Instruct model card
—Qwen2 1.5B Instruct Alibabanot yet calibratedlocal~2.4GB32kLM Studio › Qwen2-0.5B-Instruct model card
—Step 3.7 Flash (198B-A11B) StepFunnot scored yetapi $0.44/M +local~104GB256kplatform.stepfun.ai ↗

+ 26 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

Includes 80 comparison-only models with no verified API or download · hide them

563 of 598 catalogue models listed here · sources and licences

Models outside this leaderboard (9 models)

No current run option verified: Ministral 3B (2410) · Magistral Medium 1.0 (2506) · GLM-4 (0520) · MiniMax-M1 (40k) · ERNIE 4.5 21B-A3B · Magistral Medium 1.2 (2509) · Magistral Medium 1.1 (2507) · GLM-4 (0116) · Ring-2.6-1T

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory