whatmodel.sh · find the best AI models you can run

Best AI models for agents for a 64GB graphics card

A leaderboard of the best AI models for agents for a 64GB graphics card (with 32GB of system RAM). The best one that runs on it right now is Qwen3.8-Flash-Next, followed by Qwen3.8 27B and Ornith 1.5 35B-A3B. Models are ranked on agent tests like SWE-bench and Terminal-Bench, where they use tools, write code and work in a terminal. Models that run on it show the download that fits and the command to run it.

300 models · scores combine 76 benchmarks · updated how it works · find a leaderboard · benchmark your machine

Scores use reviewed agent tasks evaluations only. Compare scores within this task. Models without task measurements are excluded.

Models for 32GB RAM · 64GB graphics card → ~64GB fast · reset
+ more
$ terminal

detecting your OS…

Score vs API price

#modeltask score↓where↓speed↓RAM↓context↓get itsources
1Claude Opus 5.5 Anthropic88±1api $8.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
2Claude Sonnet 5.5 Anthropic86±1api $4.00/M1Mplatform.claude.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard dataset
3Claude Fable 5.1 Anthropic85±1api $20.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard dataset
4GPT-6.1 Sol OpenAI85±1*api $4.00/M1.1Mdevelopers.openai.com ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard dataset
5GPT-6 Astra OpenAI84±1api $20.00/M1.1Mplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
6Claude Opus 5 Anthropic83±1api $10.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)+3
7GPT-6 Sol OpenAI83±1api $4.00/M1.1Mplatform.openai.com ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
8Claude Fable 5 Anthropic82±1api $20.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)τ²/τ³-bench leaderboard: Sierra's own evaluation runs
9Gemini 4 Argon (limited preview) Google81±2*limited previewLMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)
10Claude Mythos Preview (limited preview) Anthropic81±2†limited preview1MAnthropic system cards and model announcements (lab-reported scores)
11GPT-5.6 Sol OpenAI80±1api $8.00/M1.1Mplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+4
12DeepSeek V4.1 Flash DeepSeek79±1api $0.26/M~537GB1Mapi-docs.deepseek.com ↗deepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
13Claude Haiku 5.5 Anthropic79±1†api $0.20/M1Mplatform.claude.com ↗Anthropic system cards and model announcements (lab-reported scores)
14GLM-5.3 Z.ai79±1*api $2.15/M~239GB1Mdocs.z.ai ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard dataset
15GPT-5.5 Pro OpenAI79±3*†api $67.50/M1.1Mdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
16GPT-5.4 Pro OpenAI79±3*†api $67.50/M1.1Mdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
17MiMo-V2.6-Pro Xiaomi78±2api $0.54/M~443GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
18Qwen3.8 Max (2.4T-A95B) Alibaba77±2api $3.00/M~380GB262kdeepinfra.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runs
19Claude Opus 4.8 Anthropic77±1api $10.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Legal Research Bench (Vals AI, arXiv:2610.00609v1)τ²/τ³-bench leaderboard: Sierra's own evaluation runs+3
20Grok 4.7 SpaceXAI (xAI)77±2*api $3.00/M500kdocs.x.ai ↗LMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
21Hy4 preview (770B-A49B) Tencent77±2api $1.25/M~248GB1Mcloud.tencent.com ↗LMArena leaderboard datasetTencent Hy4 preview model card
22Qwen3.8 Max (0902) Alibaba76±2*api $3.00/M1Mwww.alibabacloud.com ↗Terminal-Bench 4.0 official leaderboard submissionsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
23Kimi K3 Moonshot AI76±1api $6.00/M~884GB1Mplatform.moonshot.ai ↗LMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)τ²/τ³-bench leaderboard: Sierra's own evaluation runsVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
24GLM-5.3-Flash Z.ai76±1api $0.24/M~103GB1Mdocs.z.ai ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetzai-org/GLM-5.3-Flash model cardVideo2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)
25GPT-5.5 OpenAI76±1api $11.25/M1.1Mplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Legal Research Bench (Vals AI, arXiv:2610.00609v1)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)Agents' Last Exam — authors' own fixed-harness model comparison+6
26GPT-5.6 Terra OpenAI76±1api $4.50/M1.1Mplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
27Gemini 3.8 Flash Google75±1api $1.50/M1Mai.google.dev ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashGoogle DeepMind Gemini model page and model cards (lab-reported scores)Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos (arXiv:2610.04432v2)+1
28MiMo-V2.6-Flash (309B-A15B) Xiaomi75±2api $0.17/M~136GB1Mmimo.mi.com ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentation
29Muse Spark 1.3 Meta75±1api $2.00/M1Mdev.meta.ai ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)
30Grok 4.6 SpaceXAI (xAI)74±1api $3.00/M500kdocs.x.ai ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
31DeepSeek V4 Pro (0813) DeepSeek74±1api $1.63/M~1010GB1Mdeepinfra.com ↗deepseek-ai/DeepSeek-V4.1-Flash model cardLMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)NVIDIA ModelOpt quantized-checkpoint cards: baseline (unquantized) measurements of third-party modelsdeepseek-ai/DeepSeek-V4-Pro-0813 model card
33GPT-5.4 OpenAI74±1api $5.63/M1.1Mplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)LMArena leaderboard datasetSWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Agents' Last Exam — authors' own fixed-harness model comparisonτ²/τ³-bench leaderboard: Sierra's own evaluation runs+4
34Claude Opus 4.7 Anthropic74±2api $10.00/M1Mdocs.anthropic.com ↗SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)Agents' Last Exam — authors' own fixed-harness model comparisonτ²/τ³-bench leaderboard: Sierra's own evaluation runsEpoch AI — internal evaluationsAnthropic system cards and model announcements (lab-reported scores)TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks — authors' own Terminus-2 model comparison+1
35GPT-6 Luna OpenAI74±1api $0.20/M1.1Mplatform.openai.com ↗Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
36Claude Sonnet 5 Anthropic74±1api $4.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
37GPT-5.6 Luna OpenAI74±1api $0.45/M1.1Mplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Terminal-Bench 4.0 official leaderboard submissionsLMArena leaderboard datasetKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
38Grok 4.5 SpaceXAI (xAI)73±1api $3.00/M500kdocs.x.ai ↗LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsτ²/τ³-bench leaderboard: Sierra's own evaluation runsEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
40GLM-5.2 Z.ai73±1api $2.15/M~121GB1Mdocs.z.ai ↗LMArena leaderboard datasetzai-org/GLM-5.2 model cardE2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)zai-org/GLM-5.3-Flash model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Legal Research Bench (Vals AI, arXiv:2610.00609v1)+3
41Qwen3.8-Flash-Next · 2-bit Alibaba72±2local~59GB262kllama.cpp › LMArena leaderboard datasetQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
42Gemini 3.7 Flash Google72±1api $1.50/M1Mai.google.dev ↗LMArena leaderboard datasetTerminal-Bench 4.0 official leaderboard submissionsGoogle DeepMind Gemini model page and model cards (lab-reported scores)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 FlashKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
43Qwen3.8-Omni-Flash Alibaba72±3†api $0.23/M1Mwww.alibabacloud.com ↗Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
44DeepSeek V4 Flash (0731) DeepSeek71±2api $0.11/M~92GB1Mdeepinfra.com ↗deepseek-ai/DeepSeek-V4.1-Flash model carddeepseek-ai/DeepSeek-V4-Flash-Vision-Exp model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)deepseek-ai/DeepSeek-V4-Pro-0813 model card
45Claude Sonnet 4.6 Anthropic71±1api $6.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)Legal Research Bench (Vals AI, arXiv:2610.00609v1)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Agents' Last Exam — authors' own fixed-harness model comparisonEpoch AI — internal evaluations+1
46Qwen3.8 27B · 8-bit Alibaba71±1local~36GB262kOllama › LMArena leaderboard datasetQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)
47GPT-5.3-Codex OpenAI71±2api $4.81/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)Epoch AI — internal evaluationsKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
48Claude Opus 4.6 Anthropic70±1api $10.00/M1Mdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)+6
49Hunyuan Hy3 (295B-A21B) Tencent70±1api $0.23/M~96GB256kdeepinfra.com ↗tencent/Hy3 model cardLMArena leaderboard datasetTencent Hy4 preview model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
50Gemini 3.5 Flash Google70±3api $3.38/M1Mai.google.dev ↗Legal Research Bench (Vals AI, arXiv:2610.00609v1)Epoch AI — internal evaluationsEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
51Muse Spark 1.2 Meta69±2*api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard dataset
52Muse Spark 1.1 Meta69±2*api $2.00/M1Mdev.meta.ai ↗LMArena leaderboard datasetτ²/τ³-bench leaderboard: Sierra's own evaluation runs
53Kimi K2.6 Moonshot AI68±1api $1.71/M~317GB262kplatform.moonshot.ai ↗NVIDIA — Nemotron 3 Ultra technical reportOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Legal Research Bench (Vals AI, arXiv:2610.00609v1)Agents' Last Exam — authors' own fixed-harness model comparisonEpoch AI — internal evaluationsTerminalWorld: Benchmarking Agents on Real-World Terminal Tasks — authors' own Terminus-2 model comparison+2
54Qwen3.7 Max Alibaba68±2api $3.75/M1Mdeepinfra.com ↗LMArena leaderboard datasetLegal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardAgents' Last Exam — authors' own fixed-harness model comparisonEpoch AI — internal evaluationsEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
55Claude Opus 4.5 Anthropic68±1api $10.00/M200kdocs.anthropic.com ↗τ²/τ³-bench leaderboard: Sierra's own evaluation runsTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsAnthropic system cards and model announcements (lab-reported scores)Berkeley Function Calling Leaderboard result archiveQwen/Qwen3.6-27B model cardQwen/Qwen3.5-397B-A17B model card+4
56GLM-5.1 Z.ai67±1api $2.15/M~119GB200kdocs.z.ai ↗NVIDIA — Nemotron 3 Ultra technical reportzai-org/GLM-5.1 model cardzai-org/GLM-5.2 model cardOne Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)Legal Research Bench (Vals AI, arXiv:2610.00609v1)+5
58MiniMax-M3 MiniMax67±2api $0.48/M~137GB1Mdeepinfra.com ↗LMArena leaderboard datasetGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)Legal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
59GPT-5.2 OpenAI67±1api $4.81/M400kplatform.openai.com ↗τ²/τ³-bench leaderboard: Sierra's own evaluation runsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Berkeley Function Calling Leaderboard result archiveQwen/Qwen3.5-397B-A17B model card+6
60Gemini 3.1 Pro (Preview) Google66±1api $4.50/M1Mai.google.dev ↗LMArena leaderboard datasetVibe Code Bench (Vals AI, arXiv:2603.04601v3)Google DeepMind Gemini model page and model cards (lab-reported scores)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)Legal Research Bench (Vals AI, arXiv:2610.00609v1)zai-org/GLM-5.2 model card+4
61Qwen3.7 Plus Alibaba66±2api $0.70/M1Mwww.alibabacloud.com ↗LMArena leaderboard datasetQwen/Qwen3.8-27B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
62GPT-5.4 mini OpenAI66±2api $1.69/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)Legal Research Bench (Vals AI, arXiv:2610.00609v1)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)
64Mistral Large 4 (public preview) Mistral66±2*api $2.06/M1Mdocs.mistral.ai ↗LMArena leaderboard dataset
66MiMo-V2.5 (310B-A15B) Xiaomi65±2api $0.21/M~100GB1Mnovita.ai ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)XiaomiMiMo/MiMo-V2.5 model cardAgents' Last Exam — authors' own fixed-harness model comparison
68Qwen3.5 397B-A17B Alibaba65±1api $1.09/M +local~65GB262kdeepinfra.com ↗Qwen/Qwen3.5-397B-A17B model cardNVIDIA — Nemotron 3 Ultra technical reportτ²/τ³-bench leaderboard: Sierra's own evaluation runsQwen/Qwen3.6-27B model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
69Ornith 1.5 35B-A3B · 8-bit DeepReinforce65±2†local~41GB262kllama.cpp › ornith-ai/Ornith-1.5-35B-A3B model card
70Qwen3.6 Max Preview Alibaba65±5*api $2.92/M256kwww.alibabacloud.com ↗Epoch AI — internal evaluations
72Muse Glimmer 30B · 8-bit Meta65±2local~40GB131kOllama › meta-models/Muse-Glimmer-30B model card (speed and benchmark tables)Qwen/Qwen3.8-27B model card
73Gemini 3.6 Flash Google64±2api $1.50/M1Mai.google.dev ↗LMArena leaderboard datasetGoogle DeepMind Gemini model page and model cards (lab-reported scores)E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch (arXiv:2609.38335v1)Google Cloud developer's guides to Gemini 3.8 Flash and 3.7 Flash
74LongCat 2.0 (1.6T-A48B) Meituan64±6*api $1.30/M~2155GB1Mlongcat.chat ↗meituan-longcat/LongCat-2.0 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
75GLM-5 Z.ai64±1api $1.55/M~119GB205kdocs.z.ai ↗zai-org/GLM-5 model cardzai-org/GLM-5.1 model cardτ²/τ³-bench leaderboard: Sierra's own evaluation runsGLM-5 Team (Zhipu AI) — GLM-5 technical reportVibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)+1
76Qwen3.6 27B · 8-bit Alibaba64±1local~37GB262kOllama › Qwen/Qwen3.6-27B model cardQwen/Qwen3.8-27B model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
77GPT-5 Pro OpenAI64±5*†api $41.25/M400kdevelopers.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)
78MiMo-V2.5-Pro (1T-A42B) Xiaomi64±1api $0.65/M~163GB1Mnovita.ai ↗LMArena leaderboard datasetXiaomi MiMo official model cards and documentationXiaomiMiMo/MiMo-V2.5-Pro model cardGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions (arXiv:2609.21562v2)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
80Inkling Small (266B-A12B) Thinking Machines64±2api $0.64/M +local~86GB1Mdeepinfra.com ↗LMArena leaderboard datasetthinkingmachines/Inkling-Small model card
81MiniMax-M2.7 MiniMax64±2api $0.53/M +local~75GB205kplatform.minimax.io ↗NVIDIA — Nemotron 3 Ultra technical reportTerminalWorld: Benchmarking Agents on Real-World Terminal Tasks — authors' own Terminus-2 model comparison
82Gemini 3 Flash (Preview) Google64±2api $1.13/M1Mai.google.dev ↗τ²/τ³-bench leaderboard: Sierra's own evaluation runsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Epoch AI — internal evaluations
83Inkling (975B-A41B) Thinking Machines63±2api $1.73/M~156GB1Mdeepinfra.com ↗LMArena leaderboard datasetthinkingmachines/Inkling-Small model cardτ²/τ³-bench leaderboard: Sierra's own evaluation runs
84Step 3.5 Flash (196B-A11B) StepFun63±2†api $0.15/M~126GB256kplatform.stepfun.ai ↗stepfun-ai/Step-3.5-Flash model card
85GPT-5 OpenAI63±2api $3.44/M400kplatform.openai.com ↗Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)τ²/τ³-bench leaderboard: Sierra's own evaluation runsOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Epoch AI — internal evaluationsServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)+2
87Nemotron 3 Ultra 550B-A55B NVIDIA63±2†api $1.35/M~89GB1Mdocs.together.ai ↗NVIDIA — Nemotron 3 Ultra technical report
88KAT-Coder-V2.5-Dev (35B-A3B) · 8-bit Kuaishou (Kwaipilot)62±2†local~40GB262kllama.cpp › Kwaipilot/KAT-Coder-V2.5-Dev model card
89Qwen3.5 27B · 8-bit Alibaba62±1local~37GB262kOllama › Qwen/Qwen3.5-122B-A10B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.6-27B model cardQwen/Qwen3.5-27B model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)
90Qwen3.5 122B-A10B Alibaba62±2api $1.10/M +local~79GB262kwww.alibabacloud.com ↗Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.5-27B model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
92Kimi K2.5 Moonshot AI62±1api $1.20/M~160GB262kdocs.aws.amazon.com ↗stepfun-ai/Step-3.5-Flash model cardVibe Code Bench (Vals AI, arXiv:2603.04601v3)Qwen/Qwen3.5-397B-A17B model cardSWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)Epoch AI — internal evaluationsSEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages (arXiv:2606.28715v2)+2
93Qwen3.6 35B-A3B · 8-bit Alibaba61±1local~40GB262kllama.cpp › Kwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3.6-35B-A3B model cardQwen/Qwen3.6-27B model cardK-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)
94Claude Sonnet 4.5 Anthropic61±1api $6.00/M200kdocs.anthropic.com ↗τ²/τ³-bench leaderboard: Sierra's own evaluation runsTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Berkeley Function Calling Leaderboard result archiveServiceNow-AI/Apriel-1.6-15b-Thinker model cardEpoch AI — internal evaluations+3
95GLM-4.7 · 1-bit Z.ai61±1†local~58GB205kllama.cpp › zai-org/GLM-4.7 model cardzai-org/GLM-5 model cardGLM-5 Team (Zhipu AI) — GLM-5 technical report
96GPT-5.4 nano OpenAI61±2api $0.46/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)
97GPT-5.1 OpenAI61±2api $3.44/M400kplatform.openai.com ↗HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Epoch AI — internal evaluationsAmazon AGI — τ²-Bench-VerifiedKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
98Ornith 1.0 35B (35B-A3B) · 8-bit DeepReinforce60±1local~40GB262kOllama › ornith-ai/Ornith-1.5-35B-A3B model cardKwaipilot/KAT-Coder-V2.5-Dev model card
99Nova 2 Pro (Preview) Amazon60±8*†api $3.78/M1Maws.amazon.com ↗Amazon AGI — τ²-Bench-Verified
100Qwen3.5 35B-A3B · 8-bit Alibaba60±1local~47GB262kOllama › Qwen/Qwen3.5-122B-A10B model cardQwen/Qwen3.6-35B-A3B model cardKwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3.5-27B model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
101MiniMax-M2.1 · 2-bit MiniMax59±2local (slow)~79GBllama.cpp › stepfun-ai/Step-3.5-Flash model cardMiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)
103SmolLM3 3B · 8-bit Hugging Face59±11*†local~5.9GB66kllama.cpp › HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
105DeepSeek V3.2 DeepSeek59±2api $0.29/M~110GB128kdeepinfra.com ↗stepfun-ai/Step-3.5-Flash model cardVibe Code Bench (Vals AI, arXiv:2603.04601v3)zai-org/GLM-5 model cardGLM-5 Team (Zhipu AI) — GLM-5 technical reportEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
106MiniMax-M2.5 MiniMax59±2api $0.53/M +local~75GB205kplatform.minimax.io ↗ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Vibe Code Bench (Vals AI, arXiv:2603.04601v3)SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)
109Mistral Medium 3.5 (128B) · 2-bit Mistral58±2local~42GB262kllama.cpp › LMArena leaderboard datasetSolar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
110MiniMax-M2 MiniMax58±2api $0.53/M +local~75GB205kdocs.aws.amazon.com ↗MiniMax vendor-reported benchmark scores for its own models (Hugging Face model cards)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
111Qwen3.5 Plus Alibaba58±4*api $0.90/M1Mwww.alibabacloud.com ↗Vibe Code Bench (Vals AI, arXiv:2603.04601v3)
112Grok 4.20 (0309 reasoning) SpaceXAI (xAI)58±6*api $1.56/M1Mdocs.x.ai ↗τ²/τ³-bench leaderboard: Sierra's own evaluation runs
114Gemma 4 31B · 8-bit Google57±1local~36GB262kllama.cpp › Gemma Team — Gemma 4 Technical ReportKwaipilot/KAT-Coder-V2.5-Dev model cardmeta-models/Muse-Glimmer-30B model card (speed and benchmark tables)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Qwen/Qwen3.6-27B model cardgoogle/gemma-4-31B-it model card+4
115Grok 4.3 SpaceXAI (xAI)57±3api $1.56/M1Mdocs.x.ai ↗One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)Legal Research Bench (Vals AI, arXiv:2610.00609v1)Agents' Last Exam — authors' own fixed-harness model comparisonKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
116GLM-4.6 · 1-bit Z.ai57±1local~58GB205kllama.cpp › zai-org/GLM-4.7 model cardzai-org/GLM-4.6 model cardBerkeley Function Calling Leaderboard result archiveTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
117Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)57±2api $0.28/M128kazure.microsoft.com ↗Berkeley Function Calling Leaderboard result archiveVibe Code Bench (Vals AI, arXiv:2603.04601v3)τ²/τ³-bench leaderboard: Sierra's own evaluation runsAmazon AGI — τ²-Bench-Verified
119Spark-X2.5 4B · 16-bit iFLYTEK (SparkLLM)56±2†local~10GB1MOllama › XHToken/Spark-X2.5-4B model card
120Qwen3.6 Plus Alibaba56±4*api $1.13/M1Mdocs.together.ai ↗Epoch AI — internal evaluationsAgents' Last Exam — authors' own fixed-harness model comparison
121Qwen3-Max (2025-09-23) Alibaba56±3†api $2.40/M262kwww.alibabacloud.com ↗τ²-bench leaderboard: Qwen (Alibaba) own submissions
122Kimi K2 Thinking Moonshot AI56±2api $1.07/M~160GB262kdocs.aws.amazon.com ↗stepfun-ai/Step-3.5-Flash model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
123EXAONE 4.5 33B · 8-bit LG AI Research56±3*†local~45GB262kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
124Kimi K2 Instruct (0711) Moonshot AI56±1api $1.00/M~160GB131knovita.ai ↗Meituan LongCat-Flash-Chat model card (evaluation table)Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
125Solar Pro 4 Upstage56±2*api $0.53/M524kdevelopers.upstage.ai ↗LMArena leaderboard dataset
127MiMo-V2.6-Distill-Qwen-9B · 8-bit Xiaomi56±3†local~12GBllama.cpp › XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B model card
129o3-pro OpenAI56±5*api $35.00/M200kdevelopers.openai.com ↗One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)
130GPT-5 mini OpenAI55±1api $0.69/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Berkeley Function Calling Leaderboard result archiveServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Qwen/Qwen3.5-27B model cardSEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages (arXiv:2606.28715v2)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials+5
134Claude Haiku 4.5 Anthropic55±2api $2.00/M200kdocs.anthropic.com ↗Anthropic system cards and model announcements (lab-reported scores)Berkeley Function Calling Leaderboard result archiveTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsVibe Code Bench (Vals AI, arXiv:2603.04601v3)Amazon AGI — τ²-Bench-VerifiedKlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)+1
136GLM-4.5 · 1-bit Z.ai55±2local~58GB131kllama.cpp › GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportMeituan LongCat-Flash-Thinking model card (evaluation table)zai-org/GLM-4.6 model cardBerkeley Function Calling Leaderboard result archive
137MiniCPM5 2B · 8-bit OpenBMB55±3†local~4.3GB131kllama.cpp › openbmb/MiniCPM5-2B model card
139Nemotron 3 Super 120B-A12B NVIDIA54±2†api $0.28/M +local~78GB262kdocs.aws.amazon.com ↗NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
140GPT-4o (2024-05-13) OpenAI54±11*api $7.50/M128kdevelopers.openai.com ↗ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)
141Gemini 3.1 Flash-Lite Google54±6*api $0.56/M1Mai.google.dev ↗K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)
142Granite 4.2 30B · 8-bit IBM54±2†local~39GB131kOllama › ibm-granite/granite-4.2-30b model card
144Qwen3-Coder 480B-A35B Instruct Alibaba53±2api $0.67/M +local~78GB262knovita.ai ↗Qwen/Qwen3-Coder-30B-A3B-Instruct model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
146o3 OpenAI53±1api $3.50/M200kplatform.openai.com ↗Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Meituan LongCat-Flash-Thinking model card (evaluation table)Berkeley Function Calling Leaderboard result archiveEpoch AI — internal evaluationsFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)+1
147Gemini 2.5 Pro Google53±2api $3.44/M1Mai.google.dev ↗Meituan LongCat-Flash-Thinking model card (evaluation table)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsEpoch AI — internal evaluationsBerkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)+3
148Nemotron-Labs-3 Puzzle 75B-A9B · 4-bit NVIDIA53±2†local~56GBllama.cpp › NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
149Seed-OSS 36B Instruct · 8-bit ByteDance Seed52±3†local~48GB524kllama.cpp › ByteDance-Seed/Seed-OSS-36B-Instruct model card
151Mistral Small 4 (119B-A6.5B) · 2-bit Mistral52±3*local~40GB262kllama.cpp › ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)
153Command A+ (05-2026) Cohere52±2api price unknown +local~71GB128kdocs.cohere.com ↗Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Cohere vendor-reported benchmark scores for its own models (cohere.com blog)
155GLM-4.5-Air · 2-bit Z.ai51±2local~35GB131kllama.cpp › GLM-4.5 Team (Zhipu AI & Tsinghua) — GLM-4.5 technical reportSolar Open Technical Report (arXiv:2601.07022v1)Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
156Jamba 1.5 Mini · 4-bit AI21 Labs51±10*†local~34GBllama.cpp › Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)
157Qwen3.5 9B · 8-bit Alibaba51±1local~12GB262kllama.cpp › JetBrains/Mellum2.1-12B-A2.5B-Thinking model cardXiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B model cardXHToken/Spark-X2.5-4B model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Qwen/Qwen3.5-9B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model card+3
158DeepSeek V3.1 DeepSeek51±1api $0.42/M~110GB131kdeepinfra.com ↗Meituan LongCat-Flash-Chat model card (evaluation table)deepseek-ai/DeepSeek-V3.1 model cardMeituan LongCat-Flash-Thinking model card (evaluation table)DeepSeek-V3.1-Terminus model card
159Mellum2.1 12B-A2.5B Thinking · 16-bit JetBrains50±3†local~27GBhow to run › JetBrains/Mellum2.1-12B-A2.5B-Thinking model card
160Qwen3 235B-A22B Thinking 2507 Alibaba50±1api $0.97/M +local~76GB131knovita.ai ↗Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Meituan LongCat-Flash-Thinking model card (evaluation table)Qwen/Qwen3.5-27B model cardGalileo Agent Leaderboard (v1 and v2 results)
161gpt-oss-120b OpenAI49±2api $0.26/M +local~67GB131kconsole.groq.com ↗OpenAI — gpt-oss-120b & gpt-oss-20b Model CardServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardTerminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsMultiverse Computing HyperNova 60B 2602 model card: evaluation table (own runs incl. gpt-oss baselines)Solar Open Technical Report (arXiv:2601.07022v1)+1
162Qwen3-Next 80B-A3B Thinking · 4-bit Alibaba49±2†local~52GB131kOllama › Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3.5-9B model card
163Command A Reasoning (08-2025) · 2-bit Cohere49±3local~34GBllama.cpp › Berkeley Function Calling Leaderboard result archiveCohere vendor-reported benchmark scores for its own models (cohere.com blog)
164Granite 4.1 30B · 4-bit IBM49±5*†local~19GBOllama › ibm-granite/granite-4.1-8b model card
165o4-mini OpenAI49±1api $1.93/M200kplatform.openai.com ↗Qwen/Qwen3-235B-A22B-Thinking-2507 model cardOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)τ²/τ³-bench leaderboard: Sierra's own evaluation runsBerkeley Function Calling Leaderboard result archiveFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Galileo Agent Leaderboard (v1 and v2 results)
166Gemma 4 12B · 8-bit Google48±2local~15GB262kllama.cpp › Gemma Team — Gemma 4 Technical ReportXHToken/Spark-X2.5-4B model cardgoogle/gemma-4-31B-it model card
168Gemma 4 26B-A4B · 8-bit Google48±1local~30GB262kllama.cpp › Gemma Team — Gemma 4 Technical ReportKwaipilot/KAT-Coder-V2.5-Dev model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)google/gemma-4-31B-it model cardgoogle/diffusiongemma-26B-A4B-it model cardQwen/Qwen3.6-35B-A3B model card+1
169GPT-4.1 OpenAI48±1api $3.50/M1Mplatform.openai.com ↗Berkeley Function Calling Leaderboard result archiveMeituan LongCat-Flash-Chat model card (evaluation table)MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table)OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)τ²/τ³-bench leaderboard: Sierra's own evaluation runs+4
170Qwen3-VL 32B Thinking · 8-bit Alibaba48±1local~39GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)
171Granite 4.2 8B · 8-bit IBM48±2†local~11GB131kllama.cpp › ibm-granite/granite-4.2-30b model card
172Nova Premier Amazon47±3*†api $5.00/M1Maws.amazon.com ↗MASSIVE-Agents — Kulkarni et al. (2025)Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)
174Qwen3.5 4B · 8-bit Alibaba47±1local~6.2GB262kllama.cpp › openbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardQwen/Qwen3.5-9B model cardJetBrains/Mellum2-12B-A2.5B-Instruct model cardLiquidAI/LFM2.5-2.6B model cardJetBrains/Mellum2-12B-A2.5B-Thinking model card+1
175Qwen3 235B-A22B Instruct 2507 Alibaba47±1api $0.20/M +local~76GB262kdeepinfra.com ↗Meituan LongCat-Flash-Chat model card (evaluation table)Qwen/Qwen3-Next-80B-A3B-Instruct model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardSEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages (arXiv:2606.28715v2)Berkeley Function Calling Leaderboard result archiveQwen3-VL-235B-A22B-Instruct model card+1
177Qwen3-VL 32B Instruct · 8-bit Alibaba46±3*†local~39GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
180Solar Open 100B (102B-A12B) · 2-bit Upstage46±3†local~41GB131kllama.cpp › Solar Open 2 Technical Report (Upstage, arXiv:2607.20062v2)Solar Open Technical Report (arXiv:2601.07022v1)
181GPT-4o (2024-11-20) OpenAI46±1api $4.38/M128kdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archiveQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Epoch AI — internal evaluationsGalileo Agent Leaderboard (v1 and v2 results)+1
182LFM2.5 2.6B · 8-bit Liquid AI46±2local~4.5GB131kllama.cpp › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
184GPT-4.1 mini OpenAI46±1api $0.70/M1Mplatform.openai.com ↗Berkeley Function Calling Leaderboard result archiveOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)τ²/τ³-bench leaderboard: Sierra's own evaluation runsGalileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)ServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)+1
185Qwen3 4B Thinking 2507 · 8-bit Alibaba46±2†local~7.1GB262kOllama › Qwen/Qwen3-4B-Thinking-2507 model card
187GPT-4 Turbo (2024-04-09) OpenAI46±1*api $15.00/M128kdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archiveChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)
188Nemotron-Cascade-14B-Thinking · 8-bit NVIDIA46±4*†local~18GBllama.cpp › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
189Qwen3 14B · 8-bit Alibaba46±1local~21GB33kOllama › Berkeley Function Calling Leaderboard result archiveMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)
191Qwen3 Coder 30B-A3B Instruct · 8-bit Alibaba45±2local~41GB160kOllama › Kwaipilot/KAT-Coder-V2.5-Dev model cardQwen/Qwen3-Coder-30B-A3B-Instruct model card
192GPT-5 nano OpenAI45±2api $0.14/M400kplatform.openai.com ↗OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Berkeley Function Calling Leaderboard result archiveServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials
194Qwen3 30B-A3B Thinking 2507 · 8-bit Alibaba45±1local~41GB262kOllama › Qwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)ByteDance-Seed/Seed-OSS-36B-Instruct model cardQwen/Qwen3.5-9B model cardLiquidAI/LFM2.5-8B-A1B model card
195Nemotron 3 Nano 30B-A3B · 8-bit NVIDIA45±3*†local~42GB262kOllama › NVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
196Granite 4.2 3B · 8-bit IBM45±2local~5.6GB131kllama.cpp › openbmb/MiniCPM5-2B model cardibm-granite/granite-4.2-30b model card
197Mellum2 12B-A2.5B Instruct · 16-bit JetBrains45±4*†local~27GB131khow to run › JetBrains/Mellum2-12B-A2.5B-Instruct model card
198Nemotron Nano 9B v2 · 8-bit NVIDIA45±4*†local~13GB131kllama.cpp › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
200Qwen3-VL 4B Thinking · 8-bit Alibaba45±3*†local~6.9GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
201Gemini 2.5 Flash Google45±1api $0.85/M1Mai.google.dev ↗Meituan LongCat-Flash-Chat model card (evaluation table)Berkeley Function Calling Leaderboard result archiveContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsGalileo Agent Leaderboard (v1 and v2 results)Amazon AGI — τ²-Bench-Verified
202EXAONE 4.0 32B · 8-bit LG AI Research45±1local~42GB131kllama.cpp › HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
203Qwen3-VL 30B-A3B Thinking · 8-bit Alibaba44±2*†local~37GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
204Qwen3-VL 30B-A3B Instruct · 8-bit Alibaba44±3*†local~37GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
205Qwen3 32B · 8-bit Alibaba44±1local~43GB131kOllama › Berkeley Function Calling Leaderboard result archiveQwen/Qwen3-Next-80B-A3B-Thinking model cardQwen/Qwen3-Next-80B-A3B-Instruct model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingMUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)+1
206Command A (03-2025) · 2-bit Cohere44±1local~37GB256kllama.cpp › Berkeley Function Calling Leaderboard result archiveCohere — Command A technical reportFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)
207Falcon-H1R 7B · 8-bit TII44±2*†local~12GB262kllama.cpp › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
208Qwen3 235B-A22B (Apr 2025, hybrid) Alibaba44±1api $1.23/M +local~76GB131kwww.alibabacloud.com ↗Berkeley Function Calling Leaderboard result archiveQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-235B-A22B-Thinking-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-235B-A22B-Instruct-2507 model cardHyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)+1
209Qwen3 8B · 8-bit Alibaba44±1local~12GB33kOllama › Berkeley Function Calling Leaderboard result archiveFalcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time ScalingSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)+1
210GPT-4o (2024-08-06) OpenAI44±2api $4.38/M128kdevelopers.openai.com ↗ServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)
211Gemini 2.5 Flash-Lite Google43±3*api $0.17/M1Mai.google.dev ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
212Qwen3-Next 80B-A3B Instruct · 4-bit Alibaba43±2†local~52GB131kOllama › Qwen/Qwen3-Next-80B-A3B-Instruct model card
213Qwen3-VL 8B Instruct · 8-bit Alibaba43±3*local~12GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)allenai/Olmo-3-7B-Instruct model card
214DeepSeek R1 0528 DeepSeek43±1api $0.91/M~110GB164kdeepinfra.com ↗Berkeley Function Calling Leaderboard result archivedeepseek-ai/DeepSeek-V3.1 model cardServiceNow-AI/Apriel-1.6-15b-Thinker model carddeepseek-ai/DeepSeek-R1-0528 model card
215Nemotron-Cascade-8B · 8-bit NVIDIA43±4*†local~11GBllama.cpp › NVIDIA — Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
217Apriel 1.6 15B Thinker · 8-bit ServiceNow43±4†local~21GBllama.cpp › ServiceNow-AI/Apriel-1.6-15b-Thinker model card
218Spark-X2.5 1.7B · 16-bit iFLYTEK (SparkLLM)43±2†local~5.1GB1MOllama › XHToken/Spark-X2.5-4B model card
219Qwen2.5 72B Instruct · 4-bit Alibaba43±1local~48GB33kllama.cpp › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Cohere — Command A technical reportMulti-SWE-bench experiments (ByteDance Seed baseline runs)
220Qwen2.5 32B Instruct · 8-bit Alibaba43±1*local~43GB131kOllama › Berkeley Function Calling Leaderboard result archive
222Qwen3-VL 4B Instruct · 8-bit Alibaba42±3*†local~6.9GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
223Qwen3 30B-A3B Instruct 2507 · 8-bit Alibaba42±1local~41GB262kOllama › Qwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-Next-80B-A3B-Instruct model cardBerkeley Function Calling Leaderboard result archive
226Qwen3-VL 8B Thinking · 8-bit Alibaba42±3*†local~12GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
227gpt-oss-20b · MXFP4 OpenAI42±2local~15GB131kOllama › OpenAI — gpt-oss-120b & gpt-oss-20b Model CardServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trialsMultiverse Computing HyperNova 60B 2602 model card: evaluation table (own runs incl. gpt-oss baselines)T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)
229Llama 3.1 405B Instruct Meta42±2api $2.40/M~160GB128kdocs.aws.amazon.com ↗ServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)MASSIVE-Agents — Kulkarni et al. (2025)Cohere — Command A technical report
230Ministral 3 14B · 8-bit Mistral42±3local~20GB262kOllama › JetBrains/Mellum2-12B-A2.5B-Instruct model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)
231Qwen3 30B-A3B (Apr 2025, hybrid) · 8-bit Alibaba42±1local~41GB33kOllama › Berkeley Function Calling Leaderboard result archiveQwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Instruct-2507 model cardQwen/Qwen3-30B-A3B-Thinking-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
233Granite 4.1 8B · 8-bit IBM41±2*†local~11GB131kllama.cpp › Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)ibm-granite/granite-4.1-8b model card
234GPT-4o mini (2024-07-18) OpenAI41±1api $0.26/M128kdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archiveServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)Galileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
235Granite 4.1 3B · 4-bit IBM41±5*†local~3.7GBOllama › ibm-granite/granite-4.1-8b model card
237Mistral Large 3 (675B-A41B) Mistral41±8*api $0.75/M~108GB262kdocs.mistral.ai ↗One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)
239Ministral 3 8B · 8-bit Mistral41±4*local~13GB262kOllama › ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)
240Mistral Small 3.2 24B · 8-bit Mistral41±3*local~32GB128kOllama › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
241Nova Pro Amazon41±1api $1.40/M300kdocs.aws.amazon.com ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports)
242LFM2.5 8B-A1B · 8-bit Liquid AI41±2local~13GB128kOllama › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-8B-A1B model card
243Qwen3 4B · 8-bit Alibaba41±1local~7.1GB33kOllama › Berkeley Function Calling Leaderboard result archiveQwen/Qwen3-4B-Instruct-2507 model cardQwen/Qwen3-4B-Thinking-2507 model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
244Qwen2.5 14B Instruct · 8-bit Alibaba40±1*local~21GB131kOllama › Berkeley Function Calling Leaderboard result archiveReinforcing Real-world Service Agents: InteractCS-RL (arXiv:2602.22697v1)
245LFM2.5 1.2B Thinking · 8-bit Liquid AI40±4*†local~3.5GB33kOllama › LiquidAI/LFM2.5-1.2B-Thinking model card
247DeepSeek V3 (Dec '24) DeepSeek40±1api $0.46/M~337GB131kdeepinfra.com ↗Berkeley Function Calling Leaderboard result archiveCohere — Command A technical reportdeepseek-ai/DeepSeek-R1 model card
248DeepSeek R1 (Jan '25) DeepSeek40±1api $4.00/M~110GB128kdocs.together.ai ↗Berkeley Function Calling Leaderboard result archivedeepseek-ai/DeepSeek-R1-0528 model carddeepseek-ai/DeepSeek-R1 model cardMulti-SWE-bench experiments (ByteDance Seed baseline runs)
249Mistral Small 3.1 24B (2503) · 8-bit Mistral40±1local~32GB128kOllama › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)
250Mistral Large 2 (2411) · 4-bit Mistral40±1local (slow)~78GB128kllama.cpp › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
251Mistral Large 2 (2407) · 2-bit Mistral40±1*local~45GB128kllama.cpp › Berkeley Function Calling Leaderboard result archiveJamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1)
252QwQ-32B · 8-bit Alibaba40±1*local~43GB131kOllama › Berkeley Function Calling Leaderboard result archive
253Qwen3 4B Instruct 2507 · 8-bit Alibaba40±1local~7.1GB262kOllama › Qwen/Qwen3-4B-Instruct-2507 model cardSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Berkeley Function Calling Leaderboard result archive
254o1 (2024-12-17) OpenAI40±1api $26.25/M200kdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
257Llama 3.1 Nemotron Ultra 253B v1 · 2-bit NVIDIA39±1*local (slow)~82GB128kllama.cpp › Berkeley Function Calling Leaderboard result archive
259Gemma 4 E4B · 8-bit Google39±1local~9.9GB131kllama.cpp › Gemma Team — Gemma 4 Technical Reportopenbmb/MiniCPM5-2B model cardJetBrains/Mellum2.1-12B-A2.5B-Thinking model cardXHToken/Spark-X2.5-4B model cardLiquidAI/LFM2.5-2.6B model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)+3
260Phi-3.5-mini-instruct · 8-bit Microsoft39±11*†local~6.8GB128kOllama › Microsoft — Phi-4-Mini Technical Report
261Qwen3-VL 2B Thinking · 8-bit Alibaba39±3*†local~4.2GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
262Nova 2 Lite Amazon39±3*api $0.94/M1Mdocs.aws.amazon.com ↗Berkeley Function Calling Leaderboard result archiveAmazon AGI — τ²-Bench-Verified
263Nova Lite Amazon39±1api $0.10/M300kdocs.aws.amazon.com ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
264Qwen2.5 7B Instruct · 8-bit Alibaba38±1local~12GB131kOllama › Berkeley Function Calling Leaderboard result archiveSynthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)Reinforcing Real-world Service Agents: InteractCS-RL (arXiv:2602.22697v1)allenai/Olmo-3-7B-Instruct model card
265Granite 4.0 1B · 16-bit IBM38±3*local~6.1GB128kOllama › LiquidAI/LFM2.5-1.2B-Thinking model cardibm-granite/granite-4.0-h-1b model card
268Qwen3 1.7B · 8-bit Alibaba38±1local~4.6GB33kOllama › Berkeley Function Calling Leaderboard result archiveQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)LiquidAI/LFM2.5-1.2B-Thinking model cardEEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)HuggingFaceTB/SmolLM3-3B model card (Evaluation section)
269Qwen3-VL 2B Instruct · 8-bit Alibaba38±3*†local~4.2GB256kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
271Gemma 3 27B · 8-bit Google38±1local~37GB131kOllama › Berkeley Function Calling Leaderboard result archiveGemma Team — Gemma 4 Technical Reportgoogle/gemma-4-31B-it model card
272Ministral 3 14B Reasoning 2512 · 4-bit Mistral38±5*local~10GB262kllama.cpp › JetBrains/Mellum2-12B-A2.5B-Thinking model card
273o3-mini OpenAI37±1api $1.93/M200kdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)ServiceNow-AI/Apriel-1.6-15b-Thinker model cardFinance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
274Llama 4 Maverick (402B-A17B) Meta37±1api $0.42/M +local~65GB1Mdocs.aws.amazon.com ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)Multi-SWE-bench experiments (ByteDance Seed baseline runs)
275GPT-4.1 nano OpenAI37±1api $0.17/M1Mdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archiveOpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards)Galileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)
277Llama 3.1 70B Instruct · 4-bit Meta37±1local~46GB128kOllama › Berkeley Function Calling Leaderboard result archiveServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)MASSIVE-Agents — Kulkarni et al. (2025)
278Nemotron 3 Nano 4B · 8-bit NVIDIA37±2local~7GB262kOllama › openbmb/MiniCPM5-2B model cardNVIDIA vendor-reported benchmark scores for its own Nemotron models (Hugging Face model cards)
279GPT-3.5 Turbo (0125) OpenAI37±1*api $0.75/M16kdevelopers.openai.com ↗Berkeley Function Calling Leaderboard result archive
280Sky-T1 32B Preview · 4-bit NovaSky (UC Berkeley)37±1*local~22GBllama.cpp › Berkeley Function Calling Leaderboard result archive
281Granite 4.0 H 1B · 8-bit IBM37±3*local~3.2GB131kOllama › LiquidAI/LFM2.5-1.2B-Thinking model cardibm-granite/granite-4.0-h-1b model card
282Llama 3.3 70B Instruct · 4-bit Meta36±1local~46GB128kOllama › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Cohere — Command A technical reportContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)
284LFM2.5 1.2B Instruct · 4-bit Liquid AI36±4*†local~2.3GB33kllama.cpp › LiquidAI/LFM2.5-1.2B-Thinking model card
285Command R7B (12-2024) · 8-bit Cohere36±1local~12GB128kOllama › Berkeley Function Calling Leaderboard result archiveCohere — Command A technical report
286Mellum2 12B-A2.5B Thinking · 16-bit JetBrains36±3†local~27GB131khow to run › JetBrains/Mellum2.1-12B-A2.5B-Thinking model cardJetBrains/Mellum2-12B-A2.5B-Thinking model card
287Gemma 3 12B · 8-bit Google36±1*local~17GB131kOllama › Berkeley Function Calling Leaderboard result archive
288Llama 3 70B Instruct · 4-bit Meta36±1*local~47GB8kOllama › Berkeley Function Calling Leaderboard result archive
289Nova Micro Amazon36±1api $0.061/M128kdocs.aws.amazon.com ↗Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)MASSIVE-Agents — Kulkarni et al. (2025)
290Ministral 8B (2410) · 16-bit Mistral36±1*local~22GB128khow to run › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
291Qwen2.5 3B Instruct · 8-bit Alibaba36±1*local~5.9GB33kOllama › Berkeley Function Calling Leaderboard result archive
292Mistral Small Instruct 22B (2409) · 8-bit Mistral AI36±4*local~30GB32kOllama › Galileo Agent Leaderboard (v1 and v2 results)
293Mistral NeMo Instruct (2407) · 8-bit Mistral35±1local~18GB128kOllama › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)
294Granite 3.1 8B Instruct · 8-bit IBM35±4*local~12GB128kOllama › Berkeley Function Calling Leaderboard result archive
295MiniCPM3-4B · 4-bit OpenBMB35±1*local~4.1GB32kllama.cpp › Berkeley Function Calling Leaderboard result archive
296Granite 3.2 8B Instruct · 8-bit IBM35±4*local~12GB131kOllama › Berkeley Function Calling Leaderboard result archive
297Llama 3.1 8B Instruct · 8-bit Meta35±1local~12GB128kOllama › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)MASSIVE-Agents — Kulkarni et al. (2025)
298Command R Plus (04-2024) · 2-bit Cohere34±1*local~43GB128kllama.cpp › Berkeley Function Calling Leaderboard result archive
299DeepSeek R1 0528 Qwen3 8B · 8-bit DeepSeek34±5*local~12GB128kOllama › Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
300Qwen2.5 1.5B Instruct · 8-bit Alibaba34±1*local~4GB33kOllama › Berkeley Function Calling Leaderboard result archive
301Phi-4 14B · 8-bit Microsoft34±1*local~21GB16kOllama › Berkeley Function Calling Leaderboard result archive
302Command R Plus (08-2024) Cohere34±7*†api $4.38/M +local~67GB128kdocs.cohere.com ↗Cohere — Command A technical report
303Qwen3 0.6B · 8-bit Alibaba34±1*local~3GB33kOllama › Berkeley Function Calling Leaderboard result archive
304Hermes 2 Pro Llama 3 8B · 4-bit Nous Research34±1*local~18GB8khow to run › Berkeley Function Calling Leaderboard result archive
305Granite 4.0 H 350M · 8-bit IBM34±5*†local~1.9GB33kOllama › ibm-granite/granite-4.0-h-1b model card
307EXAONE 4.0 1.2B · 8-bit LG AI Research33±3†local~3.6GB66kllama.cpp › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
308Falcon3 10B Instruct · 8-bit TII33±1*local~15GB32kOllama › Berkeley Function Calling Leaderboard result archive
309Llama 4 Scout (109B-A17B) · 2-bit Meta33±1local~36GB328kllama.cpp › Berkeley Function Calling Leaderboard result archiveGalileo Agent Leaderboard (v1 and v2 results)Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)
310Llama 3.2 3B Instruct · 8-bit Meta33±1local~6.1GB131kOllama › Berkeley Function Calling Leaderboard result archiveMASSIVE-Agents — Kulkarni et al. (2025)
312Gemma 4 E2B · 8-bit Google32±2local~6.8GB131kllama.cpp › Gemma Team — Gemma 4 Technical Reportopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardLiquidAI/LFM2.5-2.6B model cardgoogle/gemma-4-31B-it model cardContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)+2
313Falcon3 7B Instruct · 8-bit TII32±1*local~11GB32kOllama › Berkeley Function Calling Leaderboard result archive
314Qwen3.5 2B · 8-bit Alibaba32±2local~4.9GB262kOllama › XHToken/Spark-X2.5-4B model cardopenbmb/MiniCPM5-2B model cardQwen/Qwen3.5-2B model card
315Magistral Small 1.0 (2506) · 8-bit Mistral31±7*local~32GB128kOllama › Galileo Agent Leaderboard (v1 and v2 results)
316Mixtral 8x7B Instruct v0.1 · 8-bit Mistral31±1*local~61GB32kOllama › Berkeley Function Calling Leaderboard result archive
317Mixtral 8x22B Instruct v0.1 · 2-bit Mistral31±1*local~56GB64kllama.cpp › Berkeley Function Calling Leaderboard result archive
318Qwen2 7B Instruct · 8-bit Alibaba31±1*local~12GB131kOllama › Berkeley Function Calling Leaderboard result archive
319Bielik 11B v2.3 Instruct · 8-bit SpeakLeash30±1*local~14GBllama.cpp › Berkeley Function Calling Leaderboard result archive
320Command R (08-2024) · 8-bit Cohere30±8*†local~43GB128kOllama › Cohere — Command A technical report
321Ministral 3 3B · 8-bit Mistral30±6*local~6.9GB262kOllama › ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)
322ChatGLM3-6B · 4-bit Zhipu AI (THUDM)30±10*†local~15GBhow to run › ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)
323GLM-4 9B Chat · 8-bit Zhipu AI (Z.ai)30±1local~14GB128kOllama › Berkeley Function Calling Leaderboard result archiveChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)
324Gemma 3 4B · 8-bit Google30±1*local~7.4GB131kOllama › Berkeley Function Calling Leaderboard result archive
325DBRX Instruct · 4-bit Databricks30±1*local (slow)~85GB32kOllama › Berkeley Function Calling Leaderboard result archive
326Granite 4.0 350M · 16-bit IBM30±3*local~2.9GB32kOllama › Berkeley Function Calling Leaderboard result archiveibm-granite/granite-4.0-h-1b model card
327Olmo 3 7B Instruct · 8-bit Ai229±4local~11GB66kOllama › JetBrains/Mellum2-12B-A2.5B-Instruct model cardallenai/Olmo-3-7B-Instruct model card
329NexusRaven V2 13B · 8-bit Nexusflow28±1*local~16GB16kllama.cpp › Berkeley Function Calling Leaderboard result archive
330Qwen2.5 0.5B Instruct · 8-bit Alibaba28±1*local~2.6GB33kOllama › Berkeley Function Calling Leaderboard result archive
331Qwen3.5 0.8B · 8-bit Alibaba27±5*†local~3.1GB262kOllama › Qwen/Qwen3.5-2B model card
332Llama 3 8B Instruct · 8-bit Meta26±1*local~12GB8kOllama › Berkeley Function Calling Leaderboard result archiveChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2)
333Falcon3 3B Instruct · 8-bit TII26±1*local~6.1GB32kOllama › Berkeley Function Calling Leaderboard result archive
334Phi-4-mini 3.8B · 8-bit Microsoft23±1local~6.9GB128kOllama › Berkeley Function Calling Leaderboard result archiveMicrosoft — Phi-4-Mini Technical Report
335Qwen2 1.5B Instruct · 8-bit Alibaba23±2*local~4GB32kOllama › Berkeley Function Calling Leaderboard result archive
336Llama 3.2 1B Instruct · 4-bit Meta18±1local~5.2GB131kOllama › Berkeley Function Calling Leaderboard result archiveLiquidAI/LFM2.5-1.2B-Thinking model cardMASSIVE-Agents — Kulkarni et al. (2025)
337DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) · 8-bit DeepSeek18±2*local~22GB128kOllama › Berkeley Function Calling Leaderboard result archive
338QwQ-32B-Preview · 8-bit Alibaba17±2*local~43GB33kOllama › Berkeley Function Calling Leaderboard result archive
339Falcon3 1B Instruct · 8-bit TII17±2*local~4.1GB8kOllama › Berkeley Function Calling Leaderboard result archive
340Gemma 3 1B · 4-bit Google13±2local~2.7GB32kOllama › Berkeley Function Calling Leaderboard result archiveLiquidAI/LFM2.5-1.2B-Thinking model card
—Devstral 2 (2512) · q4_k_m Mistralnot yet calibratedlocal (slow)~80GB256kOllama › mistralai/Devstral-Small-2-24B-Instruct-2512 model card
—Devstral Small 2 24B · q8_0 Mistralnot yet calibratedlocal~32GB262kOllama › mistralai/Devstral-Small-2-24B-Instruct-2512 model card
—GLM-4.7-Flash · Q8_0 Z.ainot scored yetlocal~35GB200kllama.cpp ›
—Granite 4.0 H Micro 3B · q4_k_m IBMnot yet calibratedlocal~3.5GB131kllama.cpp › ibm-granite/granite-4.0-h-small model card
—Granite 4.0 H Small (32B-A9B) · Q8_0 IBMnot yet calibratedlocal~38GB131kllama.cpp › ibm-granite/granite-4.0-h-small model card
—Granite 4.0 H Tiny (7B-A1B) · q4_k_m IBMnot yet calibratedlocal~5.9GB131kllama.cpp › ibm-granite/granite-4.0-h-small model card
—Granite 4.0 Micro 3B · Q8_0 IBMnot yet calibratedlocal~5.3GB131kllama.cpp › ibm-granite/granite-4.0-h-small model card
—K2 Horizon 7B · gguf-q8_0 Institute of Foundation Models (MBZUAI)not scored yetlocal~13GB524kllama.cpp ›
—Laguna M.1 Poolsidenot yet calibratedapi price unknown~144GB262khuggingface.co ↗Poolside — Laguna M.1/XS.2 Technical Report
—Laguna XS 2.1 (33B-A3B) · q8_0 Poolsidenot yet calibratedlocal~39GB262kOllama › Poolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)
—Laguna XS.2 · q4_k_m Poolsidenot yet calibratedlocal~26GB262kOllama › Poolside vendor-reported benchmark scores for its own Laguna models (Hugging Face model cards)Poolside — Laguna M.1/XS.2 Technical Report
—Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)not yet calibratedapi $0.090/M +local~80GB131kdeepinfra.com ↗inclusionAI/Ling-3.0-flash model card
—Llama 3.3 Nemotron Super 49B v1 · q4_k_m NVIDIAnot scored yetlocal~33GBllama.cpp ›
—MiniMax-M1 (80k) MiniMaxnot yet calibratedapi $0.96/M~959GB1Mnovita.ai ↗MiniMaxAI/MiniMax-M1-80k model card
—Nemotron Cascade 2 30B-A3B · Q8_0 NVIDIAnot scored yetlocal~37GB256kllama.cpp ›
—Qwen3-Coder-Next (80B-A3B) · q4_k_m Alibabanot scored yetlocal~52GB262kOllama ›
—Step 3.7 Flash (198B-A11B) StepFunnot scored yetapi $0.44/M~104GB256kplatform.stepfun.ai ↗
—Trinity-Large-Thinking Arcee AInot scored yetapi $0.39/M~141GB512kdocs.arcee.ai ↗

+ 62 more don't fit your machine or budget · show them

+ 32 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

362 of 398 catalogue models listed here · sources and licences

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory