whatmodel.sh / GLM-5.1

GLM-5.1

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

GLM-5.1 is an open-weights language model from Z.ai, released April 2026. Its published size is 744B parameters (40B active per token) and a 200k-token context window. Licence: mit.

70.8±0.5#76 of 719 LLMsfrom 81 benchmark results across 17 sources

$ how it compares

The LLMs ranked either side of GLM-5.1, out of 828 with a score, then more from Z.ai.

#modelscorewhereRAMreleased
79Mistral Large 4 (public preview) Mistral71±2api $2.06/M2026-10
80GPT-5.1 OpenAI71±1api $3.44/M2025-11
81GLM-5.1 Z.ai71±1api $2.15/M +local~119GB2026-04
82Qwen3.5 397B-A17B Alibaba71±0api $1.09/M +local~65GB2026-02
83Ornith 1.5 35B-A3B DeepReinforce71±2local~22GB2026-08
more from Z.ai
33GLM-5.3 Z.ai76±1api $2.15/M +local~239GB2026-08
52GLM-5.3-Flash Z.ai74±1api $0.24/M +local~103GB2026-08
57GLM-5.2 Z.ai74±1api $2.15/M +local~121GB2026-06

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #55 of 300 · 26% of its score · 19 tests

Beats Gemini 3.1 Pro (Preview), GPT-5.2, Gemini 3.6 Flash here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
SWE-bench Verified +274.2% ✎▁··▁·▁▂▁▄█▄▁ beats 16/29hard · avg 19%▇▇▇ 7.3%sharp · at its level · several runs
Terminal-Bench 2.1 +159.3% ✎█··▄█·▄····▄ beats 5/6very hard · avg 12%▇ 2.6%at its level · several runs
MCP-Atlas (public set) lab +171.8% ✎█··█·██·████ beats 1/7hard · avg 29%▇ 1.8%at its level · several runs
SWE-bench Multilingual74.8% ✎▄··▄▄···▄▄·█ beats 3/6hard · avg 30%▇ 1.7%at its level
BrowseComp +159.4% ✎▄·▄▄···▄··█▄ beats 5/6hard · avg 16%▇ 1.6%at its level · several runs
Humanity's Last Exam with tools +150.4% ✎▄····▄·█·▄·▄ beats 4/5hard · avg 16%▇ 1.4%at its level · several runs
K-BrowseComp-Verified (Korean web-browsing agent questions, pass@1)30.7% ✎▃····█··▃█·▃ beats 6/8very hard · avg 3.4%▇ 1.2%typical
SWE-bench Pro (split unspecified) lab58.4% ✎▃·▁▁▁▂▂▃█▇▃▁ beats 38/55very hard · avg 9.6%▇ 1.2%sharp · at its level · small run
11 more tests
testresultvs the others in its rundifficultycountswhy
τ³-bench +169.7% ✎▄····▄·█·▄·█ beats 1/6medium · avg 40%▇ 1.2%blunt · at its level · several runs
Toolathlon (Tool Decathlon) lab +140.7% ✎▂▂▄▂▄▄▆▂█▆██ beats 11/27very hard · avg 12%▇ 1.0%at its level · small run · several runs
Terminal-Bench 2.0 lab63.5% ✎▂▂▅▅████▆▂▃▃ beats 28/36very hard · avg 14%▇ 1.0%at its level · small run
Thinkingbox-bench pass@1 (507 stateful business-workflow agent tasks, N=20)33.2% ✎▅··▃·▃·█▃▅██ beats 6/15very hard · avg 4.9%▇ 0.8%at its level
Legal Research Bench (agentic, all-pass)27.6% ✎▄█··▄▄▄██·▄█ beats 6/12very hard · avg 5.8%▇ 0.6%typical
EEGAgentBench overall (tool-based EEG analysis agents, 6 tasks)46.7% ✎▂·▁▁··▇▇█▂·▁ beats 14/25hard · avg 29%▇ 0.5%blunt · at its level
τ³-bench telecom (tau2-bench ≥ v1.0.1)96.9% ✎▄·▄···▄·▄·█▄ beats 3/6easy · avg 77%▇ 0.5%maxed out
BrowseComp with context management lab79.3% ✎▅·▅▃·█▃·█▅·▃ beats 8/14medium · avg 43%▇ 0.5%small run
TerminalWorld-Verified (200 tasks) with Terminus-257.0% ✎▄··▄·▄·█···▄ beats 3/5hard · avg 27%▇ 0.4%blunt · at its level
KlinikeBench strict pass@1 (333 interactive clinical encounters, Terminus-2)4.8% ✎▁▁·▁▁▁▃▅▃█▅█ beats 5/27very hard · avg 2.4%▇ 0.4%typical
SWE-bench Pro lab58.4% ✎▅▃▃·▃·▅▅▅█▃▃ beats 11/15hard · avg 31%▇ 0.3%blunt · at its level · small run
coding▁▁▁▃▆██▇▆▄▃▁ #53 of 458 · 18% of its score · 9 tests

Beats GPT-5.4, DeepSeek V4 Flash Vision (Exp), Gemini 3.1 Pro (Preview) here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
SWE-bench Verified +274.2% ✎▁··▁·▁▂▁▄█▄▁ beats 16/29hard · avg 19%▇▇▇ 7.3%sharp · at its level · several runs
Arena WebDev (Code Arena)1508.5 ✎▁▂▄▃▅█▄▇▆▄▁▁ beats 71/119avg 981▇ 3.2%sharp
SWE-bench Multilingual74.8% ✎▄··▄▄···▄▄·█ beats 3/6hard · avg 30%▇ 1.7%at its level
SWE-bench Pro (split unspecified) lab58.4% ✎▃·▁▁▁▂▂▃█▇▃▁ beats 38/55very hard · avg 9.6%▇ 1.2%sharp · at its level · small run
SciCode subproblems47.7% ✎▃····▃··█·▃▃ beats 2/6hard · avg 17%▇ 1.1%at its level
LiveCodeBench v6 (window unspecified)85.7% ✎██···█··████ beats 2/6medium · avg 45%▇ 1.0%typical
SciCode (level unspecified)43.8% ✎▄·····▄█▄▄·▄ beats 4/6hard · avg 16%▇ 0.9%at its level
LiveCodeBench v6 (Feb–May 2025)86.4% ✎▄··█····▄▄·█ beats 5/6medium · avg 45%▇ 0.9%typical
1 more test
testresultvs the others in its rundifficultycountswhy
SWE-bench Pro lab58.4% ✎▅▃▃·▃·▅▅▅█▃▃ beats 11/15hard · avg 31%▇ 0.3%blunt · at its level · small run
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #39 of 393 * · 2.3% of its score · 2 tests

Beats GPT-6 Sol, MiMo-V2.6-Flash (309B-A15B), GPT-6 Luna here, though they rank higher overall. Trails MiMo-V2.5-Pro (1T-A42B) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1464.6 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 334/379avg 1268▇ 1.8%blunt
IFEval (aggregation unspecified)93.9% ✎▁▁▃▁▂▄▄▆█▃▃▁ beats 136/141easy · avg 80%▇ 0.5%maxed out
math▁▂▃▅▆▅▄█▄▁▁▁ #73 of 398 · 33% of its score · 16 tests

Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.7 Plus here, though they rank higher overall. Trails Granite 4.2 30B, INTELLECT-3, EXAONE 4.5 33B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
FrontierMath tiers 1–3 v236.8% ✎▂··▁▁▃▄▄█▅▃▃ beats 27/82very hard · avg 1.2%▇▇▇▇ 8.8%sharp · at its level
MathArena ArXivMath April 2026 (40 problems)36.3% ✎▁·····▅██▄·▁ beats 4/20very hard · avg 7.2%▇▇ 4.1%sharp · at its level
AIME 2026 +295.4% ✎▄·▄·▄▄··█··▄ beats 5/6medium · avg 61%▇ 2.8%sharp · maxed out · small run · several runs
HMMT February 2026 lab +182.6% ✎▃▃▂▃·██▆█▅▃▅ beats 21/33hard · avg 30%▇ 2.3%sharp · small run · several runs
MathArena ArXivMath February 2026 (32 problems)39.1% ✎▁▁▁▁█▇▄▃▁▃·▃ beats 11/24very hard · avg 9.1%▇ 2.3%at its level
MathArena ArXivMath March 2026 (30 problems)49.2% ✎█▃▅·█▃▃·▅·▃▃ beats 3/14very hard · avg 15%▇ 2.1%at its level
IMO-AnswerBench +186.8% ✎▄····▄·▄▄·█▄ beats 2/6medium · avg 39%▇ 1.9%small run · several runs
MathArena ArXivMath January 2026 (23 problems)65.2% ✎▂··▄▂▄█▆▆▄▄█ beats 16/23hard · avg 22%▇ 1.6%at its level
8 more tests
testresultvs the others in its rundifficultycountswhy
MathArena ArXivMath May 2026 (40 problems)52.5% ✎▁··▁▄█▄▁·▃·▁ beats 6/17very hard · avg 10%▇ 1.5%at its level
MathArena BrokenArXiv April 2026 (61 problems)8.2% ✎▂▃▆█▃▃▂▂▂··▂ beats 1/18very hard · avg 0.7%▇ 1.1%sharp · noisy
MathArena BrokenArXiv May 2026 (50 problems)10.0% ✎██·▄▄▄····▄▄ beats 1/8very hard · avg 1.6%▇ 0.8%sharp
OTIS mock AIME 2024–202593.3% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 156/193very hard · avg 8.5%▇ 0.8%sharp · noisy
MathArena BrokenArXiv March 2026 (56 problems)6.5% ✎█·▆█▂·▂▂···▂ beats 2/14very hard · avg 0.8%▇ 0.7%sharp · noisy
MathArena BrokenArXiv February 2026 (31 problems)9.3% ✎▃▂▂▃█▃▂·▃··▂ beats 4/16very hard · avg 0.7%▇ 0.6%sharp · noisy
HMMT November 2025 lab94.0% ✎▁▁····▁▅▃█▇▁ beats 20/23easy · avg 70%▇ 0.6%maxed out · small run
MathArena ArXivLean March 2026 (41 problems)2.4% ✎▂····▃·····█ beats 1/5very hard · avg 1.0%▇ 0.5%too hard for it
web search▂▁▂▆▄█▃▃▁▁·▁ #48 of 80 · 5.4% of its score · 5 tests

Beats Gemini 3.1 Pro (Preview), GPT-5.2, Kimi K2.6 here, though they rank higher overall. Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), Llama 4 Scout (109B-A17B) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
BrowseComp +159.4% ✎▄·▄▄···▄··█▄ beats 5/6hard · avg 16%▇ 1.6%at its level · several runs
Humanity's Last Exam with tools +150.4% ✎▄····▄·█·▄·▄ beats 4/5hard · avg 16%▇ 1.4%at its level · several runs
K-BrowseComp-Verified (Korean web-browsing agent questions, pass@1)30.7% ✎▃····█··▃█·▃ beats 6/8very hard · avg 3.4%▇ 1.2%typical
Legal Research Bench (agentic, all-pass)27.6% ✎▄█··▄▄▄██·▄█ beats 6/12very hard · avg 5.8%▇ 0.6%typical
BrowseComp with context management lab79.3% ✎▅·▅▃·█▃·█▅·▃ beats 8/14medium · avg 43%▇ 0.5%small run
other tests18 not in a task · 32% of its score
testresultvs the others in its rundifficultycountswhy
GPQA Diamond +389.9% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 173/208hard · avg 45%▇▇▇ 8.2%sharp · several runs
Humanity's Last Exam +327.2% ✎▄··█▄··▄·▄·▄ beats 2/6very hard · avg 5.6%▇▇ 5.1%at its level · several runs
GPQA (subset unspecified)86.1% ✎▄█▄▄·▄·····▄ beats 0/6hard · avg 47%▇ 2.9%sharp
MMLU-Pro +185.9% ✎▄·····▄▄▄·▄█ beats 1/6medium · avg 53%▇ 2.5%several runs
Global-MMLU-Lite90.3% ✎▃······▅··█▃ beats 3/6medium · avg 60%▇ 1.6%typical
NL2Repo lab +142.7% ✎▂▂·▂▂▄█▆▆▂▄▆ beats 12/21very hard · avg 11%▇ 1.3%at its level · small run · several runs
EQ-Bench 3 (emotional intelligence, judge Elo)1309.6 ✎▁·▂▁▄▅▅▃█▄▁▁ beats 51/62avg 1044▇ 1.1%typical
Chess Puzzles (Epoch)19.0% ✎█▁▁▁▁▂▃▄▂▂▁▁ beats 75/135very hard · avg 5.9%▇ 1.1%sharp · noisy
10 more tests
testresultvs the others in its rundifficultycountswhy
WildClawBench48.2% ✎▃·▅▅█·▅█▅▃▃▃ beats 13/17very hard · avg 11%▇ 1.0%at its level
AgentWorldBench (open-ended, overall)51.3% ✎▄▄··▄▄·▄█▄██ beats 3/11hard · avg 25%▇ 1.0%blunt · at its level
MMLU-ProX85.8% ✎▄·····▄·▄▄█▄ beats 5/6medium · avg 57%▇ 1.0%typical
IFBench (aggregation unspecified)76.3% ✎▃······█▃▃·▃ beats 4/6medium · avg 44%▇ 0.9%typical
IFBench prompt-level (loose)76.6% ✎▄▄·▄··▄▄···█ beats 2/6medium · avg 45%▇ 0.8%typical
CritPt +13.7% ✎▄····▄█···█▄ beats 3/6very hard · avg 1.0%▇ 0.8%too hard for it · noisy · several runs
MultiChallenge63.0% ✎▁··········█ beats 1/6hard · avg 27%▇ 0.8%at its level
SimpleQA Verified34.0% ✎▁▁▃▂▃▃▆▆█▃▂▅ beats 27/78very hard · avg 8.0%▇ 0.7%at its level
Legal Research Bench (single-shot, no tools, all-pass)3.1% ✎▃▃█▃·▅···█▃▃ beats 4/12very hard · avg 0.8%▇ 0.6%too hard for it
CyberGym lab68.7% ✎▄···▄··▄█▄█▄ beats 2/8hard · avg 27%▇ 0.4%at its level · small run

$ where to run it

Through an API

api · $2.15/M · $1.4 in / $4.4 out per 1M tokens · 200k context · docs.z.ai ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run GLM-5.1 · needs ~446.9 GB · from huggingface.co/unsloth
    chat in the terminal
    llama cli -hf unsloth/GLM-5.1-GGUF:UD-Q4_K_XL
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf unsloth/GLM-5.1-GGUF:UD-Q4_K_XL

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run GLM-5.1 · needs ~446.9 GB · from huggingface.co/unsloth

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/unsloth/GLM-5.1-GGUF@UD-Q4_K_XL -y
  4. Then run lms ls to see its name, and lms chat followed by that name

MLX (Mac)

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run GLM-5.1 · needs ~418.6 GB, Mac only · from huggingface.co/catalystsec
    chat in the terminal
    mlx_lm.chat --model catalystsec/GLM-5.1-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model catalystsec/GLM-5.1-4bit

Weights and documentation: huggingface.co/zai-org ↗

$ how the score was worked out · NVIDIA — Nemotron 3 Ultra tec… 21%, Epoch AI — internal evaluatio… 17%, MathArena ArXivMath / BrokenA… 15% and 14 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
NVIDIA — Nemotron 3 Ultra technical report17▇▇▇▇▇▇ 21%CC BY 4.0
Epoch AI — internal evaluations6▇▇▇▇▇ 17%CC BY 4.0
MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets10▇▇▇▇▇ 15%CC BY-SA 4.0
zai-org/GLM-5.1 model card · lab-reported13▇▇▇ 11%mit
zai-org/GLM-5.2 model card · lab-reported13▇▇▇ 10%mit
A.X K2 Technical Report (SK Telecom), arXiv 2608.30181v17▇▇▇ 10%CC BY 4.0
LMArena leaderboard dataset2▇▇ 5%CC BY 4.0
LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card3▇ 2%Apache-2.0
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)1▇ 1%CC BY 4.0
EQ-Bench 3 canonical leaderboard results1▇ 1%MIT
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)1▇ 1%CC BY 4.0
Qwen/Qwen-AgentWorld-35B-A3B model card1▇ 1%Apache-2.0
One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)1▇ 1%CC BY 4.0
EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)1▇ 1%CC BY 4.0
TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks — authors' own Terminus-2 model comparison1▇ 0%CC BY 4.0
KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)1▇ 0%CC BY 4.0
prior · pull toward the middle▇ 0%
weighted average= 70.8

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json