whatmodel.sh / GLM-4 (0116)

GLM-4 (0116)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

GLM-4 (0116) is a closed-weights language model from Z.ai, released January 2024. Licence: proprietary.

Catalogue identity notes

Comparison-only. Zhipu AI's original GLM-4 API model, GLM-4 (0116): the ChatGLM report (Team GLM, arXiv 2406.12793v2, CC BY 4.0) states 'Since Janurary 16, 2024, GLM-4 (0116) has been made available through the GLM-4 API at https://bigmodel.cn' and that 'GLM-4 (0520) is slightly better than its original 0116 version'. LMSYS/LMArena key 'glm-4-0116' ('GLM-4-0116', Zhipu AI, Proprietary). Superseded by glm-4-0520; no longer offered.

Model information sources

Release date ↗

49.7±1.3comparison only: no current run option verifiedfrom 7 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of GLM-4 (0116), out of 828 with a score, then more from Z.ai.

#modelscorewhereRAMreleased
418Molmo 72B (0924) Ai250±2local~309GB2024-09
419Codestral 22B v0.1 Mistral AI50±1local~15GB2024-05
420GLM-4 (0116) Z.ai50±1comparison only2024-01
421Llama 3 70B Instruct Meta50±0local~24GB2024-04
422WizardLM-2 8x22B Microsoft50±1local~91GB2024-04
more from Z.ai
33GLM-5.3 Z.ai76±1api $2.15/M +local~239GB2026-08
52GLM-5.3-Flash Z.ai74±1api $0.24/M +local~103GB2026-08
57GLM-5.2 Z.ai74±1api $2.15/M +local~121GB2026-06

see all LLMs · find the ones that fit your machine

$ what it's good at

coding█··········█ #2 of 2 * · 11% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
HumanEval lab72.0% ✎▁▁▁▁▁▃▅▅█▃▂▁ beats 127/190easy · avg 75%▇▇▇▇▇ 11%sharp · at its level · small run
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #293 of 463 * · 43% of its score · 1 test

Beats Mistral Small 4 (119B-A6.5B), Ministral 3 14B Reasoning 2512, Qwen2.5 Coder 32B Instruct here, though they rank higher overall. Trails Flan-T5 XXL, Granite 4.0 H 1B, Granite 4.0 1B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Arena-Hard (v0.1, GPT-4-1106 judge)55.7% ✎▂▁▁▅█▅▄▃▃▅▃▁ beats 38/60medium · avg 52%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 43%sharp · at its level
math█··········█ #2 of 2 * · 16% of its score · 2 tests
testresultvs the others in its rundifficultycountswhy
GSM8K lab87.6% ✎▂▂▂▅▅▆▅▇█▇▄▃ beats 116/146very easy · avg 88%▇▇▇▇ 9.3%sharp · small run
MATH (subset unspecified) lab47.9% ✎▂▄▄▃█▇▅▆▄▃▄▁ beats 22/80medium · avg 60%▇▇▇ 6.8%at its level · small run
other tests3 not in a task · 28% of its score
testresultvs the others in its rundifficultycountswhy
BIG-Bench Hard lab82.3% ✎▁▁▁▁▂▄▅▅█▃▄▁ beats 112/129easy · avg 80%▇▇▇▇ 11%sharp · at its level · small run
GPQA (subset unspecified) lab35.7% ✎▂▁▂▂▄▃▆▃▅█▄▃ beats 31/104hard · avg 38%▇▇▇▇ 10%sharp · small run
MMLU lab81.5% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 143/183easy · avg 79%▇▇▇ 7.7%at its level · small run

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · ChatGLM: A Family of Large La… 56%, Arena-Hard-Auto v0.1 leaderbo… 43%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Team GLM, arXiv 2406.12793v2) · lab-reported6▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 56%CC BY 4.0
Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)1▇▇▇▇▇▇▇▇▇▇▇▇▇ 43%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 49.7

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json