whatmodel.sh / ChatGLM-6B
ChatGLM-6B
↗ share
+ contribute
ChatGLM-6B is an open-weights language model from Zhipu AI (THUDM), released March 2023. Its published size is 6.2B parameters. Licence: chatglm-6b-license.
Catalogue identity notes
Comparison-only. Arena entry 'chatglm-6b' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'chatglm-6b' as 'ChatGLM-6B' (Tsinghua, Non-commercial), linking https://huggingface.co/THUDM/chatglm-6b. Maker repo zai-org/chatglm-6b @bf0f5cf: licence tag none tagged. Parameters: card: 'with 6.2 billion parameters'. Released 2023-03: month the maker's Hugging Face repo was created (createdAt 2023-03-13).
Model information sources
$ how it compares
The LLMs ranked either side of ChatGLM-6B, out of 828 with a score, then more from Zhipu AI (THUDM).
| # | model | score | where | ||
|---|---|---|---|---|---|
| 792 |  Phi-1 (1.3B) | 21±4 | local | ||
| 793 |  Qwen1.5 0.5B Chat | 21±3 | local | ||
| 794 | ChatGLM-6B | 20±2 | local | ||
| 795 |  Pythia 2.8B (deduped) | 20±1 | local | ||
| 796 |  Flan-T5 Base | 20±3 | local | ||
| more from Zhipu AI (THUDM) | |||||
| 349 | GLM-4 9B Chat 1M | 53±5 | local | ||
| 485 | CogVLM2 Llama3 Chinese Chat 19B | 47±7 | comparison only | ||
| 569 | CogVLM2 Llama3 Chat 19B | 43±3 | local | ||
$ what it's good at
coding▁▃▁▃▆██▇▆▄▃▁ #456 of 458 * · 13% of its score · 1 test
Trails GPT-Neo 2.7B here, though they rank lower overall.
chat & writing▃▁▁▃▅▄▅█▆▆▇▃ #391 of 393 · 67% of its score · 4 tests
Trails Gemma 3 270M IT, SmolLM2 360M Instruct here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024) | 874.0 ✎ | ▃▁▂▁▃▇█▆▇▄▂▂ beats 3/95 | 24% |
| LMArena (text) | 995.8 ✎ | ▃▁▁▂▃▃▃▆▆▇█▃ beats 4/379 | 23% |
| MT-Bench (GPT-4 single-answer grading, LMSYS runs) | 4.5000 MT-Bench score (1-10) ✎ | ▆▂▂▆▆▄██▂▄▄▆ beats 6/28 | 21% |
| MixInstruct GPT-Rank (LLM-Blender, mean rank among 11 instruction-tuned LLMs, ChatGPT pairwise judge) | 5.6300 mean rank (1-11) ✎ | █···▄··▄···▄ beats 3/4 | 0.0% |
other tests2 not in a task · 18% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| BIG-Bench Hard | 31.3% ✎ | ███··█·█·█·█ beats 2/6 | 12% |
| MMLU | 36.1% ✎ | ▄▄·▄···▄▄·██ beats 4/8 | 6.2% |
$ where to run it
Other ways to run it
- Run it as published · needs ~13.42 GB, NVIDIA only · from huggingface.co/zai-org
pip install protobuf==3.20.0 transformers==4.27.1 icetk cpm_kernels torch && python -c "from transformers import AutoTokenizer, AutoModel; t=AutoTokenizer.from_pretrained('THUDM/chatglm-6b', trust_remote_code=True); m=AutoModel.from_pretrained('THUDM/chatglm-6b', trust_remote_code=True).half().cuda().eval(); print(m.chat(t, '你好', history=[])[0])"
Weights and documentation: huggingface.co/zai-org ↗
$ how the score was worked out · InstructEval (DeCLaRe Lab) RE… 32%, Polyrating: A Cost-Effective… 24%, LMArena leaderboard dataset 23% and 2 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 InstructEval (DeCLaRe Lab) README results table | 3 | 32% |
 Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3) | 1 | 24% |
 LMArena leaderboard dataset | 1 | 23% |
 LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table | 1 | 21% |
| LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (arXiv:2306.02561v3) | 1 | 0% |
| prior · pull toward the middle | 1% | |
| weighted average | = 20.3 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json