whatmodel.sh / Baichuan2-7B-Base

Baichuan2-7B-Base

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Baichuan2-7B-Base is an open-weights language model from Baichuan, released August 2023. Licence: other.

Catalogue identity notes

Maker repo baichuan-inc/Baichuan2-7B-Base @f9d4d8d: licence tag other. Released 2023-08: month the maker's Hugging Face repo was created (createdAt 2023-08-30). Pretrained base checkpoint (not an instruct/chat version).

29.6±0.9#645 of 719 LLMsfrom 11 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of Baichuan2-7B-Base, out of 828 with a score.

#modelscorewhereRAMreleased
740Falcon 7B (base) TII30±1local~5.9GB2023-04
741Llama 2 7B Chat Meta30±1local~5.8GB2023-07
742Baichuan2-7B-Base Baichuan30±1local~17GB2023-08
743LLaMA 7B Meta30±1comparison only2023-02
744MPT-7B (base) MosaicML30±1comparison only2023-05

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests11 not in a task · 100% of its score
testresultvs the others in its rundifficultycountswhy
ARC Challenge43.2% ✎▁·▁··▁▁▇▂█▆▃ beats 18/26easy · avg 85%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 47%sharp
WinoGrande67.6% ✎▁····▁··▁▂█▄ beats 18/26easy · avg 89%▇▇▇▇▇ 12%at its level
BoolQ73.1% ✎▂▃▂▅▆▂▅▆█▂▂▂ beats 17/26easy · avg 91%▇▇▇▇ 9.5%at its level
ARC Easy72.8% ✎▁▁·▁▁▄▅▄█▅▁▃ beats 21/26easy · avg 88%▇▇▇ 6.8%at its level
HellaSwag72.3% ✎▃···▁▇▅█▇▃▁▁ beats 15/26very easy · avg 92%▇▇ 5.4%at its level
SciQ94.6% ✎▁·▁·▁·▃▅▃█▄▁ beats 24/26very easy · avg 98%▇▇ 5.1%typical
PIQA76.2% ✎▁·▁··▁▁▃█▅█▂ beats 10/26easy · avg 88%▇▇ 4.3%blunt · at its level
OpenBookQA30.4% ✎█·▁▁▃·▁▃▅▇▃▄ beats 16/26medium · avg 53%▇ 3.6%too hard for it
3 more tests
testresultvs the others in its rundifficultycountswhy
LAMBADA (OpenAI version), last-word accuracy71.0% ✎▁··▂▁▁▁▂▆█▅▃ beats 17/26very easy · avg 89%▇ 2.6%blunt · at its level
Social IQA41.8% ✎▁···▁▃█▁▁▁·▁ beats 8/26medium · avg 59%▇ 2.5%blunt
Open PT-LLM Leaderboard average (9 Portuguese tasks)46.8% ✎▃▁▁▁▁▁▃▁▂█▅▁ beats 61/276medium · avg 73%▇ 1.2%noisy

$ where to run it

Other ways to run it

  1. Run it as published · needs ~15.01 GB · from huggingface.co/baichuan-inc (official)
    uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='baichuan-inc/Baichuan2-7B-Base', torch_dtype='auto', device_map='auto', trust_remote_code=True)('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/baichuan-inc ↗

$ how the score was worked out · Stability AI — original Stabl… 98%, Open Portuguese LLM Leaderboa… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Stability AI — original StableLM zero-shot evaluation outputs10▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 98%Apache-2.0
Open Portuguese LLM Leaderboard (archived)1▇ 1%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 29.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json