Zephyr 7B beta

+ contribute

Zephyr 7B beta is an open-weights language model from Hugging Face, released October 2023. Its published size is 7.24B parameters. Licence: mit.

Catalogue identity notes

Arena entry 'zephyr-7b-beta' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'zephyr-7b-beta' as 'Zephyr-7B-beta' (HuggingFace, MIT), linking https://huggingface.co/HuggingFaceH4/zephyr-7b-beta. Maker repo HuggingFaceH4/zephyr-7b-beta @892b3d7: licence tag mit. 7,241,732,096 parameters from the Hugging Face safetensors metadata (dense, so active = total). Released 2023-10: month the maker's Hugging Face repo was created (createdAt 2023-10-26).

33.5±0.6#612 of 719 LLMsfrom 20 benchmark results across 9 sources

$ how it compares

The LLMs ranked either side of Zephyr 7B beta, out of 828 with a score, then more from Hugging Face.

#	model	score	where	RAM	released
705	Nous-Hermes-13b Nous Research	34±2	local	~9.8GB	2023-06
706	ChatGLM3 6B 128K Zhipu AI (THUDM)	34±4	local	~5.8GB	2024-01
707	Zephyr 7B beta Hugging Face	34±1	local	~6.1GB	2023-10
708	PaLM 2 Chat (chat-bison@001) Google	34±2	comparison only		2023-05
709	Qwen3 0.6B Alibaba	33±1	local	~1.8GB	2025-04
	more from Hugging Face
451	StarChat2 15B v0.1 Hugging Face	48±1	local	~12GB	2024-03
532	Zephyr ORPO 141B-A35B v0.1 Hugging Face	45±2	local	~56GB	2024-04
577	SmolLM3 3B Hugging Face	43±1	local	~3.3GB	2025-07

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #427 of 458 · 61% of its score · 4 tests

Beats DeepSeek R1 Distill Qwen 7B, Qwen3.5 2B, DeepSeek R1 Distill Llama 8B here, though they rank higher overall. Trails Llama 3.2 1B Instruct, SmolLM2 1.7B Instruct, Mistral 7B Instruct v0.1 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
HumanEval +1	30.0% ✎	▁·▁▁▃▄▆▅█▃▁▁ beats 15/99	easy · avg 79%	▇▇▇▇▇▇▇▇▇▇▇▇▇ 33%	sharp · at its level · several runs
HumanEval+	23.2% ✎	▁·▁▁▁▃▄▄▄█▃▁ beats 11/99	easy · avg 74%	▇▇▇▇▇▇▇▇ 20%	sharp · at its level
MBPP+	34.7% ✎	▁···▁▃▂▅▇█▆▂ beats 3/75	easy · avg 68%	▇▇ 4.3%	at its level
Tulu 2: CodexEval pass@10	54.3% ✎	▃▃▃·▆▃▃█▃▂▂▅ beats 9/25	very easy · avg 86%	▇ 3.0%	sharp · at its level

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #361 of 393 · 14% of its score · 4 tests

Beats Gemma 4 E2B, Code Llama 70B Instruct, Phi-3-mini-128k-instruct (Apr 2024) here, though they rank higher overall. Trails Gemma 3 270M IT, SmolLM2 360M Instruct, Granite 4.0 350M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
AlpacaEval 2 length-controlled	13.2% ✎	▁▂▂▃▆▆█▆▄▄▁▁ beats 32/122	medium · avg 39%	▇▇ 5.4%	typical
Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024)	1049.0 ✎	▂▁▂▁▃▇█▆▇▄▂▂ beats 25/95	avg 1201	▇ 3.6%	blunt
LMArena (text)	1130.8 ✎	▁▁▁▃▃▃▃▆▆▇█▃ beats 28/379	avg 1268	▇ 3.5%	blunt
AlpacaEval 1.0 win rate (vs text-davinci-003, GPT-4 judge)	90.6% ✎	▂▂·▆·▇▃▆█▆▄▂ beats 32/45	very easy · avg 96%	▇ 1.8%	sharp · at its level · noisy

math▁▂▃▅▆▅▄█▄▁▁▁ #360 of 398 * · 1.6% of its score · 1 test

Beats Yi 34B Chat, Llama 3 8B Instruct, Command R (03-2024) here, though they rank higher overall. Trails Gemma 3 1B, Stable Code 3B, Qwen3 0.6B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
GSM8K +1	28.0% ✎	▆▂▅█▅▅▃▆···▂ beats 8/25	easy · avg 73%	▇ 1.6%	sharp · at its level

other tests9 not in a task · 23% of its score

test	result	vs the others in its run	difficulty	counts	why
LiveBench 2024-07-26	16.7% ✎	▁▁▁▂▃▃▂█▆▃▅▃ beats 10/61	medium · avg 41%	▇▇▇ 7.7%	typical
Natural Questions (closed book)	25.2% ✎	▁·▁·▁▂▁▂·▅▂█ beats 20/28	medium · avg 58%	▇▇ 5.1%	typical
BIG-Bench Hard +1	44.9% ✎	▁·····▂█▂▅·▁ beats 5/25	easy · avg 77%	▇▇ 4.3%	sharp
MMLU +1	58.6% ✎	▁▁▃▃▄█▄▁█··▁ beats 17/25	easy · avg 74%	▇ 3.2%	at its level · several runs
Open PT-LLM Leaderboard average (9 Portuguese tasks)	64.5% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 111/276	medium · avg 73%	▇ 0.6%	at its level · noisy
TriviaQA	64.4% ✎	▁··▁█▃▃▁▄▅▄█ beats 24/28	easy · avg 79%	▇ 0.6%	at its level · noisy
Tulu 2: TyDiQA gold passage, 1-shot F1	23.7% ✎	▃··▁▃▂▂▂█▅▁▂ beats 1/25	medium · avg 61%	▇ 0.6%	blunt · at its level
HaluEval QA hallucination detection (accuracy)	51.8% ✎	█▁▁·▃▁·▁▁▁▁▁ beats 19/27	medium · avg 70%	▇ 0.3%	sharp · noisy

1 more test

test	result	vs the others in its run	difficulty	counts	why
TruthfulQA MC2	55.1% ✎	▅▃█▆▁▂▅···▁▁ beats 26/31	medium · avg 64%	▇ 0.3%	blunt · at its level

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Zephyr 7B beta · needs ~4.4 GB · from huggingface.co/TheBloke
chat in the terminal
llama cli -hf TheBloke/zephyr-7B-beta-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf TheBloke/zephyr-7B-beta-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Zephyr 7B beta · needs ~4.4 GB · from huggingface.co/TheBloke
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Zephyr 7B beta · needs ~4.84 GB · from ollama.com
chat in the terminal
ollama run zephyr:7b-beta-q4_K_M

Weights and documentation: huggingface.co/HuggingFaceH4 ↗

$ how the score was worked out · EvalPlus / EvalPerf published… 55%, Tulu 2 — base and instruction… 11%, LiveBench model judgments (HF… 8% and 6 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
EvalPlus / EvalPerf published evaluation runs	3	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 55%	Apache-2.0
Tulu 2 — base and instruction-tuned evaluations	5	▇▇▇ 11%	CC BY 4.0
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets	1	▇▇ 8%	Apache-2.0 (stated in the LiveBench datasheet)
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)	2	▇▇ 7%	Apache-2.0
Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)	4	▇▇ 6%	Apache-2.0
OpenChat 3.5 model card (openchat/openchat_3.5)	2	▇ 4%	Apache-2.0
Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)	1	▇ 4%	CC-BY-4.0
LMArena leaderboard dataset	1	▇ 4%	CC BY 4.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 33.5

not counted (2)

OpenChat 3.5 model card (openchat/openchat_3.5)	BIG-Bench Hard [correct]	16.0%	failed run: Few-shot CoT format failure in a chat-mode harness: 16.0 vs 44.9 for the same checkpoint in the Tulu 2 paper (3-shot CoT), and below BBH's published random average of 25.7.
OpenChat 3.5 model card (openchat/openchat_3.5)	GSM8K [correct]	5.1%	failed run: Same few-shot CoT format failure: 5.1 vs 28.0 in the Tulu 2 paper's 8-shot CoT run.

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json