whatmodel.sh / Yi 1.5 34B Chat

Yi 1.5 34B Chat

+ contribute

Yi 1.5 34B Chat is an open-weights language model from 01.AI, released May 2024. Its published size is 34.39B parameters and a 4k-token context window. Licence: apache-2.0.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-05-13). Yi-1.5-34B-Chat (May 2024). 34,388,917,248 parameters.

45.8±0.7#427 of 719 LLMsfrom 20 benchmark results across 11 sources

$ how it compares

The LLMs ranked either side of Yi 1.5 34B Chat, out of 828 with a score, then more from 01.AI.

#	model	score	where	RAM	released
506	Qwen2-VL 7B Instruct Alibaba	46±1	local	~6.4GB	2024-08
507	Qwen2.5-Math 1.5B Instruct Alibaba	46±2	local	~2.4GB	2024-09
508	Yi 1.5 34B Chat 01.AI	46±1	local	~22GB	2024-05
509	DBRX Instruct Databricks	46±1	local	~85GB	2024-03
510	Granite 34B Code Instruct (8K) IBM	46±1	local	~22GB	2024-04
	more from 01.AI
308	Yi-Lightning 01.AI	55±1	api price unknown		2024-10
373	Yi-Large Preview 01.AI	52±1	comparison only		2024-05
396	Yi-Large 01.AI	51±1	comparison only		2024-05

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #294 of 458 · 17% of its score · 5 tests

Beats Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B), GLM-4.7-Flash here, though they rank higher overall. Trails DeepSeek LLM 67B Chat, WizardLM 70B V1.0, Yi 1.5 9B Chat here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CRUXEval (ZeroEval, 800 problems)	44.1% ✎	▁▂▆▃▆█▆▃▂·▁▁ beats 7/47	medium · avg 61%	▇▇▇▇▇ 13%	sharp · at its level
CodeReviewQA: solution identification (hard), invariant accuracy	57.0% ✎	▃··▁▂▂▃▂█▃▄▂ beats 32/47	medium · avg 54%	▇ 1.2%	sharp · at its level · noisy
CodeReviewQA: change localisation (hard), invariant accuracy	24.6% ✎	█▂▁▁▂▂▂▃▅▅▅▇ beats 23/47	medium · avg 36%	▇ 1.2%	sharp · noisy
CodeReviewQA: automated code refinement, exact match rate	31.1% ✎	▁▁▁▁▁▁▂▂▃█▅▄ beats 24/47	hard · avg 33%	▇ 0.9%	at its level
CodeReviewQA: change type recognition, invariant accuracy	66.1% ✎	▁▁▁······▁▃█ beats 15/47	easy · avg 67%	▇ 0.0%	blunt · at its level · noisy

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #299 of 393 * · 9.1% of its score · 2 tests

Beats Ministral 3 14B Reasoning 2512, Gemma 4 E4B, OpenThinker3 7B here, though they rank higher overall. Trails LFM2 350M, Flan-T5 XXL, LFM2 700M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024)	1126.0 ✎	▂▁▂▁▃▇█▆▇▄▂▂ beats 59/95	avg 1201	▇▇ 4.6%	blunt
LMArena (text)	1212.8 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 74/379	avg 1268	▇▇ 4.5%	blunt

math▁▂▃▅▆▅▄█▄▁▁▁ #312 of 398 · 19% of its score · 3 tests

Beats Athene-70B, OLMo 2 32B Instruct (0325), Hermes 2 Theta Llama-3 70B here, though they rank higher overall. Trails Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 13B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
GSM8K (ZeroEval, zero-shot)	84.1% ✎	▁▄·▁▄█▅▅▅███ beats 17/43	very easy · avg 89%	▇▇▇ 7.9%	sharp
MATH level 5	25.5% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 24/92	hard · avg 32%	▇▇▇ 6.6%	sharp
MATH Level 5 (ZeroEval, 721 problems)	18.2% ✎	▁▁▁▆█▂▇▇▄··▂ beats 5/38	hard · avg 34%	▇▇ 4.9%	sharp

other tests10 not in a task · 54% of its score

test	result	vs the others in its run	difficulty	counts	why
MMLU-Redux (ZeroEval, 2,778 questions)	72.8% ✎	▁·▁▁▄▄▄▅█▄▂▁ beats 17/53	easy · avg 77%	▇▇▇▇▇▇▇▇▇ 23%	at its level
BigCodeBench (Instruct)	33.5% ✎	▃▂·▆▆██▆▅▃▃▃ beats 12/33	medium · avg 41%	▇▇ 5.5%	at its level
BigCodeBench (Complete)	43.3% ✎	▂▁▁▄▂▅▇█▆▂▂▂ beats 37/58	medium · avg 50%	▇▇ 5.5%	at its level
WildBench v2 WB-Score (task macro)	45.6135 WB-Score ✎	▁·▁▁·▃█▄▇▆▅▆ beats 35/52	avg 45.199 WB-Score	▇▇ 5.2%	sharp
ZebraLogic (ZeroEval, 1,000 puzzles)	11.5% ✎	▁▁▃▆██▁▁·▁▁▁ beats 11/57	hard · avg 18%	▇▇ 4.5%	sharp
GPQA Diamond	32.0% ✎	▁▁▁▁▃▄▆▄▅▇█▅ beats 20/208	hard · avg 40%	▇ 3.7%	sharp
CPTUB average (SpeakLeash, 0-5 judge scale)	46.6% ✎	▁▂▂▃▃█▃▆▅▄▂▃ beats 14/95	easy · avg 66%	▇ 3.3%	at its level
Polish EQ-Bench v2 (SpeakLeash)	46.3% ✎	▁··▁▁▂▂▆▅▆█▆ beats 21/70	medium · avg 64%	▇ 2.7%	at its level

2 more tests

test	result	vs the others in its run	difficulty	counts	why
Open PT-LLM Leaderboard average (9 Portuguese tasks)	73.1% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 203/276	medium · avg 73%	▇ 1.0%	at its level · noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	96.3% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 90/140	very easy · avg 94%	▇ 0.3%	blunt · maxed out · noisy

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Yi 1.5 34B Chat · needs ~20.7 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Yi-1.5-34B-Chat-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Yi-1.5-34B-Chat-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Yi 1.5 34B Chat · needs ~20.7 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Yi-1.5-34B-Chat-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Yi 1.5 34B Chat · needs ~21.13 GB · from ollama.com
chat in the terminal
ollama run yi:34b-chat-v1.5-q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Yi 1.5 34B Chat · needs ~19.3 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Yi-1.5-34B-Chat-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Yi-1.5-34B-Chat-4bit

Official weights and documentation: huggingface.co/01-ai ↗

$ how the score was worked out · WildEval ZeroEval published r… 54%, BigCodeBench paper (Zhuo et a… 11%, Epoch AI — internal evaluatio… 10% and 8 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
WildEval ZeroEval published results	5	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 54%	Apache-2.0
BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4	2	▇▇▇ 11%	CC BY 4.0
Epoch AI — internal evaluations	2	▇▇▇ 10%	CC BY 4.0
WildBench v2 leaderboard data (Ai2)	1	▇▇ 5%	Apache-2.0
Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)	1	▇ 5%	CC-BY-4.0
LMArena leaderboard dataset	1	▇ 5%	CC BY 4.0
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)	4	▇ 3%	CC-BY-4.0
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 3%	MIT
SpeakLeash Polish EQ-Bench leaderboard	1	▇ 3%	Apache-2.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 45.8