whatmodel.sh / DeepSeek R1 0528

DeepSeek R1 0528

+ contribute

DeepSeek R1 0528 is an open-weights language model from DeepSeek, released May 2025. Its published size is 685B parameters (37B active per token) and a 164k-token context window. Licence: mit.

65.2±0.6#142 of 719 LLMsfrom 51 benchmark results across 12 sources

$ how it compares

The LLMs ranked either side of DeepSeek R1 0528, out of 828 with a score, then more from DeepSeek.

#	model	score	where	RAM	released
161	MiMo-V2.6-Distill-Qwen-9B Xiaomi	65±2	local	~7.8GB	2026-09
162	K-EXAONE (236B-A23B) LG AI Research	65±1	local	~152GB	2025-12
163	DeepSeek R1 0528 DeepSeek	65±1	api $0.91/M +local	~110GB	2025-05
164	EXAONE 4.5 33B LG AI Research	65±2	local	~24GB	2026-04
165	Qwen3.5 Flash Alibaba	65±1	api $0.17/M		2026-02
	more from DeepSeek
21	DeepSeek V4.1 Flash DeepSeek	78±1	api $0.26/M +local	~537GB	2026-09
31	DeepSeek V4 Pro (0813) DeepSeek	76±1	api $1.63/M +local	~1010GB	2026-08
35	DeepSeek V3.2 Speciale DeepSeek	76±3	comparison only

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #193 of 300 · 13% of its score · 13 tests

Beats o1 (2024-12-17), o3-mini here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Mellum2 12B-A2.5B Instruct here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified +1	29.6% ✎	█·█▄·······█ beats 2/6	hard · avg 19%	▇ 3.4%	sharp · at its level · small run · several runs
τ²-bench Retail	59.9% ✎	█▄··▄··▄·▄·▄ beats 0/6	medium · avg 37%	▇ 1.9%	at its level
τ²-bench Airline	47.3% ✎	▄▄······▄█▄▄ beats 1/6	hard · avg 29%	▇ 1.4%	at its level
BrowseComp-ZH lab	35.7% ✎	▁▁▁▁▁▃·▃▄▅▄█ beats 7/23	very hard · avg 15%	▇ 1.3%	at its level · small run
BFCL V4 (2025-08-26 archive, models absent from later archives)	49.0% ✎	▄▄····█▄···▄ beats 4/5	hard · avg 27%	▇ 1.1%	at its level
SWE-bench Multilingual lab	30.5% ✎	▁··▁▃▂▅▇█▃▃▁ beats 5/60	hard · avg 19%	▇ 1.0%	at its level · small run
Humanity's Last Exam with tools lab	24.8% ✎	▂▂▃▁▄▂▃▆█▇▂▆ beats 6/54	very hard · avg 13%	▇ 0.8%	at its level · small run
BrowseComp lab	8.9% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 4/42	very hard · avg 11%	▇ 0.7%	at its level · small run

5 more tests

test	result	vs the others in its run	difficulty	counts	why
BFCL v3	63.8% ✎	▁▁▁▁▁▃▅█▅▄▁▁ beats 92/114	medium · avg 52%	▇ 0.6%	blunt · at its level
Terminal-Bench (pre-2.0) with Terminus 1 lab	5.7% ✎	█····█·····█ beats 0/2	very hard · avg 7.1%	▇ 0.3%	noisy · small run
τ-bench Airline (GPT-4.1 user) lab	53.5% ✎	█········█·█ beats 0/2		—	not counted yet: no other model in the same run to compare it with
τ-bench Retail (GPT-4.1 user) lab	63.9% ✎	██·········█ beats 1/2		—	not counted yet: no other model in the same run to compare it with
BFCL v3 multi-turn lab	37.0% ✎	no other model yet		—	not counted yet: no other model in the same run to compare it with

coding▁▁▁▃▆██▇▆▄▃▁ #115 of 458 · 38% of its score · 7 tests

Beats Qwen3.5 Plus, Gemini 2.5 Pro, Kimi K2 Thinking here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Solar Pro 4 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Aider polyglot (225 exercises, two attempts) +3	71.4% ✎	▂·▂▃▅▂▃█▆▄▂▂ beats 39/49	very hard · avg 8.5%	▇▇▇▇▇▇▇▇▇▇ 26%	sharp · at its level · several runs
SWE-bench Verified +1	29.6% ✎	█·█▄·······█ beats 2/6	hard · avg 19%	▇ 3.4%	sharp · at its level · small run · several runs
Luau Leetcode fully solved (Torpedo Software)	59.6% ✎	▁▁▁▁·▁▂▃▅▃█▇ beats 14/49	hard · avg 18%	▇ 2.6%	at its level
LiveCodeBench v6 (Aug 2024–May 2025) +1	62.5% ✎	▄▄·▄·····█·▄ beats 1/5	medium · avg 45%	▇ 1.9%	at its level · several runs
LiveCodeBench 2024-08–2025-05 (version unstated) lab +1	73.3% ✎	▃···█··▅··▃▃ beats 4/7	medium · avg 42%	▇ 1.5%	at its level · small run · several runs
WebApp1K-Duo-React pass@1 (onekq-ai)	68.5% ✎	▁··▁·▁▁▁▆█▇█ beats 27/45	hard · avg 32%	▇ 1.4%	at its level
SWE-bench Multilingual lab	30.5% ✎	▁··▁▃▂▅▇█▃▃▁ beats 5/60	hard · avg 19%	▇ 1.0%	at its level · small run

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #87 of 393 * · 3.0% of its score · 1 test

Beats LongCat 2.0 (1.6T-A48B), Inkling Small (266B-A12B), Qwen3.5 122B-A10B here, though they rank higher overall. Trails Gemma 4 12B, Gemma 4 26B-A4B, Mistral Medium 3.5 (128B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1421.6 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 271/379	avg 1268	▇ 3.0%	blunt

math▁▂▃▅▆▅▄█▄▁▁▁ #100 of 398 · 20% of its score · 6 tests

Beats Claude Sonnet 4.6, MiniMax-M3, Nemotron 3 Ultra 550B-A55B here, though they rank higher overall. Trails Gemini 2.5 Flash, GLM-4.5-Air, Granite 4.2 30B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
AIME 2024 +3	83.2% ✎	█···███·█··█ beats 1/5	medium · avg 49%	▇▇▇ 6.8%	sharp · several runs
AIME 2025 +2	73.4% ✎	█·█···███··█ beats 0/5	medium · avg 43%	▇▇ 5.4%	sharp · several runs
OTIS mock AIME 2024–2025	66.4% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 93/193	very hard · avg 8.5%	▇ 2.5%	sharp · at its level · noisy
MATH level 5	96.6% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 86/92	hard · avg 32%	▇ 2.5%	sharp
HMMT 2025 (edition unspecified) lab +1	79.4% ✎	▃█▇▅▇▅▁·▁··▁ beats 25/32	hard · avg 22%	▇ 2.1%	sharp · at its level · noisy · small run · several runs
MATH-500	87.3% ✎	▄······▄▄▄·█ beats 0/5	very easy · avg 86%	▇ 0.3%	maxed out · noisy

web search█········█·█ #3 of 3 · 2.8% of its score · 3 tests

test	result	vs the others in its run	difficulty	counts	why
BrowseComp-ZH lab	35.7% ✎	▁▁▁▁▁▃·▃▄▅▄█ beats 7/23	very hard · avg 15%	▇ 1.3%	at its level · small run
Humanity's Last Exam with tools lab	24.8% ✎	▂▂▃▁▄▂▃▆█▇▂▆ beats 6/54	very hard · avg 13%	▇ 0.8%	at its level · small run
BrowseComp lab	8.9% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 4/42	very hard · avg 11%	▇ 0.7%	at its level · small run

other tests10 not in a task · 31% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond +3	76.3% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 118/208	hard · avg 42%	▇▇▇▇▇▇ 16%	sharp · at its level · several runs
Humanity's Last Exam +2	15.9% ✎	▄····▄·█·▄·▄ beats 4/5	very hard · avg 4.3%	▇▇ 5.5%	several runs
MMLU-Pro +1	75.3% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 171/188	medium · avg 51%	▇ 2.8%	several runs
MMLU-Redux lab +1	93.4% ✎	▁▁▃▁▂▃▄▅▇▅█▄ beats 56/63	easy · avg 80%	▇ 2.3%	small run · several runs
SimpleQA lab	27.8% ✎	▄▄▄▆█▆███·▂█ beats 26/32	very hard · avg 9.6%	▇ 1.1%	at its level · small run
RobloxQA v2.0 multiple choice (Torpedo Software)	89.2% ✎	▁▁▃▁█▄▃▃▆▂▂▂ beats 11/49	easy · avg 78%	▇ 1.1%	typical
Multi-IF (aggregation unspecified)	73.8% ✎	▄·····▄█▄▄·▄ beats 0/6	easy · avg 66%	▇ 0.7%	blunt
CNMO 2024 lab	86.9% ✎	█··········█ beats 1/1	medium · avg 58%	▇ 0.7%	small run

2 more tests

test	result	vs the others in its run	difficulty	counts	why
FRAMES lab	83.0% ✎	█··········█ beats 1/1	medium · avg 64%	▇ 0.6%	small run
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	92.3% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 38/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy

$ where to run it

Through an API

api · Deep Infra · $0.91/M · $0.5 in / $2.15 out per 1M tokens · 164k context · deepinfra.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run DeepSeek R1 0528 · needs ~206 GB · from huggingface.co/unsloth
chat in the terminal
llama cli -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q2_K_XL
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q2_K_XL

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run DeepSeek R1 0528 · needs ~377.6 GB, Mac only · from huggingface.co/mlx-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/mlx-community/DeepSeek-R1-0528-4bit -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run DeepSeek R1 0528 · needs ~411.5 GB · from ollama.com
chat in the terminal
ollama run deepseek-r1:671b-0528-q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run DeepSeek R1 0528 · needs ~377.6 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/DeepSeek-R1-0528-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/DeepSeek-R1-0528-4bit

Weights and documentation: huggingface.co/deepseek-ai ↗

$ how the score was worked out · deepseek-ai/DeepSeek-R1-0528… 19%, Prime Intellect INTELLECT-3 m… 18%, deepseek-ai/DeepSeek-V3.1 mod… 15% and 9 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
deepseek-ai/DeepSeek-R1-0528 model card · lab-reported	13	▇▇▇▇▇▇ 19%	mit
Prime Intellect INTELLECT-3 model card: Evaluation table (Environments Hub runs)	7	▇▇▇▇▇ 18%	MIT
deepseek-ai/DeepSeek-V3.1 model card · lab-reported	11	▇▇▇▇▇ 15%	MIT
ServiceNow-AI/Apriel-1.6-15b-Thinker model card	5	▇▇▇▇ 14%	MIT
Epoch AI — internal evaluations	3	▇▇▇ 10%	CC BY 4.0
Aider polyglot benchmark runs	1	▇▇ 8%	Apache-2.0
deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card · lab-reported	4	▇▇ 6%	MIT
Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode	2	▇ 4%	MIT
LMArena leaderboard dataset	1	▇ 3%	CC BY 4.0
Berkeley Function Calling Leaderboard result archive	2	▇ 2%	Apache-2.0
WebApp1K models leaderboard (onekq-ai)	1	▇ 1%	MIT
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 65.2

not counted (3)

deepseek-ai/DeepSeek-R1-0528 model card	BFCL v3 multi-turn [correct]	37.0%	fewer than three models have this test yet, or this source ran it for this model alone
deepseek-ai/DeepSeek-R1-0528 model card	τ-bench Airline (GPT-4.1 user) [correct]	53.5%	fewer than three models have this test yet, or this source ran it for this model alone
deepseek-ai/DeepSeek-R1-0528 model card	τ-bench Retail (GPT-4.1 user) [correct]	63.9%	fewer than three models have this test yet, or this source ran it for this model alone

$ quantization evidence · 3 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
INT4 W4A16 (weights only; GPTQ via MoE-Quant, Red Hat/ISTA)	GPQA Diamond pass@1 (LightEval/open-r1, mean of 10 seeds)	0.796 → 0.806	101.2%	source ↗
INT4 W4A16 (weights only; GPTQ via MoE-Quant, Red Hat/ISTA)	MATH-500 pass@1 (LightEval/open-r1, mean of 10 seeds)	0.975 → 0.974	99.9%	source ↗
INT4 W4A16 (weights only; GPTQ via MoE-Quant, Red Hat/ISTA)	AIME 2024 pass@1 (LightEval/open-r1, mean of 10 seeds)	0.887 → 0.873	98.5%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json