Tulu 2 DPO 13B

+ contribute

Tulu 2 DPO 13B is an open-weights language model from Allen Institute for AI, released November 2023. Its published size is 13B parameters. Licence: AI2 ImpACT Low-risk; Llama 2 base terms.

Catalogue identity notes

Exact full-precision model evaluated in the Tulu 2 paper (arXiv:2311.10702v2). QLoRA ablations are separate and not imported.

Model information sources

Release date ↗ · Weights licence ↗

37.4±1.3#571 of 719 LLMsfrom 8 benchmark results across 3 sources

$ how it compares

The LLMs ranked either side of Tulu 2 DPO 13B, out of 828 with a score, then more from Allen Institute for AI.

#	model	score	where	RAM	released
662	SOLAR 10.7B Instruct v1.0 Upstage	37±1	local	~8.3GB	2023-12
663	Phi-3.5-vision-instruct Microsoft	37±2	local	~10GB	2024-08
664	Tulu 2 DPO 13B Allen Institute for AI	37±1	local	~9.8GB	2023-11
665	Yi-1.5 6B (base) 01.AI	37±3	local	~5.1GB	2024-05
666	Qwen2-VL 2B Instruct Alibaba	37±1	local	~2.8GB	2024-08
	more from Allen Institute for AI
493	OLMo 2 13B Instruct (1124) Allen Institute for AI	46±1	local	~9.6GB	2024-12
516	Tulu 2 70B Allen Institute for AI	46±2	local	~28GB	2023-11
557	Code Tulu 2 34B Allen Institute for AI	44±2	local	~72GB	2023-11

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #431 of 458 * · 13% of its score · 1 test

Beats Qwen3.5 2B, Molmo 7B-D (0924) here, though they rank higher overall. Trails Llama 3.2 1B Instruct, SmolLM2 1.7B Instruct, Mistral 7B Instruct v0.1 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Tulu 2: CodexEval pass@10 lab	48.9% ✎	▃▃▃·▆▃▃█▃▂▂▅ beats 6/25	very easy · avg 86%	▇▇▇▇▇ 13%	sharp · at its level

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #346 of 393 * · 40% of its score · 2 tests

Beats Gemma 4 E2B, Code Llama 70B Instruct, Phi-3-mini-4k-instruct (June 2024 update) here, though they rank higher overall. Trails Gemma 3 270M IT, SmolLM2 360M Instruct, Granite 4.0 H 350M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
AlpacaEval 2 length-controlled	11.6% ✎	▁▂▂▃▆▆█▆▄▄▁▁ beats 28/122	medium · avg 39%	▇▇▇▇▇▇▇▇▇▇▇▇▇ 34%	typical
AlpacaEval 1.0 win rate (vs text-davinci-003, GPT-4 judge)	88.1% ✎	▂▂·▆·▇▃▆█▆▄▂ beats 26/45	very easy · avg 96%	▇▇ 6.1%	sharp · noisy

math▁▂▃▅▆▅▄█▄▁▁▁ #309 of 398 * · 8.2% of its score · 1 test

Beats Athene-70B, Llama 3 70B Instruct, OLMo 2 32B Instruct (0325) here, though they rank higher overall. Trails Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
GSM8K lab	49.5% ✎	▆▂▅█▅▅▃▆···▂ beats 16/25	easy · avg 73%	▇▇▇ 8.2%	sharp · at its level

other tests4 not in a task · 38% of its score

test	result	vs the others in its run	difficulty	counts	why
BIG-Bench Hard lab	49.4% ✎	▁·····▂█▂▅·▁ beats 12/25	easy · avg 77%	▇▇▇▇▇▇▇▇▇ 24%	sharp · at its level
MMLU lab	55.3% ✎	▁▁▃▃▄█▄▁█··▁ beats 15/25	easy · avg 74%	▇▇▇ 8.7%	at its level
Open PT-LLM Leaderboard average (9 Portuguese tasks)	65.8% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 120/276	medium · avg 73%	▇ 3.2%	at its level · noisy
Tulu 2: TyDiQA gold passage, 1-shot F1 lab	39.7% ✎	▂··▁▃▃▂▂█▅▁▂ beats 6/25	medium · avg 61%	▇ 2.8%	blunt · at its level

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Tulu 2 DPO 13B · needs ~7.9 GB · from huggingface.co/TheBloke
chat in the terminal
llama cli -hf TheBloke/tulu-2-dpo-13B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf TheBloke/tulu-2-dpo-13B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Tulu 2 DPO 13B · needs ~7.9 GB · from huggingface.co/TheBloke
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/TheBloke/tulu-2-dpo-13B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Tulu 2 DPO 13B · needs ~7.9 GB · from huggingface.co/TheBloke
chat in the terminal
ollama run hf.co/TheBloke/tulu-2-dpo-13B-GGUF:Q4_K_M

Other ways to run it

Run it as published · needs ~26.03 GB · from huggingface.co/allenai
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='allenai/tulu-2-dpo-13b', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Weights and documentation: huggingface.co/allenai ↗

$ how the score was worked out · Tulu 2 — base and instruction… 56%, AlpacaEval 1.0 and 2.0 leader… 40%, Open Portuguese LLM Leaderboa… 3%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Tulu 2 — base and instruction-tuned evaluations · lab-reported	5	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 56%	CC BY 4.0
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)	2	▇▇▇▇▇▇▇▇▇▇▇▇ 40%	Apache-2.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 3%	Apache-2.0
prior · pull toward the middle		▇ 1%
weighted average		= 37.4

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json