whatmodel.sh / Llama 3.1 Tulu 3 8B

Llama 3.1 Tulu 3 8B

+ contribute

Llama 3.1 Tulu 3 8B is an open-weights language model from Ai2, released November 2024. Its published size is 8.03B parameters and a 131k-token context window. Licence: llama3.1.

Catalogue identity notes

Arena entry 'llama-3.1-tulu-3-8b' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'llama-3.1-tulu-3-8b' as 'Llama-3.1-Tulu-3-8B' (Ai2, Llama 3.1), linking https://huggingface.co/allenai/Llama-3.1-Tulu-3-8B. Maker repo allenai/Llama-3.1-Tulu-3-8B @6669437: licence tag llama3.1. 8,030,326,784 parameters from the Hugging Face safetensors metadata (dense, so active = total). Released 2024-11: month the maker's Hugging Face repo was created (createdAt 2024-11-20).

44.1±0.7#457 of 719 LLMsfrom 26 benchmark results across 4 sources

$ how it compares

The LLMs ranked either side of Llama 3.1 Tulu 3 8B, out of 828 with a score, then more from Ai2.

#	model	score	where	RAM	released
544	Gemma 3 4B Google	44±1	api $0.063/M +local	~3.8GB	2025-03
545	Eurus-2-7B-PRIME PRIME-RL	44±2	local	~6.4GB	2024-12
546	Llama 3.1 Tulu 3 8B Ai2	44±1	local	~6.2GB	2024-11
547	Qwen2.5-VL 3B Instruct Alibaba	44±1	local	~3.5GB	2025-01
548	Qwen1.5 32B Chat Alibaba	44±1	local	~22GB	2024-04
	more from Ai2
259	Olmo 3.1 32B Think Ai2	58±1	local	~21GB	2025-12
262	Olmo 3 32B Think Ai2	58±1	local	~21GB	2025-11
314	Molmo 2 8B Ai2	55±2	local	~6.8GB	2025-12

see all LLMs · find the ones that fit your machine

$ what it's good at

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #290 of 393 · 8.7% of its score · 3 tests

Beats Ministral 3 14B Reasoning 2512, Gemma 4 E4B, OpenThinker3 7B here, though they rank higher overall. Trails Flan-T5 XXL, LFM2 700M, EXAONE 3.5 2.4B Instruct here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1220.5 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 76/379	avg 1268	▇▇ 4.8%	blunt
AlpacaEval 2 length-controlled lab	34.0% ✎	▃▃·▁▁█▇▅·▁·▁ beats 10/22	medium · avg 42%	▇ 2.8%	at its level
IFEval prompt-level loose lab	82.4% ✎	▁▃▁▃▁▄▁▁█▃▃▁ beats 14/22	easy · avg 80%	▇ 1.1%	at its level

math▁▂▃▅▆▅▄█▄▁▁▁ #313 of 398 · 22% of its score · 4 tests

Beats Athene-70B, OLMo 2 32B Instruct (0325), Hermes 2 Theta Llama-3 70B here, though they rank higher overall. Trails Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 13B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
GSM8K lab +1	87.6% ✎	▂▃··▃▂▅█▆▃▂▃ beats 17/22	very easy · avg 91%	▇▇▇▇▇ 11%	sharp · several runs
MGSM	58.6% ✎	▅▃▃█·▅···▃·▃ beats 6/10	easy · avg 71%	▇▇ 5.7%	at its level
MATH	46.1% ✎	█▄██···▄▄▄·▄ beats 2/10	medium · avg 64%	▇ 3.6%	at its level
MATH (subset unspecified) lab	43.7% ✎	▃▃█▂▂▆·····▁ beats 12/22	medium · avg 61%	▇ 1.9%	at its level

other tests15 not in a task · 69% of its score

test	result	vs the others in its run	difficulty	counts	why
MMLU lab +1	68.2% ✎	▁▂·▁·▁▁█·▃▃▅ beats 6/22	easy · avg 77%	▇▇▇▇▇▇ 14%	at its level · several runs
BIG-Bench Hard lab +1	69.0% ✎	▁▄·▁▁·▅█·▄▄▁ beats 10/22	easy · avg 75%	▇▇▇▇▇▇ 14%	sharp · at its level · several runs
ARC Challenge	79.4% ✎	▂▃▂█▂···▄··▂ beats 1/10	very easy · avg 91%	▇▇▇▇ 11%	sharp
OpenBookQA	79.8% ✎	▂··▄█▂·▂···▄ beats 4/10	easy · avg 87%	▇▇▇ 6.3%	at its level
MMLU-Pro	40.9% ✎	█▄▄·█▄▄▄·▄·▄ beats 3/10	medium · avg 57%	▇▇ 4.4%	at its level
Multilingual MMLU (Microsoft)	54.5% ✎	█▄▄██··█···▄ beats 5/10	easy · avg 65%	▇▇ 4.1%	at its level
WinoGrande	65.4% ✎	▃····▃▃█▃·▅▅ beats 5/10	medium · avg 74%	▇ 2.7%	at its level
HellaSwag	80.9% ✎	▃▅█·▃▃▃···▃▃ beats 8/10	easy · avg 84%	▇ 2.4%	at its level

7 more tests

test	result	vs the others in its run	difficulty	counts	why
BoolQ	79.3% ✎	▃··▃··██▃·▃▃ beats 3/10	easy · avg 85%	▇ 2.3%	at its level
Social IQA	73.4% ✎	▃·▃▅▅█·▃···▃ beats 3/10	easy · avg 78%	▇ 2.2%	blunt · at its level
TruthfulQA MC2 lab +1	55.0% ✎	▃▃▂·▃▃·▅█▂▅▂ beats 5/21	medium · avg 63%	▇ 1.8%	blunt · at its level · several runs
GPQA (subset unspecified)	29.9% ✎	█··▄▂··▃▂··▄ beats 7/10	very hard · avg 33%	▇ 1.2%	sharp · too hard for it · noisy
PopQA lab	29.1% ✎	▁▁▁▃▃▃▁█▁▃·▄ beats 13/21	hard · avg 32%	▇ 1.1%	blunt
PIQA	78.3% ✎	▃·▅·▃█·▃▅··▃ beats 6/10	easy · avg 83%	▇ 0.7%	blunt · at its level
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy	83.9% ✎	▁▁▁▁▁▁▁▂▄█▄▃ beats 92/146	easy · avg 83%	▇ 0.7%	at its level · noisy

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Llama 3.1 Tulu 3 8B · needs ~4.9 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Llama-3.1-Tulu-3-8B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Llama-3.1-Tulu-3-8B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Llama 3.1 Tulu 3 8B · needs ~4.9 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Llama-3.1-Tulu-3-8B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Llama 3.1 Tulu 3 8B · needs ~5.32 GB · from ollama.com
chat in the terminal
ollama run tulu3:8b-q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Llama 3.1 Tulu 3 8B · needs ~4.5 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Llama-3.1-Tulu-3-8B-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Llama-3.1-Tulu-3-8B-4bit

Weights and documentation: huggingface.co/allenai ↗

$ how the score was worked out · microsoft/Phi-4-mini-instruct… 69%, 2 OLMo 2 Furious (Team OLMo,… 26%, LMArena leaderboard dataset 5% and 1 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
microsoft/Phi-4-mini-instruct model card	16	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 69%	MIT
2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models · lab-reported	8	▇▇▇▇▇▇▇▇ 26%	CC BY 4.0
LMArena leaderboard dataset	1	▇ 5%	CC BY 4.0
EuroEval leaderboards (results collection), English generative leaderboard	1	▇ 1%	MIT
prior · pull toward the middle		▇ 0%
weighted average		= 44.1

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json