whatmodel.sh / Mistral Large 2 (2411)

Mistral Large 2 (2411)

+ contribute

Mistral Large 2 (2411) is an open-weights language model from Mistral, released November 2024. Its published size is 123B parameters and a 128k-token context window. Licence: mistral-research.

55.2±1.0#258 of 719 LLMsfrom 18 benchmark results across 14 sources

$ how it compares

The LLMs ranked either side of Mistral Large 2 (2411), out of 828 with a score, then more from Mistral.

#	model	score	where	RAM	released
308	Yi-Lightning 01.AI	55±1	api price unknown		2024-10
309	Qwen2.5 72B Instruct Alibaba	55±1	api $0.37/M +local	~30GB	2024-09
310	Mistral Large 2 (2411) Mistral	55±1	local	~78GB	2024-11
311	Ministral 3 8B Reasoning 2512 Mistral	55±1	local	~6.5GB	2025-12
312	Magistral Small 1.0 (2506) Mistral	55±1	local	~16GB	2025-06
	more from Mistral
79	Mistral Large 4 (public preview) Mistral	71±2	api $2.06/M		2026-10
174	Mistral Medium 3.5 (128B) Mistral	65±1	api $3.00/M +local	~42GB	2026-04
180	Mistral Medium 3.1 (2508) Mistral	64±2	comparison only		2025-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #220 of 300 · 7.0% of its score · 3 tests

Beats o1 (2024-12-17), o3-mini, Llama 3.1 Nemotron Ultra 253B v1 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, LFM2.5 1.2B Thinking here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BFCL V4 (December 2025 archive)	38.4% ✎	▁▁▁▂██▅▅▆▂▂▂ beats 44/77	hard · avg 32%	▇▇ 3.9%	at its level
BFCL v3	58.0% ✎	▁▁▁▁▁▃▅█▅▄▁▁ beats 73/114	medium · avg 52%	▇ 1.8%	blunt · at its level
Galileo Agent Leaderboard v1 (tool selection quality)	81.0% ✎	▃▃▃▃▆▅▆▅█▂▅▂ beats 10/31	easy · avg 79%	▇ 1.3%	blunt

coding▁▁▁▃▆██▇▆▄▃▁ #213 of 458 * · 14% of its score · 1 test

Beats Qwen3.5 4B, Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B) here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 3B Reasoning 2512, DeepSeek-Coder-V2 Instruct (0724) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Aider code editing (133 Exercism Python exercises, two attempts)	65.4% ✎	▁·▁▂▄▁▆▄█▆▂▂ beats 35/62	medium · avg 54%	▇▇▇▇▇▇ 14%	at its level

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #217 of 393 * · 8.2% of its score · 1 test

Beats Granite 4.2 8B, Mistral Small 4 (119B-A6.5B), Olmo 3.1 32B Think here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, Granite 4.0 H Micro 3B, Gemma 3n E4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1305.5 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 134/379	avg 1268	▇▇▇ 8.2%	blunt

math▁▂▃▅▆▅▄█▄▁▁▁ #245 of 398 * · 26% of its score · 2 tests

Beats GPT-4o (2024-11-20), GPT-4 Turbo (2024-04-09) here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 70B, OpenThinker 7B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH level 5	50.3% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 43/92	hard · avg 32%	▇▇▇▇▇▇▇▇ 20%	sharp · at its level
OTIS mock AIME 2024–2025	7.8% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 44/193	very hard · avg 8.5%	▇▇ 5.8%	sharp · noisy

other tests11 not in a task · 44% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond	51.3% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 72/208	hard · avg 40%	▇▇▇▇▇ 13%	sharp · at its level
EQ-Bench 3 (emotional intelligence, judge Elo)	1108.9 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 13/62	avg 1044	▇▇ 5.0%	typical
CPTUB average (SpeakLeash, 0-5 judge scale)	80.1% ✎	▁▂▂▂▃█▃▆▅▄▂▃ beats 81/95	easy · avg 66%	▇▇ 4.9%	at its level
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	52.0% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 47/66	hard · avg 31%	▇▇ 4.4%	at its level
BRIDGE clinical text, zero-shot average (87 tasks)	42.3% ✎	▁▁▁▁▁▂▂▃▄▆█▃ beats 87/97	hard · avg 32%	▇▇ 4.4%	blunt · at its level
Polish EQ-Bench v2 (SpeakLeash)	77.3% ✎	▁··▁▁▂▂▆▅▆█▆ beats 68/70	medium · avg 64%	▇▇ 3.8%	at its level
Open Persian LLM Leaderboard v2 average (21 tasks)	64.3% ✎	▂▄▂▃█▄▃▆▃▅▂▂ beats 61/81	medium · avg 58%	▇ 3.3%	blunt · at its level
Judgemark v2 (LLM-as-judge for creative writing)	65.6% ✎	▃▃··▃▅▅···█▃ beats 6/10	medium · avg 55%	▇ 3.0%	at its level

3 more tests

test	result	vs the others in its run	difficulty	counts	why
RULER 64k	85.9% ✎	▁▁·▁·▂▃▄█▂·▁ beats 17/22	easy · avg 83%	▇ 1.4%	typical
RULER 128k	48.1% ✎	▄··▂▄▂▆██▄▄▂ beats 4/21	easy · avg 68%	▇ 0.6%	noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	95.5% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 93/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Mistral Large 2 (2411) · needs ~73.2 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Mistral-Large-Instruct-2411-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Mistral-Large-Instruct-2411-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Mistral Large 2 (2411) · needs ~73.2 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Mistral-Large-Instruct-2411-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Mistral Large 2 (2411) · needs ~74.3 GB · from ollama.com
chat in the terminal
ollama run mistral-large:123b-instruct-2411-q4_K_M

MLX (Mac)

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Mistral Large 2 (2411) · needs ~137.9 GB, Mac only · from huggingface.co/sjug
chat in the terminal
mlx_lm.chat --model sjug/Mistral-Large-Instruct-2411-8bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model sjug/Mistral-Large-Instruct-2411-8bit

Weights and documentation: huggingface.co/mistralai ↗

$ how the score was worked out · Epoch AI — internal evaluatio… 40%, Aider code editing benchmark… 14%, LMArena leaderboard dataset 8% and 11 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Epoch AI — internal evaluations	3	▇▇▇▇▇▇▇▇▇▇▇▇ 40%	CC BY 4.0
Aider code editing benchmark runs (old edit leaderboard)	1	▇▇▇▇ 14%	Apache-2.0
LMArena leaderboard dataset	1	▇▇ 8%	CC BY 4.0
Berkeley Function Calling Leaderboard result archive	2	▇▇ 6%	Apache-2.0
EQ-Bench 3 canonical leaderboard results	1	▇▇ 5%	MIT
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 5%	MIT
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	1	▇ 4%	Apache-2.0
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)	1	▇ 4%	MIT
SpeakLeash Polish EQ-Bench leaderboard	1	▇ 4%	Apache-2.0
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)	1	▇ 3%	Apache-2.0
Judgemark v2 results (EQ-Bench)	1	▇ 3%	MIT
NVIDIA RULER repository README results table	2	▇ 2%	Apache-2.0
Galileo Agent Leaderboard (v1 and v2 results)	1	▇ 1%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 55.2