whatmodel.sh / Qwen3 14B

Qwen3 14B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Qwen3 14B is an open-weights language model from Alibaba, released April 2025. Its published size is 14.8B parameters and a 33k-token context window. Licence: apache-2.0.

58.5±1.1#212 of 719 LLMsfrom 17 benchmark results across 12 sources

$ how it compares

The LLMs ranked either side of Qwen3 14B, out of 828 with a score, then more from Alibaba.

#modelscorewhereRAMreleased
252Phi-4-reasoning-plus 14B Microsoft59±1local~10GB2025-04
253Nemotron-Cascade-8B NVIDIA59±1local~6.3GB2025-12
254Qwen3 14B Alibaba59±1api $0.61/M +local~10GB2025-04
255Mercury 2 Inception59±2comparison only
256DiffusionGemma 26B-A4B Google59±1comparison only2026-06
more from Alibaba
18Qwen3.8 Max (2.4T-A95B) Alibaba78±1api $3.00/M +local~380GB2026-08
34Qwen3.8 Max (0902) Alibaba76±1api $3.00/M2026-09
37Qwen3.8-Flash-Next Alibaba75±1local~59GB2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #170 of 300 · 31% of its score · 3 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 0528 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
τ²-bench Telecom29.9% ✎█··██··██··█ beats 3/5very hard · avg 10%▇▇▇▇▇▇▇▇▇ 23%sharp · at its level
BFCL V4 (December 2025 archive)41.0% ✎▁▁▁▂██▅▅▆▂▂▂ beats 45/77hard · avg 32%▇▇ 5.2%at its level
BFCL v368.0% ✎▁▁▁▁▁▃▅█▅▄▁▁ beats 105/114medium · avg 52%▇ 2.2%blunt · at its level
math▁▂▃▅▆▅▄█▄▁▁▁ #145 of 398 * · 10% of its score · 1 test

Beats Claude Sonnet 4.6, MiMo-V2.5 (310B-A15B), MiniMax-M2.7 here, though they rank higher overall. Trails Nemotron 3 Nano 30B-A3B, Olmo 3 32B Think, Olmo 3.1 32B Think here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
OTIS mock AIME 2024–202566.4% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 94/193very hard · avg 8.5%▇▇▇▇ 10%sharp · at its level · noisy
other tests13 not in a task · 59% of its score
testresultvs the others in its rundifficultycountswhy
GPQA Diamond63.8% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 91/208hard · avg 40%▇▇▇▇▇▇▇▇ 19%sharp · at its level
Medmarks-V weighted mean win rate (61 models, 71 configurations)53.7% ✎▂▃▅▂▆▃█▅█▅▃▆ beats 23/34medium · avg 42%▇▇▇ 7.9%blunt · at its level
BRIDGE clinical text, zero-shot average (87 tasks)40.2% ✎▁▁▁▁▁▂▂▃▄▆█▃ beats 77/97hard · avg 32%▇▇ 5.7%blunt · at its level
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)29.7% ✎▁▁·▁▂▂▄█▄▅▄▄ beats 25/66hard · avg 31%▇▇ 5.7%at its level
CPTUB average (SpeakLeash, 0-5 judge scale)70.2% ✎▁▂▂▂▃█▃▆▅▄▂▃ beats 53/95easy · avg 66%▇▇ 5.6%typical
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items26.0% ✎▁▁▂▁▃▄█▆▄▅▂▃ beats 11/39very hard · avg 22%▇▇ 5.5%blunt
Open Persian LLM Leaderboard v2 average (21 tasks)57.8% ✎▂▄▂▃█▄▃▆▃▅▂▂ beats 47/81medium · avg 58%▇▇ 4.0%blunt · at its level
Open PT-LLM Leaderboard average (9 Portuguese tasks)79.9% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 258/276medium · avg 73%▇ 1.8%at its level · noisy
5 more tests
testresultvs the others in its rundifficultycountswhy
Chess Puzzles (Epoch)4.0% ✎█▁▁▁▁▂▁▄▂▂▁▁ beats 42/135very hard · avg 5.9%▇ 1.4%sharp · too hard for it · noisy
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy79.8% ✎▃·▁▁▂▁▂▅█▇▄▁ beats 72/100medium · avg 68%▇ 1.4%noisy
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy93.5% ✎▁▁▁▁▁▁▁▂▄█▄▃ beats 130/146easy · avg 83%▇ 0.7%noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)97.8% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 119/140very easy · avg 94%▇ 0.4%blunt · maxed out · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)94.6% ✎▁▂▁▂▅█▂▃▄▃▁▁ beats 84/98very easy · avg 90%▇ 0.0%blunt

$ where to run it

Through an API

api · $0.61/M · $0.35 in / $1.4 out per 1M tokens · 33k context · www.alibabacloud.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Qwen3 14B · needs ~9 GB · from huggingface.co/Qwen (official)
    chat in the terminal
    llama cli -hf Qwen/Qwen3-14B-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf Qwen/Qwen3-14B-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Qwen3 14B · needs ~9 GB · from huggingface.co/Qwen (official)

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/Qwen/Qwen3-14B-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Qwen3 14B · needs ~9.38 GB · from ollama.com
    chat in the terminal
    ollama run qwen3:14b

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Qwen3 14B · needs ~8.3 GB, Mac only · from huggingface.co/lmstudio-community
    chat in the terminal
    mlx_lm.chat --model lmstudio-community/Qwen3-14B-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model lmstudio-community/Qwen3-14B-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Epoch AI — internal evaluatio… 30%, MUA-RL: Multi-turn User-inter… 23%, Medmarks: A Comprehensive Ope… 8% and 9 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Epoch AI — internal evaluations3▇▇▇▇▇▇▇▇▇ 30%CC BY 4.0
MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)1▇▇▇▇▇▇▇ 23%CC-BY-4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models1▇▇ 8%CC BY 4.0
Berkeley Function Calling Leaderboard result archive2▇▇ 7%Apache-2.0
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)1▇▇ 6%MIT
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)1▇▇ 6%Apache-2.0
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)1▇▇ 6%MIT
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)1▇▇ 6%CC BY 4.0
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)1▇ 4%Apache-2.0
EuroEval leaderboards (results collection), English generative leaderboard2▇ 2%MIT
Open Portuguese LLM Leaderboard (archived)1▇ 2%Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)2▇ 0%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 58.5
$ quantization evidence · 35 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precisiontestfull → quantizedkept
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)MMLU-Redux, thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.886 → 0.88599.9%source ↗
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)MMLU-Redux, non-thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.82 → 0.81599.4%source ↗
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)LiveBench 2024-11-25, thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.713 → 0.798.2%source ↗
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)GPQA (subset not stated in card), non-thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.548 → 0.53898.2%source ↗
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)AIME 2024, thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.793 → 0.7797.1%source ↗
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)GPQA (subset not stated in card), thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.64 → 0.62197.0%source ↗
AWQ INT4 (w4, group_size 128, zero_point, GEMM; checkpoint Qwen/Qwen3-14B-AWQ)LiveBench 2024-11-25, non-thinking mode (Qwen-reported in quantized model card; harness, sampling and number of runs not stated)0.596 → 0.57496.3%source ↗
FP8 block (weights FP8 128x128 blocks, activations FP8 dynamic per-group 128; llm-compressor QuantizationModifier)OpenLLM v1 average of 6 tasks (ARC-Challenge (Acc-Norm, 25-shot); GSM8K (Strict-Match, 5-shot); HellaSwag (Acc-Norm, 10-shot); MMLU (Acc, 5-shot); TruthfulQA (MC2, 0-shot); Winogrande (Acc, 5-shot)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 16384, add_bos_token)0.748 → 0.749100.2%source ↗
FP8 block (weights FP8 128x128 blocks, activations FP8 dynamic per-group 128; llm-compressor QuantizationModifier)GSM8K — published row 'GSM8K (Strict-Match, 5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 16384, add_bos_token)0.884 → 0.884100.0%source ↗
FP8 block (weights FP8 128x128 blocks, activations FP8 dynamic per-group 128; llm-compressor QuantizationModifier)MMLU — published row 'MMLU (Acc, 5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 16384, add_bos_token)0.788 → 0.78799.8%source ↗
FP8 block (weights FP8 128x128 blocks, activations FP8 dynamic per-group 128; llm-compressor QuantizationModifier)OpenLLM v2 average of 6 tasks (IFEval (Inst Level Strict Acc, 0-shot); BBH (Acc-Norm, 3-shot); Math-Hard (Exact-Match, 4-shot); GPQA (Acc-Norm, 0-shot); MUSR (Acc-Norm, 0-shot); MMLU-Pro (Acc, 5-shot)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 16384, chat template, few-shot as multiturn); per-task scale (raw vs Open LLM Leaderboard-normalized) not stated in card0.315 → 0.31198.8%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)GPQA Diamond — published row 'GPQA diamond' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.616 → 0.641104.1%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)AIME 2025 — published row 'AIME 2025' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.663 → 0.67100.9%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)AIME 2024 — published row 'AIME 2024' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.766 → 0.769100.4%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)OpenLLM v1 average of 6 tasks (MMLU (5-shot); ARC Challenge (25-shot); GSM-8K (5-shot, strict-match); Hellaswag (10-shot); Winogrande (5-shot); TruthfulQA (0-shot, mc2)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 8192, chat template, few-shot as multiturn)0.632 → 0.634100.3%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)OpenLLM v2 average of 6 tasks (MMLU-Pro (5-shot); IFEval (0-shot); BBH (3-shot); Math-lvl-5 (4-shot); GPQA (0-shot); MuSR (0-shot)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 16384, chat template, few-shot as multiturn); per-task scale (raw vs Open LLM Leaderboard-normalized) not stated in card0.388 → 0.388100.1%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)MMLU — published row 'MMLU (5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 8192, chat template, few-shot as multiturn)0.768 → 0.76599.6%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)LiveCodeBench — published row 'LiveCodeBench' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.608 → 0.60699.5%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)GSM8K — published row 'GSM-8K (5-shot, strict-match)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 8192, chat template, few-shot as multiturn)0.676 → 0.66297.9%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)GSM8K — published row 'GSM-8K (5-shot, strict-match)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 8192, chat template, few-shot as multiturn)0.676 → 0.716105.8%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)OpenLLM v1 average of 6 tasks (MMLU (5-shot); ARC Challenge (25-shot); GSM-8K (5-shot, strict-match); Hellaswag (10-shot); Winogrande (5-shot); TruthfulQA (0-shot, mc2)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 8192, chat template, few-shot as multiturn)0.632 → 0.644102.0%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)AIME 2024 — published row 'AIME 2024' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.766 → 0.779101.8%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)OpenLLM v2 average of 6 tasks (MMLU-Pro (5-shot); IFEval (0-shot); BBH (3-shot); Math-lvl-5 (4-shot); GPQA (0-shot); MuSR (0-shot)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 16384, chat template, few-shot as multiturn); per-task scale (raw vs Open LLM Leaderboard-normalized) not stated in card0.388 → 0.38699.5%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)GPQA Diamond — published row 'GPQA diamond' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.616 → 0.61199.2%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)MMLU — published row 'MMLU (5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 8192, chat template, few-shot as multiturn)0.768 → 0.75998.8%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)AIME 2025 — published row 'AIME 2025' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.663 → 0.6598.0%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)LiveCodeBench — published row 'LiveCodeBench' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 327680.608 → 0.58696.3%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)GSM8K — published row 'gsm8k' (OpenLLM v1 section); lm-evaluation-harness (neuralmagic fork) + vLLM (backend vllm, max_model_len 4096, chat template, few-shot as multiturn)0.887 → 0.88299.5%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)OpenLLM v1 average of 6 tasks (arc_challenge; gsm8k; hellaswag; mmlu; truthfulqa_mc2; winogrande) as published; lm-evaluation-harness (neuralmagic fork) + vLLM (backend vllm, max_model_len 4096, chat template, few-shot as multiturn)0.745 → 0.73999.2%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)MMLU — published row 'mmlu' (OpenLLM v1 section); lm-evaluation-harness (neuralmagic fork) + vLLM (backend vllm, max_model_len 4096, chat template, few-shot as multiturn)0.789 → 0.77297.9%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)OpenLLM v2 average of 6 tasks (BBH (3-shot); MMLU-Pro (5-shot); MuSR (0-shot); IFEval (0-shot); GPQA (0-shot); Math-|v|-5 (4-shot)) as published; lm-evaluation-harness (neuralmagic fork) + vLLM (backend vllm, max_model_len 4096, chat template, few-shot as multiturn); per-task scale (raw vs Open LLM Leaderboard-normalized) not stated in card0.528 → 0.5196.7%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)AIME 2025 — published row 'AIME25 (0-shot)' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32768, seed 420.69 → 0.65595.0%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)GPQA Diamond — published row 'GPQA (Diamond, 0-shot)' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32768, seed 420.65 → 0.60493.0%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)Reasoning average of 3 tasks (AIME24 (0-shot); AIME25 (0-shot); GPQA (Diamond, 0-shot)) as published; lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32768, seed 420.699 → 0.63891.2%source ↗
NVFP4 W4A4 (FP4 weights and activations, group 16; llm-compressor QuantizationModifier)AIME 2024 — published row 'AIME24 (0-shot)' (Reasoning section); lighteval + vLLM, temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32768, seed 420.759 → 0.65586.4%source ↗

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json