whatmodel.sh / Qwen2.5 7B Instruct

Qwen2.5 7B Instruct

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Qwen2.5 7B Instruct is an open-weights language model from Alibaba, released September 2024. Its published size is 7.62B parameters and a 131k-token context window. Licence: apache-2.0.

48.4±0.5#375 of 719 LLMsfrom 51 benchmark results across 18 sources

$ how it compares

The LLMs ranked either side of Qwen2.5 7B Instruct, out of 828 with a score, then more from Alibaba.

#modelscorewhereRAMreleased
446InternLM2.5 7B Chat Shanghai AI Laboratory (InternLM)49±2local~6.1GB2024-06
447Hermes 3 Llama 3.1 70B Nous Research49±1local~44GB2024-07
448Qwen2.5 7B Instruct Alibaba48±1api $0.30/M +local~6GB2024-09
449DeepSeek R1 Distill Qwen 7B DeepSeek48±1local~6GB2025-01
450XwinCoder-34B Xwin-LM48±1local~23GB2023-11
more from Alibaba
18Qwen3.8 Max (2.4T-A95B) Alibaba78±1api $3.00/M +local~380GB2026-08
34Qwen3.8 Max (0902) Alibaba76±1api $3.00/M2026-09
37Qwen3.8-Flash-Next Alibaba75±1local~59GB2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #231 of 300 · 4.7% of its score · 3 tests

Beats o3-mini, Ministral 3 14B Reasoning 2512, DeepSeek R1 0528 Qwen3 8B here, though they rank higher overall. Trails SmolLM3 3B, Phi-3.5-mini-instruct, Jamba 1.5 Mini here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
τ²-bench Telecom +116.7% ✎▁·▅█▆▅▂▃▃▂▃▁ beats 7/89very hard · avg 13%▇ 3.5%sharp · small run · several runs
BFCL (version not stated)55.8% ✎▄▆▄▂·█▆·▄··▂ beats 1/17easy · avg 66%▇ 0.6%blunt · at its level
BFCL v355.8% ✎▁▁▁▁▁▃▅█▅▄▁▁ beats 59/114medium · avg 52%▇ 0.6%blunt · at its level
coding▁▁▁▃▆██▇▆▄▃▁ #264 of 458 · 15% of its score · 6 tests

Beats Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B), GLM-4.7-Flash here, though they rank higher overall. Trails DeepSeek LLM 67B Chat, QwQ-32B-Preview, Phi-3 Medium 128K Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
CRUXEval (ZeroEval, 800 problems)52.8% ✎▁▂▆▃▆█▆▃▂·▁▁ beats 14/47medium · avg 61%▇▇▇ 7.5%sharp · at its level
LiveCodeBench official (Jan 1–Nov 1, 2024)25.3% ✎▁▁·▁▂▄▆█▅▂·▂ beats 12/31hard · avg 26%▇▇ 5.1%sharp
CodeReviewQA: change localisation (hard), invariant accuracy34.2% ✎█▂▁▁▂▂▂▃▅▅▅▇ beats 28/47medium · avg 36%▇ 0.8%sharp · at its level · noisy
CodeReviewQA: solution identification (hard), invariant accuracy51.6% ✎▃··▁▂▂▃▂█▃▄▂ beats 27/47medium · avg 54%▇ 0.7%sharp · at its level · noisy
CodeReviewQA: automated code refinement, exact match rate35.7% ✎▁▁▁▁▁▁▂▂▃█▅▄ beats 34/47hard · avg 33%▇ 0.5%at its level
CodeReviewQA: change type recognition, invariant accuracy75.9% ✎▁▁▁······▁▃█ beats 33/47easy · avg 67%▇ 0.0%blunt · at its level · noisy
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #260 of 393 · 4.5% of its score · 3 tests

Beats Ministral 3 14B Reasoning 2512, Ministral 3 14B, OpenThinker2 32B here, though they rank higher overall. Trails Flan-T5 XXL, Granite 4.0 H 1B, Granite 4.0 1B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Arabic IFEval prompt-level (Inception)50.2% ✎▁·▁▁▂▁▄▄▆▄█▄ beats 26/38medium · avg 49%▇ 2.2%sharp · at its level
AlpacaEval 2 length-controlled29.7% ✎▃▃·▁▁█▇▅·▁·▁ beats 7/22medium · avg 42%▇ 1.7%at its level
IFEval prompt-level loose74.7% ✎▁▃▁▃▁▄▃▁█▃▃▁ beats 10/22easy · avg 80%▇ 0.5%typical
math▁▂▃▅▆▅▄█▄▁▁▁ #252 of 398 · 15% of its score · 7 tests

Beats GPT-4 Turbo (2024-04-09), Molmo 2 8B, Llama 3.3 70B Instruct here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Mixtral 8x7B v0.1 (base), Tulu 2 DPO 70B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K +183.8% ✎▂▃··▃▂▅█▆▃▂▃ beats 11/22very easy · avg 91%▇▇ 4.4%sharp · several runs
MATH Level 5 (ZeroEval, 721 problems)51.5% ✎▁▁▁▆█▂▇▇▄··▂ beats 26/38hard · avg 34%▇ 3.3%sharp · at its level
MGSM64.5% ✎▅▃▃█·▅···▃·▃ beats 8/10easy · avg 71%▇ 2.8%at its level
MATH60.4% ✎█▄██···▄▄▄·▄ beats 7/10medium · avg 64%▇ 1.9%at its level
MATH (subset unspecified) +169.9% ✎▃▃█▂▃▆·····▁ beats 16/22medium · avg 61%▇ 1.6%at its level · several runs
OMEGA (math generalisation)13.7% ✎█···▄·▄·▄··█ beats 3/6hard · avg 20%▇ 0.8%typical
OTIS mock AIME 2024–20252.5% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 21/193very hard · avg 8.5%▇ 0.6%sharp · too hard for it · noisy
other tests26 not in a task · 61% of its score
testresultvs the others in its rundifficultycountswhy
MMLU-Redux (ZeroEval, 2,778 questions)75.1% ✎▁·▁▁▄▄▄▅█▄▂▁ beats 20/53easy · avg 77%▇▇▇▇▇ 12%at its level
BIG-Bench Hard +170.2% ✎▁▄·▁▁·▅█·▄▄▁ beats 11/22easy · avg 75%▇▇▇ 7.1%sharp · at its level · several runs
MMLU +176.6% ✎▁▂·▁·▁▁█·▃▃▅ beats 13/22easy · avg 77%▇▇▇ 7.0%at its level · several runs
ARC Challenge90.1% ✎▂▂▂█▂···▄··▂ beats 9/10very easy · avg 91%▇▇ 4.3%sharp
ZebraLogic (ZeroEval, 1,000 puzzles)12.0% ✎▁▁▃▆██▁▁·▁▁▁ beats 15/57hard · avg 18%▇ 3.3%sharp
OpenBookQA82.6% ✎▂··▄█▃·▂···▄ beats 7/10easy · avg 87%▇ 2.7%typical
GPQA Diamond35.5% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 28/208hard · avg 40%▇ 2.6%sharp
MMLU-Pro56.2% ✎█▄▄·█▄▄▄·▄·▄ beats 9/10medium · avg 57%▇ 2.5%at its level
18 more tests
testresultvs the others in its rundifficultycountswhy
Multilingual MMLU (Microsoft)64.4% ✎█▄▄██··█···▄ beats 9/10easy · avg 65%▇ 2.1%at its level
CPTUB average (SpeakLeash, 0-5 judge scale)61.3% ✎▁▂▂▂▃█▃▆▅▄▂▃ beats 36/95easy · avg 66%▇ 1.8%at its level
EvalPerf correctness (118 tasks, 20240328)80.0% ✎▂·▂▂▂▆▅▆▆▆▃█ beats 17/29easy · avg 76%▇ 1.7%at its level
WinoGrande71.1% ✎▃····▃▃█▃·▅▅ beats 7/10medium · avg 74%▇ 1.6%at its level
Polish EQ-Bench v2 (SpeakLeash)58.6% ✎▁··▁▁▂▂▆▅▆█▆ beats 38/70medium · avg 64%▇ 1.5%at its level
BRIDGE clinical text, zero-shot average (87 tasks)31.3% ✎▁▁▁▁▁▂▂▃▄▆█▃ beats 34/97hard · avg 32%▇ 1.2%blunt · at its level
BoolQ80.0% ✎▃··▃··██▃·▃▃ beats 5/10easy · avg 85%▇ 1.1%at its level
HellaSwag80.0% ✎▃▅█·▃▃▃···▃▃ beats 7/10easy · avg 84%▇ 1.1%typical
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)26.3% ✎▁▁·▁▂▂▄█▄▅▄▄ beats 22/66hard · avg 31%▇ 1.1%at its level
Social IQA75.3% ✎▃·▃▅▅█·▃···▃ beats 7/10easy · avg 78%▇ 1.1%blunt · at its level
Open Persian LLM Leaderboard v2 average (21 tasks)52.5% ✎▂▄▂▃█▄▃▆▃▅▂▂ beats 37/81medium · avg 58%▇ 1.1%blunt · at its level
GPQA (subset unspecified)30.6% ✎█··▄▂··▂▃··▄ beats 8/10very hard · avg 33%▇ 1.0%sharp · noisy
TruthfulQA MC2 +163.1% ✎▃▃▂·▃▃·▅█▂▅▂ beats 11/21medium · avg 63%▇ 0.9%blunt · at its level · several runs
PopQA18.1% ✎▁▁▁▃▃▃▁█▁▃·▄ beats 5/21hard · avg 32%▇ 0.7%blunt · at its level
Open PT-LLM Leaderboard average (9 Portuguese tasks)73.8% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 211/276medium · avg 73%▇ 0.6%at its level · noisy
PIQA76.2% ✎▃·▅·▃█·▃▅··▃ beats 3/10easy · avg 83%▇ 0.4%blunt · at its level
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)97.2% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 109/140very easy · avg 94%▇ 0.2%blunt · maxed out · noisy
Chess Puzzles (Epoch)0.0% ✎█▁▁▁▁▂▁▄▂▂▁▁ beats 41/135very hard · avg 5.9%▇ 0.1%sharp · too hard for it · noisy

$ where to run it

Through an API

api · Together AI (Turbo) · $0.30/M · $0.3 in / $0.3 out per 1M tokens · 131k context · www.together.ai ↗ · price sourceUSD per 1M tokens, Together AI serverless inference, "Qwen2.5 7B Instruct Turbo" (Together optimized serving endpoint of Qwen2.5-7B-Instruct; its quantization may differ from the reference BF16 weights). Batch and dedicated endpoints excluded. Currency: USD. Together AI official pricing page, observed 2026-09-25.

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Qwen2.5 7B Instruct · needs ~4.7 GB · from huggingface.co/Qwen (official)
    chat in the terminal
    llama cli -hf Qwen/Qwen2.5-7B-Instruct-GGUF:q4_k_m
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf Qwen/Qwen2.5-7B-Instruct-GGUF:q4_k_m

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Qwen2.5 7B Instruct · needs ~4.7 GB · from huggingface.co/Qwen (official)

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF@q4_k_m -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Qwen2.5 7B Instruct · needs ~5.07 GB · from ollama.com
    chat in the terminal
    ollama run qwen2.5:7b

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Qwen2.5 7B Instruct · needs ~4.3 GB, Mac only · from huggingface.co/lmstudio-community
    chat in the terminal
    mlx_lm.chat --model lmstudio-community/Qwen2.5-7B-Instruct-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model lmstudio-community/Qwen2.5-7B-Instruct-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · microsoft/Phi-4-mini-instruct… 33%, WildEval ZeroEval published r… 26%, 2 OLMo 2 Furious (Team OLMo,… 13% and 15 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
microsoft/Phi-4-mini-instruct model card16▇▇▇▇▇▇▇▇▇▇ 33%MIT
WildEval ZeroEval published results4▇▇▇▇▇▇▇▇ 26%Apache-2.0
2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models8▇▇▇▇ 13%CC BY 4.0
LiveCodeBench official website results (2024-12-26)1▇▇ 5%CC-BY-SA-4.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)2▇ 3%Apache-2.0
Epoch AI — internal evaluations3▇ 3%CC BY 4.0
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)4▇ 2%CC-BY-4.0
Reinforcing Real-world Service Agents: InteractCS-RL (arXiv:2602.22697v1)1▇ 2%CC-BY-4.0
allenai/Olmo-3-7B-Instruct model card3▇ 2%Apache-2.0
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)1▇ 2%MIT
EvalPlus / EvalPerf published evaluation runs1▇ 2%Apache-2.0
SpeakLeash Polish EQ-Bench leaderboard1▇ 1%Apache-2.0
Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments (arXiv:2606.03892v2)1▇ 1%CC-BY-4.0
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)1▇ 1%MIT
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)1▇ 1%Apache-2.0
Berkeley Function Calling Leaderboard result archive1▇ 1%Apache-2.0
Open Portuguese LLM Leaderboard (archived)1▇ 1%Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)1▇ 0%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 48.4
$ published speed measurements · 1

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

Apple M4 Pro (12-core CPU / 16-core GPU, 48GB)GGUF Q4_K_M (Ollama registry default)59 tok/sOllama v0.21.2 (llama.cpp, Metal); macOS 15.4 · throughput = eval_count / eval_duration (Ollama API; decode phase only); temperature 0; one warm-up per batch; one model resident at a time; Table IV model-level value over the MMLU (~15 output tokens), summarization and code-generation (~200 output tokens) tasks; aggregation across tasks not further specified

Model-level figure from Table IV; per-task rates differ substantially (paper text).

source ↗
$ quantization evidence · 30 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precisiontestfull → quantizedkept
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)MMLU — published row 'MMLU (5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.742 → 0.7499.7%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)OpenLLM v1 average of 6 tasks (MMLU (5-shot); ARC Challenge (25-shot); GSM-8K (5-shot, strict-match); Hellaswag (10-shot); Winogrande (5-shot); TruthfulQA (0-shot, mc2)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.732 → 0.72999.7%source ↗
FP8 dynamic (weights FP8 per-channel static, activations FP8 per-token dynamic; llm-compressor QuantizationModifier)GSM8K — published row 'GSM-8K (5-shot, strict-match)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.804 → 0.80199.6%source ↗
GGUF Q2_K_S (row label 'q2ks'; AutoRound default GGUF path (tuning mode not restated in the row label); auto-round >0.12.3)Average accuracy of the 10 listed lm-eval tasks (lm-eval 0.4.9.1, transformers 4.57.6)0.657 → 0.63997.2%source ↗
GGUF Q2_K_S (row label 'q2ks'; AutoRound default GGUF path (tuning mode not restated in the row label); auto-round >0.12.3)MMLU (lm-eval 0.4.9.1 default mmlu task, transformers 4.57.6)0.716 → 0.67494.1%source ↗
GGUF Q2_K_S produced by AutoRound 'Optimized RTN' (--iters 0, no tuning), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.62795.4%source ↗
GGUF Q2_K_S produced by AutoRound + alg_ext (tuned), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.63596.6%source ↗
GGUF Q3_K_M produced by AutoRound 'Optimized RTN' (--iters 0, no tuning), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.64297.7%source ↗
GGUF Q3_K_M produced by AutoRound + alg_ext (tuned), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.65399.4%source ↗
GGUF Q3_K_S (row label 'q3ks-opt-rtn'; AutoRound 'Optimized RTN'; auto-round >0.12.3)Average accuracy of the 10 listed lm-eval tasks (lm-eval 0.4.9.1, transformers 4.57.6)0.657 → 0.63897.0%source ↗
GGUF Q3_K_S (row label 'q3ks-opt-rtn'; AutoRound 'Optimized RTN'; auto-round >0.12.3)MMLU (lm-eval 0.4.9.1 default mmlu task, transformers 4.57.6)0.716 → 0.69196.5%source ↗
GGUF Q3_K_S (row label 'q3ks'; AutoRound default GGUF path (tuning mode not restated in the row label); auto-round >0.12.3)Average accuracy of the 10 listed lm-eval tasks (lm-eval 0.4.9.1, transformers 4.57.6)0.657 → 0.64898.6%source ↗
GGUF Q3_K_S (row label 'q3ks'; AutoRound default GGUF path (tuning mode not restated in the row label); auto-round >0.12.3)MMLU (lm-eval 0.4.9.1 default mmlu task, transformers 4.57.6)0.716 → 0.70398.2%source ↗
GGUF Q3_K_S produced by AutoRound 'Optimized RTN' (--iters 0, no tuning), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.63897.1%source ↗
GGUF Q3_K_S produced by AutoRound + alg_ext (tuned), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.6599.0%source ↗
GGUF Q4_K_M (row label 'q4km'; AutoRound default GGUF path (tuning mode not restated in the row label); auto-round >0.12.3)Average accuracy of the 10 listed lm-eval tasks (lm-eval 0.4.9.1, transformers 4.57.6)0.657 → 0.658100.2%source ↗
GGUF Q4_K_M (row label 'q4km'; AutoRound default GGUF path (tuning mode not restated in the row label); auto-round >0.12.3)MMLU (lm-eval 0.4.9.1 default mmlu task, transformers 4.57.6)0.716 → 0.71499.7%source ↗
GGUF Q4_K_M produced by AutoRound 'Optimized RTN' (--iters 0, no tuning), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.65699.8%source ↗
GGUF Q4_K_M produced by AutoRound + alg_ext (tuned), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.657100.1%source ↗
GGUF Q4_K_S produced by AutoRound 'Optimized RTN' (--iters 0, no tuning), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.65699.8%source ↗
GGUF Q4_K_S produced by AutoRound + alg_ext (tuned), auto-round <=0.12.3Average accuracy of 10 lm-eval tasks (lambada_openai, hellaswag, piqa, winogrande, truthfulqa_mc1, openbookqa, boolq, arc_easy, arc_challenge, mmlu); lm-eval defaults0.657 → 0.66100.5%source ↗
GPTQ INT4 (AutoGPTQ 0.7.1, group size 128, no act-order, 128 WikiText-2 calibration samples)Clinical accuracy, unweighted mean of 4 benchmarks (MedQA n=1273, MedMCQA, Med-HALT, HealthBench risk-stratified n=150); greedy decoding; Transformers on H1000.435 → 0.43198.9%source ↗
GPTQ INT8 (AutoGPTQ 0.7.1, group size 128, no act-order, 128 WikiText-2 calibration samples)Clinical accuracy, unweighted mean of 4 benchmarks (MedQA n=1273, MedMCQA, Med-HALT, HealthBench risk-stratified n=150); greedy decoding; Transformers on H1000.435 → 0.43499.8%source ↗
GPTQ-Int8 (Qwen2.5-7B-Instruct-GPTQ-Int8 checkpoint, as named in Table 1)Bangla zero-shot accuracy, unweighted mean of 5 benchmarks (Bangla MMLU, CommonsenseQA-BN, OpenBookQA-BN, PIQA-BN, BoolQ-BN)0.581 → 0.583100.3%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)GSM8K — published row 'GSM-8K (5-shot, strict-match)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.804 → 0.806100.3%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)OpenLLM v1 average of 6 tasks (MMLU (5-shot); ARC Challenge (25-shot); GSM-8K (5-shot, strict-match); Hellaswag (10-shot); Winogrande (5-shot); TruthfulQA (0-shot, mc2)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.732 → 0.72799.4%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)MMLU — published row 'MMLU (5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.742 → 0.73298.6%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)GSM8K — published row 'GSM-8K (5-shot, strict-match)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.804 → 0.807100.5%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)OpenLLM v1 average of 6 tasks (MMLU (5-shot); ARC Challenge (25-shot); GSM-8K (5-shot, strict-match); Hellaswag (10-shot); Winogrande (5-shot); TruthfulQA (0-shot, mc2)) as published; lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.732 → 0.73199.8%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)MMLU — published row 'MMLU (5-shot)' (OpenLLM v1 section); lm-evaluation-harness + vLLM (backend vllm, max_model_len 4096, add_bos_token)0.742 → 0.73999.5%source ↗

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json