whatmodel.sh / Phi-2 (2.7B)

Phi-2 (2.7B)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Phi-2 (2.7B) is an open-weights language model from Microsoft, released December 2023. Its published size is 2.78B parameters and a 2k-token context window. Licence: mit.

Catalogue identity notes

Phi-2 2.7B base model (December 2023); not Phi-3.

38.6±0.6#550 of 719 LLMsfrom 19 benchmark results across 8 sources

$ how it compares

The LLMs ranked either side of Phi-2 (2.7B), out of 828 with a score, then more from Microsoft.

#modelscorewhereRAMreleased
640speechless-code-mistral-7b-v1.0 uukuguy39±1local~17GB2023-10
641Falcon-H1 3B Instruct TII39±4local~3.4GB2025-05
642Phi-2 (2.7B) Microsoft39±1local~3.2GB2023-12
643Granite 3.0 2B Instruct IBM39±1local~3.2GB2024-10
644Llama 2 70B Chat Meta39±1local~32GB2023-07
more from Microsoft
252Phi-4-reasoning-plus 14B Microsoft59±1local~10GB2025-04
261Phi-4-reasoning 14B Microsoft58±1api $0.22/M +local~10GB2025-04
365Phi-4 14B Microsoft53±1api $0.087/M +local~10GB2024-12

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #374 of 458 · 72% of its score · 6 tests

Beats LFM2.5 8B-A1B, DeepSeek R1 Distill Qwen 7B, Qwen3.5 2B here, though they rank higher overall. Trails Qwen2.5 1.5B Instruct, WizardLM 13B V1.2, Granite 3B Code Instruct (2K) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval +249.4% ✎▁·▁▁▃▄▆▅█▃▁▁ beats 35/99easy · avg 79%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 45%sharp · at its level · several runs
HumanEval+45.1% ✎▁·▁▁▁▃▄▄▄█▃▁ beats 37/99easy · avg 74%▇▇▇▇▇▇▇▇ 19%sharp · at its level
MBPP+54.2% ✎▁···▁▁▂▅▇█▆▂ beats 25/75easy · avg 68%▇ 3.6%at its level
MBPP55.0% ✎▁▁▁▄▅█▄▅▃▁▂▁ beats 42/81easy · avg 70%▇ 1.5%at its level
CRUXEval-I31.6% ✎▁··▁▅▂█▇▁··▁ beats 5/22medium · avg 54%▇ 1.1%at its level
CRUXEval-O33.5% ✎▁·▂▃▁█▁▂···▁ beats 4/22medium · avg 53%▇ 1.0%blunt · at its level
chat & writing▁▃▁▃▅▄▅█▆▆▇▃ #389 of 393 * · 7.6% of its score · 2 tests

Trails Gemma 3 270M IT, SmolLM2 360M Instruct, Qwen1.5 1.8B Chat here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
AlpacaEval 2 length-controlled4.4% ✎▁▃▂▃▆▆█▆▄▄▁▁ beats 7/122medium · avg 39%▇▇▇ 6.6%typical
AlpacaEval 1.0 win rate (vs text-davinci-003, GPT-4 judge)30.7% ✎▂▃·▆·▇▃▆█▆▄▂ beats 3/45very easy · avg 96%▇ 1.0%sharp · noisy
math▁▂▃▅▆▅▄█▄▁▁▁ #281 of 398 * · 1.8% of its score · 1 test

Beats Molmo 2 8B, Llama 3.1 Nemotron 70B Instruct, GPT-4 (0613) here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K57.2% ✎▂▂▂▅▅▆▅▇█▇▄▃ beats 53/146easy · avg 74%▇ 1.8%sharp · at its level
other tests8 not in a task · 18% of its score
testresultvs the others in its rundifficultycountswhy
EvoEval (overall)31.8% ✎▂▂▃▄▄█▁▄▅▃▁▃ beats 16/42medium · avg 55%▇▇▇ 7.6%at its level
BIG-Bench Hard43.4% ✎▁▁▁▁▂▄▅▅█▃▄▁ beats 31/129medium · avg 68%▇▇ 4.5%sharp
ARC Challenge71.3% ✎▆▁▁▂▂▅▇█▄▃▃▂ beats 83/97easy · avg 88%▇ 2.6%sharp · at its level
MMLU52.7% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 45/183easy · avg 80%▇ 1.9%at its level
C-Eval23.4% ✎▃···▁▁▂▁▂▂▃█ beats 0/33medium · avg 62%▇ 0.8%at its level · noisy
Open PT-LLM Leaderboard average (9 Portuguese tasks)36.5% ✎▃▁▁▁▃▁▁▁▂█▅▁ beats 45/276medium · avg 73%▇ 0.6%at its level · noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)93.3% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 45/140very easy · avg 94%▇ 0.3%blunt · noisy
COPA90.2% ✎▅·▃·▃··█▃··▃ beats 7/8easy · avg 92%▇ 0.1%at its level · noisy

$ where to run it

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Phi-2 (2.7B) · needs ~1.8 GB · from huggingface.co/TheBloke
    chat in the terminal
    llama cli -hf TheBloke/phi-2-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf TheBloke/phi-2-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Phi-2 (2.7B) · needs ~1.8 GB · from huggingface.co/TheBloke

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/TheBloke/phi-2-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Phi-2 (2.7B) · needs ~1.6 GB · from ollama.com
    chat in the terminal
    ollama run phi:2.7b

MLX (Mac)

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Phi-2 (2.7B) · needs ~1.6 GB, Mac only · from huggingface.co/jesusoctavioas
    chat in the terminal
    mlx_lm.chat --model jesusoctavioas/phi-2-mlx-4Bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model jesusoctavioas/phi-2-mlx-4Bit

Other ways to run it

  1. Run it as published · needs ~5.56 GB · from huggingface.co/microsoft (official)
    uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='microsoft/phi-2')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/microsoft ↗

$ how the score was worked out · EvalPlus / EvalPerf published… 51%, Top Leaderboard Ranking = Top… 22%, MiniCPM: Unveiling the Potent… 16% and 5 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
EvalPlus / EvalPerf published evaluation runs3▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 51%Apache-2.0
Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs2▇▇▇▇▇▇▇ 22%CC BY 4.0
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)7▇▇▇▇▇ 16%CC BY 4.0
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)2▇▇ 8%Apache-2.0
CRUXEval published evaluation results (authors' generations and scores)2▇ 2%MIT
Open Portuguese LLM Leaderboard (archived)1▇ 1%Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)1▇ 0%Apache-2.0
From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1)1▇ 0%CC-BY-4.0
prior · pull toward the middle▇ 0%
weighted average= 38.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json