whatmodel.sh / Phi-3.5-MoE-instruct

Phi-3.5-MoE-instruct

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Phi-3.5-MoE-instruct is an open-weights language model from Microsoft, released August 2024. Its published size is 41.87B parameters (6.6B active per token) and a 128k-token context window. Licence: mit.

Catalogue identity notes

Maker repo microsoft/Phi-3.5-MoE-instruct @4368845: licence tag mit. 41,873,153,344 parameters from the Hugging Face safetensors metadata. Mixture-of-experts: pinned README: 'Phi-3.5-MoE has 16x3.8B parameters with 6.6B active parameters when using 2 experts'. Released 2024-08: month the maker's Hugging Face repo was created (createdAt 2024-08-17).

48.0±1.2#382 of 719 LLMsfrom 6 benchmark results across 4 sources

$ how it compares

The LLMs ranked either side of Phi-3.5-MoE-instruct, out of 828 with a score, then more from Microsoft.

#modelscorewhereRAMreleased
455T5 Large Google48±9local~4.6GB2019-10
456WaveCoder-Ultra-6.7B Microsoft48±1local~5.8GB2024-04
457Phi-3.5-MoE-instruct Microsoft48±1local~26GB2024-08
458Claude 3 Haiku Anthropic48±0comparison only2024-03
459GPT-4o mini Audio Preview (2024-12-17) OpenAI48±2comparison only2024-12
more from Microsoft
252Phi-4-reasoning-plus 14B Microsoft59±1local~10GB2025-04
261Phi-4-reasoning 14B Microsoft58±1api $0.22/M +local~10GB2025-04
365Phi-4 14B Microsoft53±1api $0.087/M +local~10GB2024-12

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests6 not in a task · 99% of its score
testresultvs the others in its rundifficultycountswhy
LiveBench 2024-07-2635.2% ✎▁▁▁▂▂▃▂█▆▃▅▃ beats 38/61medium · avg 41%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 53%at its level
CLIcK Korean (Daekeun Kim)56.4% ✎▆▄▄▄█▄▂·▄█·▄ beats 4/23medium · avg 58%▇▇▇▇▇▇▇▇ 19%at its level
HAE-RAE 1.0 (Daekeun Kim)61.8% ✎▄▄▄▂▂█▂▄▂█▂▄ beats 5/22medium · avg 63%▇▇▇▇▇▇▇ 17%at its level
BRIDGE clinical text, zero-shot average (87 tasks)29.5% ✎▁▁▁▁▁▂▂▃▄▆█▃ beats 30/97hard · avg 32%▇▇ 6.2%blunt · at its level
KMMLU-HARD (Daekeun Kim, zero-shot)25.3% ✎█···▁▁▁▂▃·▆▂ beats 3/23very hard · avg 29%▇ 3.0%sharp · too hard for it
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)97.5% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 115/140very easy · avg 94%▇ 0.9%blunt · maxed out · noisy

$ where to run it

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Phi-3.5-MoE-instruct · needs ~25.3 GB · from huggingface.co/lmstudio-community
    chat in the terminal
    llama cli -hf lmstudio-community/Phi-3.5-MoE-instruct-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf lmstudio-community/Phi-3.5-MoE-instruct-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Phi-3.5-MoE-instruct · needs ~25.3 GB · from huggingface.co/lmstudio-community

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/lmstudio-community/Phi-3.5-MoE-instruct-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Phi-3.5-MoE-instruct · needs ~25.3 GB · from huggingface.co/lmstudio-community
    chat in the terminal
    ollama run hf.co/lmstudio-community/Phi-3.5-MoE-instruct-GGUF:Q4_K_M

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Phi-3.5-MoE-instruct · needs ~23.6 GB, Mac only · from huggingface.co/mlx-community
    chat in the terminal
    mlx_lm.chat --model mlx-community/Phi-3.5-MoE-instruct-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model mlx-community/Phi-3.5-MoE-instruct-4bit

Other ways to run it

  1. Run it as published · needs ~100.99 GB, NVIDIA only · from huggingface.co/microsoft (official)
    vllm serve microsoft/Phi-3.5-MoE-instruct

Official weights and documentation: huggingface.co/microsoft ↗

$ how the score was worked out · LiveBench model judgments (HF… 53%, Daekeun Kim Korean LLM benchm… 39%, BRIDGE Medical Leaderboard (H… 6% and 1 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets1▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 53%Apache-2.0 (stated in the LiveBench datasheet)
Daekeun Kim Korean LLM benchmark runs3▇▇▇▇▇▇▇▇▇▇▇▇ 39%MIT
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)1▇▇ 6%MIT
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)1▇ 1%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 48.0

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json