whatmodel.sh / stt / Phi-4-multimodal-instruct

Phi-4-multimodal-instruct

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Phi-4-multimodal-instruct is an open-weights speech-to-text model from Microsoft, released February 2025. Its published size is 5.57B parameters. Licence: mit.

Catalogue identity notes

Exact checkpoint microsoft/Phi-4-multimodal-instruct (5.6B multimodal model; ASR uses its speech-lora folder). Distinct from catalogue LLMs phi-4, phi-4-mini. Params: HF safetensors total. Serve command combines the card's vLLM server flags with the speech-lora path used in vLLM's audio example.

57.9±1.8#48 of 162 speech-to-text modelsfrom 7 benchmark results across 5 sources

Score within the Open ASR English short-form WER (March 2026) comparison set only.

$ how it compares

The speech-to-text models ranked either side of Phi-4-multimodal-instruct, out of 176 with a score, then more from Microsoft.

#modelscorewhereRAMreleased
55Chirp 2 Google58±3api price unknown
56Canary-1B-v2 NVIDIA58±1local~5.2GB2025-08
57Phi-4-multimodal-instruct Microsoft58±2local~14GB2025-02
58Omnilingual ASR LLM 7B Meta58±2local~19GB2025-11
59Omnilingual ASR LLM 7B v2 Meta58±3local~19GB2025-12
more from Microsoft
102VibeVoice-ASR-HF Microsoft52±3local~21GB2026-03
109VibeVoice-ASR-BitNet Microsoft50±4local~1.7GB2026-07
125VibeVoice-ASR-Streaming-7B Microsoft47±5local~17GB2026-09

see all speech-to-text models · find the ones that fit your machine

$ what it's good at

multilingual transcription▂··▃▄█▆▅▄▂·▂ #20 of 36 * · 41% of its score · 2 tests

Beats Qwen3-ASR-1.7B, Voxtral Mini Transcribe V2, Canary-1B-v2 here, though they rank higher overall. Trails Omnilingual ASR CTC 1B v2, Omnilingual ASR CTC 7B v2, SeamlessM4T v2 Large here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Open ASR multilingual WER4.4% ✎▂··▂▃▂█▃█▂▁▁ beats 23/29avg 8.9%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 39%typical
Iberian ASR benchmark · 7-language macro WER (eu, ca, gl, pt, es + de, tr)56.3% ✎█▃▃··▅▃▃··▃▃ beats 1/10avg 38%▇ 1.9%noisy
other tests5 not in a task · 58% of its score
testresultvs the others in its rundifficultycountswhy
AfriSpeech-MultiBench · 7-region African-accent mean WER25.9% ✎▂····▂···▃▆█ beats 2/11avg 46%▇▇▇▇▇▇▇▇▇ 22%typical
AfriSpeech-MultiBench · Parliament (legal domain) WER36.7% ✎▂▂····▃·▅·▃█ beats 2/12avg 52%▇▇▇▇▇▇▇ 18%sharp
Open ASR English short-form WER (March 2026)6.0% ✎▁▁▁▃··▁▁▂██▆ beats 78/86avg 8.2%▇▇▇▇▇ 11%blunt
Open ASR Leaderboard – Chilean Spanish · 3-dataset average WER9.4% ✎▂▂·····▂▃·█▂ beats 3/8avg 13%▇▇ 5.4%typical
Speech Accessibility Project dysarthric speech · zero-shot WER (rated subset, N=11,218)15.8% ✎▂·▂··▂····▆█ beats 4/9avg 18%▇ 1.3%blunt

$ where to run it

Other ways to run it

  1. Run it as published · needs ~13.87 GB, NVIDIA only · from huggingface.co/microsoft (official)
    vllm serve microsoft/Phi-4-multimodal-instruct --trust-remote-code --max-model-len 12800 --enable-lora --max-lora-rank 320 --limit-mm-per-prompt '{"audio":1}' --lora-modules speech=$(hf download microsoft/Phi-4-multimodal-instruct)/speech-lora

Official weights and documentation: huggingface.co/microsoft ↗

$ how the score was worked out · Open ASR Leaderboard 50%, AfriSpeech-MultiBench (arXiv:… 40%, Open ASR Leaderboard – Chilea… 5% and 2 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Open ASR Leaderboard2▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 50%Apache-2.0
AfriSpeech-MultiBench (arXiv:2511.14255v1) · African-accented English ASR benchmark suite2▇▇▇▇▇▇▇▇▇▇▇▇ 40%CC BY 4.0
Open ASR Leaderboard – Chilean Spanish (idsudd / Alonso Astroza)1▇▇ 5%Apache-2.0
Benchmarking Automatic Speech Recognition Tools for Iberian Languages (arXiv:2609.36920v1), Table 31▇ 2%CC BY 4.0
Dysarthric ASR with clinical context on the Speech Accessibility Project (arXiv:2605.02782v2)1▇ 1%CC BY 4.0
prior · pull toward the middle▇ 1%
weighted average= 57.9

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json