whatmodel.sh / Phi-3-small-8k-instruct
Phi-3-small-8k-instruct
↗ share
+ contribute
Phi-3-small-8k-instruct is an open-weights language model from Microsoft, released May 2024. Its published size is 7.39B parameters. Licence: mit.
Catalogue identity notes
Comparison-only. Arena entry 'phi-3-small-8k-instruct' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'phi-3-small-8k-instruct' as 'Phi-3-Small-8k-Instruct' (Microsoft, MIT), linking https://huggingface.co/microsoft/Phi-3-small-8k-instruct. Maker repo microsoft/Phi-3-small-8k-instruct @188b876: licence tag mit. 7,392,274,432 parameters from the Hugging Face safetensors metadata (dense, so active = total). Released 2024-05: card: 'The model weight is released on May 21, 2024'.
$ how it compares
The LLMs ranked either side of Phi-3-small-8k-instruct, out of 828 with a score, then more from Microsoft.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 581 |  DeepSeek-Coder 1.3B Instruct | 43±1 | local | ||
| 582 |  Aya 23 8B | 43±3 | local | ||
| 583 | Phi-3-small-8k-instruct | 43±2 | local | ||
| 584 |  CodeGemma 7B IT (v1.0) | 43±1 | local | ||
| 585 |  Aya Vision 32B | 42±3 | api price unknown +local | ||
| more from Microsoft | |||||
| 252 | Phi-4-reasoning-plus 14B | 59±1 | local | ||
| 261 | Phi-4-reasoning 14B | 58±1 | api $0.22/M +local | ||
| 365 | Phi-4 14B | 53±1 | api $0.087/M +local | ||
$ what it's good at
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #326 of 393 · 94% of its score · 3 tests
Beats OpenThinker3 7B, Gemma 4 E2B, Code Llama 70B Instruct here, though they rank higher overall. Trails LFM2 350M, Gemma 3 1B, Flan-T5 XXL here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena-Hard (v0.1, GPT-4-1106 judge) | 29.8% ✎ | ▂▁▁▅█▅▄▃▃▅▃▁ beats 25/60 | 47% |
| Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024) | 1103.0 ✎ | ▂▁▂▁▃▇█▆▇▄▂▂ beats 48/95 | 24% |
| LMArena (text) | 1171.0 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 51/379 | 23% |
other tests1 not in a task · 5.2% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Open PT-LLM Leaderboard average (9 Portuguese tasks) | 70.8% ✎ | ▃▁▁▁▁▁▁▁▂█▅▁ beats 172/276 | 5.2% |
$ where to run it
Other ways to run it
- Run it as published · needs ~14.78 GB, NVIDIA only · from huggingface.co/microsoft (official)
pip install tiktoken==0.6.0 triton==2.3.0 transformers==4.40.2 torch && python -c "import torch; from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline; r='microsoft/Phi-3-small-8k-instruct'; d=torch.cuda.current_device(); m=AutoModelForCausalLM.from_pretrained(r, torch_dtype='auto', trust_remote_code=True).to(d); t=AutoTokenizer.from_pretrained(r, trust_remote_code=True); print(pipeline('text-generation', model=m, tokenizer=t, device=d)([{'role': 'user', 'content': 'What about solving an 2x + 3 = 7 equation?'}], max_new_tokens=500, return_full_text=False, do_sample=False)[0]['generated_text'])"
Official weights and documentation: huggingface.co/microsoft ↗
$ how the score was worked out · Arena-Hard-Auto v0.1 leaderbo… 47%, Polyrating: A Cost-Effective… 24%, LMArena leaderboard dataset 23% and 1 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2) | 1 | 47% |
 Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3) | 1 | 24% |
 LMArena leaderboard dataset | 1 | 23% |
 Open Portuguese LLM Leaderboard (archived) | 1 | 5% |
| prior · pull toward the middle | 1% | |
| weighted average | = 42.5 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json