whatmodel.sh / speechless-code-mistral-7b-v1.0

speechless-code-mistral-7b-v1.0

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

speechless-code-mistral-7b-v1.0 is an open-weights language model from uukuguy, released October 2023. Licence: apache-2.0.

Catalogue identity notes

Maker repo uukuguy/speechless-code-mistral-7b-v1.0 @1862e0a: licence tag apache-2.0. Released 2023-10: month the maker's Hugging Face repo was created (createdAt 2023-10-10).

38.7±0.8#548 of 719 LLMsfrom 4 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of speechless-code-mistral-7b-v1.0, out of 828 with a score, then more from uukuguy.

#modelscorewhereRAMreleased
638Granite 8B Code Base (4K) IBM39±1local~6.6GB2024-04
639StarCoder2 15B BigCode39±1local~11GB2024-02
640speechless-code-mistral-7b-v1.0 uukuguy39±1local~17GB2023-10
641Falcon-H1 3B Instruct TII39±4local~3.4GB2025-05
642Phi-2 (2.7B) Microsoft39±1local~3.2GB2023-12
more from uukuguy
435speechless-codellama-34b-v2.0 uukuguy49±1local~23GB2023-10
480speechless-coder-ds-6.7b uukuguy47±1local~5.8GB2023-12
523speechless-starcoder2-15b uukuguy45±1local~12GB2024-03

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #388 of 458 · 99% of its score · 3 tests

Beats DeepSeek R1 Distill Qwen 7B, Qwen3.5 2B, DeepSeek R1 Distill Llama 8B here, though they rank higher overall. Trails Qwen2.5 1.5B Instruct, WizardLM 13B V1.2, Falcon3 3B Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval48.2% ✎▁·▁▁▃▄▆▅█▃▁▁ beats 33/99easy · avg 79%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 54%sharp · at its level
HumanEval+41.5% ✎▁·▁▁▁▃▄▄▄█▃▁ beats 31/99easy · avg 74%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 38%sharp · at its level
MBPP+48.7% ✎▁···▁▁▂▅▇█▆▂ beats 18/75easy · avg 68%▇▇▇ 6.9%at its level
other tests1 not in a task · 1.2% of its score
testresultvs the others in its rundifficultycountswhy
Open PT-LLM Leaderboard average (9 Portuguese tasks)64.4% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 110/276medium · avg 73%▇ 1.2%at its level · noisy

$ where to run it

Other ways to run it

  1. Run it as published · needs ~14.48 GB · from huggingface.co/uukuguy (official)
    uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='uukuguy/speechless-code-mistral-7b-v1.0', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/uukuguy ↗

$ how the score was worked out · EvalPlus / EvalPerf published… 99%, Open Portuguese LLM Leaderboa… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
EvalPlus / EvalPerf published evaluation runs3▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 99%Apache-2.0
Open Portuguese LLM Leaderboard (archived)1▇ 1%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 38.7

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json