whatmodel.sh / StarCoderBase 3B

StarCoderBase 3B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

StarCoderBase 3B is an open-weights language model from BigCode, released June 2023. Its published size is 3B parameters. Licence: bigcode-openrail-m.

Catalogue identity notes

Comparison-only. Maker repo bigcode/starcoderbase-3b @e1c5ef4: licence tag bigcode-openrail-m. Parameters: nominal size from the model name (no safetensors metadata in the repo) (dense, so active = total). Released 2023-06: month the maker's Hugging Face repo was created (createdAt 2023-06-30). StarCoderBase (2023) 3B; not StarCoder2.

27.6±0.8#658 of 719 LLMsfrom 12 benchmark results across 3 sources

$ how it compares

The LLMs ranked either side of StarCoderBase 3B, out of 828 with a score, then more from BigCode.

#modelscorewhereRAMreleased
757LLaMA 13B Meta28±1comparison only2023-02
758Koala 13B UC Berkeley (BAIR)28±1comparison only2023-04
759StarCoderBase 3B BigCode28±1local~3.4GB2023-06
760Granite 3.0 1B-A400M Instruct IBM27±1local~2.3GB2024-10
761RedPajama-INCITE 7B Base Together27±1local~16GB2023-06
more from BigCode
521StarCoder2-15B-Instruct-v0.1 BigCode46±1local~11GB2024-04
639StarCoder2 15B BigCode39±1local~11GB2024-02
698StarCoder2 7B BigCode35±1local~6.1GB2024-02

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▃▃▆██▇▆▄▃▁ #442 of 458 · 97% of its score · 9 tests

Beats Molmo 7B-D (0924), DeepSeek R1 Distill Qwen 1.5B, Tulu 2 7B here, though they rank higher overall. Trails Falcon3 1B Instruct, Granite 3.0 1B-A400M Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval +117.7% ✎▁·▁▃▃▄▆▅█▃▁▁ beats 6/99easy · avg 79%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 43%sharp · several runs
HumanEval+15.9% ✎▁·▁▁▃▃▄▄▄█▃▁ beats 6/99easy · avg 74%▇▇▇▇▇▇▇▇ 21%sharp
MBPP +129.4% ✎█·▅▃▃▃▅▃▃▅·▅ beats 0/15easy · avg 82%▇▇▇▇ 11%at its level · several runs
HumanEvalSynthesize (HumanEvalPack, 6-language average)21.8% ✎▁·▂▃▂▆▅█▄▄▁▂ beats 3/35medium · avg 64%▇▇▇ 6.7%typical
DS-1000 lab14.2% ✎▄▄·▄██▄█·█·▄ beats 0/12medium · avg 52%▇▇ 3.9%typical
CRUXEval-I27.5% ✎▃▅█▃▃▅·▃▃▅·▅ beats 0/15medium · avg 59%▇ 3.5%at its level
HumanEvalExplain (HumanEvalPack, 6-language average)11.9% ✎▄▇█▃▄▇▄▁▇▃·▁ beats 2/35medium · avg 49%▇ 3.0%typical
CRUXEval-O27.0% ✎▃·▃██▅▃·▃▃▅▃ beats 0/15medium · avg 55%▇ 2.9%blunt · at its level
1 more test
testresultvs the others in its rundifficultycountswhy
HumanEvalFix (HumanEvalPack, 6-language average)6.4% ✎▁·▁▃▄██▄█▁▅▅ beats 2/35medium · avg 56%▇ 2.0%sharp · too hard for it
math▃▂▃▅▆▅▄█▄▁▁▁ #398 of 398 * · 2.5% of its score · 1 test

Trails Qwen2 0.5B Instruct, Granite 4.0 350M, Granite 4.0 H 350M here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K lab8.0% ✎▄▄▄·▄█·██·█▄ beats 0/12easy · avg 75%▇ 2.5%sharp

$ where to run it

Ollama · recommended

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run StarCoderBase 3B · needs ~1.8 GB · from ollama.com
    chat in the terminal
    ollama run starcoder:3b

Other ways to run it

  1. Run it as published · needs ~12.17 GB · from huggingface.co/bigcode (official)
    uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='bigcode/starcoderbase-3b', torch_dtype='auto', device_map='auto')('def print_hello_world():', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/bigcode ↗

$ how the score was worked out · EvalPlus / EvalPerf published… 53%, StarCoder 2 and The Stack v2:… 24%, Granite Code Models: A Family… 22%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
EvalPlus / EvalPerf published evaluation runs2▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 53%Apache-2.0
StarCoder 2 and The Stack v2: The Next Generation — authors' own runs · lab-reported4▇▇▇▇▇▇▇ 24%CC BY-SA 4.0
Granite Code Models: A Family of Open Foundation Models for Code Intelligence — authors' own runs6▇▇▇▇▇▇▇ 22%CC BY 4.0
prior · pull toward the middle▇ 0%
weighted average= 27.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json