whatmodel.sh / StableLM-Base-Alpha 3B

StableLM-Base-Alpha 3B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

StableLM-Base-Alpha 3B is an open-weights language model from Stability AI, released April 2023. Its published size is 3.6B parameters and a 4k-token context window. Licence: cc-by-sa-4.0.

Catalogue identity notes

First StableLM alpha base release (April 2023), not the -v2 re-release.

3.6±1.4#713 of 719 LLMsfrom 12 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of StableLM-Base-Alpha 3B, out of 828 with a score, then more from Stability AI.

#modelscorewhereRAMreleased
820BLOOM 560M BigScience5±2local~2.1GB2022-07
821OPT-350M Meta5±2local~2.2GB2022-05
822StableLM-Base-Alpha 3B Stability AI4±1local~17GB2023-04
823Pythia 160M EleutherAI3±2local~1.6GB2023-02
824OPT-125M Meta3±2local~1.8GB2022-05
more from Stability AI
718Stable Code 3B Stability AI33±1local~3.3GB2024-01
746StableLM-3B-4E1T Stability AI29±1local~3.3GB2023-09
750StableLM-Base-Alpha 7B v2 Stability AI29±1local~30GB2023-08

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests12 not in a task · 99% of its score
testresultvs the others in its rundifficultycountswhy
SciQ lab71.7% ✎▃·▁·▁·▃▅▃█▄▁ beats 0/26very easy · avg 98%▇▇▇▇▇▇▇▇▇▇▇▇▇ 33%at its level
ARC Easy lab42.0% ✎▃▁·▁▁▄▅▄█▅▁▃ beats 0/26easy · avg 88%▇▇▇▇▇ 12%typical
HellaSwag lab38.3% ✎▃···▁▇▅█▇▃▁▁ beats 0/26very easy · avg 92%▇▇▇▇ 10%typical
PIQA lab63.8% ✎▃·▁··▁▁▃█▅█▂ beats 0/26easy · avg 88%▇▇▇▇ 8.9%blunt · at its level
ARC Challenge lab25.8% ✎▃·▁··▁▁▇▂█▆▃ beats 0/26easy · avg 85%▇▇▇ 8.6%sharp · too hard for it
BoolQ lab57.6% ✎▃▃▂▅▆▂▅▆█▂▂▂ beats 0/26easy · avg 91%▇▇▇ 7.7%typical
LAMBADA (OpenAI version), last-word accuracy lab41.7% ✎▃··▂▁▁▁▂▆█▅▃ beats 0/26very easy · avg 89%▇▇▇ 7.5%blunt · at its level
WinoGrande lab52.6% ✎▁····▃··▁▂█▄ beats 1/26easy · avg 89%▇▇▇ 6.4%too hard for it
4 more tests
testresultvs the others in its rundifficultycountswhy
Social IQA lab35.6% ✎▃···▁▃█▁▁▁·▁ beats 0/26medium · avg 59%▇ 2.1%blunt · too hard for it
TruthfulQA MC2 lab40.5% ✎█·█·█▄···▄·█ beats 9/9medium · avg 61%▇ 1.1%blunt · at its level
OpenBookQA lab17.0% ✎█·▁▁▃·▁▃▅▇▃▄ beats 5/26medium · avg 53%▇ 0.9%too hard for it
Open PT-LLM Leaderboard average (9 Portuguese tasks)27.6% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 2/276medium · avg 73%▇ 0.6%too hard for it · noisy

$ where to run it

Other ways to run it

  1. Run it as published · needs ~14.82 GB · from huggingface.co/stabilityai (official)
    uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='stabilityai/stablelm-base-alpha-3b')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/stabilityai ↗

$ how the score was worked out · Stability AI — original Stabl… 99%, Open Portuguese LLM Leaderboa… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Stability AI — original StableLM zero-shot evaluation outputs · lab-reported11▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 99%Apache-2.0
Open Portuguese LLM Leaderboard (archived)1▇ 1%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 3.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json