whatmodel.sh / Mistral 7B v0.1 (base)

Mistral 7B v0.1 (base)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Mistral 7B v0.1 (base) is an open-weights language model from Mistral AI, released September 2023. Its published size is 7.24B parameters and a 8k-token context window. Licence: apache-2.0.

Catalogue identity notes

Pretrained Mistral-7B-v0.1 base model; not Mistral 7B Instruct and not v0.2/v0.3.

35.5±0.4#597 of 719 LLMsfrom 59 benchmark results across 10 sources

$ how it compares

The LLMs ranked either side of Mistral 7B v0.1 (base), out of 828 with a score, then more from Mistral AI.

#modelscorewhereRAMreleased
689xDAN-L1-Chat-RL-v1 xDAN-AI36±1local~6.1GB2023-12
690Llama 3 8B (base) Meta36±1local~6.6GB2024-04
691Mistral 7B v0.1 (base) Mistral AI36±0local~5.8GB2023-09
692Qwen2.5 1.5B Instruct Alibaba36±1local~2.4GB2024-09
693MPT-30B-Chat MosaicML35±2comparison only2023-06
more from Mistral AI
419Codestral 22B v0.1 Mistral AI50±1local~15GB2024-05
425Mistral Small Instruct 22B (2409) Mistral AI49±2local~15GB2024-09
587Mathstral 7B v0.1 Mistral AI42±1local~5.8GB2024-07

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #425 of 458 · 38% of its score · 4 tests

Beats DeepSeek R1 Distill Qwen 7B, Qwen3.5 2B, DeepSeek R1 Distill Llama 8B here, though they rank higher overall. Trails Llama 3.2 1B Instruct, SmolLM2 1.7B Instruct, Mistral 7B Instruct v0.1 here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval +328.7% ✎▁·▁▁▃▄▆▅█▃▁▁ beats 11/99easy · avg 79%▇▇▇▇▇▇▇▇▇ 22%sharp · at its level · several runs
HumanEval+23.8% ✎▁·▁▁▁▃▄▄▄█▃▁ beats 12/99easy · avg 74%▇▇▇▇ 11%sharp · at its level
MBPP lab +350.2% ✎▄··▄·▄··█··▄ beats 4/5easy · avg 71%▇ 3.3%at its level · several runs
MBPP+42.1% ✎▁···▁▁▃▅▇█▆▂ beats 8/75easy · avg 68%▇ 2.2%at its level
math▁▂▃▅▆▅▄█▄▁▁▁ #343 of 398 · 5.9% of its score · 2 tests

Beats DBRX Instruct, Command R Plus (04-2024), Mistral NeMo Instruct (2407) here, though they rank higher overall. Trails Qwen3 0.6B, Gemma 7B (base), Qwen2.5 1.5B Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K lab +350.0% ✎█···█·██··██ beats 3/5easy · avg 79%▇▇ 4.3%sharp · at its level · small run · several runs
MATH lab +112.7% ✎█·█·█·██···█ beats 3/5hard · avg 30%▇ 1.6%several runs
other tests23 not in a task · 56% of its score
testresultvs the others in its rundifficultycountswhy
ARC Challenge +354.4% ✎▁·▁··▁▁▇▂█▆▃ beats 26/26easy · avg 85%▇▇▇▇▇▇▇▇ 19%sharp · at its level · several runs
WinoGrande +274.0% ✎▁····▁··▁▂█▄ beats 26/26easy · avg 89%▇▇ 5.2%at its level · several runs
Natural Questions (closed book) +230.1% ✎▁·▁·▁▂▁▂·▅▂█ beats 26/28medium · avg 58%▇▇ 4.7%at its level · several runs
BIG-Bench Hard +141.1% ✎▁▁▁▁▂▄▅▅█▃▄▁ beats 60/129medium · avg 68%▇ 3.7%sharp · several runs
MMLU lab +262.5% ✎▄····▄▄·▄··█ beats 3/5easy · avg 82%▇ 3.4%at its level · several runs
BoolQ +183.6% ✎▂▃▂▅▆▂▅▆█▂▂▃ beats 26/26easy · avg 91%▇ 2.8%at its level · several runs
HellaSwag +281.1% ✎▃···▁▇▅█▇▃▁▁ beats 24/26very easy · avg 92%▇ 2.6%at its level · several runs
ARC Easy +279.5% ✎▁▁·▁▁▄▅▄█▅▁▃ beats 26/26easy · avg 88%▇ 2.4%at its level · several runs
15 more tests
testresultvs the others in its rundifficultycountswhy
TriviaQA +270.9% ✎▁··▁█▃▃▁▄▅▄█ beats 28/28very easy · avg 86%▇ 2.1%at its level · several runs
PIQA +280.8% ✎▁·▁··▁▁▃█▅█▃ beats 26/26easy · avg 88%▇ 1.6%blunt · at its level · several runs
Factual knowledge recall, Premium2k exact match (zero-shot)28.4% ✎▅····▃▅·██·█ beats 10/13medium · avg 59%▇ 1.3%at its level
OpenBookQA32.8% ✎█·▁▁▃·▁▃▅▇▃▄ beats 22/26medium · avg 53%▇ 1.3%typical
Social IQA +142.7% ✎▁···▁▃█▁▁▁·▁ beats 19/26medium · avg 59%▇ 1.0%blunt · several runs
Factual knowledge recall, Premium2k exact match (10-shot)40.8% ✎▃▅▃··▅▃·█▃▃▅ beats 7/13easy · avg 67%▇ 0.9%at its level
SciQ95.9% ✎▁·▁·▁·▃▅▃█▄▃ beats 26/26very easy · avg 98%▇ 0.9%maxed out
AGIEval English (Gemma report)41.2% ✎▄··········█ beats 1/2easy · avg 65%▇ 0.5%at its level · small run
LAMBADA (OpenAI version), last-word accuracy75.7% ✎▁··▂▁▁▁▂▆█▅▃ beats 25/26very easy · avg 89%▇ 0.5%blunt
C-Eval46.1% ✎▁···▁▁▃▁▂▂▃█ beats 6/33medium · avg 62%▇ 0.4%noisy
CommonsenseQA66.3% ✎▄·▄▄·▄····▄█ beats 5/6easy · avg 84%▇ 0.4%at its level · small run
Open PT-LLM Leaderboard average (9 Portuguese tasks)61.1% ✎▃▁▁▁▁▁▁▁▃█▅▁ beats 92/276medium · avg 73%▇ 0.3%at its level · noisy
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy76.3% ✎▁▁▁▁▁▁▁▂▄█▄▃ beats 53/146easy · avg 83%▇ 0.3%at its level · noisy
HaluEval QA hallucination detection (accuracy)54.5% ✎█▁▁·▁▁·▃▁▁▁▁ beats 21/27medium · avg 70%▇ 0.2%sharp · noisy
TruthfulQA MC242.6% ✎▅▃█▆▁▂▅···▁▁ beats 15/31medium · avg 64%▇ 0.1%blunt · at its level

$ where to run it

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Mistral 7B v0.1 (base) · needs ~4.4 GB · from huggingface.co/MaziyarPanahi
    chat in the terminal
    llama cli -hf MaziyarPanahi/Mistral-7B-v0.1-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf MaziyarPanahi/Mistral-7B-v0.1-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Mistral 7B v0.1 (base) · needs ~4.4 GB · from huggingface.co/MaziyarPanahi

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/MaziyarPanahi/Mistral-7B-v0.1-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Mistral 7B v0.1 (base) · needs ~4.4 GB · from huggingface.co/MaziyarPanahi
    chat in the terminal
    ollama run hf.co/MaziyarPanahi/Mistral-7B-v0.1-GGUF:Q4_K_M

MLX (Mac)

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Mistral 7B v0.1 (base) · needs ~4.1 GB, Mac only · from huggingface.co/en3rypt
    chat in the terminal
    mlx_lm.chat --model en3rypt/Mistral-7B-v0.1-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model en3rypt/Mistral-7B-v0.1-4bit

Other ways to run it

  1. Run it as published · needs ~14.48 GB · from huggingface.co/mistralai (official)
    uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='mistralai/Mistral-7B-v0.1', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/mistralai ↗

$ how the score was worked out · EvalPlus / EvalPerf published… 29%, Stability AI — original Stabl… 26%, Mixtral of Experts — base-mod… 16% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
EvalPlus / EvalPerf published evaluation runs3▇▇▇▇▇▇▇▇▇ 29%Apache-2.0
Stability AI — original StableLM zero-shot evaluation outputs10▇▇▇▇▇▇▇▇ 26%Apache-2.0
Mixtral of Experts — base-model evaluations · lab-reported12▇▇▇▇▇ 16%CC BY 4.0
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies (Hu et al.)7▇▇▇ 9%CC BY 4.0
Mistral 7B — authors' own runs · lab-reported12▇▇▇ 9%CC BY 4.0
Gemma report — original evaluations7▇▇ 5%CC BY 4.0
Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)4▇ 3%Apache-2.0
Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)2▇ 2%CC-BY-4.0
Open Portuguese LLM Leaderboard (archived)1▇ 0%Apache-2.0
EuroEval leaderboards (results collection), English generative leaderboard1▇ 0%MIT
prior · pull toward the middle▇ 0%
weighted average= 35.5

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json