whatmodel.sh / Guanaco 33B
Guanaco 33B
↗ share
+ contribute
Guanaco 33B is an open-weights language model from University of Washington, released May 2023. Its published size is 33B parameters. Licence: llama-1-noncommercial-research.
Catalogue identity notes
Comparison-only. Arena entry 'guanaco-33b' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'guanaco-33b' as 'Guanaco-33B' (UW, Non-commercial), linking https://huggingface.co/timdettmers/guanaco-33b-merged. Maker repo timdettmers/guanaco-33b-merged @b2e78a9: licence tag none tagged. Parameters: nominal size from the model name. Released 2023-05: month the maker's Hugging Face repo was created (createdAt 2023-05-22). QLoRA fine-tune of LLaMA 33B (merged weights repo linked by LMArena's table); LLaMA-based, non-commercial per the Arena licence field.
Model information sources
$ how it compares
The LLMs ranked either side of Guanaco 33B, out of 828 with a score.
| # | model | score | where |
|---|---|---|---|
| 715 |  StarCoderBase 15.5B | 33±1 | local |
| 716 |  Vicuna 7B v1.5 | 33±1 | local |
| 717 | Guanaco 33B | 33±2 | local |
| 718 |  Stable Code 3B | 33±1 | local |
| 719 |  OLMo 2 1B Instruct (0425) | 33±2 | local |
$ what it's good at
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #370 of 393 · 99% of its score · 5 tests
Beats Phi-2 (2.7B), Qwen2.5 1.5B Instruct, Nous-Hermes-13b here, though they rank higher overall. Trails Gemma 3 270M IT, SmolLM2 360M Instruct, Granite 4.0 350M here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AlpacaEval 2 length-controlled | 5.7% ✎ | ▁▂▃▃▆▆█▆▄▄▁▁ beats 11/122 | 29% |
| Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024) | 1035.0 ✎ | ▂▁▂▁▃▇█▆▇▄▂▂ beats 20/95 | 21% |
| LMArena (text) | 1127.4 ✎ | ▁▁▁▃▃▃▃▆▆▇█▃ beats 25/379 | 20% |
| MT-Bench (GPT-4 single-answer grading, LMSYS runs) | 6.5300 MT-Bench score (1-10) ✎ | ▆▂▂▆▆▄██▂▄▄▆ beats 17/28 | 18% |
| AlpacaEval 1.0 win rate (vs text-davinci-003, GPT-4 judge) | 66.0% ✎ | ▂▂·▆·▇▃▆█▆▄▂ beats 12/45 | 11% |
$ where to run it
Other ways to run it
- Run it as published · needs ~65.06 GB · from huggingface.co/timdettmers
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='timdettmers/guanaco-33b-merged', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
Weights and documentation: huggingface.co/timdettmers ↗
$ how the score was worked out · AlpacaEval 1.0 and 2.0 leader… 40%, Polyrating: A Cost-Effective… 21%, LMArena leaderboard dataset 20% and 1 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository) | 2 | 40% |
 Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3) | 1 | 21% |
 LMArena leaderboard dataset | 1 | 20% |
 LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table | 1 | 18% |
| prior · pull toward the middle | 1% | |
| weighted average | = 32.7 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json