whatmodel.sh · find the best AI models you can run

Best AI models to run locally on a 4GB Mac

A leaderboard of the best AI models you can run locally on a Mac with 4GB of memory. MiniCPM5 2B is #1 right now, followed by Spark-X2.5 1.7B and LFM2.5 2.6B. Models are ranked by one score that combines 575 published benchmarks. Each model shows the download that fits and the command to run it.

65 models · 575 benchmarks · updated
Filter by your hardware and budget → ~3GB fast · local only · reset
ram[2][4][8][16][24][32][48][64][96][128][192][256]
[]
GB
gpu[apple silicon][graphics card][unified memory][none]
price[local][$0.02][$0.1][$0.5][$2][$10][$50][any]max per M
Filter by task
task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]
New #1s for your hardware:
orRSS

detecting your OS…

$ terminal

detecting your OS…

Run curl "whatmodel.sh/?ram=4&gpu=apple&max=0"to get this list in your terminal (add &format=json for agents).

Score vs release date

#modelscore↓where↓RAM↓speed↓context↓get itsources
#modelscore↓where↓RAM↓speed↓context↓get itsources
192MiniCPM5 2B · 4-bit OpenBMB63±1†local~2.5GB131kllama.cpp › openbmb/MiniCPM5-2B model card
329Spark-X2.5 1.7B · 4-bit iFLYTEK (SparkLLM)55±1†local~1.9GB~166 tok/s1Mllama.cpp › XHToken/Spark-X2.5-4B model card
386LFM2.5 2.6B · 4-bit Liquid AI53±1local~2.4GB131kllama.cpp › openbmb/MiniCPM5-2B model cardLiquidAI/LFM2.5-2.6B model card
450LFM2.5 1.2B Thinking · 8-bit Liquid AI50±2local~2.5GB33kOllama › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
475Qwen2.5 Coder 3B Instruct · 4-bit Alibaba49±1local~2.6GB~9.5 tok/s33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
504Qwen3.5 2B · 4-bit Alibaba47±1local~2.6GB~174 tok/s262kLM Studio › Qwen/Qwen3.5-2B model cardopenbmb/MiniCPM5-2B model cardXHToken/Spark-X2.5-4B model cardCohereLabs/North-Micro-Vision-Instruct model card (benchmark table)EuroEval leaderboards (results collection), English generative leaderboardMathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets+7
523EXAONE 4.0 1.2B · 4-bit LG AI Research47±1†local~2.2GB66kLM Studio › LG AI Research vendor-reported benchmark scores for its own EXAONE models (Hugging Face model cards)
526Qwen2.5-Math 1.5B Instruct · 4-bit Alibaba47±2†local~1.9GB4kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
546Qwen2-Math 1.5B Instruct · 4-bit Alibaba46±2†local~2.6GB4kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)
583LFM2.5 1.2B Instruct · 8-bit Liquid AI45±2local~1.8GB33kLM Studio › LiquidAI/LFM2.5-1.2B-Thinking model cardEuroEval leaderboards (results collection), English generative leaderboard
584Qwen3 1.7B · 4-bit Alibaba45±1local~2.6GB~96 tok/s33kLM Studio › Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient UpcyclingQwen/Qwen3.5-2B model cardLiquidAI/LFM2-1.2B model cardQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)HuggingFaceTB/SmolLM3-3B model card (Evaluation section)LiquidAI/LFM2.5-1.2B-Thinking model card+11
590Qwen2.5 3B Instruct · 4-bit Alibaba44±1local~2.6GB~61 tok/s33kLM Studio › microsoft/Phi-4-mini-instruct model cardWildEval ZeroEval published resultsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)SpeakLeash Polish EQ-Bench leaderboard+5
594LFM2 2.6B · 4-bit Liquid AI44±1local~2.2GB33kllama.cpp › LFM2 Technical Report — Liquid AI own SmolLM3 evaluationEuroEval leaderboards (results collection), English generative leaderboard
604Qwen2.5 Coder 1.5B Instruct · 8-bit Alibaba44±1local~2.5GB33kOllama › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
635Granite 4.0 1B · 4-bit IBM42±1local~2.6GB128kllama.cpp › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model cardEpoch AI — internal evaluations
638Granite 4.0 H 1B · 4-bit IBM42±1local~2.4GB131kllama.cpp › ibm-granite/granite-4.0-h-1b model cardLiquidAI/LFM2.5-1.2B-Thinking model card
650Bielik 1.5B v3.0 Instruct · 4-bit SpeakLeash41±3local~2.6GBllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)EuroEval leaderboards (results collection), English generative leaderboard
657LFM2 1.2B · 8-bit Liquid AI41±1local~1.8GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
697Qwen2.5 Coder 0.5B Instruct · 8-bit Alibaba39±1local~1.2GB33kLM Studio › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Aider code editing benchmark runs (old edit leaderboard)
702Qwen2-VL 2B Instruct · 4-bit Alibaba38±1local~2.6GB33kllama.cpp › Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)
704Qwen2.5 1.5B Instruct · 8-bit Alibaba38±1local~2.5GB~98 tok/s33kLM Studio › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)SpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)+6
707LFM2 700M · 8-bit Liquid AI38±1local~1.4GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
708Qwen3.5 0.8B · 8-bit Alibaba38±1local~2.5GB~319 tok/s262kOllama › Qwen/Qwen3.5-2B model cardEuroEval leaderboards (results collection), English generative leaderboardQwen/Qwen3.5-0.8B model cardChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)Knowledge Index of Noah's Ark (arXiv:2606.05104v2)BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)+2
723DeepSeek R1 Distill Qwen 1.5B · 4-bit DeepSeek37±2local~2.4GB131kOllama › deepseek-ai/DeepSeek-R1 model cardBRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)EuroEval leaderboards (results collection), English generative leaderboardTHINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models (arXiv:2505.22113v1)Epoch AI — internal evaluations+1
731Qwen3 0.6B · 4-bit Alibaba36±1local~2.5GB~191 tok/s33kOllama › Olmo 3LiquidAI/LFM2-1.2B model cardUnsupervised Confidence Calibration for Reasoning LLMs from a Single Generation (arXiv:2604.19444v1)Berkeley Function Calling Leaderboard result archiveOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)+5
742OLMo 2 1B Instruct (0425) · 4-bit Allen Institute for AI35±1local~2.4GB4kllama.cpp › 2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsOpen Portuguese LLM Leaderboard (archived)
750DeepSeek-Coder 1.3B Base · 4-bit DeepSeek34±1local~2.4GB16kllama.cpp › EvalPlus / EvalPerf published evaluation runsTop Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runsStarCoder 2 and The Stack v2: The Next Generation — authors' own runsBigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4CRUXEval published evaluation results (authors' generations and scores)
756Qwen2 1.5B Instruct · 8-bit Alibaba34±1local~2.5GB32kllama.cpp › WildEval ZeroEval published resultsQwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card)Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsSpeakLeash Polish EQ-Bench leaderboardOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)+1
758Falcon-H1 1.5B Instruct · 4-bit TII33±4local~2.4GB131kllama.cpp › Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Open Portuguese LLM Leaderboard (archived)
759Gemma 3 1B · 8-bit Google33±1local~2.4GB~180 tok/s32kLM Studio › Google DeepMind Gemma 3 270M IT model cardLiquidAI/LFM2-1.2B model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsLiquidAI/LFM2.5-1.2B-Thinking model cardGemma Team — Gemma 3 Technical ReportMIMIC Clinical Decision Making leaderboard+7
762LFM2 350M · 8-bit Liquid AI33±2local~1.9GB33kLM Studio › LiquidAI/LFM2-1.2B model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
767Llama 3.2 1B Instruct · 8-bit Meta33±1local~2.2GB~251 tok/s131kllama.cpp › IBM Granite 3.0 Language Models technical report (GitHub)LiquidAI/LFM2-1.2B model card2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct modelsSmolLM2 paper (Hugging Face) — authors' own instruct-model evaluationsLiquidAI/LFM2.5-1.2B-Thinking model cardLMArena leaderboard dataset+13
768Phi-1.5 (1.3B) · 4-bit Microsoft33±1local~2.4GB2kllama.cpp › Stability AI — original StableLM zero-shot evaluation outputsTextbooks Are All You Need II: phi-1.5 technical report — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
777Granite 3.1 1B-A400M Instruct · 4-bit IBM31±2†local~2.3GB~20 tok/s131kllama.cpp › ibm-granite/granite-3.1-3b-a800m-instruct model card
788Qwen2.5 0.5B Instruct · 8-bit Alibaba30±2local~1.8GB33kOllama › HuggingFaceTB/SmolLM2-360M-Instruct model cardSpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)Berkeley Function Calling Leaderboard result archiveOpen Portuguese LLM Leaderboard (archived)+1
793Granite 3.0 1B-A400M Instruct · 4-bit IBM30±1local~2.4GB4kOllama › IBM Granite 3.0 Language Models technical report (GitHub)EuroEval leaderboards (results collection), English generative leaderboard
799EuroLLM 1.7B Instruct · 4-bit UTTER (EuroLLM)29±3local~2.6GB4kllama.cpp › SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
800Granite 4.0 H 350M · 8-bit IBM29±1†local~1.9GB33kLM Studio › ibm-granite/granite-4.0-h-1b model card
804Granite 4.0 350M · 8-bit IBM28±1local~1.9GB32kllama.cpp › ibm-granite/granite-4.0-h-1b model cardBerkeley Function Calling Leaderboard result archiveEpoch AI — internal evaluations
808Qwen2 0.5B Instruct · 8-bit Alibaba28±2local~1.8GB32kOllama › Qwen2-0.5B-Instruct model cardLiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsOpen Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)Open Portuguese LLM Leaderboard (archived)
814Falcon3 1B Instruct · 4-bit TII26±4local~2.7GB8kllama.cpp › Berkeley Function Calling Leaderboard result archiveCodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)Open Portuguese LLM Leaderboard (archived)
815Qwen1.5 1.8B Chat · 4-bit Alibaba26±2local~2.7GB33kOllama › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsAlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)Open Portuguese LLM Leaderboard (archived)EuroEval leaderboards (results collection), English generative leaderboard
821Flan-T5 Base · 4-bit Google25±1local~2.5GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsModel-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsAugmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In — authors' own runsSpecializing Smaller Language Models towards Multi-Step Reasoning — authors' own runs+3
831Qwen1.5 0.5B Chat · 4-bit Alibaba23±3local~1.9GB33kllama.cpp › LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasetsEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
835Phi-1 (1.3B) · 4-bit Microsoft22±4local~2.4GB2kllama.cpp › CRUXEval published evaluation results (authors' generations and scores)Open Portuguese LLM Leaderboard (archived)
837SmolLM2 360M Instruct · 4-bit Hugging Face22±3local~1.5GB8kOllama › HuggingFaceTB/SmolLM2-360M-Instruct model cardEuroEval leaderboards (results collection), English generative leaderboard
839OLMo 1B · 4-bit Allen Institute for AI21±2†local~2.2GB2kllama.cpp › OLMo: Accelerating the Science of Language Models — authors' own runs
849Gemma 3 270M IT · 8-bit Google17±4local~1.8GB32kOllama › Google DeepMind Gemma 3 270M IT model cardEuroEval leaderboards (results collection), English generative leaderboardOpen Portuguese LLM Leaderboard (archived)
850Flan-T5 Small · 4-bit Google16±3local~1.8GBhow to run › Scaling Instruction-Finetuned Language Models — authors' own runsZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runsLarge Language Models Are Reasoning Teachers — authors' own runsPOSQA: Probe the World Models of LLMs with Size Comparisons — authors' own runsAn Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs
855T5 Base · 4-bit Google14±3local~2.4GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
856Pythia 1.4B · 4-bit EleutherAI14±1local~2.4GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)
858Pythia 1B · 8-bit EleutherAI12±1local~2.7GB2kllama.cpp › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOLMo: Accelerating the Science of Language Models — authors' own runsOpen Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)Open Portuguese LLM Leaderboard (archived)
861T5 Small · 4-bit Google12±3local~1.8GB1khow to run › Large Language Models Are Reasoning Teachers — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluations
863Zyte-1B · 4-bit venkycs (aihub-app)11±3local~2.2GBllama.cpp › EvalPlus / EvalPerf published evaluation runs
866Pythia 410M · 4-bit EleutherAI8±2local~2.5GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)
868BLOOM 1.1B · 4-bit BigScience7±1local~2.6GB2kllama.cpp › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsBLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Crosslingual Generalization through Multitask Finetuning — authors' own runs
872OPT-350M · 4-bit Meta1±1local~2.2GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsRevisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? — authors' own runsDo Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models — authors' own runs+1
875BLOOM 560M · 4-bit BigScience0±1local~2.7GB2khow to run › EleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsGPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)BLOOM: A 176B-Parameter Open-Access Multilingual Language Model — authors' own runsCrosslingual Generalization through Multitask Finetuning — authors' own runs+1
876Pythia 160M · 4-bit EleutherAI-2±2local~1.9GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runstrlX: A Framework for Large Scale Reinforcement Learning from Human Feedback — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsOpen Portuguese LLM Leaderboard (archived)
877GPT-2 Large (774M) · 8-bit OpenAI-3±1local~2.4GB~137 tok/s1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluationsGenerative Calibration for In-context Learning — authors' own runs+3
878OPT-125M · 4-bit Meta-3±1local~1.8GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsEleutherAI lm-evaluation-harness results folder (third-party runs committed to the repo)GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runsPreserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories — authors' own runsCan Language Models Understand Physical Concepts? — authors' own runs+1
879GPT-Neo 125M · 4-bit EleutherAI-3±2local~2.1GB2khow to run › Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models — authors' own runsPrompting Language Models for Linguistic Structure — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluationsHallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results)Open Portuguese LLM Leaderboard (archived)TruthfulQA: Measuring How Models Mimic Human Falsehoods — authors' own runs
880GPT-2 Medium (355M) · 8-bit OpenAI-5±1local~1.9GB1kllama.cpp › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsRankGen: Improving Text Generation with Large Ranking Models — authors' own runsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsCOPEN — EMNLP 2022 original zero-shot evaluations+3
881Pythia 70M · 4-bit EleutherAI-13±2local~1.7GB2khow to run › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runsOpen Portuguese LLM Leaderboard (archived)
882GPT-2 Small (124M) · 4-bit OpenAI-13±2local~2.1GB1khow to run › Coherence boosting: When your pretrained language model is not paying enough attention — authors' own runsSurface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runsCASE: Commonsense-Augmented Score with an Expanded Answer Space — authors' own runsCOPEN — EMNLP 2022 original zero-shot evaluationsPROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runsLarge Language Models Are Reasoning Teachers — authors' own runs+3

+ 848 more don't fit your machine or budget · show them

+ 15 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

913 of 1219 catalogue models listed here · sources and licences

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory

65 models · 575 benchmarks · updated