whatmodel.sh · find the best AI models you can run

Best AI models to run locally on a 96GB PC without a graphics card

A leaderboard of the best AI models you can run locally on a computer with 96GB of RAM and no graphics card. Qwen3.8-Flash-Next is #1 right now, followed by Qwen3.8 27B and Qwen3.5 397B-A17B. Models are ranked by one score that combines 566 published benchmarks. Each model shows the download that fits and the command to run it.

529 models · scores combine 566 benchmarks · updated 2026-10-14 08:02 UTChow it works · find a leaderboard · benchmark your machine

task[all tasks][agent tasks][coding][chat & writing][math][vision & documents][web search]

Models for 96GB RAM · no GPU → ~74GB for models (CPU, slow) · local only · reset

+ more

set[all sets][LMArena (text) · 1182][ARC Challenge · 8][ANLI Round 1 (prompted) · 8][LAMBADA (OpenAI version), last-word accuracy · 7][ARC Challenge · 3][ARC Challenge · 2]

Filter the model list while keeping overall scores. Models without a shared score scale are not yet calibrated; select a set to see its own scores.

Detect your exact chip or graphics card:

curl -G whatmodel.sh/ --data-urlencode "mac=$(sysctl -n hw.memsize machdep.cpu.brand_string)"Mac

curl -G whatmodel.sh/ --data-urlencode "ram=$(awk '/MemT/{print $2}' /proc/meminfo)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null)"Linux

curl.exe -G whatmodel.sh/ --data-urlencode "ram=$((gcim Win32_ComputerSystem).TotalPhysicalMemory)" --data-urlencode "nv=$(nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>$null)"Windows

$ terminal

ram[8][16][24][32][48][64][96][128][192][256]GB

gpu[apple silicon][graphics card][unified memory][none]

price[local][$0.02][$0.1][$0.5][$2][$10][$50][any]max per M

detecting your OS…

Score vs memory to run

     │                                   Spark-X2.5 4B        Qwen3.8 27B ●────∘∘─Qwen3.8-Flash-Next─● ∘
  50 ┤ · ●───────────────●────────●─────●────∘───∘───∘──∘──∘─∘─∘─∘─∘─∘∘─∘∘∘·∘∘∘∘∘∘∘∘∘∘∘∘∘∘∘∘·∘∘∘∘∘∘·∘∘·∘∘∘∘···
     │   ∘               ∘        ∘     ∘    ∘   ∘   ∘  ∘  ∘ ∘ ∘ ∘ ∘ ∘∘ ∘∘     ∘∘∘  ∘∘∘∘∘        ∘  ∘  ∘ ∘
   0 ┤ · ∘ · · · · · · · ∘ · · · ·∘· · ·∘· · ∘ · · · ∘ · · · · · · ∘ · · ∘ · · · · · · · ∘ ··· · · · ··· · ···
     │                   ∘
     └───────────────────────────────────────────────────────────────────────┬──────────────┬─────────┬─────┬─
                                                                           20GB           40GB      60GB  80GB

#	model	score↓	where↓	RAM↓	speed↓	context↓	get it	sources

#	model	score↓	where↓	RAM↓	speed↓	context↓	get it	sources
40	Qwen3.8-Flash-Next · 2-bit Alibaba	74±1	local (slow)	~59GB	~27 tok/s	262k	llama.cpp ›
61	Qwen3.8 27B · 4-bit Alibaba	72±1	local (slow)	~18GB	~8.6 tok/s	262k	Ollama ›	+3
85	Qwen3.5 397B-A17B · 1-bit Alibaba	70±0	local (slow)	~65GB		262k	llama.cpp ›	+15
87	KAT-Coder-V2.5-Dev (35B-A3B) · 4-bit Kuaishou (Kwaipilot)	70±2†	local (slow)	~22GB		262k	llama.cpp ›
88	Ornith 1.5 35B-A3B · 4-bit DeepReinforce	70±1†	local (slow)	~23GB	~80 tok/s	262k	llama.cpp ›
98	Muse Glimmer 30B · 4-bit Meta	69±1	local (slow)	~18GB	~27 tok/s	131k	llama.cpp ›
102	Qwen3.6 27B · 4-bit Alibaba	69±1	local (slow)	~19GB	~29 tok/s	262k	Ollama ›	+4
113	GLM-4.7 · 1-bit Z.ai	68±1	local (slow)	~58GB		205k	llama.cpp ›	+5
115	Qwen3.5 27B · 4-bit Alibaba	68±0	local (slow)	~18GB	~29 tok/s	262k	llama.cpp ›	+8
122	Qwen3.6 35B-A3B · 4-bit Alibaba	67±1	local (slow)	~21GB	~63 tok/s	262k	llama.cpp ›	+4
124	Qwen3-Coder-Next (80B-A3B) · 4-bit Alibaba	67±2	local (slow)	~49GB	~62 tok/s	262k	Ollama ›
127	Ornith 1.0 35B (35B-A3B) · 4-bit DeepReinforce	67±1	local (slow)	~24GB	~103 tok/s	262k	Ollama ›
131	Gemma 4 31B · 4-bit Google	67±1	local (slow)	~24GB	~11 tok/s	262k	llama.cpp ›	+20
144	Qwen3.5 35B-A3B · 4-bit Alibaba	66±0	local (slow)	~22GB	~15 tok/s	262k	llama.cpp ›	+12
150	Spark-X2.5 4B · 4-bit iFLYTEK (SparkLLM)	66±1†	local (slow)	~3.8GB	~279 tok/s	1M	llama.cpp ›
153	GLM-4.6 · 1-bit Z.ai	66±1	local (slow)	~58GB		205k	llama.cpp ›	+1
162	Nemotron Cascade 2 30B-A3B · 4-bit NVIDIA	65±1	local (slow)	~22GB	~100 tok/s	256k	Ollama ›
164	MiMo-V2.6-Distill-Qwen-9B · 4-bit Xiaomi	65±2†	local (slow)	~8.6GB			llama.cpp ›
165	Laguna XS 2.1 (33B-A3B) · 4-bit Poolside	65±2†	local (slow)	~22GB		262k	Ollama ›
167	Qwen3 235B-A22B Instruct 2507 · 2-bit Alibaba	65±1	local (slow)	~73GB	~15 tok/s	262k	llama.cpp ›	+3
168	INTELLECT-3 · 2-bit Prime Intellect	65±1	local (slow)	~48GB			llama.cpp ›
169	GLM-4.5 · 1-bit Z.ai	65±1	local (slow)	~58GB		131k	llama.cpp ›	+3
170	EXAONE 4.5 33B · 4-bit LG AI Research	64±2†	local (slow)	~24GB		262k	llama.cpp ›
175	Mistral Medium 3.5 (128B) · 2-bit Mistral	64±1	local (slow)	~43GB		262k	llama.cpp ›
176	Laguna XS.2 · 4-bit Poolside	64±1	local (slow)	~23GB		262k	llama.cpp ›
179	Command A+ (05-2026) · 2-bit Cohere	64±1	local (slow)	~71GB		128k	llama.cpp ›
181	Gemma 4 26B-A4B · 4-bit QAT Google	64±1	local (slow)	~18GB	~35 tok/s	262k	Ollama ›	+12
184	Nemotron-Labs-3 Puzzle 75B-A9B · 4-bit NVIDIA	64±1†	local (slow)	~56GB			llama.cpp ›
188	MiniCPM5 2B · 4-bit OpenBMB	64±1†	local (slow)	~2.5GB		131k	llama.cpp ›
192	Qwen3-Next 80B-A3B Instruct · 4-bit Alibaba	63±1	local (slow)	~50GB	~4.7 tok/s	131k	Ollama ›
194	Qwen3-Next 80B-A3B Thinking · 4-bit Alibaba	63±1	local (slow)	~49GB		131k	Ollama ›	+3
195	Granite 4.2 30B · 4-bit IBM	63±1	local (slow)	~20GB		131k	llama.cpp ›
199	GLM-4.5-Air · 4-bit Z.ai	63±1	local (slow)	~66GB	~23 tok/s	131k	llama.cpp ›	+3
200	Gemma 4 12B · 4-bit Google	63±1	local (slow)	~9.9GB	~6.5 tok/s	262k	llama.cpp ›
202	Qwen3.5 9B · 4-bit Alibaba	62±0	local (slow)	~6.7GB	~15 tok/s	262k	llama.cpp ›	+17
203	Seed-OSS 36B Instruct · 4-bit ByteDance Seed	62±1	local (slow)	~25GB		524k	llama.cpp ›
206	GLM-4.6V · 4-bit Z.ai	62±2	local (slow)	~69GB		128k	llama.cpp ›
209	Nemotron 3.5 Lightning 30B-A3B · 4-bit NVIDIA	62±2	local (slow)	~22GB	~87 tok/s	262k	Ollama ›
211	Qwen3-VL 32B Thinking · 4-bit Alibaba	62±1	local (slow)	~22GB		256k	llama.cpp ›
213	gpt-oss-120b · 4-bit OpenAI	62±1	local (slow)	~64GB	~44 tok/s	131k	llama.cpp ›	+19
216	Apriel 1.6 15B Thinker · 4-bit ServiceNow	62±1†	local (slow)	~11GB			llama.cpp ›
221	GLM-4.5V · 4-bit Z.ai	61±2	local (slow)	~69GB		64k	llama.cpp ›
228	Qwen3 30B-A3B Thinking 2507 · 4-bit Alibaba	61±1	local (slow)	~20GB		262k	llama.cpp ›	+8
230	Qwen3.5 4B · 4-bit Alibaba	61±1	local (slow)	~3.8GB		262k	llama.cpp ›	+10
231	Qwen3 30B-A3B Instruct 2507 · 4-bit Alibaba	61±1	local (slow)	~20GB	~100 tok/s	262k	llama.cpp ›	+4
234	Mellum2.1 12B-A2.5B Thinking · 4-bit JetBrains	60±1†	local (slow)	~10GB			llama.cpp ›
236	Nemotron-Cascade-14B-Thinking · 4-bit NVIDIA	60±1†	local (slow)	~11GB			llama.cpp ›
237	Qwen3-VL 32B Instruct · 4-bit Alibaba	60±1†	local (slow)	~23GB		256k	llama.cpp ›
238	Devstral Small 2 24B · 4-bit Mistral	60±3†	local (slow)	~16GB		262k	llama.cpp ›
239	Gemma 2 9B IT SimPO · 4-bit Princeton NLP	60±1	local (slow)	~7.6GB			llama.cpp ›
240	Gemma 2 9B IT DPO (Princeton) · 4-bit Princeton NLP	60±1	local (slow)	~7.6GB			llama.cpp ›
241	Falcon-H1R 7B · 4-bit TII	60±1†	local (slow)	~6GB		262k	llama.cpp ›
242	Qwen3 235B-A22B (Apr 2025, hybrid) · 2-bit Alibaba	60±0	local (slow)	~73GB		131k	llama.cpp ›	+14
244	Mistral Small 4 (119B-A6.5B) · 4-bit Mistral	60±2	local (slow)	~73GB		262k	llama.cpp ›
246	Qwen3-VL 30B-A3B Thinking · 4-bit Alibaba	60±1†	local (slow)	~20GB		256k	llama.cpp ›
247	Granite 4.2 8B · 4-bit IBM	60±1	local (slow)	~7.2GB		131k	llama.cpp ›
248	QwQ-32B · 4-bit Alibaba	60±1	local (slow)	~23GB		131k	llama.cpp ›	+7
249	Solar Open 100B (102B-A12B) · 4-bit Upstage	60±1†	local (slow)	~67GB		131k	llama.cpp ›
250	OpenReasoning-Nemotron-32B · 4-bit NVIDIA	59±2†	local (slow)	~22GB			llama.cpp ›
252	GLM-4.7-Flash · 4-bit Z.ai	59±1	local (slow)	~19GB	~44 tok/s	200k	llama.cpp ›
253	EXAONE 4.0 32B · 4-bit LG AI Research	59±1	local (slow)	~22GB		131k	llama.cpp ›
254	Phi-4-reasoning-plus 14B · 4-bit Microsoft	59±1	local (slow)	~11GB			llama.cpp ›
256	Qwen3 14B · 4-bit Alibaba	59±1	local (slow)	~11GB		33k	llama.cpp ›	+6
257	Olmo 3.1 32B Think · 4-bit Ai2	59±1	local (slow)	~22GB		66k	llama.cpp ›
258	Nemotron-Cascade-8B · 4-bit NVIDIA	59±1†	local (slow)	~6.8GB			llama.cpp ›
260	gpt-oss-20b · 4-bit OpenAI	58±1	local (slow)	~13GB	~8.7 tok/s	131k	llama.cpp ›	+15
261	Llama 3.3 Nemotron Super 49B v1 · 4-bit NVIDIA	58±2	local (slow)	~33GB			llama.cpp ›
262	DeepSeek R1 Distill Llama 70B · 4-bit DeepSeek	58±1	local (slow)	~46GB		131k	Ollama ›	+1
264	Command A (03-2025) · 4-bit Cohere	58±1	local (slow)	~72GB		256k	Ollama ›
265	Olmo 3 32B Think · 4-bit Ai2	58±1	local (slow)	~22GB		66k	llama.cpp ›
266	Phi-4-reasoning 14B · 4-bit Microsoft	58±1	local (slow)	~11GB		33k	llama.cpp ›
268	Qwen3-VL 30B-A3B Instruct · 4-bit Alibaba	58±1†	local (slow)	~20GB		256k	llama.cpp ›
269	Qwen3-VL 8B Thinking · 4-bit Alibaba	58±1	local (slow)	~6.9GB		256k	llama.cpp ›
271	Magistral Small 1.2 · 4-bit Mistral	58±1	local (slow)	~17GB		131k	llama.cpp ›
274	Qwen3 32B · 4-bit Alibaba	58±0	local (slow)	~23GB		131k	llama.cpp ›	+17
275	Nemotron-Cascade-8B-Thinking · 4-bit NVIDIA	58±2†	local (slow)	~6.8GB			llama.cpp ›
276	Qwen3 Coder 30B-A3B Instruct · 4-bit Alibaba	58±1	local (slow)	~20GB	~93 tok/s	160k	llama.cpp ›
280	Llama 4 Maverick (402B-A17B) · 1-bit Meta	58±1	local (slow)	~65GB		1M	llama.cpp ›	+14
281	Llama 3.3 Nemotron Super 49B v1.5 · 4-bit NVIDIA	58±1	local (slow)	~33GB		131k	llama.cpp ›
282	Qwen2.5 Coder 32B Instruct · 4-bit Alibaba	58±1	local (slow)	~23GB		131k	llama.cpp ›	+5
283	DeepSeek R1 0528 Qwen3 8B · 4-bit DeepSeek	58±1	local (slow)	~6.9GB	~3.6 tok/s	128k	llama.cpp ›
284	Granite 4.2 3B · 4-bit IBM	57±1	local (slow)	~3.4GB		131k	llama.cpp ›
286	Nemotron 3 Nano 30B-A3B · 4-bit NVIDIA	57±1	local (slow)	~22GB	~105 tok/s	262k	Ollama ›	+3
288	Athene-V2-Chat 72B · 4-bit Nexusflow	57±1	local (slow)	~48GB		33k	Ollama ›
290	Granite 4.1 30B · 4-bit IBM	57±1†	local (slow)	~19GB	~25 tok/s		Ollama ›
291	Qwen2.5-Omni 7B · 4-bit Alibaba	57±2	local (slow)	~5.8GB			llama.cpp ›
293	Ministral 3 14B Reasoning 2512 · 4-bit Mistral	57±1	local (slow)	~10GB		262k	llama.cpp ›
295	Sky-T1 32B Preview · 4-bit NovaSky (UC Berkeley)	57±4	local (slow)	~22GB			llama.cpp ›
296	Qwen3 4B Thinking 2507 · 4-bit Alibaba	57±1	local (slow)	~4.4GB	~36 tok/s	262k	llama.cpp ›	+2
298	QED-Nano · 4-bit LM Provers	57±2	local (slow)	~4.1GB		262k	llama.cpp ›
299	Mistral Small 3.2 24B · 4-bit Mistral	57±1	local (slow)	~16GB		128k	llama.cpp ›	+4
301	Qwen3-VL 4B Thinking · 4-bit Alibaba	56±1	local (slow)	~4.4GB		256k	llama.cpp ›
302	Qwen2.5-VL 72B Instruct · 4-bit Alibaba	56±1	local (slow)	~48GB		33k	Ollama ›	+3
305	Qwen2.5 72B Instruct · 4-bit Alibaba	56±1	local (slow)	~47GB		33k	Ollama ›	+24
308	Qwen3 8B · 4-bit Alibaba	56±1	local (slow)	~6.9GB	~25 tok/s	33k	llama.cpp ›	+14
309	DeepSeek R1 Distill Qwen 32B · 4-bit DeepSeek	56±1	local (slow)	~23GB	~7.3 tok/s	131k	llama.cpp ›	+5
310	Qwen2.5 Coder 14B Instruct · 4-bit Alibaba	56±1	local (slow)	~11GB	~18 tok/s	131k	llama.cpp ›
311	DeepSeek R1 Distill Qwen 14B · 4-bit DeepSeek	56±1	local (slow)	~11GB	~15 tok/s	131k	llama.cpp ›	+2
312	Ministral 3 14B · 4-bit Mistral	56±1	local (slow)	~10GB	~46 tok/s	262k	llama.cpp ›	+1
313	Qwen2.5 32B Instruct · 4-bit Alibaba	56±1	local (slow)	~23GB		131k	llama.cpp ›	+11
314	Nemotron Nano 9B v2 · 4-bit NVIDIA	56±1	local (slow)	~7.6GB		131k	llama.cpp ›
315	Molmo 2 8B · 4-bit Ai2	56±2	local (slow)	~6.8GB			llama.cpp ›
317	EXAONE Deep 32B · 4-bit LG AI Research	56±2†	local (slow)	~22GB		33k	Ollama ›
318	Qwen3 4B Instruct 2507 · 4-bit Alibaba	56±1	local (slow)	~4.4GB	~60 tok/s	262k	llama.cpp ›	+4
319	Qwen3 30B-A3B (Apr 2025, hybrid) · 4-bit Alibaba	56±0	local (slow)	~20GB		33k	llama.cpp ›	+10
320	Magistral Small 1.0 (2506) · 4-bit Mistral	56±1	local (slow)	~17GB		128k	llama.cpp ›
321	Qwen3-VL 8B Instruct · 4-bit Alibaba	56±1	local (slow)	~6.9GB		256k	llama.cpp ›
322	Bielik 11B v3.0 Instruct · 4-bit SpeakLeash	56±2	local (slow)	~8.5GB			llama.cpp ›
323	Spark-X2.5 1.7B · 4-bit iFLYTEK (SparkLLM)	56±1†	local (slow)	~1.9GB	~314 tok/s	1M	llama.cpp ›
325	Granite 4.1 8B · 4-bit IBM	55±1	local (slow)	~7.2GB	~70 tok/s	131k	Ollama ›
326	Llama 3.3 70B Instruct · 4-bit Meta	55±1	local (slow)	~46GB	~2.3 tok/s	128k	Ollama ›	+21
328	Mistral Large 2 (2407) · 2-bit Mistral	55±1	local (slow)	~45GB		128k	llama.cpp ›	+10
329	Falcon-H1 34B Instruct · 4-bit TII	55±3	local (slow)	~23GB		262k	llama.cpp ›
330	Qwen2.5-Math 72B Instruct · 4-bit Alibaba	55±2†	local (slow)	~51GB		4k	llama.cpp ›
331	Llama 4 Scout (109B-A17B) · 4-bit Meta	55±1	local (slow)	~68GB	~19 tok/s	328k	Ollama ›	+8
332	Bielik 11B v2.6 Instruct · 4-bit SpeakLeash	55±3	local (slow)	~8.5GB			llama.cpp ›
335	Mistral Small 3.1 24B (2503) · 4-bit Mistral	55±1	local (slow)	~16GB		128k	llama.cpp ›	+7
336	Llama 3.1 Tulu 3 70B · 4-bit Ai2	55±1	local (slow)	~46GB		131k	llama.cpp ›
337	Pixtral Large (2411) · 2-bit Mistral	55±3	local (slow)	~51GB			llama.cpp ›
338	Olmo 3.1 32B Instruct · 4-bit Ai2	55±2	local (slow)	~22GB		66k	llama.cpp ›
339	OpenThinker2 32B · 4-bit Open Thoughts	55±1	local (slow)	~22GB		33k	llama.cpp ›
342	OpenThinker 32B · 4-bit Open Thoughts	55±1†	local (slow)	~22GB		33k	llama.cpp ›
344	DeepSeek-V2 Chat (0628) · 2-bit DeepSeek	55±1	local (slow)	~66GB			llama.cpp ›
345	Command A Reasoning (08-2025) · 4-bit Cohere	55±2	local (slow)	~72GB			llama.cpp ›
346	Magistral Small 1.1 (2507) · 4-bit Mistral	55±1†	local (slow)	~17GB		40k	llama.cpp ›
347	Mellum2 12B-A2.5B Thinking · 4-bit JetBrains	55±1†	local (slow)	~10GB		131k	llama.cpp ›
348	Ministral 3 8B Reasoning 2512 · 4-bit Mistral	55±1	local (slow)	~6.9GB		262k	llama.cpp ›
349	Nemotron 3 Nano 4B · 4-bit NVIDIA	54±1	local (slow)	~3.5GB	~104 tok/s	262k	llama.cpp ›
350	Llama 3.1 Tulu 3 70B DPO · 4-bit Ai2	54±1	local (slow)	~46GB		131k	llama.cpp ›
356	EXAONE Deep 7.8B · 4-bit LG AI Research	54±2†	local (slow)	~6.5GB		33k	llama.cpp ›
357	Mistral Small 3 24B (2501) · 4-bit Mistral	54±1	local (slow)	~16GB		32k	llama.cpp ›	+6
360	GLM-4 9B Chat 1M · 4-bit Zhipu AI (THUDM)	54±4	local (slow)	~8.1GB		1M	llama.cpp ›
363	Qwen2-VL 72B Instruct · 4-bit Alibaba	54±1	local (slow)	~53GB			llama.cpp ›
364	Qwen2.5-VL 32B Instruct · 4-bit Alibaba	54±2	local (slow)	~23GB		33k	llama.cpp ›	+1
366	Llama 3.1 70B Instruct · 4-bit Meta	54±0	local (slow)	~46GB		128k	llama.cpp ›	+22
367	Gemma 3 27B · 4-bit Google	54±1	local (slow)	~20GB		131k	llama.cpp ›	+23
370	Phi-4 14B · 4-bit Microsoft	53±1	local (slow)	~11GB		16k	llama.cpp ›	+11
372	Bielik 11B v2.1 Instruct · 4-bit SpeakLeash	53±3	local (slow)	~8.5GB			llama.cpp ›
373	Ministral 3 8B · 4-bit Mistral	53±1	local (slow)	~6.9GB	~68 tok/s	262k	llama.cpp ›	+2
374	Olmo 3 7B Think · 4-bit Ai2	53±1	local (slow)	~6.2GB		66k	llama.cpp ›
375	s1.1 32B · 4-bit Simple Scaling	53±1	local (slow)	~22GB		33k	llama.cpp ›
376	Qwen3-VL 4B Instruct · 4-bit Alibaba	53±1	local (slow)	~4.4GB		256k	llama.cpp ›
377	Gemma 4 E4B · 4-bit Google	53±1	local (slow)	~6.6GB	~59 tok/s	131k	llama.cpp ›	+7
378	Qwen2.5-Math 7B Instruct · 4-bit Alibaba	53±2†	local (slow)	~5.8GB		4k	llama.cpp ›
379	Llama 3.1 Nemotron 70B Instruct · 4-bit NVIDIA	53±1	local (slow)	~46GB		128k	llama.cpp ›	+4
380	LFM2.5 2.6B · 4-bit Liquid AI	53±1	local (slow)	~2.4GB		131k	llama.cpp ›
382	Bielik 11B v2.2 Instruct · 4-bit SpeakLeash	53±3	local (slow)	~8.5GB			llama.cpp ›
386	Granite 4.0 H Small (32B-A9B) · 4-bit IBM	53±1	local (slow)	~22GB		131k	llama.cpp ›	+1
387	LFM2.5 8B-A1B · 4-bit Liquid AI	53±1	local (slow)	~5.9GB	~250 tok/s	128k	llama.cpp ›
388	Llama 3.1 Nemotron 51B Instruct · 4-bit NVIDIA	53±2	local (slow)	~34GB			llama.cpp ›
389	Aya Expanse 32B · 4-bit Cohere	53±2	local (slow)	~22GB		128k	llama.cpp ›	+1
390	Qwen2.5 Coder 7B Instruct · 4-bit Alibaba	53±1	local (slow)	~5.8GB	~48 tok/s	131k	llama.cpp ›
391	OpenCoder-8B-Instruct · 4-bit INF (infly)	53±1	local (slow)	~6.4GB			llama.cpp ›
392	Gemma 3 12B · 4-bit Google	53±1	local (slow)	~10GB	~13 tok/s	131k	llama.cpp ›	+10
393	Qwen2.5 14B Instruct · 4-bit Alibaba	53±1	local (slow)	~11GB		131k	llama.cpp ›	+7
395	Cogito v1 Preview Llama 8B · 4-bit Deep Cogito	52±4	local (slow)	~6.6GB		128k	llama.cpp ›
396	Llama-3-Instruct-8B-SimPO · 4-bit Princeton NLP	52±1	local (slow)	~6.6GB			llama.cpp ›
397	Llama-3-Instruct-8B-SimPO-ExPO · 4-bit Chujie Zheng (ExPO)	52±1	local (slow)	~6.6GB			llama.cpp ›
398	Athene-70B · 4-bit Nexusflow	52±1	local (slow)	~46GB			llama.cpp ›
400	Qwen2 72B Instruct · 4-bit Alibaba	52±1	local (slow)	~48GB		131k	Ollama ›	+14
402	CodeQwen1.5 7B Chat · 4-bit Alibaba	52±1	local (slow)	~6.4GB		64k	llama.cpp ›
403	OpenCodeInterpreter-DS-33B · 4-bit M-A-P	52±1	local (slow)	~22GB			llama.cpp ›
404	LFM2 24B-A2B · 4-bit Liquid AI	52±1	local (slow)	~15GB		33k	llama.cpp ›
406	Qwen3 4B · 4-bit Alibaba	52±1	local (slow)	~4.4GB		33k	llama.cpp ›	+8
407	Bielik 11B v2.0 Instruct · 4-bit SpeakLeash	52±3	local (slow)	~8.5GB			llama.cpp ›
409	DeepSeek-Coder 33B Instruct · 4-bit DeepSeek	51±1	local (slow)	~22GB		16k	llama.cpp ›	+1
410	Codestral 22B v0.1 · 4-bit Mistral AI	51±1	local (slow)	~16GB		32k	llama.cpp ›
412	Gemma 2 27B IT · 4-bit Google	51±1	local (slow)	~21GB		8k	llama.cpp ›	+10
413	Llama 3 70B Instruct · 4-bit Meta	51±0	local (slow)	~46GB		8k	llama.cpp ›	+17
414	Llama 3.2 90B Vision Instruct · 4-bit Meta	51±1	local (slow)	~58GB		128k	Ollama ›	+1
415	Bielik 11B v2.3 Instruct · 4-bit SpeakLeash	51±2	local (slow)	~8.6GB			llama.cpp ›
416	WizardCoder 33B V1.1 · 4-bit WizardLM	51±1	local (slow)	~22GB		16k	llama.cpp ›
418	Llama-3-8B-ProLong-512k-Instruct · 4-bit Princeton NLP	51±4	local (slow)	~35GB			how to run ›
420	Mellum2 12B-A2.5B Instruct · 4-bit JetBrains	51±2†	local (slow)	~9GB		131k	llama.cpp ›
422	Mistral Small Instruct 22B (2409) · 4-bit Mistral AI	51±2	local (slow)	~16GB		32k	llama.cpp ›
423	OpenThinker3 7B · 4-bit Open Thoughts	51±1	local (slow)	~6.4GB		33k	llama.cpp ›
424	Granite 4.1 3B · 4-bit IBM	51±1†	local (slow)	~3.7GB	~131 tok/s		Ollama ›
425	OpenCodeInterpreter-DS-6.7B · 4-bit M-A-P	51±1	local (slow)	~5.8GB			llama.cpp ›
426	Qwen2-Math 72B Instruct · 4-bit Alibaba	51±1	local (slow)	~45GB		4k	Ollama ›
428	AceReason-Nemotron-1.0-14B · 4-bit NVIDIA	51±2†	local (slow)	~11GB			llama.cpp ›
429	Artigenz-Coder-DS-6.7B · 4-bit Artigenz	51±1	local (slow)	~5.8GB			llama.cpp ›
430	Yi 34B 200K (base) · 4-bit 01.AI	51±4	local (slow)	~23GB		200k	llama.cpp ›
431	Hermes 3 Llama 3.1 70B · 4-bit Nous Research	51±1	local (slow)	~43GB		131k	Ollama ›
432	OLMo 2 32B Instruct (0325) · 4-bit Ai2	50±1	local (slow)	~21GB			llama.cpp ›
433	speechless-codellama-34b-v2.0 · 4-bit uukuguy	50±1	local (slow)	~23GB			llama.cpp ›
434	Ministral 3 3B Reasoning 2512 · 4-bit Mistral	50±1	local (slow)	~3.6GB		262k	llama.cpp ›
435	Aya 23 35B · 4-bit Cohere For AI	50±2	local (slow)	~24GB		8k	Ollama ›
436	Qwen2.5-VL 7B Instruct · 4-bit Alibaba	50±1	local (slow)	~5.8GB			llama.cpp ›
438	Mixtral 8x22B Instruct v0.1 · 2-bit Mistral	50±1	local (slow)	~56GB		64k	llama.cpp ›	+9
439	EXAONE Deep 2.4B · 4-bit LG AI Research	50±2†	local (slow)	~3.2GB		33k	Ollama ›
440	Magicoder-S-DS 6.7B · 4-bit UIUC (ise-uiuc)	50±1	local (slow)	~5.8GB			llama.cpp ›
441	Pixtral 12B (2409) · 4-bit Mistral	50±2	local (slow)	~9.6GB			llama.cpp ›
443	Qwen2.5 7B Instruct · 4-bit Alibaba	50±1	local (slow)	~5.8GB	~49 tok/s	131k	llama.cpp ›	+12
444	Yi-Coder 9B Chat · 4-bit 01.AI	50±1	local (slow)	~6.6GB		128k	llama.cpp ›
445	InternLM2.5 7B Chat · 4-bit Shanghai AI Laboratory (InternLM)	50±2	local (slow)	~6.4GB			llama.cpp ›
446	XwinCoder-34B · 4-bit Xwin-LM	50±1	local (slow)	~23GB			llama.cpp ›
447	Code Millenials 34B · 4-bit Bud Ecosystem	50±1	local (slow)	~23GB			llama.cpp ›
448	LFM2.5 1.2B Thinking · 4-bit Liquid AI	50±2	local (slow)	~1.3GB		33k	llama.cpp ›
449	StarChat2 15B v0.1 · 4-bit Hugging Face	50±1	local (slow)	~12GB		16k	llama.cpp ›
450	WaveCoder-Ultra-6.7B · 4-bit Microsoft	50±1	local (slow)	~5.8GB			llama.cpp ›
452	Qwen2-Math 7B Instruct · 4-bit Alibaba	50±2†	local (slow)	~6.1GB		4k	Ollama ›
455	Qwen3-VL 2B Thinking · 4-bit Alibaba	50±1	local (slow)	~2.7GB		256k	llama.cpp ›
456	Phi-3.5-MoE-instruct · 4-bit Microsoft	49±1	local (slow)	~27GB		128k	llama.cpp ›
457	MegaBeam-Mistral-7B-512k · 4-bit AWS (aws-prototyping)	49±4	local (slow)	~5.9GB			llama.cpp ›
458	Gemma 3n E4B · 4-bit Google	49±1	local (slow)	~5.6GB		32k	llama.cpp ›
460	OpenThinker2 7B · 4-bit Open Thoughts	49±2†	local (slow)	~6.4GB		33k	llama.cpp ›
461	DeepSeek-Coder 7B Instruct v1.5 · 4-bit DeepSeek	49±1	local (slow)	~5.9GB			llama.cpp ›
462	Hermes 2 Theta Llama-3 70B · 4-bit Nous Research	49±2	local (slow)	~47GB		8k	llama.cpp ›
463	DeepSeek-Coder 6.7B Instruct · 4-bit DeepSeek	49±1	local (slow)	~8.8GB		16k	llama.cpp ›
464	Qwen2 57B-A14B Instruct · 4-bit Alibaba	49±2	local (slow)	~38GB		33k	llama.cpp ›
465	Granite 4.0 H Tiny (7B-A1B) · 4-bit IBM	49±1	local (slow)	~4.8GB	~12 tok/s	131k	Ollama ›
467	MiniCPM3-4B · 4-bit OpenBMB	49±2	local (slow)	~4.1GB		32k	llama.cpp ›
468	Bielik 4.5B v3.0 Instruct · 4-bit SpeakLeash	49±2	local (slow)	~4.5GB			llama.cpp ›
469	Qwen2.5 Coder 3B Instruct · 4-bit Alibaba	49±1	local (slow)	~2.8GB	~9.5 tok/s	33k	Ollama ›
470	Gemma 2 9B IT · 4-bit Google	49±1	local (slow)	~9.5GB		8k	llama.cpp ›	+14
471	Yi-1.5 34B (base) · 4-bit 01.AI	49±2	local (slow)	~23GB		4k	llama.cpp ›
473	Command R Plus (08-2024) · 4-bit Cohere	49±1	local (slow)	~67GB		128k	llama.cpp ›	+1
474	GLM-4 9B Chat · 4-bit Zhipu AI (Z.ai)	49±2	local (slow)	~7.1GB		128k	Ollama ›	+1
475	AceReason-Nemotron-1.1-7B · 4-bit NVIDIA	49±2†	local (slow)	~6.4GB			llama.cpp ›
476	WhiteRabbitNeo-33B-v1 · 4-bit WhiteRabbitNeo	49±1	local (slow)	~22GB			llama.cpp ›
477	Qwen1.5 110B Chat · 4-bit Alibaba	49±1	local (slow)	~72GB		32k	Ollama ›	+1
478	speechless-coder-ds-6.7b · 4-bit uukuguy	48±1	local (slow)	~5.8GB			llama.cpp ›
479	Magicoder-S-CL-7B · 4-bit UIUC (ise-uiuc)	48±1	local (slow)	~5.8GB			llama.cpp ›
480	Phind CodeLlama 34B v2 · 4-bit Phind	48±1	local (slow)	~23GB		16k	Ollama ›
481	DeepSeek R1 Distill Qwen 7B · 4-bit DeepSeek	48±1	local (slow)	~5.8GB	~64 tok/s	131k	llama.cpp ›	+6
482	Jamba 1.5 Mini · 4-bit AI21 Labs	48±1	local (slow)	~34GB			llama.cpp ›
485	Aya Expanse 8B · 4-bit Cohere	48±2	local (slow)	~7GB		8k	llama.cpp ›
488	EuroLLM 9B Instruct · 4-bit UTTER (EuroLLM)	48±2	local (slow)	~7.4GB		4k	llama.cpp ›
491	Granite 4.0 H Micro 3B · 4-bit IBM	48±1	local (slow)	~3.5GB		131k	Ollama ›
493	Code Llama 70B Instruct · 4-bit Meta	48±1	local (slow)	~45GB		16k	llama.cpp ›	+1
494	OLMo 2 13B Instruct (1124) · 4-bit Allen Institute for AI	48±1	local (slow)	~10GB		4k	llama.cpp ›
495	InternLM2.5 7B Chat 1M · 4-bit Shanghai AI Laboratory (InternLM)	48±4	local (slow)	~6.4GB			llama.cpp ›
496	Falcon-H1 7B Instruct · 4-bit TII	48±3	local (slow)	~6.3GB		262k	llama.cpp ›
497	Qwen3.5 2B · 4-bit Alibaba	48±1	local (slow)	~2.2GB	~174 tok/s	262k	llama.cpp ›	+7
501	MedGemma 4B IT · 4-bit Google	48±3	local (slow)	~4.1GB			llama.cpp ›
503	Gemma 4 E2B · 4-bit Google	47±1	local (slow)	~4.5GB		131k	llama.cpp ›	+6
504	OpenChat 3.5 0106 · 4-bit OpenChat	47±1	local (slow)	~6.1GB		8k	llama.cpp ›	+1
505	Command R (08-2024) · 4-bit Cohere	47±1	local (slow)	~22GB		128k	llama.cpp ›	+3
506	Tulu 2 70B · 4-bit Allen Institute for AI	47±2†	local (slow)	~45GB			llama.cpp ›
507	Hermes 2 Pro Llama 3 8B · 4-bit Nous Research	47±3	local (slow)	~6.6GB		8k	llama.cpp ›
508	EXAONE 4.0 1.2B · 4-bit LG AI Research	47±1†	local (slow)	~2.8GB		66k	llama.cpp ›
511	Yi 1.5 34B Chat · 4-bit 01.AI	47±1	local (slow)	~23GB		4k	llama.cpp ›	+5
515	Granite 34B Code Instruct (8K) · 4-bit IBM	47±1	local (slow)	~22GB			Ollama ›
516	WizardCoder Python 34B V1.0 · 4-bit WizardLM	47±1	local (slow)	~22GB		16k	Ollama ›
518	Granite 4.0 Micro 3B · 4-bit IBM	47±1	local (slow)	~3.7GB	~6.2 tok/s	131k	llama.cpp ›
519	Cogito v1 Preview Llama 3B · 4-bit Deep Cogito	47±4	local (slow)	~3.8GB		128k	llama.cpp ›
520	InternLM2.5 20B Chat · 4-bit Shanghai AI Laboratory (InternLM)	47±2	local (slow)	~14GB		33k	llama.cpp ›
521	StarCoder2-15B-Instruct-v0.1 · 4-bit BigCode	47±1	local (slow)	~11GB		16k	Ollama ›
522	Qwen2.5-Math 1.5B Instruct · 4-bit Alibaba	47±2†	local (slow)	~1.9GB		4k	llama.cpp ›
524	Qwen1.5 72B Chat · 4-bit Alibaba	47±1	local (slow)	~48GB		32k	Ollama ›	+9
525	Granite 3.3 8B Instruct · 4-bit IBM	47±1	local (slow)	~6.6GB		128k	llama.cpp ›
526	speechless-starcoder2-15b · 4-bit uukuguy	47±1	local (slow)	~12GB			llama.cpp ›
527	OpenChat 3.5 0106 Gemma · 4-bit OpenChat	47±2	local (slow)	~6.9GB		8k	llama.cpp ›
528	Qwen2-VL 7B Instruct · 4-bit Alibaba	47±1	local (slow)	~6.5GB			llama.cpp ›
529	DeepSeek R1 Distill Llama 8B · 4-bit DeepSeek	47±1	local (slow)	~6.6GB		131k	llama.cpp ›	+1
531	Phi-3 Medium 128K Instruct · 4-bit Microsoft	47±1	local (slow)	~11GB		128k	llama.cpp ›
532	Zephyr ORPO 141B-A35B v0.1 · 2-bit Hugging Face	47±2	local (slow)	~56GB		66k	llama.cpp ›
535	Phi-4-mini-reasoning 3.8B · 4-bit Microsoft	47±2	local (slow)	~4.3GB		128k	llama.cpp ›
536	Phi-3-medium-4k-instruct · 4-bit Microsoft	47±1	local (slow)	~11GB		4k	llama.cpp ›	+1
537	QwQ-32B-Preview · 4-bit Alibaba	46±2	local (slow)	~22GB		33k	llama.cpp ›	+2
538	DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) · 4-bit DeepSeek	46±1	local (slow)	~11GB		128k	Ollama ›	+1
541	Llama 3.1 8B Instruct · 4-bit Meta	46±0	local (slow)	~6.6GB	~89 tok/s	128k	llama.cpp ›	+25
542	Ministral 3 3B · 4-bit Mistral	46±1	local (slow)	~3.6GB	~128 tok/s	262k	llama.cpp ›	+2
543	Olmo 3 7B Instruct · 4-bit Ai2	46±1	local (slow)	~6.2GB		66k	llama.cpp ›
544	Qwen2-Math 1.5B Instruct · 4-bit Alibaba	46±2†	local (slow)	~2.5GB		4k	Ollama ›
545	Command R Plus (04-2024) · 4-bit Cohere	46±1	local (slow)	~67GB		128k	llama.cpp ›	+6
547	Mistral NeMo Instruct (2407) · 4-bit Mistral	46±1	local (slow)	~9.4GB		128k	llama.cpp ›	+9
549	Yi 1.5 9B Chat · 4-bit 01.AI	46±1	local (slow)	~7.1GB		4k	llama.cpp ›	+1
550	Llama 3.1 Tulu 3 8B · 4-bit Ai2	46±1	local (slow)	~6.6GB		131k	llama.cpp ›
551	Phi-3-mini-4k-instruct (June 2024 update) · 4-bit Microsoft	46±1	local (slow)	~6GB			llama.cpp ›
552	Qwen1.5 32B Chat · 4-bit Alibaba	46±1	local (slow)	~22GB		32k	llama.cpp ›
553	Xwin-LM 70B v0.1 · 4-bit Xwin-LM	46±1	local (slow)	~42GB			Ollama ›
554	Eurus-2-7B-PRIME · 4-bit PRIME-RL	46±2	local (slow)	~6.4GB		4k	llama.cpp ›
555	Qwen2 7B Instruct · 4-bit Alibaba	46±1	local (slow)	~5.8GB		131k	llama.cpp ›	+4
556	Gemma 3 4B · 4-bit Google	46±1	local (slow)	~4.1GB	~74 tok/s	131k	llama.cpp ›	+11
558	Granite 20B Code Instruct (8K) · 4-bit IBM	46±1	local (slow)	~14GB			Ollama ›
559	Command R7B (12-2024) · 4-bit Cohere	46±1	local (slow)	~6.9GB		128k	llama.cpp ›
560	Dolphin 2.6 Mixtral 8x7B · 4-bit Cognitive Computations	46±1	local (slow)	~29GB			llama.cpp ›
561	Phi-4-mini 3.8B · 4-bit Microsoft	45±1	local (slow)	~4.3GB		128k	llama.cpp ›	+4
562	Ministral 8B (2410) · 4-bit Mistral	45±1	local (slow)	~6.6GB		128k	llama.cpp ›	+6
563	Granite 3.0 8B Instruct · 4-bit IBM	45±1	local (slow)	~6.6GB		4k	llama.cpp ›
564	Granite 8B Code Instruct (4K) · 4-bit IBM	45±1	local (slow)	~6.6GB			llama.cpp ›
565	Falcon3 7B Instruct · 4-bit TII	45±3	local (slow)	~6.3GB		32k	llama.cpp ›
566	Code Llama 70B Python · 4-bit Meta	45±1	local (slow)	~45GB		16k	Ollama ›
567	Code Tulu 2 34B · 16-bit Allen Institute for AI	45±2†	local (slow)	~72GB			how to run ›
568	OpenThinker 7B · 4-bit Open Thoughts	45±2†	local (slow)	~6.4GB		33k	llama.cpp ›
569	Falcon3 10B Instruct · 4-bit TII	45±3	local (slow)	~8.1GB	~13 tok/s	32k	llama.cpp ›
570	Granite 3.2 8B Instruct · 4-bit IBM	45±1	local (slow)	~6.6GB		131k	llama.cpp ›
571	Yi 34B Chat · 4-bit 01.AI	45±1	local (slow)	~23GB		4k	llama.cpp ›	+1
572	Hermes 2 Theta Llama 3 8B · 4-bit Nous Research	45±2	local (slow)	~6.6GB		8k	llama.cpp ›
573	Phi-3.5-mini-instruct · 4-bit Microsoft	45±1	local (slow)	~6GB		128k	llama.cpp ›	+4
574	Qwen2.5-VL 3B Instruct · 4-bit Alibaba	45±1	local (slow)	~2.8GB			llama.cpp ›
575	Llama 3.1 8B (base) · 4-bit Meta	45±3	local (slow)	~6.6GB			llama.cpp ›
576	Gemma 3n E2B IT · 4-bit Google	45±2	local (slow)	~4.1GB		32k	llama.cpp ›
577	Code Llama 70B (base) · 4-bit Meta	45±1	local (slow)	~45GB		16k	Ollama ›
578	Qwen3-VL 2B Instruct · 4-bit Alibaba	45±1	local (slow)	~2.7GB		256k	llama.cpp ›
581	LFM2.5 1.2B Instruct · 4-bit Liquid AI	45±2	local (slow)	~1.3GB		33k	llama.cpp ›
582	Qwen3 1.7B · 4-bit Alibaba	45±1	local (slow)	~2.9GB	~96 tok/s	33k	llama.cpp ›	+10
583	Granite 3.1 8B Instruct · 4-bit IBM	45±1	local (slow)	~6.7GB		128k	llama.cpp ›
584	Llama 3.2 11B Vision Instruct · 4-bit Meta	45±2	local (slow)	~8.9GB		128k	Ollama ›
588	Qwen2.5 3B Instruct · 4-bit Alibaba	44±1	local (slow)	~3GB	~61 tok/s	33k	llama.cpp ›	+5
589	LFM2 2.6B · 4-bit Liquid AI	44±1	local (slow)	~2.2GB		33k	llama.cpp ›
590	Qwen2-Audio 7B Instruct · 4-bit Alibaba	44±3	local (slow)	~6.5GB			llama.cpp ›
591	MiniCPM-V 2.6 · 4-bit OpenBMB	44±2	local (slow)	~5.8GB			llama.cpp ›
592	Tulu 2 DPO 70B · 4-bit Ai2	44±1	local (slow)	~45GB			llama.cpp ›	+1
594	CodeGemma 7B IT (v1.0) · 4-bit Google	44±1	local (slow)	~7.1GB			llama.cpp ›
595	Yi-1.5 9B (base) · 4-bit 01.AI	44±2	local (slow)	~7.1GB		4k	llama.cpp ›
597	DeepSeek-Coder 1.3B Instruct · 4-bit DeepSeek	44±1	local (slow)	~2.9GB		16k	Ollama ›
598	Aya 23 8B · 4-bit Cohere For AI	44±3	local (slow)	~6.9GB		8k	llama.cpp ›
599	Qwen2.5 Coder 1.5B Instruct · 4-bit Alibaba	44±1	local (slow)	~1.9GB		33k	Ollama ›
600	SmolLM3 3B · 4-bit Hugging Face	44±1	local (slow)	~3.5GB		66k	llama.cpp ›
601	WizardLM 70B V1.0 · 4-bit Microsoft	44±1	local (slow)	~42GB			Ollama ›
602	Mathstral 7B v0.1 · 4-bit Mistral AI	44±1	local (slow)	~5.9GB		32k	llama.cpp ›
603	Aya Vision 32B · 4-bit Cohere	44±3	local (slow)	~71GB		16k	how to run ›
604	Hermes 3 Llama 3.1 8B · 4-bit Nous Research	44±1	local (slow)	~6.6GB		128k	llama.cpp ›
605	Llama 3 8B Instruct · 4-bit Meta	44±0	local (slow)	~6.6GB		8k	llama.cpp ›	+19
606	WizardCoder Python 13B V1.0 · 4-bit WizardLM	44±2	local (slow)	~15GB		16k	llama.cpp ›
607	OpenChat 3.5 · 4-bit OpenChat	43±1	local (slow)	~6.1GB		8k	llama.cpp ›
608	Command R (03-2024) · 4-bit Cohere	43±1	local (slow)	~33GB		128k	llama.cpp ›	+6
609	Granite 34B Code Base (8K) · 4-bit IBM	43±1	local (slow)	~24GB			llama.cpp ›
610	speechless-starcoder2-7b · 4-bit uukuguy	43±1	local (slow)	~6.1GB			llama.cpp ›
611	Falcon2 11B (base) · 4-bit TII	43±3	local (slow)	~8.2GB		8k	Ollama ›
613	OLMo 2 7B Instruct (1124) · 4-bit Allen Institute for AI	43±1	local (slow)	~6.2GB		4k	llama.cpp ›
614	WizardCoder-15B-V1.0 · 4-bit WizardLM	43±1	local (slow)	~12GB			llama.cpp ›
618	DeepSeek LLM 67B Chat · 4-bit DeepSeek	43±1	local (slow)	~44GB		4k	llama.cpp ›
619	Qwen1.5 14B Chat · 4-bit Alibaba	43±1	local (slow)	~11GB		32k	Ollama ›	+3
620	Llama 2 70B (base) · 4-bit Meta	43±1	local (slow)	~43GB		4k	Ollama ›
621	Mixtral 8x7B v0.1 (base) · 4-bit Mistral AI	43±1	local (slow)	~29GB		32k	llama.cpp ›
623	Code Llama 34B Python · 4-bit Meta	43±1	local (slow)	~23GB		16k	Ollama ›
624	DeepSeek-Coder 33B Base · 4-bit DeepSeek	43±1	local (slow)	~22GB		16k	llama.cpp ›
625	speechless-coding-7b-16k-tora · 16-bit uukuguy	43±1	local (slow)	~16GB			how to run ›
626	InternLM2 Chat 20B · 4-bit Shanghai AI Laboratory (InternLM)	43±1	local (slow)	~14GB			llama.cpp ›
627	CodeQwen1.5 7B (base) · 4-bit Alibaba	43±1	local (slow)	~6.4GB		66k	llama.cpp ›
628	Mixtral 8x7B Instruct v0.1 · 4-bit Mistral	42±0	local (slow)	~28GB		32k	llama.cpp ›	+15
629	Nous Hermes 2 Mistral 7B DPO · 4-bit Nous Research	42±3	local (slow)	~6.1GB		33k	llama.cpp ›
631	Granite 4.0 1B · 4-bit IBM	42±1	local (slow)	~2.6GB		128k	llama.cpp ›
632	OpenHermes-2.5-Code-290k-13B · 4-bit ajibawa-2023	42±1	local (slow)	~9.8GB			llama.cpp ›
633	Granite 4.0 H 1B · 4-bit IBM	42±1	local (slow)	~2.4GB		131k	llama.cpp ›
634	Nous Hermes 2 Mixtral 8x7B DPO · 4-bit Nous Research	42±2	local (slow)	~30GB		33k	llama.cpp ›
637	Deita 7B v1.0 · 4-bit HKUST NLP	42±2	local (slow)	~32GB			how to run ›
639	OpenHermes 2.5 Mistral 7B · 4-bit Nous Research	42±1	local (slow)	~5.9GB			llama.cpp ›	+2
640	Code Llama 34B Instruct · 4-bit Meta	42±1	local (slow)	~23GB		16k	llama.cpp ›	+1
641	Starling-LM 7B beta · 4-bit Nexusflow	42±1	local (slow)	~6.1GB		8k	llama.cpp ›	+3
642	Code Llama 34B (base) · 4-bit Meta	42±1	local (slow)	~23GB		16k	Ollama ›	+1
643	Vicuna 33B v1.3 · 4-bit LMSYS	41±1	local (slow)	~23GB		2k	Ollama ›	+2
644	Phi-3-mini-128k-instruct (Apr 2024) · 16-bit Microsoft	41±2	local (slow)	~9.5GB			how to run ›	+1
645	Bielik 1.5B v3.0 Instruct · 4-bit SpeakLeash	41±3	local (slow)	~2.6GB			llama.cpp ›
646	Starling-LM 7B alpha · 4-bit UC Berkeley (NEST)	41±1	local (slow)	~6.1GB		8k	llama.cpp ›	+2
647	LLaVA-v1.6-34B · 4-bit LLaVA	41±2	local (slow)	~24GB		4k	Ollama ›
648	Code Tulu 2 13B · 16-bit Allen Institute for AI	41±2†	local (slow)	~29GB			how to run ›
649	Llama3-OpenBioLLM-8B · 4-bit Saama AI Labs	41±2	local (slow)	~6.6GB			llama.cpp ›
650	Mistral 7B Instruct v0.3 · 4-bit Mistral	41±1	local (slow)	~5.9GB		32k	llama.cpp ›	+4
651	Granite 3.3 2B Instruct · 4-bit IBM	41±1†	local (slow)	~3.1GB		128k	llama.cpp ›
652	DeepSeek-Coder 6.7B Base · 4-bit DeepSeek	41±1	local (slow)	~8.8GB		16k	llama.cpp ›
653	Granite 3.0 2B Instruct · 4-bit IBM	41±1	local (slow)	~3.2GB		4k	llama.cpp ›
654	LFM2 1.2B · 4-bit Liquid AI	41±1	local (slow)	~1.3GB		33k	llama.cpp ›
655	Llama 2 70B Chat · 4-bit Meta	41±1	local (slow)	~45GB		4k	llama.cpp ›	+13
656	Llama 3.2 3B Instruct · 4-bit Meta	41±1	local (slow)	~3.5GB	~162 tok/s	131k	llama.cpp ›	+11
657	Granite 8B Code Base (4K) · 4-bit IBM	41±1	local (slow)	~6.6GB			llama.cpp ›
658	StarCoder2 15B · 4-bit BigCode	41±1	local (slow)	~12GB		16k	llama.cpp ›
659	Falcon-H1 3B Instruct · 4-bit TII	41±3	local (slow)	~3.5GB		131k	llama.cpp ›
660	Phi-2 (2.7B) · 4-bit Microsoft	40±1	local (slow)	~4.7GB		2k	Ollama ›	+2
661	speechless-code-mistral-7b-v1.0 · 4-bit uukuguy	40±1	local (slow)	~17GB			how to run ›
662	Llama-3-Instruct-8B-SimPO-v0.2 · 4-bit Princeton NLP	40±1	local (slow)	~6.6GB			llama.cpp ›
663	Qwen-14B-Chat · 4-bit Alibaba	40±1	local (slow)	~11GB			llama.cpp ›
664	Tulu 2 13B · 4-bit Allen Institute for AI	40±2	local (slow)	~9.8GB			llama.cpp ›
666	Code Llama 13B Instruct · 4-bit Meta	40±1	local (slow)	~15GB		16k	llama.cpp ›
667	MistralHermes-CodePro-7B-v1 · 4-bit beowolx	40±1	local (slow)	~6.1GB			llama.cpp ›
668	Phi-3-mini-4k-instruct (Apr 2024) · 4-bit Microsoft	40±2	local (slow)	~6GB			llama.cpp ›
669	CodeGemma 7B (v1.0) · 4-bit Google	40±1	local (slow)	~7.1GB			llama.cpp ›
671	Code Llama 13B Python · 4-bit Meta	40±1	local (slow)	~9.8GB		16k	llama.cpp ›
672	Apertus 8B Instruct (2509) · 4-bit Swiss AI	40±3	local (slow)	~6.9GB		66k	llama.cpp ›
673	DeepSeek-V2 Lite Chat (16B-A2.4B) · 4-bit DeepSeek	40±1	local (slow)	~12GB		32k	Ollama ›
674	Granite 3B Code Instruct (2K) · 4-bit IBM	40±1	local (slow)	~3.7GB			llama.cpp ›
675	Tulu 2 DPO 13B · 4-bit Allen Institute for AI	40±1	local (slow)	~9.8GB			llama.cpp ›
677	BioMistral-7B · 4-bit BioMistral	39±3	local (slow)	~6.1GB			llama.cpp ›
678	SOLAR 10.7B Instruct v1.0 · 4-bit Upstage	39±1	local (slow)	~8.3GB		4k	llama.cpp ›	+2
679	Code Tulu 2 7B · 16-bit Allen Institute for AI	39±2†	local (slow)	~16GB			how to run ›
680	Gemma 2 2B IT · 4-bit Google	39±1	local (slow)	~3.4GB		8k	llama.cpp ›	+4
681	Yi-1.5 6B (base) · 4-bit 01.AI	39±3	local (slow)	~5.4GB		4k	llama.cpp ›
682	Flan-T5 XXL · 4-bit Google	39±1	local (slow)	~49GB			how to run ›	+12
683	Dolphin 2.2.1 Mistral 7B · 4-bit Cognitive Computations	39±2	local (slow)	~5.9GB			llama.cpp ›
685	Gemma 1.1 7B IT · 4-bit Google	39±1	local (slow)	~7.1GB		8k	llama.cpp ›	+2
686	Granite 3.1 2B Instruct · 4-bit IBM	39±1	local (slow)	~3.1GB		128k	llama.cpp ›
687	Granite 20B Code Base (8K) · 4-bit IBM	39±1	local (slow)	~15GB			llama.cpp ›
688	Qwen1.5 7B Chat · 4-bit Alibaba	39±1	local (slow)	~9.7GB		32k	llama.cpp ›	+4
689	Code Llama 7B Python · 4-bit Meta	39±1	local (slow)	~5.8GB		16k	llama.cpp ›
690	Granite 3.2 2B Instruct · 4-bit IBM	39±1	local (slow)	~3.1GB		131k	llama.cpp ›
691	Falcon3 3B Instruct · 4-bit TII	39±3	local (slow)	~3.6GB		32k	llama.cpp ›
692	Qwen2.5 Coder 0.5B Instruct · 4-bit Alibaba	39±1	local (slow)	~1.1GB		33k	Ollama ›
693	Vicuna 13B v1.5 · 4-bit LMSYS	38±1	local (slow)	~9.8GB		4k	llama.cpp ›
694	Zephyr 7B alpha · 4-bit Hugging Face	38±1	local (slow)	~6.1GB			llama.cpp ›
695	Code Llama 13B (base) · 4-bit Meta	38±1	local (slow)	~9.8GB		16k	llama.cpp ›	+1
696	Yi 1.5 6B Chat · 4-bit 01.AI	38±2	local (slow)	~4.7GB		4k	llama.cpp ›
697	Qwen2-VL 2B Instruct · 4-bit Alibaba	38±1	local (slow)	~2.6GB		33k	llama.cpp ›
698	WizardLM 13B V1.2 · 4-bit Microsoft	38±1	local (slow)	~9.8GB			llama.cpp ›
699	Qwen2.5 1.5B Instruct · 4-bit Alibaba	38±1	local (slow)	~2GB	~98 tok/s	33k	llama.cpp ›	+6
700	Qwen3.5 0.8B · 4-bit Alibaba	38±1	local (slow)	~1.4GB	~319 tok/s	262k	llama.cpp ›	+2
701	Mistral 7B Instruct v0.2 · 4-bit Mistral	38±1	local (slow)	~5.9GB		32k	llama.cpp ›	+9
702	Code Llama 7B Instruct · 4-bit Meta	38±1	local (slow)	~5.8GB		16k	llama.cpp ›
703	Tulu 2 7B · 4-bit Allen Institute for AI	38±2	local (slow)	~5.8GB			llama.cpp ›
705	LFM2 700M · 4-bit Liquid AI	38±1	local (slow)	~1GB		33k	llama.cpp ›
706	EXAONE 3.5 2.4B Instruct · 4-bit LG AI Research	38±2	local (slow)	~3.2GB		33k	llama.cpp ›
707	xDAN-L1-Chat-RL-v1 · 4-bit xDAN-AI	38±1	local (slow)	~6.1GB			llama.cpp ›
710	Llama 3 8B (base) · 4-bit Meta	37±1	local (slow)	~6.6GB			llama.cpp ›
711	Mistral 7B v0.1 (base) · 4-bit Mistral AI	37±1	local (slow)	~6.1GB		8k	llama.cpp ›	+4
712	Flan-T5 XL · 4-bit Google	37±1	local (slow)	~14GB			how to run ›	+8
713	Gemma 7B (base) · 4-bit Google	37±1	local (slow)	~7.1GB		8k	llama.cpp ›	+3
714	Tulu 2 DPO 7B · 4-bit Allen Institute for AI	37±1	local (slow)	~5.8GB			llama.cpp ›
715	Vicuna 13B v1.3 · 4-bit LMSYS	37±2	local (slow)	~29GB		2k	how to run ›
716	DeepSeek R1 Distill Qwen 1.5B · 4-bit DeepSeek	37±2	local (slow)	~2GB		131k	llama.cpp ›	+1
717	Granite 3.1 3B-A800M Instruct · 4-bit IBM	37±2†	local (slow)	~3.6GB	~23 tok/s	131k	llama.cpp ›
718	Granite 3.0 3B-A800M Instruct · 4-bit IBM	37±1	local (slow)	~3.7GB		4k	llama.cpp ›
719	StarCoder2 7B · 4-bit BigCode	37±1	local (slow)	~5.4GB		16k	llama.cpp ›
720	DeepSeek LLM 7B Chat · 4-bit DeepSeek	37±2†	local (slow)	~5.7GB			Ollama ›
722	StarCoder 15.5B · 4-bit BigCode	37±1	local (slow)	~11GB			Ollama ›
723	Eurus-2-7B-SFT · 4-bit PRIME-RL	37±3†	local (slow)	~6.4GB			llama.cpp ›
724	Falcon 40B (base) · 4-bit TII	36±1	local (slow)	~27GB		2k	Ollama ›	+2
725	Nous-Hermes-13b · 4-bit Nous Research	36±2	local (slow)	~9.8GB		2k	llama.cpp ›
727	Code Llama 7B (base) · 4-bit Meta	36±1	local (slow)	~5.8GB		16k	llama.cpp ›	+2
728	Llama 2 13B Chat · 4-bit Meta	36±1	local (slow)	~15GB		4k	llama.cpp ›	+10
729	Llama 2 13B (base) · 4-bit Meta	36±1	local (slow)	~9.3GB		4k	Ollama ›	+4
730	Zephyr 7B beta · 4-bit Hugging Face	36±1	local (slow)	~6.1GB			llama.cpp ›	+3
731	Guanaco 33B · 16-bit University of Washington	35±1	local (slow)	~70GB			how to run ›
732	Qwen3 0.6B · 4-bit Alibaba	35±1	local (slow)	~2.1GB	~61 tok/s	33k	llama.cpp ›	+4
733	StarCoder2 3B · 4-bit BigCode	35±1	local (slow)	~3.4GB		16k	llama.cpp ›
734	Vicuna 7B v1.5 · 4-bit LMSYS	35±1	local (slow)	~5.8GB		4k	llama.cpp ›	+1
735	Granite 3B Code Base (2K) · 4-bit IBM	35±1	local (slow)	~3.7GB			llama.cpp ›
736	StarCoderBase 15.5B · 4-bit BigCode	35±1	local (slow)	~12GB			llama.cpp ›
738	OLMo 2 1B Instruct (0425) · 4-bit Allen Institute for AI	35±1	local (slow)	~2.4GB		4k	llama.cpp ›
739	ChatGLM3 6B 128K · 4-bit Zhipu AI (THUDM)	35±4	local (slow)	~5.8GB		131k	llama.cpp ›
740	Orca 2 13B · 4-bit Microsoft	35±3	local (slow)	~9.8GB		4k	llama.cpp ›
741	Vicuna 7B v1.3 · 4-bit LMSYS	35±2	local (slow)	~5.8GB		2k	llama.cpp ›
742	Stable Code 3B · 4-bit Stability AI	35±1	local (slow)	~4.8GB		16k	llama.cpp ›
744	Qwen1.5 4B Chat · 4-bit Alibaba	34±2	local (slow)	~4.1GB		32k	llama.cpp ›
745	DeepSeek-Coder 1.3B Base · 4-bit DeepSeek	34±1	local (slow)	~2.4GB		16k	Ollama ›
746	Gemma 7B IT · 4-bit Google	34±1	local (slow)	~7.1GB		8k	llama.cpp ›	+5
747	Mistral 7B Instruct v0.1 · 4-bit Mistral	34±1	local (slow)	~5.9GB		8k	llama.cpp ›	+3
750	CodeGemma 2B (v1.0) · 4-bit Google	34±1	local (slow)	~3.2GB			llama.cpp ›
751	Qwen2 1.5B Instruct · 4-bit Alibaba	34±1	local (slow)	~1.9GB		32k	llama.cpp ›	+1
752	Ghost 7B Alpha · 4-bit Ghost X	34±2	local (slow)	~6.1GB			llama.cpp ›
753	Falcon-H1 1.5B Instruct · 4-bit TII	33±4	local (slow)	~2.4GB		131k	llama.cpp ›
754	Gemma 3 1B · 4-bit Google	33±1	local (slow)	~1.8GB	~180 tok/s	32k	llama.cpp ›	+7
755	SmolLM2 1.7B Instruct · 4-bit Hugging Face	33±1	local (slow)	~3.1GB		8k	llama.cpp ›
756	Qwen-7B (base) · 4-bit Alibaba	33±1	local (slow)	~6.6GB		33k	llama.cpp ›
757	LFM2 350M · 4-bit Liquid AI	33±1	local (slow)	~1.7GB		33k	llama.cpp ›
761	StarCoderBase 7B · 4-bit BigCode	33±1	local (slow)	~6GB			Ollama ›
762	Llama 3.2 1B Instruct · 4-bit Meta	33±1	local (slow)	~1.7GB	~251 tok/s	131k	llama.cpp ›	+13
763	Phi-1.5 (1.3B) · 4-bit Microsoft	33±1	local (slow)	~2.4GB		2k	llama.cpp ›
765	Llama 2 7B Chat · 4-bit Meta	32±1	local (slow)	~8.8GB		4k	llama.cpp ›	+13
766	Qwen-7B-Chat · 4-bit Alibaba	32±1	local (slow)	~6.6GB		33k	llama.cpp ›
767	OLMo 7B Instruct · 16-bit Ai2	32±2	local (slow)	~16GB			how to run ›
768	Llama 2 7B (base) · 4-bit Meta	32±1	local (slow)	~8.5GB		4k	Ollama ›	+6
771	Flan-T5 Large · 4-bit Google	32±1	local (slow)	~4.8GB			how to run ›	+5
772	Granite 3.1 1B-A400M Instruct · 4-bit IBM	31±2†	local (slow)	~2.3GB	~20 tok/s	131k	llama.cpp ›
773	Falcon 40B Instruct · 4-bit TII	31±2	local (slow)	~27GB			Ollama ›
774	Gemma 2B (base) · 4-bit Google	31±1	local (slow)	~3.2GB		8k	llama.cpp ›	+3
775	Gemma 1.1 2B IT · 4-bit Google	31±1	local (slow)	~3.2GB		8k	llama.cpp ›	+1
776	Falcon 7B (base) · 4-bit TII	31±1	local (slow)	~5.9GB		2k	Ollama ›	+4
778	Baichuan2-7B-Base · 16-bit Baichuan	30±1	local (slow)	~17GB			how to run ›
779	NexusRaven V2 13B · 4-bit Nexusflow	30±3	local (slow)	~9.3GB		16k	Ollama ›
782	ChatGLM3-6B · 4-bit Zhipu AI (THUDM)	30±2	local (slow)	~5.8GB			llama.cpp ›
783	Qwen2.5 0.5B Instruct · 4-bit Alibaba	30±2	local (slow)	~1.2GB		33k	llama.cpp ›	+1
784	OLMo 7B · 4-bit Allen Institute for AI	30±1†	local (slow)	~30GB		2k	how to run ›
785	OpenLLaMA 7B v2 · 8-bit OpenLM Research	30±1	local (slow)	~9.1GB			llama.cpp ›
786	StableLM-3B-4E1T · 4-bit Stability AI	30±1	local (slow)	~3.3GB		4k	llama.cpp ›
787	StarCoderBase 3B · 4-bit BigCode	30±1	local (slow)	~3.4GB			Ollama ›
788	Granite 3.0 1B-A400M Instruct · 4-bit IBM	30±1	local (slow)	~2.3GB		4k	llama.cpp ›
791	StableLM-Base-Alpha 7B v2 · 4-bit Stability AI	30±1	local (slow)	~30GB		4k	how to run ›
792	GPT4All-13B-Snoozy · 4-bit Nomic AI	29±2	local (slow)	~56GB			how to run ›
794	EuroLLM 1.7B Instruct · 4-bit UTTER (EuroLLM)	29±3	local (slow)	~2.6GB		4k	llama.cpp ›
795	Granite 4.0 H 350M · 4-bit IBM	29±1†	local (slow)	~1.7GB		33k	llama.cpp ›
798	Gemma 2B IT · 4-bit Google	29±1	local (slow)	~3.1GB		8k	llama.cpp ›	+3
799	Granite 4.0 350M · 4-bit IBM	28±1	local (slow)	~1.7GB		32k	llama.cpp ›
800	OPT-30B · 4-bit Meta	28±1	local (slow)	~64GB		2k	how to run ›	+5
802	GPT-NeoX-20B · 4-bit EleutherAI	28±1	local (slow)	~15GB		2k	llama.cpp ›	+1
803	Qwen2 0.5B Instruct · 4-bit Alibaba	28±2	local (slow)	~1.1GB		32k	llama.cpp ›
804	RedPajama-INCITE 7B Base · 4-bit Together	28±1	local (slow)	~16GB		2k	how to run ›
809	Qwen1.5 1.8B Chat · 4-bit Alibaba	26±2	local (slow)	~2.7GB		33k	Ollama ›
811	Falcon3 1B Instruct · 4-bit TII	26±4	local (slow)	~2.7GB		8k	llama.cpp ›
812	OPT-13B (13 billion parameters) · 4-bit Meta	26±1	local (slow)	~29GB		2k	how to run ›	+4
814	Falcon 7B Instruct · 4-bit TII	26±2	local (slow)	~5.9GB			Ollama ›	+1
815	OpenAssistant SFT-4 Pythia 12B · 4-bit LAION (OpenAssistant)	25±1	local (slow)	~9.5GB			llama.cpp ›	+1
817	RedPajama-INCITE Base 3B v1 · 4-bit Together	25±3	local (slow)	~3.4GB		2k	llama.cpp ›
818	Pythia 12B · 4-bit EleutherAI	25±1	local (slow)	~9.5GB		2k	llama.cpp ›	+1
819	Flan-T5 Base · 4-bit Google	25±1	local (slow)	~2.5GB			how to run ›	+3
820	OpenLLaMA 3B v2 · 4-bit OpenLM Research	25±1	local (slow)	~4.2GB			llama.cpp ›
821	GPT-J 6B · 4-bit EleutherAI	25±1	local (slow)	~14GB		2k	how to run ›	+6
823	StableLM-Base-Alpha 3B v2 · 4-bit Stability AI	24±1	local (slow)	~13GB		4k	how to run ›
824	OPT-6.7B · 4-bit Meta	24±1	local (slow)	~16GB		2k	how to run ›	+5
827	Qwen1.5 0.5B Chat · 4-bit Alibaba	23±3	local (slow)	~1.8GB		33k	Ollama ›
828	Pythia 6.9B · 4-bit EleutherAI	23±1	local (slow)	~6.1GB		2k	llama.cpp ›	+2
830	Phi-1 (1.3B) · 4-bit Microsoft	22±4	local (slow)	~2.4GB		2k	llama.cpp ›
832	SmolLM2 360M Instruct · 8-bit Hugging Face	22±3	local (slow)	~1.2GB		8k	llama.cpp ›
834	BLOOM 7.1B · 4-bit BigScience	21±1	local (slow)	~7.1GB		2k	llama.cpp ›	+7
835	OLMo 1B · 4-bit Allen Institute for AI	21±2†	local (slow)	~2.2GB		2k	llama.cpp ›
836	Pythia 2.8B · 4-bit EleutherAI	20±1	local (slow)	~3.4GB		2k	llama.cpp ›
837	Pythia 2.8B (deduped) · 4-bit EleutherAI	20±1	local (slow)	~3.4GB		2k	llama.cpp ›
838	Falcon-RW 1B · 16-bit TII	20±2	local (slow)	~4.3GB		2k	how to run ›
839	OPT-2.7B · 4-bit Meta	19±1	local (slow)	~7.1GB		2k	how to run ›	+3
840	GPT-Neo 2.7B · 4-bit EleutherAI	18±1	local (slow)	~13GB		2k	how to run ›	+8
841	StableLM-Tuned-Alpha 7B · 4-bit Stability AI	18±2	local (slow)	~6.9GB			llama.cpp ›
843	T5 Large · 4-bit Google	17±2	local (slow)	~4.6GB		1k	how to run ›
844	Gemma 3 270M IT · 8-bit Google	16±4	local (slow)	~1.2GB		32k	llama.cpp ›
845	BLOOM 3B · 4-bit BigScience	16±1	local (slow)	~4GB		2k	llama.cpp ›	+2
846	Flan-T5 Small · 4-bit Google	16±3	local (slow)	~1.8GB			how to run ›
849	OPT-1.3B · 4-bit Meta	14±1	local (slow)	~4.3GB		2k	how to run ›	+5
850	T5 Base · 4-bit Google	14±3	local (slow)	~2.4GB		1k	how to run ›
851	Pythia 1.4B · 4-bit EleutherAI	14±1	local (slow)	~2.4GB		2k	llama.cpp ›
853	Pythia 1B · 4-bit EleutherAI	12±1	local (slow)	~2.2GB		2k	llama.cpp ›
855	GPT-Neo 1.3B · 4-bit EleutherAI	12±2	local (slow)	~7.1GB		2k	how to run ›	+3
856	T5 Small · 4-bit Google	12±3	local (slow)	~1.8GB		1k	how to run ›
857	BLOOM 1.7B · 4-bit BigScience	11±1	local (slow)	~3.1GB		2k	llama.cpp ›	+1
858	Zyte-1B · 4-bit venkycs (aihub-app)	10±3	local (slow)	~2.2GB			llama.cpp ›
860	StableLM-Base-Alpha 7B · 4-bit Stability AI	8±1	local (slow)	~35GB		4k	how to run ›
861	Pythia 410M · 4-bit EleutherAI	8±2	local (slow)	~1.8GB		2k	llama.cpp ›
863	BLOOM 1.1B · 4-bit BigScience	7±1	local (slow)	~2.6GB		2k	llama.cpp ›
864	GPT-2 XL (1.5B) · 4-bit OpenAI	7±1	local (slow)	~2.8GB	~75 tok/s	1k	llama.cpp ›	+9
867	OPT-350M · 4-bit Meta	1±1	local (slow)	~2.2GB		2k	how to run ›	+1
869	StableLM-Base-Alpha 3B · 4-bit Stability AI	1±2	local (slow)	~17GB		4k	how to run ›
870	BLOOM 560M · 4-bit BigScience	0±1	local (slow)	~2.1GB		2k	llama.cpp ›	+1
871	Pythia 160M · 4-bit EleutherAI	-2±2	local (slow)	~1.6GB		2k	llama.cpp ›
872	GPT-2 Large (774M) · 4-bit OpenAI	-3±1	local (slow)	~2GB	~137 tok/s	1k	llama.cpp ›	+3
873	OPT-125M · 4-bit Meta	-3±1	local (slow)	~1.8GB		2k	how to run ›	+1
874	GPT-Neo 125M · 4-bit EleutherAI	-3±2	local (slow)	~2.1GB		2k	how to run ›
875	GPT-2 Medium (355M) · 4-bit OpenAI	-5±1	local (slow)	~1.8GB		1k	llama.cpp ›	+3
876	Pythia 70M · 4-bit EleutherAI	-13±2	local (slow)	~1.6GB		2k	llama.cpp ›
877	GPT-2 Small (124M) · 4-bit OpenAI	-13±2	local (slow)	~1.6GB		1k	llama.cpp ›	+3
—	K2 Horizon 7B · gguf-q4_k_m Institute of Foundation Models (MBZUAI)	not scored yet	local (slow)	~8GB		524k	llama.cpp ›

+ 378 more don't fit your machine or budget · show them

+ 207 models with fewer than 3 results left off, since one or two tests say little about overall ability · show them

907 of 1212 catalogue models listed here · sources and licences

* new or thinly tested: ranked on too few benchmark results · † only the maker’s own numbers so far · ~RAM is estimated; full context may need more memory