whatmodel.sh / image / Stable Diffusion v1.4

Stable Diffusion v1.4

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Stable Diffusion v1.4 is an open-weights image model from CompVis, released August 2022. Its published size is 0.86B parameters. Licence: creativeml-openrail-m.

Catalogue identity notes

CompVis/stable-diffusion-v1-4 checkpoint; 860M UNet per the CompVis README.

26.8±2.0#136 of 144 image modelsfrom 7 benchmark results across 6 sources

Score within the LMArena text-to-image comparison set only.

$ how it compares

The image models ranked either side of Stable Diffusion v1.4, out of 197 with a score, then more from CompVis.

#modelscorewhereRAMreleased
184MagicBrush (InstructPix2Pix fine-tune, jul7) Ohio State University (OSU NLP)29±7comparison only2023-07
185Openjourney PromptHero28±2local~2.1GB2022-11
186Stable Diffusion v1.4 CompVis27±2local~2.1GB2022-08
187Stable Diffusion v1.3 CompVis27±4local~2.1GB2022-08
188Stable Diffusion v1.1 CompVis26±3local~2.1GB2022-08
more from CompVis
106Latent Diffusion text2img-large (LDM-KL-8-G, LAION-400M) CompVis48±15local~7.5GB2022-04
189Stable Diffusion v1.2 CompVis26±3local~2.1GB2022-08

see all image models · find the ones that fit your machine

$ what it's good at

text to image▃▅▅▃▄▄█▃▁▂▁▁ #121 of 126 · 45% of its score · 3 tests

Trails CogView2 here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GenEval42.0% ✎▅█···▅▃▃▃··▃ beats 1/10easy · avg 79%▇▇▇▇▇▇▇▇▇▇ 26%at its level
ConceptMix (k=2 full-mark score)23.0% ✎▄·▄·█▄▄▄···▄ beats 0/7easy · avg 66%▇▇▇▇▇▇▇▇ 19%at its level
HPS v2 benchmark (4-style average)27.0% ✎▃··▃▅·▅·▅▅█▅ beats 9/14hard · avg 27%▇ 0.1%blunt · at its level
other tests4 not in a task · 53% of its score
testresultvs the others in its rundifficultycountswhy
EvalAlign human text-image alignment score4.5200 mean human alignment score ✎█▄▄·█·▄█▄▄·▄ beats 1/11avg 6.285 mean human alignment score▇▇▇▇▇▇▇▇▇▇ 26%typical
ImageReward human ranking of T2I models (#wins, real user prompts)362.0000 wins ✎▄·········█▄ beats 2/3avg 526.557 wins▇▇▇▇▇▇ 16%sharp
HEIM text-image alignment (aspect win rate, v1.1.0)69.1% ✎▁····█▃·▁··▁ beats 7/9very easy · avg 92%▇▇▇ 8.4%sharp · at its level · noisy
MS-COCO 2014 zero-shot FID-30k at 256px ↓ (ImageReward protocol)17.9000 FID ✎▄··········█ beats 2/2avg 7.871 FID▇ 3.2%blunt · small run

$ where to run it

Other ways to run it

version

  1. Run it as published · needs ~2.6 GB, NVIDIA only · from huggingface.co/CompVis (official)
    uv run --with diffusers --with transformers --with accelerate --with torch python -c "import torch; from diffusers import StableDiffusionPipeline; p=StableDiffusionPipeline.from_pretrained('CompVis/stable-diffusion-v1-4', torch_dtype=torch.float16).to('cuda'); p('a cat holding a sign').images[0].save('out.png')"

Official weights and documentation: huggingface.co/CompVis ↗

$ how the score was worked out · GenEval authors' results: djg… 26%, EvalAlign: Supervised Fine-Tu… 26%, ConceptMix: A Compositional I… 19% and 3 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
GenEval authors' results: djghosh13/geneval README and arXiv 2310.11513v11▇▇▇▇▇▇▇▇ 26%MIT (README); CC-BY-4.0 (arXiv 2310.11513v1)
EvalAlign: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models (arXiv:2406.16562v3)1▇▇▇▇▇▇▇▇ 26%CC-BY-4.0
ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty (arXiv v1)1▇▇▇▇▇▇ 19%CC-BY-4.0
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation (arXiv 2304.05977v4)2▇▇▇▇▇▇ 19%CC-BY-4.0
HEIM: Holistic Evaluation of Text-to-Image Models (arXiv 2311.04287v1)1▇▇▇ 8%CC-BY-4.0
tgxs002/HPSv2 repository README, HPS v2 benchmark table1▇ 0%Apache-2.0
prior · pull toward the middle▇ 2%
weighted average= 26.8

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json