whatmodel.sh / image / Versatile Diffusion v1.0

Versatile Diffusion v1.0

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Versatile Diffusion v1.0 is an open-weights image model from SHI Labs, released November 2022. Licence: mit.

Catalogue identity notes

shi-labs/versatile-diffusion (V1.0). Comparison-only: its diffusers pipelines are deprecated upstream and no parameter/memory figure is published, so no command is listed.

29.0±4.4#133 of 144 image modelsfrom 3 benchmark results across 2 sources

Score within the LMArena text-to-image comparison set only.

$ how it compares

The image models ranked either side of Versatile Diffusion v1.0, out of 197 with a score.

#modelscorewhereRAMreleased
180Stable Diffusion v2.1 Stability AI29±2comparison only2022-12
181Stable Diffusion v2.0 Stability AI29±3comparison only2022-11
182Versatile Diffusion v1.0 SHI Labs29±4local~7.2GB2022-11
183DALL·E Mega (minDALL-E) DALL·E Mini / minDALL-E29±4local~8GB2022-07
184MagicBrush (InstructPix2Pix fine-tune, jul7) Ohio State University (OSU NLP)29±7comparison only2023-07

see all image models · find the ones that fit your machine

$ what it's good at

text to image▂▅▅▃▄▄█▃▁▂▁▁ #112 of 126 * · 0.5% of its score · 1 test

Beats Latent Diffusion text2img-large (LDM-KL-8-G, LAION-400M), VQ-Diffusion (ITHQ), VILA-U 7B (256px) here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
HPS v2 benchmark (4-style average)26.6% ✎▃··▃▅·▅·▅▅█▅ beats 7/14hard · avg 27%▇ 0.5%blunt · at its level
other tests2 not in a task · 91% of its score
testresultvs the others in its rundifficultycountswhy
ImageReward human ranking of T2I models (#wins, real user prompts)340.0000 wins ✎▄·········█▄ beats 1/3avg 526.557 wins▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 76%sharp
MS-COCO 2014 zero-shot FID-30k at 256px ↓ (ImageReward protocol)18.4000 FID ✎▄··········█ beats 1/2avg 7.871 FID▇▇▇▇▇▇ 16%blunt · small run

$ where to run it

Other ways to run it

  1. Run it as published · needs ~7.2 GB, NVIDIA only · from huggingface.co/shi-labs (official)
    uv run --with torch --with diffusers --with transformers --with accelerate python -c "import torch; from diffusers import VersatileDiffusionTextToImagePipeline; p=VersatileDiffusionTextToImagePipeline.from_pretrained('shi-labs/versatile-diffusion', torch_dtype=torch.float16); p.remove_unused_weights(); p=p.to('cuda'); p('an astronaut riding on a horse on mars', generator=torch.Generator(device='cuda').manual_seed(0)).images[0].save('out.png')"

Official weights and documentation: huggingface.co/shi-labs ↗

$ how the score was worked out · ImageReward: Learning and Eva… 91%, tgxs002/HPSv2 repository READ… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation (arXiv 2304.05977v4)2▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 91%CC-BY-4.0
tgxs002/HPSv2 repository README, HPS v2 benchmark table1▇ 1%Apache-2.0
prior · pull toward the middle▇▇ 8%
weighted average= 29.0

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json