whatmodel.sh / image / InstructPix2Pix
InstructPix2Pix
↗ share
+ contribute
InstructPix2Pix is an open-weights image model from UC Berkeley (Brooks, Holynski, Efros), released January 2023. Its published size is 0.86B parameters. Licence: mit.
Catalogue identity notes
timbrooks/instruct-pix2pix (instruct-pix2pix-00-22000), the only public InstructPix2Pix checkpoint; Stable Diffusion v1.5-based image editor (0.86B UNet as in SD 1.x). HF card licence MIT. Memory estimated as for SD 1.x. Not MagicBrush (a separate fine-tune).
Score within the LMArena text-to-image comparison set only.
$ how it compares
The image models ranked either side of InstructPix2Pix, out of 197 with a score.
| # | model | score | where |
|---|---|---|---|
| 192 | 24±5 | local | |
| 193 | 23±4 | local | |
| 194 | 19±3 | local | |
| 195 | 17±4 | comparison only | |
| 196 | 16±4 | comparison only |
$ what it's good at
image to image / editing▃▁·▃▂▃█▃▃▄▁▁ #62 of 62 · 81% of its score · 4 tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| UnicBench (EN, overall) | 29.2% ✎ | ▃····▂▄▂▄·█▂ beats 0/11 | 33% |
| Edit-Compass (EN, overall) | 23.8% ✎ | ▃▃▂·▃▆▂█·▃·▃ beats 0/20 | 22% |
| ImagenWorld text-guided editing (human overall) | 45.0% ✎ | █··█··██···█ beats 0/4 | 15% |
| MultiTextEdit cross-lingual text-in-image editing (SemAvg, LVM judge) | 23.8% ✎ | ▄·▄▄▄▄··█▄·▄ beats 0/8 | 11% |
other tests2 not in a task · 15% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| EdiVal multi-turn editing overall (EdiVal-O after turn 3) | 13.0% ✎ | ▃·▂▃·▂·▃▂·▃█ beats 0/14 | 12% |
| PaintBench precise editing (mIoU, 20-task average) | 0.3% ✎ | ▃·····█▅▃·▃▅ beats 0/9 | 2.3% |
$ where to run it
Other ways to run it
- Run it as published · needs ~2.6 GB, NVIDIA only · from huggingface.co/timbrooks
uv run --with diffusers --with transformers --with accelerate --with torch python -c "import torch; from diffusers import StableDiffusionInstructPix2PixPipeline; from diffusers.utils import load_image; p=StableDiffusionInstructPix2PixPipeline.from_pretrained('timbrooks/instruct-pix2pix', torch_dtype=torch.float16, safety_checker=None).to('cuda'); p('turn him into cyborg', image=load_image('https://raw.githubusercontent.com/timothybrooks/instruct-pix2pix/main/imgs/example.jpg'), num_inference_steps=10, image_guidance_scale=1).images[0].save('out.png')"
Weights and documentation: huggingface.co/timbrooks ↗
$ how the score was worked out · UnicEdit-10M: A Dataset and B… 33%, Edit-Compass & EditReward-Com… 22%, ImagenWorld: Stress-Testing I… 15% and 3 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json