Update README.md
Browse files
README.md
CHANGED
@@ -70,38 +70,6 @@ For more technical details of the original MMDiT architecture, please refer to t
|
|
70 |
|
71 |
See [blog](https://stability.ai/news/introducing-stable-diffusion-3-5) for our study about comparative performance in prompt adherence and aesthetic quality.
|
72 |
|
73 |
-
|
74 |
-
## File Structure
|
75 |
-
|
76 |
-
Click here to access the [Files and versions tab](https://huggingface.co/stabilityai/stable-diffusion-3.5-large/tree/main)
|
77 |
-
|
78 |
-
```β
|
79 |
-
βββ text_encoders/
|
80 |
-
β βββ README.md
|
81 |
-
β βββ clip_g.safetensors
|
82 |
-
β βββ clip_l.safetensors
|
83 |
-
β βββ t5xxl_fp16.safetensors
|
84 |
-
β βββ t5xxl_fp8_e4m3fn.safetensors
|
85 |
-
β
|
86 |
-
βββ README.md
|
87 |
-
βββ LICENSE
|
88 |
-
βββ sd3_large.safetensors
|
89 |
-
βββ SD3.5L_example_workflow.json
|
90 |
-
βββ sd3_large_demo.png
|
91 |
-
|
92 |
-
** File structure below is for diffusers integration**
|
93 |
-
βββ scheduler/
|
94 |
-
βββ text_encoder/
|
95 |
-
βββ text_encoder_2/
|
96 |
-
βββ text_encoder_3/
|
97 |
-
βββ tokenizer/
|
98 |
-
βββ tokenizer_2/
|
99 |
-
βββ tokenizer_3/
|
100 |
-
βββ transformer/
|
101 |
-
βββ vae/
|
102 |
-
βββ model_index.json
|
103 |
-
```
|
104 |
-
|
105 |
## Using with Diffusers
|
106 |
Upgrade to the latest version of the [𧨠diffusers library](https://github.com/huggingface/diffusers)
|
107 |
```
|
@@ -124,86 +92,6 @@ image = pipe(
|
|
124 |
image.save("capybara.png")
|
125 |
```
|
126 |
|
127 |
-
### Quantizing the model with diffusers
|
128 |
-
|
129 |
-
Reduce your VRAM usage and have the model fit on π€ VRAM GPUs
|
130 |
-
|
131 |
-
```
|
132 |
-
pip install bitsandbytes
|
133 |
-
```
|
134 |
-
|
135 |
-
```py
|
136 |
-
from diffusers import BitsAndBytesConfig, SD3Transformer2DModel
|
137 |
-
from diffusers import StableDiffusion3Pipeline
|
138 |
-
import torch
|
139 |
-
|
140 |
-
model_id = "stabilityai/stable-diffusion-3.5-large"
|
141 |
-
|
142 |
-
nf4_config = BitsAndBytesConfig(
|
143 |
-
load_in_4bit=True,
|
144 |
-
bnb_4bit_quant_type="nf4",
|
145 |
-
bnb_4bit_compute_dtype=torch.bfloat16
|
146 |
-
)
|
147 |
-
model_nf4 = SD3Transformer2DModel.from_pretrained(
|
148 |
-
model_id,
|
149 |
-
subfolder="transformer",
|
150 |
-
quantization_config=nf4_config,
|
151 |
-
torch_dtype=torch.bfloat16
|
152 |
-
)
|
153 |
-
|
154 |
-
pipeline = StableDiffusion3Pipeline.from_pretrained(
|
155 |
-
model_id,
|
156 |
-
transformer=model_nf4,
|
157 |
-
torch_dtype=torch.bfloat16
|
158 |
-
)
|
159 |
-
pipeline.enable_model_cpu_offload()
|
160 |
-
|
161 |
-
prompt = "A whimsical and creative image depicting a hybrid creature that is a mix of a waffle and a hippopotamus, basking in a river of melted butter amidst a breakfast-themed landscape. It features the distinctive, bulky body shape of a hippo. However, instead of the usual grey skin, the creature's body resembles a golden-brown, crispy waffle fresh off the griddle. The skin is textured with the familiar grid pattern of a waffle, each square filled with a glistening sheen of syrup. The environment combines the natural habitat of a hippo with elements of a breakfast table setting, a river of warm, melted butter, with oversized utensils or plates peeking out from the lush, pancake-like foliage in the background, a towering pepper mill standing in for a tree. As the sun rises in this fantastical world, it casts a warm, buttery glow over the scene. The creature, content in its butter river, lets out a yawn. Nearby, a flock of birds take flight"
|
162 |
-
|
163 |
-
image = pipeline(
|
164 |
-
prompt=prompt,
|
165 |
-
num_inference_steps=28,
|
166 |
-
guidance_scale=4.5,
|
167 |
-
max_sequence_length=512,
|
168 |
-
).images[0]
|
169 |
-
image.save("whimsical.png")
|
170 |
-
```
|
171 |
-
|
172 |
-
### Fine-tuning
|
173 |
-
|
174 |
-
Please see the fine-tuning guide [here](https://stabilityai.notion.site/Stable-Diffusion-3-5-Large-Fine-tuning-Tutorial-11a61cdcd1968027a15bdbd7c40be8c6).
|
175 |
-
|
176 |
-
|
177 |
-
## Uses
|
178 |
-
|
179 |
-
### Intended Uses
|
180 |
-
|
181 |
-
Intended uses include the following:
|
182 |
-
* Generation of artworks and use in design and other artistic processes.
|
183 |
-
* Applications in educational or creative tools.
|
184 |
-
* Research on generative models, including understanding the limitations of generative models.
|
185 |
-
|
186 |
-
All uses of the model must be in accordance with our [Acceptable Use Policy](https://stability.ai/use-policy).
|
187 |
-
|
188 |
-
### Out-of-Scope Uses
|
189 |
-
|
190 |
-
The model was not trained to be factual or true representations of people or events. As such, using the model to generate such content is out-of-scope of the abilities of this model.
|
191 |
-
|
192 |
-
## Safety
|
193 |
-
|
194 |
-
As part of our safety-by-design and responsible AI deployment approach, we take deliberate measures to ensure Integrity starts at the early stages of development. We implement safety measures throughout the development of our models. We have implemented safety mitigations that are intended to reduce the risk of certain harms, however we recommend that developers conduct their own testing and apply additional mitigations based on their specific use cases.
|
195 |
-
For more about our approach to Safety, please visit our [Safety page](https://stability.ai/safety).
|
196 |
-
|
197 |
-
### Integrity Evaluation
|
198 |
-
|
199 |
-
Our integrity evaluation methods include structured evaluations and red-teaming testing for certain harms. Testing was conducted primarily in English and may not cover all possible harms.
|
200 |
-
|
201 |
-
### Risks identified and mitigations:
|
202 |
-
|
203 |
-
* Harmful content: We have used filtered data sets when training our models and implemented safeguards that attempt to strike the right balance between usefulness and preventing harm. However, this does not guarantee that all possible harmful content has been removed. TAll developers and deployers should exercise caution and implement content safety guardrails based on their specific product policies and application use cases.
|
204 |
-
* Misuse: Technical limitations and developer and end-user education can help mitigate against malicious applications of models. All users are required to adhere to our [Acceptable Use Policy](https://stability.ai/use-policy), including when applying fine-tuning and prompt engineering mechanisms. Please reference the Stability AI Acceptable Use Policy for information on violative uses of our products.
|
205 |
-
* Privacy violations: Developers and deployers are encouraged to adhere to privacy regulations with techniques that respect data privacy.
|
206 |
-
|
207 |
### Contact
|
208 |
|
209 |
Please report any issues with the model or contact us:
|
@@ -214,4 +102,3 @@ Please report any issues with the model or contact us:
|
|
214 |
* License and general: https://stability.ai/license
|
215 |
* Enterprise license: https://stability.ai/enterprise
|
216 |
|
217 |
-
|
|
|
70 |
|
71 |
See [blog](https://stability.ai/news/introducing-stable-diffusion-3-5) for our study about comparative performance in prompt adherence and aesthetic quality.
|
72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
## Using with Diffusers
|
74 |
Upgrade to the latest version of the [𧨠diffusers library](https://github.com/huggingface/diffusers)
|
75 |
```
|
|
|
92 |
image.save("capybara.png")
|
93 |
```
|
94 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
95 |
### Contact
|
96 |
|
97 |
Please report any issues with the model or contact us:
|
|
|
102 |
* License and general: https://stability.ai/license
|
103 |
* Enterprise license: https://stability.ai/enterprise
|
104 |
|
|