R
RunPod3mo ago
smoke

Dreambooth training taking very long (even for 1000 steps) [4090]

Hi there, I am running a Dreambooth training that uses sd-scripts under the hood but it seems like it is taking very long on a 4090. I use the save_every_n_steps flag so I can kinda see how long it takes for every 1000 steps. I feel like this is taking very long or am I wrong?
No description
3 Replies
smoke
smoke3mo ago
No description
smoke
smoke3mo ago
I disabled gradient checkpointing, set the train_batch_size to 4 and the vae_batch_size to 4 aswell to possibly improve the overall training speed and also caching latent step and also enabled xformers
digigoblin
digigoblin3mo ago
Join @YouTube.com/@SECourses discord and ask there
Want results from more Discord servers?
Add your server