source

​

What is Dreambooth?

• Dreambooth is a method of personalizing text-to-image models, such as Stable Diffusion.

• Dreambooth personalizes Stable Diffusion by fine tuning on a small set of reference images (around 3-5 images) that you provide.

• Current models such as Stable Diffusion produce high-quality images, but it struggles to make new versions of your data in different settings or poses.

​

How Does Dreambooth Work?

• During fine-tuning, Dreambooth learns to recognize unique features of your data.

• The model associates an identifier (called sks) to your unique features.

• The sks identifier allows the model to embed your unique features into the output domain of the model.

• The sks identifier will be used to synthesize photorealistic images of your features in different scenes or poses.

• The sks identifier was chosen randomly, because it is a rare word in the English dictionary.

​

Training with a sample dataset

• Please download the images of the dog found here to the --dataset-dir folder.

• --class-word describes what images in your dataset contain (in this example, the dataset contains dog photos).

• --regularization-prompt describes the type of images you want to generate.

• Make a directory with mkdir logs and pass in the directory as --log-dir to get logs in that directory.

• We set sensible defaults for the steps, batch size, and learning rate. Refer to How do I change default settings? for more information.

rubbrband train dreambooth --dataset-dir="./data" \
--class-word="dog" \
--regularization-prompt="a photo of a dog"
--log-dir logs

​

Inference on sample prompts

• After doing a training run on Dreambooth, run the rubbrband eval command.

• In the --input-prompt, make sure the class-word (e.g. dog) is in the prompt.

• In the --input-prompt, make sure the sks identifier is in the prompt.

  • For more information on what sks means, refer to How Does Dreambooth Work?

rubbrband eval dreambooth --input-prompt "sks dog on the beach"

​

Where are my outputs?

• To save the output image on your local machine

rubbrband copy-from dreambooth /home/engineering/samples/ ./data

• The checkpoint file is already on your local machine inside the --log-dir

​

How do I change default settings?

• Dreambooth fine tuning is very sensitive and may require changing the default settings to get good outputs.

• This blog post covers some of the settings that have been found to work well.