Skip to content

Latest commit

 

History

History

depth_anything_controlnet

README.md

ControlNet trained on DepthAnything

Input

  • Image used to control the reverse diffusion process

depth image

The script will perform a image generation conditioned on the control image. Images like this can be generated using the script in ailia-models/depth_estimation/depth_anything.

  • Prompt and optional negative prompt

By default, the prompt is set to "Beautiful flower garden full of tulips.", and the negative prompt is set to ""(empty string).

Output

  • Generated image

result

Usage

Internet connection is required when running the script for the first time, as it will download the necessary model files.

Running this script will generate an image conditioned on the control image. Use --help to see the help message with all the arguments.

Example 1: Inference on prepared demo image, with the default prompts and parameters.

$ python3 depth_anything_controlnet.py

Example 2: Specify input path and save path.

$ python3 depth_anything_controlnet.py -i depth_flower.png -s output.png

-i and -s options can be used to specify the input path and save path separately.

Example 3: Specify prompt, negative prompt, and sampler.

$ python3 depth_anything_controlnet.py --prompt "Flower garden full of yellow tulips" --negative_prompt "Low quality, unrealistic." --n_timesteps 50 --sampler ddim

The argument --n_timesteps can be used to specify the number of steps the model takes to generate the image. The sampling method specified in the --sampler argument is used in the reverse diffusion process. Currently supports ddpm and ddim only.

Example 4: Change the size of the generated image

$ python3 depth_anything_controlnet.py --width 512 --height 512

--width and --height argument can be used to change the size of the generated image. It may take longer or even be impossible to generate the image depending on the machine being used.

Reference

Framework

Pytorch

Model Format

ONNX opset=12

Netron