DragGAN README

The project's README from github.com/XingangPan/DragGAN. As one markdown file: README.md.

Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

Xingang Pan · Ayush Tewari · Thomas Leimkühler · Lingjie Liu · Abhimitra Meka · Christian Theobalt

SIGGRAPH 2023 Conference Proceedings


PyTorch Twitter Paper PDF Project Page Open In Colab

Web Demos

Open in OpenXLab

Huggingface

Requirements

If you have CUDA graphic card, please follow the requirements of NVlabs/stylegan3.

The usual installation steps involve the following commands, they should set up the correct CUDA version and all the python packages

conda env create -f environment.yml
conda activate stylegan3

Then install the additional requirements

pip install -r requirements.txt

Otherwise (for GPU acceleration on MacOS with Silicon Mac M1/M2, or just CPU) try the following:

cat environment.yml | \
  grep -v -E 'nvidia|cuda' > environment-no-nvidia.yml && \
    conda env create -f environment-no-nvidia.yml
conda activate stylegan3

# On MacOS
export PYTORCH_ENABLE_MPS_FALLBACK=1

Run Gradio visualizer in Docker

Provided docker image is based on NGC PyTorch repository. To quickly try out visualizer in Docker, run the following: 

# before you build the docker container, make sure you have cloned this repo, and downloaded the pretrained model by `python scripts/download_model.py`.
docker build . -t draggan:latest  
docker run -p 7860:7860 -v "$PWD":/workspace/src -it draggan:latest bash
# (Use GPU)if you want to utilize your Nvidia gpu to accelerate in docker, please add command tag `--gpus all`, like:
#   docker run --gpus all  -p 7860:7860 -v "$PWD":/workspace/src -it draggan:latest bash

cd src && python visualizer_drag_gradio.py --listen

Now you can open a shared link from Gradio (printed in the terminal console).
Beware the Docker image takes about 25GB of disk space!

Download pre-trained StyleGAN2 weights

To download pre-trained weights, simply run:

python scripts/download_model.py

If you want to try StyleGAN-Human and the Landscapes HQ (LHQ) dataset, please download weights from these links: StyleGAN-Human, LHQ, and put them under ./checkpoints.

Feel free to try other pretrained StyleGAN.

Run DragGAN GUI

To start the DragGAN GUI, simply run:

sh scripts/gui.sh

If you are using windows, you can run:

.\scripts\gui.bat

This GUI supports editing GAN-generated images. To edit a real image, you need to first perform GAN inversion using tools like PTI. Then load the new latent code and model weights to the GUI.

You can run DragGAN Gradio demo as well, this is universal for both windows and linux:

python visualizer_drag_gradio.py

Acknowledgement

This code is developed based on StyleGAN3. Part of the code is borrowed from StyleGAN-Human.

(cheers to the community as well)

License

The code related to the DragGAN algorithm is licensed under CC-BY-NC. However, most of this project are available under a separate license terms: all codes used or modified from StyleGAN3 is under the Nvidia Source Code License.

Any form of use and derivative of this code must preserve the watermarking functionality showing "AI Generated".

BibTeX

@inproceedings{pan2023draggan,
    title={Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold},
    author={Pan, Xingang and Tewari, Ayush, and Leimk{\"u}hler, Thomas and Liu, Lingjie and Meka, Abhimitra and Theobalt, Christian},
    booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
    year={2023}
}

DragGAN (XingangPan/DragGAN)

Searching Database…

35.7K ·GitHub
DragGAN
What is this?DragGAN is the official implementation of the SIGGRAPH 2023 paper that lets you interactively drag points on a GAN-generated image to move or reshape objects within it.
Is this a library I import, or a standalone app?It is a standalone research tool with its own GUI and a separate Gradio web demo, built on StyleGAN2/3 models rather than something you import as a library.
How do I get DragGAN running?Install the CUDA-based requirements following the NVlabs/stylegan3 setup, download the pretrained StyleGAN2 weights with the provided script, then launch the DragGAN GUI (or the Gradio demo) from the command line.
What hardware does DragGAN need?A CUDA-capable GPU is expected by default, but there is a separate install path for GPU acceleration on Apple Silicon Macs (M1/M2) or for CPU-only use.
Can I just drag points on any photo?No - the GUI only edits GAN-generated images directly; to edit a real photo you first need to perform GAN inversion (e.g. with PTI) to get a matching latent code.
Does it run on Windows and Mac, or only Linux with an Nvidia GPU?It supports all three: there's a dedicated Windows run command, a Gradio demo that works the same on Windows and Linux, and a CPU/Apple Silicon install path for Macs without CUDA.
Can I use DragGAN commercially?No - the DragGAN algorithm itself is licensed CC-BY-NC (non-commercial), and any use must preserve the 'AI Generated' watermark on outputs; the remaining code mixes in the Nvidia Source Code License.
What pretrained models come with DragGAN?A script downloads pretrained StyleGAN2 weights, and you can additionally fetch StyleGAN-Human and Landscapes HQ (LHQ) weights to try other domains.
Why do outputs need an 'AI Generated' watermark?The license requires any use or derivative of the code to keep the watermarking feature that marks generated images as 'AI Generated'.
Is there a Docker option?Yes, a Docker image based on the NGC PyTorch repository is provided to run the Gradio visualizer, though it takes about 25GB of disk space.
What are similar projects?pytorch-CycleGAN-and-pix2pix, CycleGAN, pix2pix, lama, stylegan2, GFPGAN, Deep-Live-Cam, Bringing-Old-Photos-Back-to-Life
Which topics is it in?image-manipulation, generative-adversarial-network, gan, gradio, image-editing, interactive, image-generation, apple-silicon, artificial-intelligence, nvidia, computer-vision, cuda
Creator
XingangPan
Size
34 MB
Stars
35.7K
Updated
CommitOct 11
Age
3y
Commits · 90 Days