DragGAN README

The project's README from github.com/XingangPan/DragGAN. As one markdown file: README.md.

Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

Xingang Pan · Ayush Tewari · Thomas Leimkühler · Lingjie Liu · Abhimitra Meka · Christian Theobalt

SIGGRAPH 2023 Conference Proceedings


PyTorch Twitter Paper PDF Project Page Open In Colab

Web Demos

Open in OpenXLab

Huggingface

Requirements

If you have CUDA graphic card, please follow the requirements of NVlabs/stylegan3.

The usual installation steps involve the following commands, they should set up the correct CUDA version and all the python packages

conda env create -f environment.yml
conda activate stylegan3

Then install the additional requirements

pip install -r requirements.txt

Otherwise (for GPU acceleration on MacOS with Silicon Mac M1/M2, or just CPU) try the following:

cat environment.yml | \
  grep -v -E 'nvidia|cuda' > environment-no-nvidia.yml && \
    conda env create -f environment-no-nvidia.yml
conda activate stylegan3

# On MacOS
export PYTORCH_ENABLE_MPS_FALLBACK=1

Run Gradio visualizer in Docker

Provided docker image is based on NGC PyTorch repository. To quickly try out visualizer in Docker, run the following: 

# before you build the docker container, make sure you have cloned this repo, and downloaded the pretrained model by `python scripts/download_model.py`.
docker build . -t draggan:latest  
docker run -p 7860:7860 -v "$PWD":/workspace/src -it draggan:latest bash
# (Use GPU)if you want to utilize your Nvidia gpu to accelerate in docker, please add command tag `--gpus all`, like:
#   docker run --gpus all  -p 7860:7860 -v "$PWD":/workspace/src -it draggan:latest bash

cd src && python visualizer_drag_gradio.py --listen

Now you can open a shared link from Gradio (printed in the terminal console).
Beware the Docker image takes about 25GB of disk space!

Download pre-trained StyleGAN2 weights

To download pre-trained weights, simply run:

python scripts/download_model.py

If you want to try StyleGAN-Human and the Landscapes HQ (LHQ) dataset, please download weights from these links: StyleGAN-Human, LHQ, and put them under ./checkpoints.

Feel free to try other pretrained StyleGAN.

Run DragGAN GUI

To start the DragGAN GUI, simply run:

sh scripts/gui.sh

If you are using windows, you can run:

.\scripts\gui.bat

This GUI supports editing GAN-generated images. To edit a real image, you need to first perform GAN inversion using tools like PTI. Then load the new latent code and model weights to the GUI.

You can run DragGAN Gradio demo as well, this is universal for both windows and linux:

python visualizer_drag_gradio.py

Acknowledgement

This code is developed based on StyleGAN3. Part of the code is borrowed from StyleGAN-Human.

(cheers to the community as well)

License

The code related to the DragGAN algorithm is licensed under CC-BY-NC. However, most of this project are available under a separate license terms: all codes used or modified from StyleGAN3 is under the Nvidia Source Code License.

Any form of use and derivative of this code must preserve the watermarking functionality showing "AI Generated".

BibTeX

@inproceedings{pan2023draggan,
    title={Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold},
    author={Pan, Xingang and Tewari, Ayush, and Leimk{\"u}hler, Thomas and Liu, Lingjie and Meka, Abhimitra and Theobalt, Christian},
    booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
    year={2023}
}

DragGAN (XingangPan/DragGAN)

Searching Database…

35.7K ·GitHub
DragGAN
这是什么？DragGAN 是 SIGGRAPH 2023 论文的官方实现，你可以交互式地拖动 GAN 生成图像上的点，从而移动或改变图中的物体形状。
这是我可以导入的库，还是独立应用？它是一个独立的研究工具，自带图形界面，还有一个单独的 Gradio 网页演示，基于 StyleGAN2/3 模型构建，并不是可以导入使用的库。
怎样才能运行 DragGAN？按照 NVlabs/stylegan3 的说明安装基于 CUDA 的依赖，用提供的脚本下载预训练的 StyleGAN2 权重，然后在命令行中启动 DragGAN 图形界面（或 Gradio 演示）。
DragGAN 需要什么硬件？默认需要支持 CUDA 的 GPU；如果使用 Apple Silicon 芯片的 Mac（M1/M2），或者只用 CPU，则有单独的安装方式。
我能直接拖动任意照片上的点吗？不能。图形界面只能直接编辑 GAN 生成的图像；要编辑真实照片，你需要先做 GAN 反演（例如使用 PTI），得到匹配的潜在编码。
它能在 Windows 和 Mac 上运行，还是只能在装有 Nvidia 显卡的 Linux 上运行？三个系统都支持：有专门的 Windows 运行命令；Gradio 演示在 Windows 和 Linux 上的用法相同；没有 CUDA 的 Mac 可以使用 CPU 或 Apple Silicon 的安装方式。
我能把 DragGAN 用于商业用途吗？不能。DragGAN 算法本身采用 CC-BY-NC（仅限非商业用途）许可，任何使用都必须保留输出结果上的 'AI Generated' 水印；其余代码混用了 Nvidia 源代码许可证。
DragGAN 自带哪些预训练模型？有一个脚本可以下载预训练的 StyleGAN2 权重，你还可以另外下载 StyleGAN-Human 和 Landscapes HQ（LHQ）的权重，尝试其他领域。
为什么输出结果需要 'AI Generated' 水印？许可证要求，任何使用或衍生该代码的行为都必须保留将生成图像标记为 'AI Generated' 的水印功能。
有 Docker 方案吗？有。提供了基于 NGC PyTorch 仓库的 Docker 镜像，用于运行 Gradio 可视化工具，但大约需要 25GB 的磁盘空间。
What are similar projects?pytorch-CycleGAN-and-pix2pix, CycleGAN, pix2pix, lama, stylegan2, GFPGAN, Deep-Live-Cam, Bringing-Old-Photos-Back-to-Life
Which topics is it in?image-manipulation, generative-adversarial-network, gan, gradio, image-editing, interactive, image-generation, apple-silicon, artificial-intelligence, nvidia, computer-vision, cuda
Creator
XingangPan
Size
34 MB
Stars
35.7K
Updated
CommitOct 11
Age
3y
Commits · 90 Days