waifu2x README

The project's README from github.com/nagadomi/waifu2x. As one markdown file: README.md.

waifu2x

Image Super-Resolution for Anime-style art using Deep Convolutional Neural Networks. And it supports photo.

The demo application can be found at https://waifu2x.udp.jp/ (Cloud version), https://unlimited.waifu2x.net/ (In-Browser version).

2023/02 PyTorch version

nunif

waifu2x development has already been moved to the repository above.

Summary

Click to see the slide show.

slide

References

waifu2x is inspired by SRCNN [1]. 2D character picture (HatsuneMiku) is licensed under CC BY-NC by piapro [2].

Public AMI

TODO

Third Party Software

Third-Party

If you are a windows user, I recommend you to use waifu2x-caffe(Just download from releases tab), waifu2x-ncnn-vulkan or waifu2x-conver-cpp.

Dependencies

Hardware

Platform

LuaRocks packages (excludes torch7's default packages)

Installation

Setting Up the Command Line Tool Environment

(on Ubuntu 16.04)

Install CUDA

See: NVIDIA CUDA Getting Started Guide for Linux

Download CUDA

sudo dpkg -i cuda-repo-ubuntu1404_7.5-18_amd64.deb
sudo apt-get update
sudo apt-get install cuda

Install Package

sudo apt-get install libsnappy-dev
sudo apt-get install libgraphicsmagick1-dev
sudo apt-get install libssl1.0-dev # for web server

Note: waifu2x requires little-cms2 linked graphicsmagick. if you use macOS/homebrew, See #174.

Install Torch7

See: Getting started with Torch.

Getting waifu2x

git clone --depth 1 https://github.com/nagadomi/waifu2x.git

and install lua modules.

cd waifu2x
./install_lua_modules.sh

Validation

Testing the waifu2x command line tool.

th waifu2x.lua

Web Application

th web.lua

View at: http://localhost:8812/

Command line tools

Notes: If you have cuDNN library, than you can use cuDNN with -force_cudnn 1 option. cuDNN is too much faster than default kernel. If you got GPU out of memory error, you can avoid it with -crop_size option (e.g. -crop_size 128).

Noise Reduction

th waifu2x.lua -m noise -noise_level 1 -i input_image.png -o output_image.png

th waifu2x.lua -m noise -noise_level 0 -i input_image.png -o output_image.png
th waifu2x.lua -m noise -noise_level 2 -i input_image.png -o output_image.png
th waifu2x.lua -m noise -noise_level 3 -i input_image.png -o output_image.png

2x Upscaling

th waifu2x.lua -m scale -i input_image.png -o output_image.png

Noise Reduction + 2x Upscaling

th waifu2x.lua -m noise_scale -noise_level 1 -i input_image.png -o output_image.png

th waifu2x.lua -m noise_scale -noise_level 0 -i input_image.png -o output_image.png
th waifu2x.lua -m noise_scale -noise_level 2 -i input_image.png -o output_image.png
th waifu2x.lua -m noise_scale -noise_level 3 -i input_image.png -o output_image.png

Batch conversion

find /path/to/imagedir -name "*.png" -o -name "*.jpg" > image_list.txt
th waifu2x.lua -m scale -l ./image_list.txt -o /path/to/outputdir/prefix_%d.png

The output format supports %s and %d(e.g. %06d). %s will be replaced the basename of the source filename. %d will be replaced a sequence number. For example, when input filename is piyo.png, %s_%03d.png will be replaced piyo_001.png.

See also th waifu2x.lua -h.

Using photo model

Please add -model_dir models/photo to command line option, if you want to use photo model. For example,

th waifu2x.lua -model_dir models/photo -m scale -i input_image.png -o output_image.png

Video Encoding

* avconv is alias of ffmpeg on Ubuntu 14.04.

Extracting images and audio from a video. (range: 00:09:00 ~ 00:12:00)

mkdir frames
avconv -i data/raw.avi -ss 00:09:00 -t 00:03:00 -r 24 -f image2 frames/%06d.png
avconv -i data/raw.avi -ss 00:09:00 -t 00:03:00 audio.mp3

Generating a image list.

find ./frames -name "*.png" |sort > data/frame.txt

waifu2x (for example, noise reduction)

mkdir new_frames
th waifu2x.lua -m noise -noise_level 1 -resume 1 -l data/frame.txt -o new_frames/%d.png

Generating a video from waifu2xed images and audio.

avconv -f image2 -framerate 24 -i new_frames/%d.png -i audio.mp3 -r 24 -vcodec libx264 -crf 16 video.mp4

Train Your Own Model

Note1: If you have cuDNN library, you can use cudnn kernel with -backend cudnn option. And, you can convert trained cudnn model to cunn model with tools/rebuild.lua.

Note2: The command that was used to train for waifu2x's pretrained models is available at appendix/train_upconv_7_art.sh, appendix/train_upconv_7_photo.sh. Maybe it is helpful.

Data Preparation

Genrating a file list.

find /path/to/image/dir -name "*.png" > data/image_list.txt

You should use noise free images. In my case, waifu2x is trained with 6000 high-resolution-noise-free-PNG images.

Converting training data.

th convert_data.lua

Train a Noise Reduction(level1) model

mkdir models/my_model
th train.lua -model_dir models/my_model -method noise -noise_level 1 -test images/miku_noisy.png
# usage
th waifu2x.lua -model_dir models/my_model -m noise -noise_level 1 -i images/miku_noisy.png -o output.png

You can check the performance of model with models/my_model/noise1_best.png.

Train a Noise Reduction(level2) model

th train.lua -model_dir models/my_model -method noise -noise_level 2 -test images/miku_noisy.png
# usage
th waifu2x.lua -model_dir models/my_model -m noise -noise_level 2 -i images/miku_noisy.png -o output.png

You can check the performance of model with models/my_model/noise2_best.png.

Train a 2x UpScaling model

th train.lua -model upconv_7 -model_dir models/my_model -method scale -scale 2 -test images/miku_small.png
# usage
th waifu2x.lua -model_dir models/my_model -m scale -scale 2 -i images/miku_small.png -o output.png

You can check the performance of model with models/my_model/scale2.0x_best.png.

Train a 2x and noise reduction fusion model

th train.lua -model upconv_7 -model_dir models/my_model -method noise_scale -scale 2 -noise_level 1 -test images/miku_small.png
# usage
th waifu2x.lua -model_dir models/my_model -m noise_scale -scale 2 -noise_level 1 -i images/miku_small.png -o output.png

You can check the performance of model with models/my_model/noise1_scale2.0x_best.png.

Docker

( Docker image is available at https://hub.docker.com/r/nagadomi/waifu2x )

Requires nvidia-docker.

docker build -t waifu2x .
docker run --gpus all -p 8812:8812 waifu2x th web.lua
docker run --gpus all -v `pwd`/images:/images waifu2x th waifu2x.lua -force_cudnn 1 -m scale -scale 2 -i /images/miku_small.png -o /images/output.png

Note that running waifu2x in without JIT caching is very slow, which is what would happen if you use docker. For a workaround, you can mount a host volume to the CUDA_CACHE_PATH, for instance,

docker run --gpus all -v $PWD/ComputeCache:/root/.nv/ComputeCache waifu2x th waifu2x.lua --help

waifu2x (nagadomi/waifu2x)

anime
Searching Database…

28.2K ·GitHub
Waifu2x
What is this?waifu2x is a command-line tool that upscales anime-style art 2x and removes noise using deep convolutional neural networks — it also supports photos with a separate model.
Is it a GUI app or something I run from the terminal?It is a command-line tool built on Torch7 — you run it as 'th waifu2x.lua' with options like -cropsize or -modeldir. If you just want to try it, the README links cloud and in-browser demo versions.
How do I get it running on Linux?The README walks through an Ubuntu 16.04 setup: install NVIDIA CUDA, install Torch7 and the LuaRocks packages (lua-csnappy, md5, uuid, csvigo, turbo), clone the repo, then run a validation test.
Do I need a specific GPU for this?Yes — an NVIDIA GPU with CUDA is a listed hardware requirement. With the cuDNN library you can add -forcecudnn 1 for a big speedup, and -cropsize (e.g. 128) helps if you hit out-of-memory errors.
Does it run natively on Windows?No — the author tells Windows users to use ready-made alternatives instead: waifu2x-caffe (download from its releases tab), waifu2x-ncnn-vulkan or waifu2x-convert-cpp.
What can it actually do to an image?Three modes: noise reduction, 2x upscaling, or both combined, plus batch conversion where %s and %d in the output name become the source basename and a sequence number.
Can I use it on real photos, not just anime art?Yes — switch to the photo model by adding -modeldir models/photo to the command.
Can it upscale video?Not in one command, but the README shows an ffmpeg workflow: extract frames and audio from the video, run waifu2x over the images, then generate the video back from the upscaled frames and audio.
Can I train my own model?Yes — training commands are included (appendix/trainupconv7art.sh and trainupconv7photo.sh) for noise-reduction and 2x upscaling models. The author trained on about 6,000 high-resolution, noise-free PNG images.
Is there a Docker image?Yes, but it requires nvidia-docker, and without JIT caching it runs very slowly — the workaround is mounting a host volume to the CUDA cache path.
Is this repo still being developed?Not really — as of February 2023 development moved to the PyTorch-based 'nunif' repository, so this Torch7 version is essentially in maintenance mode.
What are similar projects?Waifu2x-Extension-GUI, Real-ESRGAN, Anime4K, ALH-Pro, ComfyUI-VOSR2, neural-enhance, sentrysearch, CorridorKey
Which topics is it in?torch, anime, super-resolution, upscaling, lua, video-processing, neural-network, ffmpeg, image-processing, computer-vision, cuda, command-line-tool
Creator
nagadomi
Size
692.9 MB
Stars
28.2K
Updated
CommitFeb 23Releasev0.13.2Nov 18
Age
11y
Commits · 90 Days