Anime4K README

The project's README from github.com/bloc97/Anime4K. As one markdown file: README.md.

Anime4K

Anime4K is a set of open-source, high-quality real-time anime upscaling/denoising algorithms that can be implemented in any programming language.

The simplicity and speed of Anime4K allows the user to watch upscaled anime in real time, as we believe in preserving original content and promoting freedom of choice for all anime fans. Re-encoding anime into 4K should be avoided as it is non-reversible, potentially damages original content by introducing artifacts, takes up to O(n2) more disk space and more importantly, does so without any meaningful decrease in entropy (lost information is lost).

Disclaimer: All art assets used are for demonstration and educational purposes. All rights are reserved to their original owners. If you (as a person or a company) own the art and do not wish it to be associated with this project, please contact us at [email protected] and we will gladly take it down.

Foreword

Anime4K is optimized for native 1080p anime encoded with h.264, h.265 or VC-1.

Even if it might work, it is not optimized for downscaled 720p, 480p or standard definition anime (eg. DVDs). Older anime (especially pre-digital era production) have artifacts that are very difficult to remove, such as bad deinterlacing, camera blur during production, severe ringing, film grain, older MPEG compression artifacts, etc.

This is also not replacement for SRGANs, as they perform much better on low-resolution images or images with lots of degradation (albeit not in real time).

What Anime4K does provide is a way to upscale, in real time, 1080p anime for 4K screens while providing a similar effect to SRGANs and being much better than waifu2x (See comparisons).

Currently, research is being done on better real-time upscaling for lower resolution or older content.

Installation Instructions

Windows

Linux

Mac

v4.1 Low resolution experiment

Results from the experimental SRGAN shaders for 360p -> 4K: (zoom in to view details)

The images are sorted by algorithm speed, bicubic being the fastest. FSRCNNX and Anime4K are real-time while waifu2x and Real-ESRGAN are not. Comparison Comparison

v4

We introduce a line reconstruction algorithm that aims to tackle the distribution shift problem seen in 1080p anime. In the wild anime exhibit a surprising amount of variance caused by low quality compositing due to budget and time constraints that traditional super-resolution algorithms cannot handle. GANs can implicitly encode this distribution shift but are slow to use and hard to train. Our algorithm explicitly corrects this distribution shift and allows traditional "MSE" SR algorithms to work with a wide variety of anime.

Source: https://fancaps.net/anime/picture.php?/14728493 | Mode: B
Comparison

Source: https://fancaps.net/anime/picture.php?/13365760 | Mode: A
Comparison

Performance numbers are obtained using a Vega64 GPU and are tested using UL shader variants. The fast version is for M variants.
Note that CUDA accelerated SRGANs/Waifu2x using tensor cores can be much faster and close to realtime (~80ms), but their large size severely hampers non-CUDA implementations.

v3

The monolithic Anime4K shader is broken into modular components, allowing customization for specific types of anime and/or personal taste. What's new:

More information about each shader (OUTDATED).

Visits

Counting since 2021-09-19T16:02:06Z (ISO 8601)

Projects that use Anime4K

Note that the following might be using an outdated version of Anime4K. There have been significant quality improvements since v3.

Acknowledgements

OpenCV TensorFlow Keras Torch mpv MPC
OpenCV TensorFlow Keras Torch mpv MPC

Many thanks to the OpenCV, TensorFlow, Keras and Torch groups and contributors. This project would not have been possible without the existence of high quality, open source machine learning libraries.

I would also want to specially thank the creators of VDSR and FSRCNN, in addition to the open source projects waifu2x and FSRCNNX for sparking my interest in creating this project. I am also extending my gratitude to the contributors of mpv and MPC-HC/BE for their efforts on creating excellent media players with endless customization options.
Furthermore, I want to thank the people who contributed to this project in any form, be it by reporting bugs, submitting suggestions, helping others' issues or submitting code. I will forever hold you in high regard.

I also wish to express my sincere gratitude to the people of Université de Montréal, DIRO, LIGUM and MILA for providing so many opportunities to students (including me), providing the necessary infrastructure and fostering an excellent learning environment.

I would also like to thank the greater open source community, in which the assortment of concrete examples and code were of great help.

Finally, but not least, infinite thanks to my family, friends and professors for providing financial, technical, social support and expertise for my ongoing learning journey during these hard times. Your help has been beyond description, really.

This list is not final, as the project is far from done. Any future acknowledgements will be promptly added.

Anime4K (bloc97/Anime4K)

Searching Database…

21.5K ·GitHub
What is this?Anime4K is a set of open-source, real-time upscaling and denoising algorithms, implemented as GLSL shaders, that sharpen and upscale anime video on the fly inside your media player instead of permanently re-encoding the file.
Is this a standalone app or something I load into my player?It is not a standalone app by itself — it is a library of GLSL shaders you load into a shader-capable media player (mpv, IINA, Plex), though several community projects wrap it into ready-made GUI applications.
How do I get it running?Follow the installation guide for your platform: on Windows and Linux you load the shaders via GLSL/MPV or GLSL/Plex, and on Mac you additionally have GLSL/IINA; each guide walks through pointing your player's shader pipeline at the Anime4K shader files.
Do I need a specific media player instead of just any video player?Yes — since Anime4K is implemented as GLSL shaders, you need a player with GLSL shader support, such as mpv, IINA, or Plex; the README provides separate setup guides for each.
Does it work well on any anime video, or are there catches?It is tuned for native 1080p anime encoded with h.264, h.265, or VC-1; it is not optimized for downscaled 720p/480p or old standard-definition sources, and very old anime with heavy compression or blur artifacts are hard to clean up.
Do I need a powerful GPU to run this in real time?Yes, real-time shader upscaling is GPU-bound; the project's benchmarks were measured on an AMD Vega64, and it notes CUDA tensor-core implementations can be even faster but aren't available outside CUDA.
How does this compare to waifu2x or other AI upscalers like Real-ESRGAN?Anime4K is designed to run in real time during playback, while GAN-based upscalers like waifu2x and Real-ESRGAN produce better results on heavily degraded or low-resolution images but are far too slow to use live.
What changed between the v3 and v4 shader versions?v3 split the monolithic shader into modular line-art CNN upscalers, deblurring, denoising, and resampling-artifact shaders; v4 added a line reconstruction algorithm that explicitly corrects the distribution shift seen in real-world, inconsistently composited 1080p anime.
Is there a simpler app instead of manually configuring shaders?Yes, the community has built several ready-made tools on top of Anime4K, including a Windows 10/11 GUI upscaler, a Metal-based app for Apple platforms, and reimplementations in PyTorch, C#, Rust, and C++.
Creator
bloc97
Size
804.8 MB
Stars
21.5K
Age
7y
Commits · 90 Days