18.2K ·GitHub
What is this?
An AI face-restoration model and tool from a NeurIPS 2022 paper by S-Lab, Nanyang Technological University: it rebuilds clear, realistic faces from blurry, low-quality, or old photos using a codebook-lookup transformer.
An AI face-restoration model and tool from a NeurIPS 2022 paper by S-Lab, Nanyang Technological University: it rebuilds clear, realistic faces from blurry, low-quality, or old photos using a codebook-lookup transformer.Is it a ready app, or something I run from code?It's a Python/PyTorch research codebase you run from the command line, not a packaged app. If you don't want to install anything, there are official online demos, plus integrations in Stable Diffusion WebUI, ComfyUI, and ChaiNNer.
It's a Python/PyTorch research codebase you run from the command line, not a packaged app. If you don't want to install anything, there are official online demos, plus integrations in Stable Diffusion WebUI, ComfyUI, and ChaiNNer.How do I get it running?Install PyTorch >= 1.7.1 plus the requirements.txt packages, fetch the pretrained models into the weights/ folders, drop images into inputs/TestWhole, and run the inference script — results are saved to the results folder.
Install PyTorch >= 1.7.1 plus the requirements.txt packages, fetch the pretrained models into the weights/ folders, drop images into inputs/TestWhole, and run the inference script — results are saved to the results folder.The face looks great but less like the person — can I tune that trade-off?Yes, via the fidelity weight w in [0, 1]: smaller w gives a higher-quality result, larger w keeps the output closer to the original identity.
Yes, via the fidelity weight w in [0, 1]: smaller w gives a higher-quality result, larger w keeps the output closer to the original identity.Do I need an NVIDIA GPU?Yes — the README requires PyTorch >= 1.7.1 with CUDA >= 10.1, so plan on an NVIDIA GPU; otherwise use one of the hosted online demos.
Yes — the README requires PyTorch >= 1.7.1 with CUDA >= 10.1, so plan on an NVIDIA GPU; otherwise use one of the hosted online demos.How does the 'Codebook Lookup' part work?It's the idea in the paper title: the model rebuilds degraded faces by looking up plausible high-quality face fragments from a learned discrete codebook (VQGAN-style quantization), which acts as a strong prior for realistic detail.
It's the idea in the paper title: the model rebuilds degraded faces by looking up plausible high-quality face fragments from a learned discrete codebook (VQGAN-style quantization), which acts as a strong prior for realistic detail.Does it enhance the background too, or only the face?Yes — for whole images it adopts Real-ESRGAN to enhance the background and fuses it with the restored face.
Yes — for whole images it adopts Real-ESRGAN to enhance the background and fuses it with the restored face.What is CodeFormer built on?Its training framework is BasicSR, and parts of the code come from Unleashing Transformers, YOLOv5-face, and FaceXLib.
Its training framework is BasicSR, and parts of the code come from Unleashing Transformers, YOLOv5-face, and FaceXLib.Which face detector options does it have?Besides the default, dlib was added as a detector option that the authors note produces more accurate face identity.
Besides the default, dlib was added as a detector option that the authors note produces more accurate face identity.What are similar projects?GFPGAN, ComfyUI-VOSR2, IOPaint, Real-ESRGAN, Bringing-Old-Photos-Back-to-Life, neural-enhance, flux, lama
GFPGAN, ComfyUI-VOSR2, IOPaint, Real-ESRGAN, Bringing-Old-Photos-Back-to-Life, neural-enhance, flux, lamaWhich topics is it in?face-restoration, image-restoration, inpainting, face-detection, super-resolution, transformer, image-processing, video, huggingface, computer-vision, cuda, pytorch
face-restoration, image-restoration, inpainting, face-detection, super-resolution, transformer, image-processing, video, huggingface, computer-vision, cuda, pytorch