CorridorKey2.0 README

项目的 README,来自 github.com/nikopueringer/CorridorKey2.0。合并为一个 markdown 文件:README.md。

CorridorKey 2.0

Welcome to CorridorKey 2.0! You may rest easy, weary artist. The green beast has been slain. The CorridorKey 2.0 model was designed and trained by Zachary Harris, in direct collaboration with Niko and Corridor Digital.

Check out our official plugin!

The most requested feature we've heard this year- "Can you make a plugin for my favorite software?

...yes! Finally!

Led by Ale Alvaro, we now have a plugin (and stand-alone app) available for almost every media software package. A one-time purchase gets you access to ALL OF THEM. corridorkey.com

...And for those of you that want to experiment, tinker, and build your own python tools, welcome to the CorridorKey 2.0 repo! You may use this model and software freely, for commercial or personal work. See the license for more details, but that's it in a nutshell. Enjoy!

Any business or research inquiries for Zac can be directed to [email protected]

Setup

You need an NVIDIA GPU (currently RTX 30-series or newer) with at least 4 GB of VRAM for footage up to 1440p, or 6 GB for 4K, ffmpeg on your PATH, and a Python environment with CUDA-enabled PyTorch installed. On smaller cards, see Long Clips And Low VRAM. This package was tested with:

torch==2.9.1
torchvision==0.24.1

From the repository root, create an environment, install PyTorch and the remaining dependencies, and download the model weights (about 5 GB):

python -m venv venv
source venv/bin/activate
pip install torch==2.9.1 torchvision==0.24.1 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
python scripts/download_weights.py

The download script fetches:

After that everything loads from those local folders.

Drag-And-Drop Launcher

The easiest way to run CorridorKey is to drag a clip, a frame-sequence folder, or a folder of clips onto the launcher in the repository root:

It walks you through the inference mode, outputs, and GPU memory profile, then runs the job. The launcher uses the venv created during setup if it exists. It also picks up companion hint files next to the input automatically (<clip>FrameHint.png for a first-frame hint, <clip>_AlphaHint.mp4 for a full hint video).

You can also start the same guided session directly:

python run_interactive.py /path/to/clip.mp4

Quick Start

From the repository root:

python infer.py \
  --input /path/to/input.mp4 \
  --output_dir /path/to/output

This runs native-resolution inference with no external hint. By default it writes:

Native resolution is always preserved. The model reflect-pads internally to its required shape quantum and crops back to the original size for output.

Choosing An Inference Mode

CorridorKey can run with no hints, a single first-frame hint, or a full hint video. The right mode depends on what information you have and how much you want the model to decide for itself.

No Hint

python infer.py \
  --input shot.mp4 \
  --output_dir out_no_hint

Use this when no matte or object hint is available. The model decides what the foreground subject is.

Tradeoffs:

First-Frame Hint

python infer.py \
  --input shot.mp4 \
  --output_dir out_frame1_hint \
  --hint_first_frame frame_0001_matte.png \
  --hint_quality 0.95

Use this when you can provide a good matte for the first frame. The hint should be a single-channel matte or an image with an alpha channel: white means subject, black means background.

By default, Hann carry is enabled. This means the model uses its own predicted auxiliary matte from overlapping Hann windows as a continuing hint signal after the first explicit hint.

Tradeoffs:

Full Hint Video

python infer.py \
  --input shot.mp4 \
  --output_dir out_full_hint \
  --hint_video hint_matte_video.mp4 \
  --hint_quality 0.95

Use this when you have a matte, roto pass, garbage matte, or other subject hint for every frame.

Tradeoffs:

Full Hint Video Without Carry

python infer.py \
  --input shot.mp4 \
  --output_dir out_full_hint_no_carry \
  --hint_video hint_matte_video.mp4 \
  --hint_quality 0.95 \
  --no_carry_hint

Use this when the hint video is authoritative and you do not want the model's previous-window predictions to feed back as hints.

Tradeoffs:

Hint Quality

--hint_quality tells the model how much to trust an explicit hint. It does not change the hint image itself; it changes the conditioning value supplied to the model.

Practical values:

For carried Hann hints, use --carry_hint_quality. The default is 0.95.

python infer.py \
  --input shot.mp4 \
  --output_dir out_frame1_hint \
  --hint_first_frame frame_0001_matte.png \
  --hint_quality 0.95 \
  --carry_hint_quality 0.95

If a Hann window has no hints at all, the model receives the no-hint signal and is asked to decide the subject. If a window has at least one hinted frame, unhinted frames in that same window receive quality zero, meaning "there is a hinted subject in this window, but this frame does not have its own explicit hint."

Hann Windows

Stage 1 runs on overlapping temporal windows and combines its hidden states with a Hann-style taper. Stage 2 then streams the final full-resolution predictions from those combined hidden states.

By default the window size is chosen automatically for each clip from the free memory on your GPU and the clip's resolution, using the largest window that fits (from 80/40 down to a single frame). The chosen values are printed at the start of the run and saved in the settings log. To set them yourself:

--hann_chunk 80 --hann_stride 40

Passing --hann_chunk or --low_vram turns automatic selection off.

What the settings mean:

Example:

python infer.py \
  --input shot.mp4 \
  --output_dir out_window_8 \
  --hann_chunk 8 \
  --hann_stride 4

Outputs

Default outputs are alpha and fg.

Available output choices:

raw_fg and foreground are accepted aliases for fg.

Select outputs explicitly:

python infer.py \
  --input shot.mp4 \
  --output_dir out_review \
  --outputs alpha fg checker cutout

Performance note: each additional output requires more GPU rendering work and more ffmpeg encoding. For fastest normal use, keep the default alpha fg. Add checker or cutout for review renders.

Despill Strength

Use --despill_strength (value from 0.0 for no despill to 1.0 for full despill, defaults to 0.5) to configure the despill amount applied to the checker and cutout outputs.

python infer.py \
  --input shot.mp4 \
  --output_dir out_despill \
  --outputs checker cutout \
  --despill_strength 0.75

Cutout Linear

By default, the straight color for the cutout output is in sRGB color space. If you want to export the cutout as straight scene-linear RGB instead of sRGB (common for linear VFX compositing pipelines), pass:

--cutout_linear

--stage2_batch controls how many final-resolution frames are run and rendered together in stage 2. The default is 1, which uses the least memory and was no slower than larger batches in our tests.

HDR EXR Output

By default, all outputs are ffmpeg-encoded preview videos. That path is display-referred: values are clipped to [0, 1] and quantized for video encoding (10-bit yuv444p10le x264 for alpha/fg/checker, ProRes 4444 for cutout). HDR values above 1.0 do not survive the video path.

For full HDR float output, pass:

--output_format exr

Each selected output is then written as an OpenEXR image sequence in its own subdirectory of the output directory (e.g. <clip>_fg_exr/fg_000000.exr), instead of a video file:

EXR pixel type defaults to half (float16, the VFX interchange standard). Half cannot represent values above 65504, so half output is clamped to the finite float16 range to avoid inf pixels. For extreme HDR values or full float32 files, pass:

--exr_dtype float

Effective precision note: the network itself runs in bfloat16 (in the asinh HDR domain), which bounds output accuracy to roughly 1–2% relative error through most of the HDR range regardless of the EXR pixel type. half adds at most ~0.05% on top of that; float adds none.

Typical HDR round trip from linear EXR plates:

python infer.py \
  --input /path/to/linear_exr_frames \
  --frame_dir_linear \
  --frame_dir_fps 24 \
  --output_format exr \
  --outputs alpha fg cutout \
  --output_dir out_hdr

Note that --frame_dir_linear matters on the input side: without it, EXR frames are treated as display-coded and clipped to [0, 1] before the model sees them. The runtime prints a warning if EXR input is read without the flag. The ffmpeg encoding options below only apply to --output_format video.

Video Encoding

Outputs are written through ffmpeg. Defaults:

--ffmpeg_codec libx264 --crf 12 --preset medium --pix_fmt yuv444p10le

Bit depth in the video path: frames are rendered in fp32 and piped to ffmpeg as 16-bit raw video. The x264 preview outputs default to 10-bit 4:4:4 (yuv444p10le); pass --pix_fmt yuv444p if a playback target needs 8-bit H.264. The cutout output is always ProRes 4444 with alpha, encoded from 10-bit input (decodes as 12-bit yuva444p12le). Video containers remain display-referred and clipped to [0, 1] — use --output_format exr for unclipped HDR.

Override these as needed for your pipeline. For example:

python infer.py \
  --input shot.mp4 \
  --output_dir out_encoded \
  --ffmpeg_codec libx264 \
  --crf 10 \
  --preset slow \
  --pix_fmt yuv444p

Long Clips And Low VRAM

You normally do not need to set anything here: with no memory flags, CorridorKey measures the free memory on your GPU and picks the Hann window and low-VRAM mode for each clip. As a guide, this is roughly what it selects (window as chunk/stride):

GPU memory 720p 1080p 1440p 4K
4 GB 4/2 low-VRAM 2/1 low-VRAM 1/1 low-VRAM not enough memory
6 GB 24/12 low-VRAM 8/4 low-VRAM 4/2 low-VRAM 2/1 low-VRAM
8 GB 24/12 low-VRAM 8/4 low-VRAM 8/4 low-VRAM 4/2 low-VRAM
12 GB 80/40 low-VRAM 24/12 low-VRAM 8/4 8/4 low-VRAM
16 GB 80/40 48/24 low-VRAM 24/12 low-VRAM 8/4 low-VRAM
24 GB 80/40 48/24 48/24 low-VRAM 8/4
32 GB 80/40 80/40 48/24 24/12
48 GB 80/40 80/40 80/40 48/24 low-VRAM

On small cards it goes down to windows of only a few frames, or a single frame, which gives up temporal context in stage 1 to save memory. Results remain good, with somewhat less frame-to-frame consistency than larger windows.

These come from measurements on synthetic test clips; other software using the GPU reduces what is available. If you hit a CUDA out-of-memory error, pass --low_vram with a smaller --hann_chunk yourself.

The normal path streams video and does not intentionally hold an entire long clip in RAM. To force low-VRAM mode yourself, add:

python infer.py \
  --input long_shot.mp4 \
  --output_dir out_low_vram \
  --low_vram \
  --temp_dir /path/with/free/space

Low-VRAM mode is exact staged inference:

Tradeoffs:

Advanced memory knobs:

Use the defaults unless you are actively tuning memory behavior.

Frame Directories

--input can be a video file or a directory of frames. Frame directories are read in sorted filename order. Supported extensions are EXR, PNG, JPG, and JPEG.

python infer.py \
  --input /path/to/frames \
  --frame_dir_fps 24 \
  --output_dir out_frames

Frame directories are treated as display-coded by default. If the incoming image sequence is scene-linear RGB (e.g. for linear EXR sequences), pass:

--frame_dir_linear

Example:

python infer.py \
  --input /path/to/linear_exr_frames \
  --frame_dir_fps 24 \
  --frame_dir_linear \
  --output_dir out_exr

Partial Clips

For quick tests, process only part of a clip:

python infer.py \
  --input shot.mp4 \
  --output_dir out_test \
  --num_frames 100 \
  --start_mode middle

--start_mode choices:

Input Color Handling

The runtime does not inspect or convert color metadata. It uses a simple, explicit input contract:

For display-coded inputs, the runtime converts RGB to an internal scene-linear working value with the package gamma function, then applies the model's signed asinh HDR encoding. Model outputs are decoded back through the inverse path for video previews.

For --frame_dir_linear, the incoming RGB values are passed as scene-linear values directly into the model's signed asinh encoding. This is the path to use for unclipped linear image sequence workflows (e.g. EXR sequences).

The foundation models are different: C-RADIO, MoGe, and RVM receive SDR-style RGB proxies because those models expect normal image-like inputs. They do not receive unclipped HDR values.

What this runtime does not currently do:

Practical guidance:

Brightness And Contrast Controls

Leave these controls at default for standard inference:

--linear_brightness 1.0
--linear_contrast 1.0
--linear_contrast_pivot 0.18

The brightness and contrast options apply an experimental linear-light adjustment after display-coded inputs have been converted to linear light and before the model's asinh encoding. The inverse adjustment is applied to the predicted foreground before writing preview outputs. These controls are useful for controlled tests, not for ordinary production use.

Failure Behavior

The runtime intentionally does not silently switch modes. Missing weights, missing foundation assets, missing ffmpeg, unsupported inputs, and load mismatches raise errors instead of falling back to a different behavior.

Common Recipes

No hint, fastest standard outputs:

python infer.py --input shot.mp4 --output_dir out_no_hint

First-frame hint with default carry:

python infer.py \
  --input shot.mp4 \
  --output_dir out_frame1 \
  --hint_first_frame matte_frame_0001.png \
  --hint_quality 0.95

Full hint video, trusting the provided hint on every pass:

python infer.py \
  --input shot.mp4 \
  --output_dir out_full_hint \
  --hint_video hint.mp4 \
  --hint_quality 0.95 \
  --no_carry_hint

Review render with checker and cutout:

python infer.py \
  --input shot.mp4 \
  --output_dir out_review \
  --hint_first_frame matte_frame_0001.png \
  --outputs alpha fg checker cutout

Lower-memory 8-frame windows:

python infer.py \
  --input shot.mp4 \
  --output_dir out_lowmem_window \
  --hint_first_frame matte_frame_0001.png \
  --hann_chunk 8 \
  --hann_stride 4

Licensing and Permissions

Use CorridorKey for whatever you'd like, including processing footage for commercial projects. What you make with it is yours.

The full terms are in LICENSE. The license is a variation of CC BY-NC-SA 4.0 with additional terms.

Acknowledgements

CorridorKey builds on three foundation models. Please go show their creators some support:

Details are in THIRD_PARTY_NOTICES.md.

CorridorKey2.0 (nikopueringer/CorridorKey2.0)

正在搜索数据库…

这是什么?CorridorKey 2.0 是一个机器学习视频抠像模型和 Python 工具包,可以去除绿幕和蓝幕素材的背景,用 AI 驱动的抠像器取代传统的基于颜色的色键抠像。
这是我剪辑软件的插件,还是我自己运行的独立工具?这个仓库提供的是原始的 Python 模型和命令行工具,适合想自行开发工具的人;另有一款付费的独立插件或应用,覆盖大多数媒体软件,可在 corridorkey.com 获取,适合只想直接集成的用户。
我要怎样才能让它运行起来?创建 Python 环境,安装支持 CUDA 的 PyTorch 和其余依赖,然后运行下载脚本获取模型权重(约 5GB),之后再处理视频片段。
开始之前需要什么硬件?一块 NVIDIA RTX 30 系列或更新的显卡,显存至少 4GB(处理 4K 需要 6GB),PATH 中需有 ffmpeg,以及一个安装了支持 CUDA 的 PyTorch 的 Python 环境。
它总能自动选对要抠出的主体吗?如果没有提示,它会自行判断前景主体,但当画面中同时存在多个可能的主体时,判断可能出错;提供首帧或完整的提示视频,可以让你更好地掌控结果。
设置完成后,运行时还需要联网吗?不需要。模型权重和基础模型只需下载一次,之后所有内容都从本地磁盘上的文件夹加载并运行。
我可以把它用于商业项目吗?可以。根据仓库的许可条款,模型和软件可免费用于商业和个人工作。
CorridorKey 2.0 基于哪些基础模型?它基于三个冻结的预训练基础模型:NVIDIA 的 C-RADIOv4-SO400M、Microsoft 的 MoGe-2 ViT-B,以及 Google DeepMind 的 RVM small。
可以使用哪些提示模式来引导抠像?可以不加提示运行(由模型决定主体),也可以使用单个首帧蒙版提示(模型会向后续帧跟踪),或使用完整的逐帧提示视频以获得最大控制;提示的可信度通过 --hintquality 调整。
能为 VFX 合成流程输出 HDR 吗?可以。传入 --outputformat exr 即可输出未被截断的场景线性 OpenEXR 序列(half 或 float32 格式),而不是默认的截断预览视频。
它会输出哪些类型的结果?默认输出 Alpha 蒙版和预测的前景 RGB 预览(fg);你也可以启用棋盘格合成(checker)或直通色抠像,用于审阅渲染。
大小
0.2 MB
星标
2.2K
更新
提交Oct 15
时长
4个月
提交 · 90 天