High-Angle_Robust_Fast_FaceAlignment README

The project's README from github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment. As one markdown file: README.md.

HRFFA: High-Angle Robust Fast FaceAlignment

GitHub DOI

A training, distillation and ONNX deployment pipeline for whole-head face alignment that stays robust at extreme head poses while remaining light enough for CPU inference.

teacher clean_v3: extreme pitch/yaw tiles with roll 0..320 deg student vitt-256: measured pitch extremes with roll

Left: teacher vitl-320 on nine looking-up × profile tiles with a roll of 0–320° composited onto each tile.
Right: student vitt-256 on measured pitch extremes (+86° to −89°) with roll. Predictions only.

1. Results at a glance

inter-ocular NME %, lower is better; CPU latency on an i9-10900K with onnxruntime 1.22 CPU EP, batch 1.

Checkpoints and ONNX/TFLite(LiteRT): https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment/releases/tag/weights

Model Input Params GMACs GFLOPs CPU ms WFLW Full WFLW Pose 300W Full COFW
D-ViT (paper) 256 96.4M ≈73.15 ≈146.30 – 3.75 6.43 2.85 4.13
vitl-320 (teacher) 320 308.2M 131.32 262.63 520 1.51 2.54 1.03 1.21
vitt-256 256 9.0M 2.05 4.09 12.4 3.36 5.64 2.66 2.76
hg0-256 256 1.6M 0.70 1.40 5.2 5.32 9.53 3.87 3.77
vitt-096 96 9.0M 0.43 0.85 4.6 5.14 8.65 3.86 3.90
hg0-096 96 1.6M 0.13 0.26 1.5 7.46 14.71 4.77 4.99

How to read these numbers (important): all splits of the real-image datasets — including WFLW test, 300W and COFW test — are used for training. The "official" numbers above therefore measure how well the models fit the training distribution and must not be compared with published benchmark results. The only unleaked instrument is 300wlp_val (head-NME on a held-out 300W-LP subset; values in 050). The full tables — WFLW subsets, FR/AUC, breakdowns by pose (yaw / pitch / roll) and by image degradation, and the D-ViT paper values listed side by side with an explicit non-comparability note — follow in §1.1 (same content as 050 §2). The D-ViT (paper) row lists the published benchmark values of arXiv 2411.07167 (Tables 1 / 2, "Ours": 256×256 input, 8 prediction blocks = 96.4M parameters per its Table S1) for format reference only. The paper does not report FLOPs or latency; the GMACs / GFLOPs are our estimate from the official code with its default arguments (8 prediction blocks, 32×32 heatmaps, depth 256, 256×256 input, all 8 stages, batch 1, same FlopCounterMode method as the other rows) and are approximate because that configuration has 101.1M parameters, about 5 % more than Table S1. Its COFW NME is normalized by the inter-pupil distance instead of inter-ocular, and its test sets are held out from training, so no ranking against the fit measures above is implied.

1.1 Full tables

All tables below are generated from runs/<run>/eval_best_{official,stratreal,stress,style}.log by scripts/make_results_tables.py. Notes:

Table 1. NME (%, inter-ocular, lower is better)

Model WFLW
Full

Pose

Exp

Ill

Mu

Occ

Blur
COFW
Full
300W
Full

Comm

Chal
D-ViT (paper) 3.75 6.43 3.85 4.06 3.57 4.47 4.37 4.13 2.85 2.43 4.56
vitl-320 1.51 2.54 1.55 1.47 1.42 1.56 1.58 1.21 1.03 0.96 1.34
vitt-256 3.36 5.64 3.43 3.25 3.19 3.56 3.60 2.76 2.66 2.36 3.90
hg0-256 5.32 9.53 5.54 5.16 5.09 6.29 6.04 3.77 3.87 3.37 5.96
vitt-096 5.14 8.65 5.32 4.96 5.06 5.52 5.40 3.90 3.85 3.57 5.02
hg0-096 7.46 14.71 7.77 7.19 8.10 8.91 8.16 4.99 4.77 4.24 6.91

Table 2. FR10 (%, lower is better) and AUC10 (%, higher is better) on WFLW

Model FR10
Full

Pose

Exp

Ill

Mu

Occ

Blur
AUC10
Full

Pose

Exp

Ill

Mu

Occ

Blur
D-ViT (paper) 1.76 8.28 1.27 1.29 1.94 3.80 2.07 63.7 40.1 62.6 64.7 64.7 57.1 58.6
vitl-320 0.00 0.00 0.00 0.00 0.00 0.00 0.00 84.9 74.5 84.5 85.4 85.8 84.4 84.2
vitt-256 0.28 1.53 0.32 0.00 0.00 0.54 0.52 66.5 43.9 65.7 67.5 68.2 64.5 64.1
hg0-256 7.00 32.21 5.73 5.44 6.31 13.04 9.44 49.2 17.5 46.7 50.3 50.2 42.4 42.8
vitt-096 4.20 23.31 4.78 2.72 2.91 5.57 3.75 49.7 20.2 47.7 51.2 49.9 46.2 47.1
hg0-096 17.88 67.48 18.79 14.18 20.87 26.09 22.38 36.1 5.8 31.9 37.6 34.4 28.7 30.0

Table 3. NME (%) by yaw / pitch bin (effective pose bins of pose-stress)

Model Yaw
0–30
Yaw
30–60
Yaw
60–95
Pitch
−95..−45
Pitch
−45..−15
Pitch
−15..15
Pitch
15..45
Pitch
45..95
vitl-320 1.26 1.71 2.04 1.76 1.37 1.30 1.29 1.85
vitt-256 2.94 4.10 4.68 4.07 3.20 3.06 2.99 4.43
hg0-256 4.21 6.35 8.14 6.26 4.63 4.46 4.35 7.51
vitt-096 4.49 6.21 7.37 5.93 4.82 4.77 4.54 7.06
hg0-096 5.69 8.70 10.85 8.02 6.17 6.21 5.79 10.07
n 5933 1124 143 223 2503 2944 1398 48

Table 4. NME (%) by in-plane roll (pose-stress, n=300 per set)

Model Set Roll 0 45 90 135 180 225 270 315 worst−base
vitl-320 wflw 1.61 1.61 1.64 1.60 1.63 1.61 1.63 1.60 +0.02
300w 1.12 1.11 1.14 1.11 1.13 1.11 1.13 1.11 +0.01
cofw 1.29 1.30 1.31 1.30 1.30 1.30 1.30 1.29 +0.02
vitt-256 wflw 3.67 3.61 3.67 3.60 3.70 3.61 3.68 3.61 +0.03
300w 2.93 2.88 2.89 2.89 2.95 2.92 2.96 2.90 +0.03
cofw 3.01 2.96 3.02 2.97 2.98 2.96 2.98 2.97 +0.01
hg0-256 wflw 5.96 5.95 5.90 6.06 5.92 6.03 5.98 5.96 +0.10
300w 4.14 4.11 4.26 4.20 4.26 4.29 4.21 4.16 +0.15
cofw 4.12 4.08 4.06 4.06 4.03 4.11 4.12 4.06 +0.00
vitt-096 wflw 6.30 6.05 6.17 5.98 6.30 6.26 6.40 6.19 +0.10
300w 4.58 4.41 4.55 4.50 5.03 5.11 5.14 4.71 +0.57
cofw 4.74 4.61 4.72 4.49 4.73 4.49 4.56 4.55 +0.00
hg0-096 wflw 8.89 9.03 9.09 9.21 8.98 9.33 9.90 8.72 +1.01
300w 5.57 5.61 5.82 6.07 6.40 6.59 6.05 5.95 +1.03
cofw 5.84 5.79 6.05 5.71 5.76 5.51 5.68 5.73 +0.21

Table 5. NME (%) under camera pitch / yaw perturbation (pose-stress, n=300 per set)

Model Set base cam
pitch
−25


−15


+15


+25
cam
yaw
−15


+15


p+25,y+15


p−25,y−15
vitl-320 wflw 1.61 1.78 1.63 1.58 1.60 1.55 1.55 1.56 1.78
300w 1.12 1.25 1.15 1.09 1.10 1.08 1.08 1.08 1.22
cofw 1.29 1.36 1.29 1.26 1.29 1.23 1.24 1.25 1.35
vitt-256 wflw 3.67 3.84 3.67 3.52 3.48 3.52 3.50 3.44 3.94
300w 2.93 3.07 2.93 2.81 2.75 2.78 2.81 2.73 3.03
cofw 3.01 3.12 3.01 2.95 2.92 2.90 2.90 2.92 3.12
hg0-256 wflw 5.96 5.99 5.71 5.71 5.38 5.55 5.60 5.31 6.08
300w 4.14 4.46 4.24 4.05 3.98 3.97 4.04 3.90 4.39
cofw 4.12 4.27 4.11 4.02 3.96 3.98 3.97 3.99 4.23
vitt-096 wflw 6.30 5.94 5.83 5.97 5.51 5.78 5.76 5.30 6.05
300w 4.58 4.49 4.39 4.41 4.10 4.24 4.34 4.06 4.41
cofw 4.74 4.53 4.49 4.45 4.18 4.33 4.42 4.20 4.47
hg0-096 wflw 8.89 8.20 8.27 8.44 7.72 8.37 8.19 7.35 8.34
300w 5.57 5.38 5.38 5.45 4.94 5.16 5.25 4.81 5.34
cofw 5.84 5.62 5.57 5.67 5.28 5.35 5.53 5.13 5.48

Table 6. head-NME (×100) stratified by 6DRepNet-estimated pose (3,696 real images from the three sets)

Model mean Yaw
0–30
Yaw
30–60
Yaw
60–95
Pitch
−90..−30
Pitch
−30..−10
Pitch
−10..10
Pitch
10..30
Pitch
30..90
vitl-320 0.37 0.36 0.42 0.45 0.42 0.37 0.36 0.40 0.50
vitt-256 0.85 0.81 0.98 1.01 0.98 0.85 0.82 0.93 1.13
hg0-256 1.29 1.21 1.60 1.85 1.71 1.28 1.21 1.50 2.09
vitt-096 1.27 1.22 1.46 1.58 1.48 1.27 1.22 1.38 1.71
hg0-096 1.75 1.60 2.22 2.85 2.53 1.76 1.59 2.04 2.94
n 2939 672 85 143 1076 2136 232 32

Table 7. style-shift: NME (%) under image degradations (in parentheses: degradation vs clean, n=300 per set)

Model Set clean mblur9 mblur21 warm cool gamma0.6 gamma1.6 gray jpeg30
vitl-320 wflw_test 1.49 1.57 (+5.5%) 2.09 (+40.4%) 1.51 (+1.4%) 1.51 (+1.5%) 1.50 (+1.1%) 1.51 (+1.3%) 1.60 (+7.8%) 1.55 (+4.5%)
300w_valid 1.04 1.13 (+8.6%) 1.46 (+40.6%) 1.06 (+2.0%) 1.05 (+1.2%) 1.06 (+1.8%) 1.05 (+1.4%) 1.09 (+4.9%) 1.08 (+4.3%)
cofw_test 1.21 1.27 (+5.3%) 1.60 (+32.3%) 1.22 (+1.3%) 1.23 (+1.5%) 1.22 (+1.1%) 1.22 (+1.2%) 1.27 (+5.0%) 1.25 (+3.6%)
vitt-256 wflw_test 3.26 3.32 (+1.8%) 4.60 (+41.1%) 3.32 (+1.9%) 3.33 (+2.1%) 3.35 (+2.7%) 3.36 (+3.0%) 3.48 (+6.6%) 3.36 (+3.1%)
300w_valid 2.65 2.73 (+2.6%) 3.66 (+37.7%) 2.68 (+1.1%) 2.69 (+1.3%) 2.71 (+2.0%) 2.72 (+2.3%) 2.75 (+3.7%) 2.73 (+2.9%)
cofw_test 2.76 2.82 (+2.1%) 3.51 (+27.2%) 2.79 (+1.0%) 2.79 (+1.1%) 2.83 (+2.4%) 2.81 (+1.7%) 2.85 (+3.3%) 2.81 (+1.8%)
hg0-256 wflw_test 5.08 5.28 (+3.9%) 8.16 (+60.5%) 5.14 (+1.2%) 5.23 (+2.9%) 5.13 (+0.9%) 5.29 (+4.2%) 5.41 (+6.4%) 5.35 (+5.2%)
300w_valid 3.85 3.88 (+0.8%) 5.20 (+35.0%) 3.90 (+1.2%) 3.90 (+1.3%) 3.88 (+0.7%) 3.92 (+1.7%) 3.95 (+2.5%) 3.90 (+1.0%)
cofw_test 3.79 3.90 (+2.9%) 4.93 (+30.0%) 3.83 (+1.1%) 3.83 (+0.9%) 3.82 (+0.8%) 3.82 (+0.8%) 3.91 (+3.2%) 3.85 (+1.5%)
vitt-096 wflw_test 5.01 6.26 (+25.0%) 24.80 (+395.2%) 5.10 (+1.9%) 5.09 (+1.6%) 5.13 (+2.4%) 5.21 (+4.1%) 5.37 (+7.3%) 5.36 (+7.0%)
300w_valid 3.82 4.42 (+15.8%) 18.31 (+379.4%) 3.86 (+1.1%) 3.86 (+1.1%) 3.90 (+2.1%) 3.94 (+3.1%) 3.96 (+3.8%) 4.04 (+5.8%)
cofw_test 3.89 4.52 (+16.3%) 15.95 (+310.4%) 3.95 (+1.8%) 3.95 (+1.6%) 3.99 (+2.8%) 4.00 (+2.8%) 4.05 (+4.2%) 4.08 (+5.0%)
hg0-096 wflw_test 7.01 9.18 (+31.0%) 32.00 (+356.7%) 7.15 (+2.0%) 7.30 (+4.2%) 7.11 (+1.5%) 7.36 (+5.0%) 7.77 (+10.9%) 9.10 (+29.9%)
300w_valid 4.64 5.68 (+22.5%) 19.79 (+326.5%) 4.68 (+1.0%) 4.74 (+2.1%) 4.70 (+1.2%) 4.72 (+1.7%) 4.80 (+3.5%) 5.60 (+20.8%)
cofw_test 5.03 5.71 (+13.7%) 18.03 (+258.9%) 5.07 (+0.8%) 5.15 (+2.4%) 5.04 (+0.3%) 5.16 (+2.6%) 5.19 (+3.2%) 5.74 (+14.2%)

2. Features

3. Models and the ONNX I/O contract

Teacher vitl-320 Students vitt-256 / vitt-096 Students hg0-256 / hg0-096
Backbone DINOv3 ViT-L/16 (the official implementation is imported at runtime from the torch.hub cache; its code is not vendored) ViT-T/16 (own implementation with RoPE, initialized from ckpts/vitt_distill.pt) PP-HGNetV2-B0 stages 0–2 (own implementation) + FPN (stride 8)
Head Point-query Transformer decoder (d256, 4 layers) with coordinate regression and visibility Same (d256, 3 layers) Same (d128, 2 layers, 4 heads)
Input normalization ImageNet mean/std center05 (x/255 − 0.5)/0.5 (no normalization op inside the ONNX graph) center05, folded into the stem conv (no input op at all)
Preset clean_v3 student_s256_96gb_r2 / student_s096_96gb_r2 student_hg0_wsd / student_hg0_s096_wsd

ONNX (example for --scheme ibug68; wflw98 / cofw29 are exported as separate files):

# RGB, normalized as above. S = 320 / 256 / 96 (the preset's out_size)
input   images      float32 [N, 3, S, S]
# coordinates relative to the crop (0..1 inside the crop; points outside still get coordinates)
output  points      float32 [N, K, 2]
# visibility logits (0 = outside the image, 1 = occluded, 2 = visible)
output  vis_logits  float32 [N, K, 3]

4. Setup

Prerequisites: Linux, Python 3.13 (pinned in .python-version), uv, and an NVIDIA GPU (torch built for CUDA 12.8; an 8 GB GPU is enough for smoke tests, full training targets a 96 GB class GPU). Dependencies are pinned with == in pyproject.toml and hash-locked in uv.lock.

git clone https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment.git
cd High-Angle_Robust_Fast_FaceAlignment
uv sync --frozen                                   # default: onnxruntime-gpu 1.22.0 (the validated toolchain)
uv sync --frozen --no-group ort --group tensorrt   # alternative: onnxruntime-gpu 1.26.0 for TensorRT BF16 (trt_bf16_enable)
# NOTE: `uv run` re-syncs the environment with the DEFAULT groups before running, i.e. a plain
# `uv run python ...` silently switches back to 1.22.0. Keep the same group flags on `uv run`
# (or use `uv run --no-sync` / `.venv/bin/python` after the manual sync):
uv run --no-group ort --group tensorrt \
python demo/demo_hrffa_onnx.py \
-v 0 \
-o output_dir \
-d tensorrt

uv run python -m unittest tests/test_model_arms.py

4.1 Weights (ckpts/, none of them are bundled)

| File | Purpose | Source /

High-Angle_Robust_Fast_FaceAlignment (PINTO0309/High-Angle_Robust_Fast_FaceAlignment)

Searching Database…

这是什么?HRFFA 是一个面向整头人脸对齐的训练、蒸馏和 ONNX 部署流水线,在极端头部姿态(抬头/低头、完整 360° 翻滚、正侧脸)下依然稳健,同时轻量到可以在 CPU 上推理。
它处理紧贴人脸的裁剪,还是整个头部的裁剪?它刻意采用整头裁剪而不是紧贴人脸的裁剪,因为目标检测器定位头部远比定位人脸可靠,而且额外的头部/背景上下文有助于提高极端姿态下的精度。
怎么把模型跑起来?可以从结果表中选一个预训练的 ONNX/TFLite 检查点(如 vitt-256、hg0-256)直接运行,也可以用自带的训练/蒸馏脚本从头复现或重新训练这些模型。
报告的 WFLW/300W/COFW 数字能和其他论文比较吗?不能。这里把所有真实图像测试集(WFLW test、300W、COFW test)都放进了训练,所以报告的数字衡量的是模型对自身训练分布的拟合程度,而不是泛化能力,明确不能与标准公开基准比较。
导出的模型能在 CPU 上完全离线运行吗?可以。蒸馏后的 ONNX/TFLite 模型专为本地 CPU 执行设计,不依赖网络,在普通桌面 CPU 上单次推理耗时 1.5-12 毫秒。
教师模型和蒸馏出的学生模型有什么区别?教师模型是 3.08 亿参数的 DINOv3 ViT-L/16,输入 320x320,在 CPU 上约需 520 毫秒;学生模型(一个 ViT-T/16 和一个 PP-HGNetV2-B0 CNN,各有 256 和 96 两种输入版本)通过在线蒸馏训练,参数量 160 万至 900 万,运行耗时 1.5-12 毫秒。
为什么在头部裁剪上对齐人脸,而不是紧贴人脸的裁剪?因为目标检测器定位整个头部远比定位人脸可靠,而且把整个头部连同周围一圈背景一起输入能给模型更多上下文,尽管这会让它与基于人脸裁剪的基准直接比较变得不公平。
模型支持哪些极端姿态和关键点输出?它支持实测俯仰角超过 ±85°(抬头/低头)、完整的 360° 面内翻滚,以及高达 ±90° 的偏航角(正侧脸),可预测 68、98 或 29 个关键点,且每个点附带三分类可见性标签(可见、被遮挡或在图像外)。
What are similar projects?RelateAnything, yolov3, YOLOX, yolov7, yolov5, NanoJev, SolarWM, pointdit
Creator
PINTO0309
Size
1 MB
Stars
195
Age
1mo
Commits · 90 Days