High-Angle_Robust_Fast_FaceAlignment README

The project's README from github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment. As one markdown file: README.md.

HRFFA: High-Angle Robust Fast FaceAlignment

GitHub DOI

A training, distillation and ONNX deployment pipeline for whole-head face alignment that stays robust at extreme head poses while remaining light enough for CPU inference.

teacher clean_v3: extreme pitch/yaw tiles with roll 0..320 deg student vitt-256: measured pitch extremes with roll

Left: teacher vitl-320 on nine looking-up × profile tiles with a roll of 0–320° composited onto each tile.
Right: student vitt-256 on measured pitch extremes (+86° to −89°) with roll. Predictions only.

1. Results at a glance

inter-ocular NME %, lower is better; CPU latency on an i9-10900K with onnxruntime 1.22 CPU EP, batch 1.

Checkpoints and ONNX/TFLite(LiteRT): https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment/releases/tag/weights

Model Input Params GMACs GFLOPs CPU ms WFLW Full WFLW Pose 300W Full COFW
D-ViT (paper) 256 96.4M ≈73.15 ≈146.30 – 3.75 6.43 2.85 4.13
vitl-320 (teacher) 320 308.2M 131.32 262.63 520 1.51 2.54 1.03 1.21
vitt-256 256 9.0M 2.05 4.09 12.4 3.36 5.64 2.66 2.76
hg0-256 256 1.6M 0.70 1.40 5.2 5.32 9.53 3.87 3.77
vitt-096 96 9.0M 0.43 0.85 4.6 5.14 8.65 3.86 3.90
hg0-096 96 1.6M 0.13 0.26 1.5 7.46 14.71 4.77 4.99

How to read these numbers (important): all splits of the real-image datasets — including WFLW test, 300W and COFW test — are used for training. The "official" numbers above therefore measure how well the models fit the training distribution and must not be compared with published benchmark results. The only unleaked instrument is 300wlp_val (head-NME on a held-out 300W-LP subset; values in 050). The full tables — WFLW subsets, FR/AUC, breakdowns by pose (yaw / pitch / roll) and by image degradation, and the D-ViT paper values listed side by side with an explicit non-comparability note — follow in §1.1 (same content as 050 §2). The D-ViT (paper) row lists the published benchmark values of arXiv 2411.07167 (Tables 1 / 2, "Ours": 256×256 input, 8 prediction blocks = 96.4M parameters per its Table S1) for format reference only. The paper does not report FLOPs or latency; the GMACs / GFLOPs are our estimate from the official code with its default arguments (8 prediction blocks, 32×32 heatmaps, depth 256, 256×256 input, all 8 stages, batch 1, same FlopCounterMode method as the other rows) and are approximate because that configuration has 101.1M parameters, about 5 % more than Table S1. Its COFW NME is normalized by the inter-pupil distance instead of inter-ocular, and its test sets are held out from training, so no ranking against the fit measures above is implied.

1.1 Full tables

All tables below are generated from runs/<run>/eval_best_{official,stratreal,stress,style}.log by scripts/make_results_tables.py. Notes:

Table 1. NME (%, inter-ocular, lower is better)

Model WFLW
Full

Pose

Exp

Ill

Mu

Occ

Blur
COFW
Full
300W
Full

Comm

Chal
D-ViT (paper) 3.75 6.43 3.85 4.06 3.57 4.47 4.37 4.13 2.85 2.43 4.56
vitl-320 1.51 2.54 1.55 1.47 1.42 1.56 1.58 1.21 1.03 0.96 1.34
vitt-256 3.36 5.64 3.43 3.25 3.19 3.56 3.60 2.76 2.66 2.36 3.90
hg0-256 5.32 9.53 5.54 5.16 5.09 6.29 6.04 3.77 3.87 3.37 5.96
vitt-096 5.14 8.65 5.32 4.96 5.06 5.52 5.40 3.90 3.85 3.57 5.02
hg0-096 7.46 14.71 7.77 7.19 8.10 8.91 8.16 4.99 4.77 4.24 6.91

Table 2. FR10 (%, lower is better) and AUC10 (%, higher is better) on WFLW

Model FR10
Full

Pose

Exp

Ill

Mu

Occ

Blur
AUC10
Full

Pose

Exp

Ill

Mu

Occ

Blur
D-ViT (paper) 1.76 8.28 1.27 1.29 1.94 3.80 2.07 63.7 40.1 62.6 64.7 64.7 57.1 58.6
vitl-320 0.00 0.00 0.00 0.00 0.00 0.00 0.00 84.9 74.5 84.5 85.4 85.8 84.4 84.2
vitt-256 0.28 1.53 0.32 0.00 0.00 0.54 0.52 66.5 43.9 65.7 67.5 68.2 64.5 64.1
hg0-256 7.00 32.21 5.73 5.44 6.31 13.04 9.44 49.2 17.5 46.7 50.3 50.2 42.4 42.8
vitt-096 4.20 23.31 4.78 2.72 2.91 5.57 3.75 49.7 20.2 47.7 51.2 49.9 46.2 47.1
hg0-096 17.88 67.48 18.79 14.18 20.87 26.09 22.38 36.1 5.8 31.9 37.6 34.4 28.7 30.0

Table 3. NME (%) by yaw / pitch bin (effective pose bins of pose-stress)

Model Yaw
0–30
Yaw
30–60
Yaw
60–95
Pitch
−95..−45
Pitch
−45..−15
Pitch
−15..15
Pitch
15..45
Pitch
45..95
vitl-320 1.26 1.71 2.04 1.76 1.37 1.30 1.29 1.85
vitt-256 2.94 4.10 4.68 4.07 3.20 3.06 2.99 4.43
hg0-256 4.21 6.35 8.14 6.26 4.63 4.46 4.35 7.51
vitt-096 4.49 6.21 7.37 5.93 4.82 4.77 4.54 7.06
hg0-096 5.69 8.70 10.85 8.02 6.17 6.21 5.79 10.07
n 5933 1124 143 223 2503 2944 1398 48

Table 4. NME (%) by in-plane roll (pose-stress, n=300 per set)

Model Set Roll 0 45 90 135 180 225 270 315 worst−base
vitl-320 wflw 1.61 1.61 1.64 1.60 1.63 1.61 1.63 1.60 +0.02
300w 1.12 1.11 1.14 1.11 1.13 1.11 1.13 1.11 +0.01
cofw 1.29 1.30 1.31 1.30 1.30 1.30 1.30 1.29 +0.02
vitt-256 wflw 3.67 3.61 3.67 3.60 3.70 3.61 3.68 3.61 +0.03
300w 2.93 2.88 2.89 2.89 2.95 2.92 2.96 2.90 +0.03
cofw 3.01 2.96 3.02 2.97 2.98 2.96 2.98 2.97 +0.01
hg0-256 wflw 5.96 5.95 5.90 6.06 5.92 6.03 5.98 5.96 +0.10
300w 4.14 4.11 4.26 4.20 4.26 4.29 4.21 4.16 +0.15
cofw 4.12 4.08 4.06 4.06 4.03 4.11 4.12 4.06 +0.00
vitt-096 wflw 6.30 6.05 6.17 5.98 6.30 6.26 6.40 6.19 +0.10
300w 4.58 4.41 4.55 4.50 5.03 5.11 5.14 4.71 +0.57
cofw 4.74 4.61 4.72 4.49 4.73 4.49 4.56 4.55 +0.00
hg0-096 wflw 8.89 9.03 9.09 9.21 8.98 9.33 9.90 8.72 +1.01
300w 5.57 5.61 5.82 6.07 6.40 6.59 6.05 5.95 +1.03
cofw 5.84 5.79 6.05 5.71 5.76 5.51 5.68 5.73 +0.21

Table 5. NME (%) under camera pitch / yaw perturbation (pose-stress, n=300 per set)

Model Set base cam
pitch
−25


−15


+15


+25
cam
yaw
−15


+15


p+25,y+15


p−25,y−15
vitl-320 wflw 1.61 1.78 1.63 1.58 1.60 1.55 1.55 1.56 1.78
300w 1.12 1.25 1.15 1.09 1.10 1.08 1.08 1.08 1.22
cofw 1.29 1.36 1.29 1.26 1.29 1.23 1.24 1.25 1.35
vitt-256 wflw 3.67 3.84 3.67 3.52 3.48 3.52 3.50 3.44 3.94
300w 2.93 3.07 2.93 2.81 2.75 2.78 2.81 2.73 3.03
cofw 3.01 3.12 3.01 2.95 2.92 2.90 2.90 2.92 3.12
hg0-256 wflw 5.96 5.99 5.71 5.71 5.38 5.55 5.60 5.31 6.08
300w 4.14 4.46 4.24 4.05 3.98 3.97 4.04 3.90 4.39
cofw 4.12 4.27 4.11 4.02 3.96 3.98 3.97 3.99 4.23
vitt-096 wflw 6.30 5.94 5.83 5.97 5.51 5.78 5.76 5.30 6.05
300w 4.58 4.49 4.39 4.41 4.10 4.24 4.34 4.06 4.41
cofw 4.74 4.53 4.49 4.45 4.18 4.33 4.42 4.20 4.47
hg0-096 wflw 8.89 8.20 8.27 8.44 7.72 8.37 8.19 7.35 8.34
300w 5.57 5.38 5.38 5.45 4.94 5.16 5.25 4.81 5.34
cofw 5.84 5.62 5.57 5.67 5.28 5.35 5.53 5.13 5.48

Table 6. head-NME (×100) stratified by 6DRepNet-estimated pose (3,696 real images from the three sets)

Model mean Yaw
0–30
Yaw
30–60
Yaw
60–95
Pitch
−90..−30
Pitch
−30..−10
Pitch
−10..10
Pitch
10..30
Pitch
30..90
vitl-320 0.37 0.36 0.42 0.45 0.42 0.37 0.36 0.40 0.50
vitt-256 0.85 0.81 0.98 1.01 0.98 0.85 0.82 0.93 1.13
hg0-256 1.29 1.21 1.60 1.85 1.71 1.28 1.21 1.50 2.09
vitt-096 1.27 1.22 1.46 1.58 1.48 1.27 1.22 1.38 1.71
hg0-096 1.75 1.60 2.22 2.85 2.53 1.76 1.59 2.04 2.94
n 2939 672 85 143 1076 2136 232 32

Table 7. style-shift: NME (%) under image degradations (in parentheses: degradation vs clean, n=300 per set)

Model Set clean mblur9 mblur21 warm cool gamma0.6 gamma1.6 gray jpeg30
vitl-320 wflw_test 1.49 1.57 (+5.5%) 2.09 (+40.4%) 1.51 (+1.4%) 1.51 (+1.5%) 1.50 (+1.1%) 1.51 (+1.3%) 1.60 (+7.8%) 1.55 (+4.5%)
300w_valid 1.04 1.13 (+8.6%) 1.46 (+40.6%) 1.06 (+2.0%) 1.05 (+1.2%) 1.06 (+1.8%) 1.05 (+1.4%) 1.09 (+4.9%) 1.08 (+4.3%)
cofw_test 1.21 1.27 (+5.3%) 1.60 (+32.3%) 1.22 (+1.3%) 1.23 (+1.5%) 1.22 (+1.1%) 1.22 (+1.2%) 1.27 (+5.0%) 1.25 (+3.6%)
vitt-256 wflw_test 3.26 3.32 (+1.8%) 4.60 (+41.1%) 3.32 (+1.9%) 3.33 (+2.1%) 3.35 (+2.7%) 3.36 (+3.0%) 3.48 (+6.6%) 3.36 (+3.1%)
300w_valid 2.65 2.73 (+2.6%) 3.66 (+37.7%) 2.68 (+1.1%) 2.69 (+1.3%) 2.71 (+2.0%) 2.72 (+2.3%) 2.75 (+3.7%) 2.73 (+2.9%)
cofw_test 2.76 2.82 (+2.1%) 3.51 (+27.2%) 2.79 (+1.0%) 2.79 (+1.1%) 2.83 (+2.4%) 2.81 (+1.7%) 2.85 (+3.3%) 2.81 (+1.8%)
hg0-256 wflw_test 5.08 5.28 (+3.9%) 8.16 (+60.5%) 5.14 (+1.2%) 5.23 (+2.9%) 5.13 (+0.9%) 5.29 (+4.2%) 5.41 (+6.4%) 5.35 (+5.2%)
300w_valid 3.85 3.88 (+0.8%) 5.20 (+35.0%) 3.90 (+1.2%) 3.90 (+1.3%) 3.88 (+0.7%) 3.92 (+1.7%) 3.95 (+2.5%) 3.90 (+1.0%)
cofw_test 3.79 3.90 (+2.9%) 4.93 (+30.0%) 3.83 (+1.1%) 3.83 (+0.9%) 3.82 (+0.8%) 3.82 (+0.8%) 3.91 (+3.2%) 3.85 (+1.5%)
vitt-096 wflw_test 5.01 6.26 (+25.0%) 24.80 (+395.2%) 5.10 (+1.9%) 5.09 (+1.6%) 5.13 (+2.4%) 5.21 (+4.1%) 5.37 (+7.3%) 5.36 (+7.0%)
300w_valid 3.82 4.42 (+15.8%) 18.31 (+379.4%) 3.86 (+1.1%) 3.86 (+1.1%) 3.90 (+2.1%) 3.94 (+3.1%) 3.96 (+3.8%) 4.04 (+5.8%)
cofw_test 3.89 4.52 (+16.3%) 15.95 (+310.4%) 3.95 (+1.8%) 3.95 (+1.6%) 3.99 (+2.8%) 4.00 (+2.8%) 4.05 (+4.2%) 4.08 (+5.0%)
hg0-096 wflw_test 7.01 9.18 (+31.0%) 32.00 (+356.7%) 7.15 (+2.0%) 7.30 (+4.2%) 7.11 (+1.5%) 7.36 (+5.0%) 7.77 (+10.9%) 9.10 (+29.9%)
300w_valid 4.64 5.68 (+22.5%) 19.79 (+326.5%) 4.68 (+1.0%) 4.74 (+2.1%) 4.70 (+1.2%) 4.72 (+1.7%) 4.80 (+3.5%) 5.60 (+20.8%)
cofw_test 5.03 5.71 (+13.7%) 18.03 (+258.9%) 5.07 (+0.8%) 5.15 (+2.4%) 5.04 (+0.3%) 5.16 (+2.6%) 5.19 (+3.2%) 5.74 (+14.2%)

2. Features

3. Models and the ONNX I/O contract

Teacher vitl-320 Students vitt-256 / vitt-096 Students hg0-256 / hg0-096
Backbone DINOv3 ViT-L/16 (the official implementation is imported at runtime from the torch.hub cache; its code is not vendored) ViT-T/16 (own implementation with RoPE, initialized from ckpts/vitt_distill.pt) PP-HGNetV2-B0 stages 0–2 (own implementation) + FPN (stride 8)
Head Point-query Transformer decoder (d256, 4 layers) with coordinate regression and visibility Same (d256, 3 layers) Same (d128, 2 layers, 4 heads)
Input normalization ImageNet mean/std center05 (x/255 − 0.5)/0.5 (no normalization op inside the ONNX graph) center05, folded into the stem conv (no input op at all)
Preset clean_v3 student_s256_96gb_r2 / student_s096_96gb_r2 student_hg0_wsd / student_hg0_s096_wsd

ONNX (example for --scheme ibug68; wflw98 / cofw29 are exported as separate files):

# RGB, normalized as above. S = 320 / 256 / 96 (the preset's out_size)
input   images      float32 [N, 3, S, S]
# coordinates relative to the crop (0..1 inside the crop; points outside still get coordinates)
output  points      float32 [N, K, 2]
# visibility logits (0 = outside the image, 1 = occluded, 2 = visible)
output  vis_logits  float32 [N, K, 3]

4. Setup

Prerequisites: Linux, Python 3.13 (pinned in .python-version), uv, and an NVIDIA GPU (torch built for CUDA 12.8; an 8 GB GPU is enough for smoke tests, full training targets a 96 GB class GPU). Dependencies are pinned with == in pyproject.toml and hash-locked in uv.lock.

git clone https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment.git
cd High-Angle_Robust_Fast_FaceAlignment
uv sync --frozen                                   # default: onnxruntime-gpu 1.22.0 (the validated toolchain)
uv sync --frozen --no-group ort --group tensorrt   # alternative: onnxruntime-gpu 1.26.0 for TensorRT BF16 (trt_bf16_enable)
# NOTE: `uv run` re-syncs the environment with the DEFAULT groups before running, i.e. a plain
# `uv run python ...` silently switches back to 1.22.0. Keep the same group flags on `uv run`
# (or use `uv run --no-sync` / `.venv/bin/python` after the manual sync):
uv run --no-group ort --group tensorrt \
python demo/demo_hrffa_onnx.py \
-v 0 \
-o output_dir \
-d tensorrt

uv run python -m unittest tests/test_model_arms.py

4.1 Weights (ckpts/, none of them are bundled)

| File | Purpose | Source /

High-Angle_Robust_Fast_FaceAlignment (PINTO0309/High-Angle_Robust_Fast_FaceAlignment)

Searching Database…

What is this?HRFFA is a training, distillation and ONNX deployment pipeline for whole-head face alignment that stays robust at extreme head poses (looking up/down, full 360° roll, profile views) while being light enough for CPU inference.
Does it work on tightly cropped faces or on whole-head crops?It deliberately works on whole-head crops rather than tight face crops, since object detectors can localize heads far more reliably than faces, and the extra head/background context helps accuracy at extreme poses.
How do I get a model running?You can take one of the pretrained ONNX/TFLite checkpoints (e.g. vitt-256, hg0-256) listed in the results table and run it directly, or use the included training/distillation scripts to reproduce or retrain the models from scratch.
Can I compare the reported WFLW/300W/COFW numbers to other papers?No — all real-image test splits (WFLW test, 300W, COFW test) are included in training here, so the reported numbers measure how well the models fit their own training distribution, not generalization, and are explicitly not comparable to standard published benchmarks.
Can the exported models run fully offline on CPU?Yes — the distilled ONNX/TFLite models are designed for local CPU execution with no network dependency, running in 1.5-12 ms per inference on a standard desktop CPU.
What is the difference between the teacher model and the distilled student models?The teacher is a 308M-parameter DINOv3 ViT-L/16 at 320x320 that takes about 520 ms on CPU, while the students (a ViT-T/16 and a PP-HGNetV2-B0 CNN, each in 256 and 96 input variants) are trained by online distillation and run in 1.5-12 ms with 1.6-9M parameters.
Why align faces on head crops instead of tightly cropped face crops?Because object detectors localize whole heads far more reliably than faces, and feeding the whole head plus a margin of background gives the model more context, even though it makes direct comparison with face-crop benchmarks unfair.
What pose extremes and landmark outputs does the model support?It handles measured pitch beyond ±85° (looking up/down), a full 360° in-plane roll, and yaw up to ±90° (profile views), predicting 68, 98, or 29 landmarks plus a 3-class visibility label (visible, occluded, or outside the image) per point.
What are similar projects?RelateAnything, yolov3, YOLOX, yolov7, yolov5, NanoJev, SolarWM, pointdit
Creator
PINTO0309
Size
1 MB
Stars
195
Age
1mo
Commits · 90 Days