The project's README from github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment. As one markdown file: README.md.
A training, distillation and ONNX deployment pipeline for whole-head face alignment that stays robust at extreme head poses while remaining light enough for CPU inference.
Left: teacher vitl-320 on nine looking-up × profile tiles with a roll of 0–320° composited onto each tile.
Right: student vitt-256 on measured pitch extremes (+86° to −89°) with roll. Predictions only.
Of the total inference time, 25 ms is spent on CUDA inference for DEIMv2-Wholebody49-S, while the remainder is spent on FaceAlignment inference.
https://github.com/user-attachments/assets/144e485e-ec5c-460d-9cfb-58334accb92a
TensorRT BF16
https://github.com/user-attachments/assets/1226ea6c-7f49-4dcd-8c35-39701a192b5d
WebGPU + YOLOv9-MIT Demo
https://github.com/user-attachments/assets/1266c08b-3c2d-4aa6-8804-0427716a56d1
360° Yaw Estimation Test
https://github.com/user-attachments/assets/7eaab9fc-5402-41fb-93f3-49b707422bc9
inter-ocular NME %, lower is better; CPU latency on an i9-10900K with onnxruntime 1.22 CPU EP, batch 1.
Checkpoints and ONNX/TFLite(LiteRT): https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment/releases/tag/weights
| Model | Input | Params | GMACs | GFLOPs | CPU ms | WFLW Full | WFLW Pose | 300W Full | COFW |
|---|---|---|---|---|---|---|---|---|---|
| D-ViT (paper) | 256 | 96.4M | ≈73.15 | ≈146.30 | – | 3.75 | 6.43 | 2.85 | 4.13 |
| vitl-320 (teacher) | 320 | 308.2M | 131.32 | 262.63 | 520 | 1.51 | 2.54 | 1.03 | 1.21 |
| vitt-256 | 256 | 9.0M | 2.05 | 4.09 | 12.4 | 3.36 | 5.64 | 2.66 | 2.76 |
| hg0-256 | 256 | 1.6M | 0.70 | 1.40 | 5.2 | 5.32 | 9.53 | 3.87 | 3.77 |
| vitt-096 | 96 | 9.0M | 0.43 | 0.85 | 4.6 | 5.14 | 8.65 | 3.86 | 3.90 |
| hg0-096 | 96 | 1.6M | 0.13 | 0.26 | 1.5 | 7.46 | 14.71 | 4.77 | 4.99 |
How to read these numbers (important): all splits of the real-image datasets — including WFLW test, 300W and COFW test — are used for training. The "official" numbers above therefore measure how well the models fit the training distribution and must not be compared with published benchmark results. The only unleaked instrument is 300wlp_val (head-NME on a held-out 300W-LP subset; values in 050). The full tables — WFLW subsets, FR/AUC, breakdowns by pose (yaw / pitch / roll) and by image degradation, and the D-ViT paper values listed side by side with an explicit non-comparability note — follow in §1.1 (same content as 050 §2). The D-ViT (paper) row lists the published benchmark values of arXiv 2411.07167 (Tables 1 / 2, "Ours": 256×256 input, 8 prediction blocks = 96.4M parameters per its Table S1) for format reference only. The paper does not report FLOPs or latency; the GMACs / GFLOPs are our estimate from the official code with its default arguments (8 prediction blocks, 32×32 heatmaps, depth 256, 256×256 input, all 8 stages, batch 1, same FlopCounterMode method as the other rows) and are approximate because that configuration has 101.1M parameters, about 5 % more than Table S1. Its COFW NME is normalized by the inter-pupil distance instead of inter-ocular, and its test sets are held out from training, so no ranking against the fit measures above is implied.
All tables below are generated from runs/<run>/eval_best_{official,stratreal,stress,style}.log by scripts/make_results_tables.py. Notes:
D-ViT (paper) rows in Tables 1 / 2 are the published test-benchmark values of arXiv 2411.07167 (p.6, "Ours") and are listed for format reference only — the conditions differ, so no ranking is implied.| Model | WFLW Full |
Pose |
Exp |
Ill |
Mu |
Occ |
Blur |
COFW Full |
300W Full |
Comm |
Chal |
|---|---|---|---|---|---|---|---|---|---|---|---|
| D-ViT (paper) | 3.75 | 6.43 | 3.85 | 4.06 | 3.57 | 4.47 | 4.37 | 4.13 | 2.85 | 2.43 | 4.56 |
| vitl-320 | 1.51 | 2.54 | 1.55 | 1.47 | 1.42 | 1.56 | 1.58 | 1.21 | 1.03 | 0.96 | 1.34 |
| vitt-256 | 3.36 | 5.64 | 3.43 | 3.25 | 3.19 | 3.56 | 3.60 | 2.76 | 2.66 | 2.36 | 3.90 |
| hg0-256 | 5.32 | 9.53 | 5.54 | 5.16 | 5.09 | 6.29 | 6.04 | 3.77 | 3.87 | 3.37 | 5.96 |
| vitt-096 | 5.14 | 8.65 | 5.32 | 4.96 | 5.06 | 5.52 | 5.40 | 3.90 | 3.85 | 3.57 | 5.02 |
| hg0-096 | 7.46 | 14.71 | 7.77 | 7.19 | 8.10 | 8.91 | 8.16 | 4.99 | 4.77 | 4.24 | 6.91 |
| Model | FR10 Full |
Pose |
Exp |
Ill |
Mu |
Occ |
Blur |
AUC10 Full |
Pose |
Exp |
Ill |
Mu |
Occ |
Blur |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| D-ViT (paper) | 1.76 | 8.28 | 1.27 | 1.29 | 1.94 | 3.80 | 2.07 | 63.7 | 40.1 | 62.6 | 64.7 | 64.7 | 57.1 | 58.6 |
| vitl-320 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 84.9 | 74.5 | 84.5 | 85.4 | 85.8 | 84.4 | 84.2 |
| vitt-256 | 0.28 | 1.53 | 0.32 | 0.00 | 0.00 | 0.54 | 0.52 | 66.5 | 43.9 | 65.7 | 67.5 | 68.2 | 64.5 | 64.1 |
| hg0-256 | 7.00 | 32.21 | 5.73 | 5.44 | 6.31 | 13.04 | 9.44 | 49.2 | 17.5 | 46.7 | 50.3 | 50.2 | 42.4 | 42.8 |
| vitt-096 | 4.20 | 23.31 | 4.78 | 2.72 | 2.91 | 5.57 | 3.75 | 49.7 | 20.2 | 47.7 | 51.2 | 49.9 | 46.2 | 47.1 |
| hg0-096 | 17.88 | 67.48 | 18.79 | 14.18 | 20.87 | 26.09 | 22.38 | 36.1 | 5.8 | 31.9 | 37.6 | 34.4 | 28.7 | 30.0 |
| Model | Yaw 0–30 |
Yaw 30–60 |
Yaw 60–95 |
Pitch −95..−45 |
Pitch −45..−15 |
Pitch −15..15 |
Pitch 15..45 |
Pitch 45..95 |
|---|---|---|---|---|---|---|---|---|
| vitl-320 | 1.26 | 1.71 | 2.04 | 1.76 | 1.37 | 1.30 | 1.29 | 1.85 |
| vitt-256 | 2.94 | 4.10 | 4.68 | 4.07 | 3.20 | 3.06 | 2.99 | 4.43 |
| hg0-256 | 4.21 | 6.35 | 8.14 | 6.26 | 4.63 | 4.46 | 4.35 | 7.51 |
| vitt-096 | 4.49 | 6.21 | 7.37 | 5.93 | 4.82 | 4.77 | 4.54 | 7.06 |
| hg0-096 | 5.69 | 8.70 | 10.85 | 8.02 | 6.17 | 6.21 | 5.79 | 10.07 |
| n | 5933 | 1124 | 143 | 223 | 2503 | 2944 | 1398 | 48 |
| Model | Set | Roll 0 | 45 | 90 | 135 | 180 | 225 | 270 | 315 | worst−base |
|---|---|---|---|---|---|---|---|---|---|---|
| vitl-320 | wflw | 1.61 | 1.61 | 1.64 | 1.60 | 1.63 | 1.61 | 1.63 | 1.60 | +0.02 |
| 300w | 1.12 | 1.11 | 1.14 | 1.11 | 1.13 | 1.11 | 1.13 | 1.11 | +0.01 | |
| cofw | 1.29 | 1.30 | 1.31 | 1.30 | 1.30 | 1.30 | 1.30 | 1.29 | +0.02 | |
| vitt-256 | wflw | 3.67 | 3.61 | 3.67 | 3.60 | 3.70 | 3.61 | 3.68 | 3.61 | +0.03 |
| 300w | 2.93 | 2.88 | 2.89 | 2.89 | 2.95 | 2.92 | 2.96 | 2.90 | +0.03 | |
| cofw | 3.01 | 2.96 | 3.02 | 2.97 | 2.98 | 2.96 | 2.98 | 2.97 | +0.01 | |
| hg0-256 | wflw | 5.96 | 5.95 | 5.90 | 6.06 | 5.92 | 6.03 | 5.98 | 5.96 | +0.10 |
| 300w | 4.14 | 4.11 | 4.26 | 4.20 | 4.26 | 4.29 | 4.21 | 4.16 | +0.15 | |
| cofw | 4.12 | 4.08 | 4.06 | 4.06 | 4.03 | 4.11 | 4.12 | 4.06 | +0.00 | |
| vitt-096 | wflw | 6.30 | 6.05 | 6.17 | 5.98 | 6.30 | 6.26 | 6.40 | 6.19 | +0.10 |
| 300w | 4.58 | 4.41 | 4.55 | 4.50 | 5.03 | 5.11 | 5.14 | 4.71 | +0.57 | |
| cofw | 4.74 | 4.61 | 4.72 | 4.49 | 4.73 | 4.49 | 4.56 | 4.55 | +0.00 | |
| hg0-096 | wflw | 8.89 | 9.03 | 9.09 | 9.21 | 8.98 | 9.33 | 9.90 | 8.72 | +1.01 |
| 300w | 5.57 | 5.61 | 5.82 | 6.07 | 6.40 | 6.59 | 6.05 | 5.95 | +1.03 | |
| cofw | 5.84 | 5.79 | 6.05 | 5.71 | 5.76 | 5.51 | 5.68 | 5.73 | +0.21 |
| Model | Set | base | cam pitch −25 |
−15 |
+15 |
+25 |
cam yaw −15 |
+15 |
p+25,y+15 |
p−25,y−15 |
|---|---|---|---|---|---|---|---|---|---|---|
| vitl-320 | wflw | 1.61 | 1.78 | 1.63 | 1.58 | 1.60 | 1.55 | 1.55 | 1.56 | 1.78 |
| 300w | 1.12 | 1.25 | 1.15 | 1.09 | 1.10 | 1.08 | 1.08 | 1.08 | 1.22 | |
| cofw | 1.29 | 1.36 | 1.29 | 1.26 | 1.29 | 1.23 | 1.24 | 1.25 | 1.35 | |
| vitt-256 | wflw | 3.67 | 3.84 | 3.67 | 3.52 | 3.48 | 3.52 | 3.50 | 3.44 | 3.94 |
| 300w | 2.93 | 3.07 | 2.93 | 2.81 | 2.75 | 2.78 | 2.81 | 2.73 | 3.03 | |
| cofw | 3.01 | 3.12 | 3.01 | 2.95 | 2.92 | 2.90 | 2.90 | 2.92 | 3.12 | |
| hg0-256 | wflw | 5.96 | 5.99 | 5.71 | 5.71 | 5.38 | 5.55 | 5.60 | 5.31 | 6.08 |
| 300w | 4.14 | 4.46 | 4.24 | 4.05 | 3.98 | 3.97 | 4.04 | 3.90 | 4.39 | |
| cofw | 4.12 | 4.27 | 4.11 | 4.02 | 3.96 | 3.98 | 3.97 | 3.99 | 4.23 | |
| vitt-096 | wflw | 6.30 | 5.94 | 5.83 | 5.97 | 5.51 | 5.78 | 5.76 | 5.30 | 6.05 |
| 300w | 4.58 | 4.49 | 4.39 | 4.41 | 4.10 | 4.24 | 4.34 | 4.06 | 4.41 | |
| cofw | 4.74 | 4.53 | 4.49 | 4.45 | 4.18 | 4.33 | 4.42 | 4.20 | 4.47 | |
| hg0-096 | wflw | 8.89 | 8.20 | 8.27 | 8.44 | 7.72 | 8.37 | 8.19 | 7.35 | 8.34 |
| 300w | 5.57 | 5.38 | 5.38 | 5.45 | 4.94 | 5.16 | 5.25 | 4.81 | 5.34 | |
| cofw | 5.84 | 5.62 | 5.57 | 5.67 | 5.28 | 5.35 | 5.53 | 5.13 | 5.48 |
| Model | mean | Yaw 0–30 |
Yaw 30–60 |
Yaw 60–95 |
Pitch −90..−30 |
Pitch −30..−10 |
Pitch −10..10 |
Pitch 10..30 |
Pitch 30..90 |
|---|---|---|---|---|---|---|---|---|---|
| vitl-320 | 0.37 | 0.36 | 0.42 | 0.45 | 0.42 | 0.37 | 0.36 | 0.40 | 0.50 |
| vitt-256 | 0.85 | 0.81 | 0.98 | 1.01 | 0.98 | 0.85 | 0.82 | 0.93 | 1.13 |
| hg0-256 | 1.29 | 1.21 | 1.60 | 1.85 | 1.71 | 1.28 | 1.21 | 1.50 | 2.09 |
| vitt-096 | 1.27 | 1.22 | 1.46 | 1.58 | 1.48 | 1.27 | 1.22 | 1.38 | 1.71 |
| hg0-096 | 1.75 | 1.60 | 2.22 | 2.85 | 2.53 | 1.76 | 1.59 | 2.04 | 2.94 |
| n | 2939 | 672 | 85 | 143 | 1076 | 2136 | 232 | 32 |
| Model | Set | clean | mblur9 | mblur21 | warm | cool | gamma0.6 | gamma1.6 | gray | jpeg30 |
|---|---|---|---|---|---|---|---|---|---|---|
| vitl-320 | wflw_test | 1.49 | 1.57 (+5.5%) | 2.09 (+40.4%) | 1.51 (+1.4%) | 1.51 (+1.5%) | 1.50 (+1.1%) | 1.51 (+1.3%) | 1.60 (+7.8%) | 1.55 (+4.5%) |
| 300w_valid | 1.04 | 1.13 (+8.6%) | 1.46 (+40.6%) | 1.06 (+2.0%) | 1.05 (+1.2%) | 1.06 (+1.8%) | 1.05 (+1.4%) | 1.09 (+4.9%) | 1.08 (+4.3%) | |
| cofw_test | 1.21 | 1.27 (+5.3%) | 1.60 (+32.3%) | 1.22 (+1.3%) | 1.23 (+1.5%) | 1.22 (+1.1%) | 1.22 (+1.2%) | 1.27 (+5.0%) | 1.25 (+3.6%) | |
| vitt-256 | wflw_test | 3.26 | 3.32 (+1.8%) | 4.60 (+41.1%) | 3.32 (+1.9%) | 3.33 (+2.1%) | 3.35 (+2.7%) | 3.36 (+3.0%) | 3.48 (+6.6%) | 3.36 (+3.1%) |
| 300w_valid | 2.65 | 2.73 (+2.6%) | 3.66 (+37.7%) | 2.68 (+1.1%) | 2.69 (+1.3%) | 2.71 (+2.0%) | 2.72 (+2.3%) | 2.75 (+3.7%) | 2.73 (+2.9%) | |
| cofw_test | 2.76 | 2.82 (+2.1%) | 3.51 (+27.2%) | 2.79 (+1.0%) | 2.79 (+1.1%) | 2.83 (+2.4%) | 2.81 (+1.7%) | 2.85 (+3.3%) | 2.81 (+1.8%) | |
| hg0-256 | wflw_test | 5.08 | 5.28 (+3.9%) | 8.16 (+60.5%) | 5.14 (+1.2%) | 5.23 (+2.9%) | 5.13 (+0.9%) | 5.29 (+4.2%) | 5.41 (+6.4%) | 5.35 (+5.2%) |
| 300w_valid | 3.85 | 3.88 (+0.8%) | 5.20 (+35.0%) | 3.90 (+1.2%) | 3.90 (+1.3%) | 3.88 (+0.7%) | 3.92 (+1.7%) | 3.95 (+2.5%) | 3.90 (+1.0%) | |
| cofw_test | 3.79 | 3.90 (+2.9%) | 4.93 (+30.0%) | 3.83 (+1.1%) | 3.83 (+0.9%) | 3.82 (+0.8%) | 3.82 (+0.8%) | 3.91 (+3.2%) | 3.85 (+1.5%) | |
| vitt-096 | wflw_test | 5.01 | 6.26 (+25.0%) | 24.80 (+395.2%) | 5.10 (+1.9%) | 5.09 (+1.6%) | 5.13 (+2.4%) | 5.21 (+4.1%) | 5.37 (+7.3%) | 5.36 (+7.0%) |
| 300w_valid | 3.82 | 4.42 (+15.8%) | 18.31 (+379.4%) | 3.86 (+1.1%) | 3.86 (+1.1%) | 3.90 (+2.1%) | 3.94 (+3.1%) | 3.96 (+3.8%) | 4.04 (+5.8%) | |
| cofw_test | 3.89 | 4.52 (+16.3%) | 15.95 (+310.4%) | 3.95 (+1.8%) | 3.95 (+1.6%) | 3.99 (+2.8%) | 4.00 (+2.8%) | 4.05 (+4.2%) | 4.08 (+5.0%) | |
| hg0-096 | wflw_test | 7.01 | 9.18 (+31.0%) | 32.00 (+356.7%) | 7.15 (+2.0%) | 7.30 (+4.2%) | 7.11 (+1.5%) | 7.36 (+5.0%) | 7.77 (+10.9%) | 9.10 (+29.9%) |
| 300w_valid | 4.64 | 5.68 (+22.5%) | 19.79 (+326.5%) | 4.68 (+1.0%) | 4.74 (+2.1%) | 4.70 (+1.2%) | 4.72 (+1.7%) | 4.80 (+3.5%) | 5.60 (+20.8%) | |
| cofw_test | 5.03 | 5.71 (+13.7%) | 18.03 (+258.9%) | 5.07 (+0.8%) | 5.15 (+2.4%) | 5.04 (+0.3%) | 5.16 (+2.6%) | 5.19 (+3.2%) | 5.74 (+14.2%) |
license_tag). Head boxes and face parts come from DEIMv2-Wholebody49 (Apache-2.0) pseudo labels; extreme poses are reinforced with depth-reprojection synthesis and generated image pools.epochs and resuming.hrffa.train.evaluate): official (inter-ocular NME / [email protected] / [email protected]), stratify-real (stratified by 6DRepNet-estimated pose), pose-stress (real images + exact GT geometric transforms to measure equivariance under 360° roll and camera pose), and style-shift (motion blur, color temperature, gamma, grayscale, JPEG).hrffa.export.export_onnx): a static batch-1 graph is optimized with onnxslim → onnxsim (without Gemm fusion), checked for parity against PyTorch, and an N-batch variant (<stem>_n.onnx) is derived from it. The batch axis stays the leading axis of every Reshape and no rank-5 tensors are produced.| Teacher vitl-320 | Students vitt-256 / vitt-096 | Students hg0-256 / hg0-096 | |
|---|---|---|---|
| Backbone | DINOv3 ViT-L/16 (the official implementation is imported at runtime from the torch.hub cache; its code is not vendored) | ViT-T/16 (own implementation with RoPE, initialized from ckpts/vitt_distill.pt) |
PP-HGNetV2-B0 stages 0–2 (own implementation) + FPN (stride 8) |
| Head | Point-query Transformer decoder (d256, 4 layers) with coordinate regression and visibility | Same (d256, 3 layers) | Same (d128, 2 layers, 4 heads) |
| Input normalization | ImageNet mean/std | center05 (x/255 − 0.5)/0.5 (no normalization op inside the ONNX graph) |
center05, folded into the stem conv (no input op at all) |
| Preset | clean_v3 |
student_s256_96gb_r2 / student_s096_96gb_r2 |
student_hg0_wsd / student_hg0_s096_wsd |
ONNX (example for --scheme ibug68; wflw98 / cofw29 are exported as separate files):
# RGB, normalized as above. S = 320 / 256 / 96 (the preset's out_size)
input images float32 [N, 3, S, S]
# coordinates relative to the crop (0..1 inside the crop; points outside still get coordinates)
output points float32 [N, K, 2]
# visibility logits (0 = outside the image, 1 = occluded, 2 = visible)
output vis_logits float32 [N, K, 3]
*.onnx and *_n.onnx with a symbolic batch axis N. Both are verified with onnxruntime to agree at batch sizes 1 / 2 / 3.Prerequisites: Linux, Python 3.13 (pinned in .python-version), uv, and an NVIDIA GPU (torch built for CUDA 12.8; an 8 GB GPU is enough for smoke tests, full training targets a 96 GB class GPU). Dependencies are pinned with == in pyproject.toml and hash-locked in uv.lock.
git clone https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment.git
cd High-Angle_Robust_Fast_FaceAlignment
uv sync --frozen # default: onnxruntime-gpu 1.22.0 (the validated toolchain)
uv sync --frozen --no-group ort --group tensorrt # alternative: onnxruntime-gpu 1.26.0 for TensorRT BF16 (trt_bf16_enable)
# NOTE: `uv run` re-syncs the environment with the DEFAULT groups before running, i.e. a plain
# `uv run python ...` silently switches back to 1.22.0. Keep the same group flags on `uv run`
# (or use `uv run --no-sync` / `.venv/bin/python` after the manual sync):
uv run --no-group ort --group tensorrt \
python demo/demo_hrffa_onnx.py \
-v 0 \
-o output_dir \
-d tensorrt
uv run python -m unittest tests/test_model_arms.py
ckpts/, none of them are bundled)| File | Purpose | Source /