High-Angle_Robust_Fast_FaceAlignment README

The project's README from github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment. As one markdown file: README.md.

HRFFA: High-Angle Robust Fast FaceAlignment

GitHub DOI

A training, distillation and ONNX deployment pipeline for whole-head face alignment that stays robust at extreme head poses while remaining light enough for CPU inference.

teacher clean_v3: extreme pitch/yaw tiles with roll 0..320 deg student vitt-256: measured pitch extremes with roll

Left: teacher vitl-320 on nine looking-up × profile tiles with a roll of 0–320° composited onto each tile.
Right: student vitt-256 on measured pitch extremes (+86° to −89°) with roll. Predictions only.

1. Results at a glance

inter-ocular NME %, lower is better; CPU latency on an i9-10900K with onnxruntime 1.22 CPU EP, batch 1.

Checkpoints and ONNX/TFLite(LiteRT): https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment/releases/tag/weights

Model Input Params GMACs GFLOPs CPU ms WFLW Full WFLW Pose 300W Full COFW
D-ViT (paper) 256 96.4M ≈73.15 ≈146.30 – 3.75 6.43 2.85 4.13
vitl-320 (teacher) 320 308.2M 131.32 262.63 520 1.51 2.54 1.03 1.21
vitt-256 256 9.0M 2.05 4.09 12.4 3.36 5.64 2.66 2.76
hg0-256 256 1.6M 0.70 1.40 5.2 5.32 9.53 3.87 3.77
vitt-096 96 9.0M 0.43 0.85 4.6 5.14 8.65 3.86 3.90
hg0-096 96 1.6M 0.13 0.26 1.5 7.46 14.71 4.77 4.99

How to read these numbers (important): all splits of the real-image datasets — including WFLW test, 300W and COFW test — are used for training. The "official" numbers above therefore measure how well the models fit the training distribution and must not be compared with published benchmark results. The only unleaked instrument is 300wlp_val (head-NME on a held-out 300W-LP subset; values in 050). The full tables — WFLW subsets, FR/AUC, breakdowns by pose (yaw / pitch / roll) and by image degradation, and the D-ViT paper values listed side by side with an explicit non-comparability note — follow in §1.1 (same content as 050 §2). The D-ViT (paper) row lists the published benchmark values of arXiv 2411.07167 (Tables 1 / 2, "Ours": 256×256 input, 8 prediction blocks = 96.4M parameters per its Table S1) for format reference only. The paper does not report FLOPs or latency; the GMACs / GFLOPs are our estimate from the official code with its default arguments (8 prediction blocks, 32×32 heatmaps, depth 256, 256×256 input, all 8 stages, batch 1, same FlopCounterMode method as the other rows) and are approximate because that configuration has 101.1M parameters, about 5 % more than Table S1. Its COFW NME is normalized by the inter-pupil distance instead of inter-ocular, and its test sets are held out from training, so no ranking against the fit measures above is implied.

1.1 Full tables

All tables below are generated from runs/<run>/eval_best_{official,stratreal,stress,style}.log by scripts/make_results_tables.py. Notes:

Table 1. NME (%, inter-ocular, lower is better)

Model WFLW
Full

Pose

Exp

Ill

Mu

Occ

Blur
COFW
Full
300W
Full

Comm

Chal
D-ViT (paper) 3.75 6.43 3.85 4.06 3.57 4.47 4.37 4.13 2.85 2.43 4.56
vitl-320 1.51 2.54 1.55 1.47 1.42 1.56 1.58 1.21 1.03 0.96 1.34
vitt-256 3.36 5.64 3.43 3.25 3.19 3.56 3.60 2.76 2.66 2.36 3.90
hg0-256 5.32 9.53 5.54 5.16 5.09 6.29 6.04 3.77 3.87 3.37 5.96
vitt-096 5.14 8.65 5.32 4.96 5.06 5.52 5.40 3.90 3.85 3.57 5.02
hg0-096 7.46 14.71 7.77 7.19 8.10 8.91 8.16 4.99 4.77 4.24 6.91

Table 2. FR10 (%, lower is better) and AUC10 (%, higher is better) on WFLW

Model FR10
Full

Pose

Exp

Ill

Mu

Occ

Blur
AUC10
Full

Pose

Exp

Ill

Mu

Occ

Blur
D-ViT (paper) 1.76 8.28 1.27 1.29 1.94 3.80 2.07 63.7 40.1 62.6 64.7 64.7 57.1 58.6
vitl-320 0.00 0.00 0.00 0.00 0.00 0.00 0.00 84.9 74.5 84.5 85.4 85.8 84.4 84.2
vitt-256 0.28 1.53 0.32 0.00 0.00 0.54 0.52 66.5 43.9 65.7 67.5 68.2 64.5 64.1
hg0-256 7.00 32.21 5.73 5.44 6.31 13.04 9.44 49.2 17.5 46.7 50.3 50.2 42.4 42.8
vitt-096 4.20 23.31 4.78 2.72 2.91 5.57 3.75 49.7 20.2 47.7 51.2 49.9 46.2 47.1
hg0-096 17.88 67.48 18.79 14.18 20.87 26.09 22.38 36.1 5.8 31.9 37.6 34.4 28.7 30.0

Table 3. NME (%) by yaw / pitch bin (effective pose bins of pose-stress)

Model Yaw
0–30
Yaw
30–60
Yaw
60–95
Pitch
−95..−45
Pitch
−45..−15
Pitch
−15..15
Pitch
15..45
Pitch
45..95
vitl-320 1.26 1.71 2.04 1.76 1.37 1.30 1.29 1.85
vitt-256 2.94 4.10 4.68 4.07 3.20 3.06 2.99 4.43
hg0-256 4.21 6.35 8.14 6.26 4.63 4.46 4.35 7.51
vitt-096 4.49 6.21 7.37 5.93 4.82 4.77 4.54 7.06
hg0-096 5.69 8.70 10.85 8.02 6.17 6.21 5.79 10.07
n 5933 1124 143 223 2503 2944 1398 48

Table 4. NME (%) by in-plane roll (pose-stress, n=300 per set)

Model Set Roll 0 45 90 135 180 225 270 315 worst−base
vitl-320 wflw 1.61 1.61 1.64 1.60 1.63 1.61 1.63 1.60 +0.02
300w 1.12 1.11 1.14 1.11 1.13 1.11 1.13 1.11 +0.01
cofw 1.29 1.30 1.31 1.30 1.30 1.30 1.30 1.29 +0.02
vitt-256 wflw 3.67 3.61 3.67 3.60 3.70 3.61 3.68 3.61 +0.03
300w 2.93 2.88 2.89 2.89 2.95 2.92 2.96 2.90 +0.03
cofw 3.01 2.96 3.02 2.97 2.98 2.96 2.98 2.97 +0.01
hg0-256 wflw 5.96 5.95 5.90 6.06 5.92 6.03 5.98 5.96 +0.10
300w 4.14 4.11 4.26 4.20 4.26 4.29 4.21 4.16 +0.15
cofw 4.12 4.08 4.06 4.06 4.03 4.11 4.12 4.06 +0.00
vitt-096 wflw 6.30 6.05 6.17 5.98 6.30 6.26 6.40 6.19 +0.10
300w 4.58 4.41 4.55 4.50 5.03 5.11 5.14 4.71 +0.57
cofw 4.74 4.61 4.72 4.49 4.73 4.49 4.56 4.55 +0.00
hg0-096 wflw 8.89 9.03 9.09 9.21 8.98 9.33 9.90 8.72 +1.01
300w 5.57 5.61 5.82 6.07 6.40 6.59 6.05 5.95 +1.03
cofw 5.84 5.79 6.05 5.71 5.76 5.51 5.68 5.73 +0.21

Table 5. NME (%) under camera pitch / yaw perturbation (pose-stress, n=300 per set)

Model Set base cam
pitch
−25


−15


+15


+25
cam
yaw
−15


+15


p+25,y+15


p−25,y−15
vitl-320 wflw 1.61 1.78 1.63 1.58 1.60 1.55 1.55 1.56 1.78
300w 1.12 1.25 1.15 1.09 1.10 1.08 1.08 1.08 1.22
cofw 1.29 1.36 1.29 1.26 1.29 1.23 1.24 1.25 1.35
vitt-256 wflw 3.67 3.84 3.67 3.52 3.48 3.52 3.50 3.44 3.94
300w 2.93 3.07 2.93 2.81 2.75 2.78 2.81 2.73 3.03
cofw 3.01 3.12 3.01 2.95 2.92 2.90 2.90 2.92 3.12
hg0-256 wflw 5.96 5.99 5.71 5.71 5.38 5.55 5.60 5.31 6.08
300w 4.14 4.46 4.24 4.05 3.98 3.97 4.04 3.90 4.39
cofw 4.12 4.27 4.11 4.02 3.96 3.98 3.97 3.99 4.23
vitt-096 wflw 6.30 5.94 5.83 5.97 5.51 5.78 5.76 5.30 6.05
300w 4.58 4.49 4.39 4.41 4.10 4.24 4.34 4.06 4.41
cofw 4.74 4.53 4.49 4.45 4.18 4.33 4.42 4.20 4.47
hg0-096 wflw 8.89 8.20 8.27 8.44 7.72 8.37 8.19 7.35 8.34
300w 5.57 5.38 5.38 5.45 4.94 5.16 5.25 4.81 5.34
cofw 5.84 5.62 5.57 5.67 5.28 5.35 5.53 5.13 5.48

Table 6. head-NME (×100) stratified by 6DRepNet-estimated pose (3,696 real images from the three sets)

Model mean Yaw
0–30
Yaw
30–60
Yaw
60–95
Pitch
−90..−30
Pitch
−30..−10
Pitch
−10..10
Pitch
10..30
Pitch
30..90
vitl-320 0.37 0.36 0.42 0.45 0.42 0.37 0.36 0.40 0.50
vitt-256 0.85 0.81 0.98 1.01 0.98 0.85 0.82 0.93 1.13
hg0-256 1.29 1.21 1.60 1.85 1.71 1.28 1.21 1.50 2.09
vitt-096 1.27 1.22 1.46 1.58 1.48 1.27 1.22 1.38 1.71
hg0-096 1.75 1.60 2.22 2.85 2.53 1.76 1.59 2.04 2.94
n 2939 672 85 143 1076 2136 232 32

Table 7. style-shift: NME (%) under image degradations (in parentheses: degradation vs clean, n=300 per set)

Model Set clean mblur9 mblur21 warm cool gamma0.6 gamma1.6 gray jpeg30
vitl-320 wflw_test 1.49 1.57 (+5.5%) 2.09 (+40.4%) 1.51 (+1.4%) 1.51 (+1.5%) 1.50 (+1.1%) 1.51 (+1.3%) 1.60 (+7.8%) 1.55 (+4.5%)
300w_valid 1.04 1.13 (+8.6%) 1.46 (+40.6%) 1.06 (+2.0%) 1.05 (+1.2%) 1.06 (+1.8%) 1.05 (+1.4%) 1.09 (+4.9%) 1.08 (+4.3%)
cofw_test 1.21 1.27 (+5.3%) 1.60 (+32.3%) 1.22 (+1.3%) 1.23 (+1.5%) 1.22 (+1.1%) 1.22 (+1.2%) 1.27 (+5.0%) 1.25 (+3.6%)
vitt-256 wflw_test 3.26 3.32 (+1.8%) 4.60 (+41.1%) 3.32 (+1.9%) 3.33 (+2.1%) 3.35 (+2.7%) 3.36 (+3.0%) 3.48 (+6.6%) 3.36 (+3.1%)
300w_valid 2.65 2.73 (+2.6%) 3.66 (+37.7%) 2.68 (+1.1%) 2.69 (+1.3%) 2.71 (+2.0%) 2.72 (+2.3%) 2.75 (+3.7%) 2.73 (+2.9%)
cofw_test 2.76 2.82 (+2.1%) 3.51 (+27.2%) 2.79 (+1.0%) 2.79 (+1.1%) 2.83 (+2.4%) 2.81 (+1.7%) 2.85 (+3.3%) 2.81 (+1.8%)
hg0-256 wflw_test 5.08 5.28 (+3.9%) 8.16 (+60.5%) 5.14 (+1.2%) 5.23 (+2.9%) 5.13 (+0.9%) 5.29 (+4.2%) 5.41 (+6.4%) 5.35 (+5.2%)
300w_valid 3.85 3.88 (+0.8%) 5.20 (+35.0%) 3.90 (+1.2%) 3.90 (+1.3%) 3.88 (+0.7%) 3.92 (+1.7%) 3.95 (+2.5%) 3.90 (+1.0%)
cofw_test 3.79 3.90 (+2.9%) 4.93 (+30.0%) 3.83 (+1.1%) 3.83 (+0.9%) 3.82 (+0.8%) 3.82 (+0.8%) 3.91 (+3.2%) 3.85 (+1.5%)
vitt-096 wflw_test 5.01 6.26 (+25.0%) 24.80 (+395.2%) 5.10 (+1.9%) 5.09 (+1.6%) 5.13 (+2.4%) 5.21 (+4.1%) 5.37 (+7.3%) 5.36 (+7.0%)
300w_valid 3.82 4.42 (+15.8%) 18.31 (+379.4%) 3.86 (+1.1%) 3.86 (+1.1%) 3.90 (+2.1%) 3.94 (+3.1%) 3.96 (+3.8%) 4.04 (+5.8%)
cofw_test 3.89 4.52 (+16.3%) 15.95 (+310.4%) 3.95 (+1.8%) 3.95 (+1.6%) 3.99 (+2.8%) 4.00 (+2.8%) 4.05 (+4.2%) 4.08 (+5.0%)
hg0-096 wflw_test 7.01 9.18 (+31.0%) 32.00 (+356.7%) 7.15 (+2.0%) 7.30 (+4.2%) 7.11 (+1.5%) 7.36 (+5.0%) 7.77 (+10.9%) 9.10 (+29.9%)
300w_valid 4.64 5.68 (+22.5%) 19.79 (+326.5%) 4.68 (+1.0%) 4.74 (+2.1%) 4.70 (+1.2%) 4.72 (+1.7%) 4.80 (+3.5%) 5.60 (+20.8%)
cofw_test 5.03 5.71 (+13.7%) 18.03 (+258.9%) 5.07 (+0.8%) 5.15 (+2.4%) 5.04 (+0.3%) 5.16 (+2.6%) 5.19 (+3.2%) 5.74 (+14.2%)

2. Features

3. Models and the ONNX I/O contract

Teacher vitl-320 Students vitt-256 / vitt-096 Students hg0-256 / hg0-096
Backbone DINOv3 ViT-L/16 (the official implementation is imported at runtime from the torch.hub cache; its code is not vendored) ViT-T/16 (own implementation with RoPE, initialized from ckpts/vitt_distill.pt) PP-HGNetV2-B0 stages 0–2 (own implementation) + FPN (stride 8)
Head Point-query Transformer decoder (d256, 4 layers) with coordinate regression and visibility Same (d256, 3 layers) Same (d128, 2 layers, 4 heads)
Input normalization ImageNet mean/std center05 (x/255 − 0.5)/0.5 (no normalization op inside the ONNX graph) center05, folded into the stem conv (no input op at all)
Preset clean_v3 student_s256_96gb_r2 / student_s096_96gb_r2 student_hg0_wsd / student_hg0_s096_wsd

ONNX (example for --scheme ibug68; wflw98 / cofw29 are exported as separate files):

# RGB, normalized as above. S = 320 / 256 / 96 (the preset's out_size)
input   images      float32 [N, 3, S, S]
# coordinates relative to the crop (0..1 inside the crop; points outside still get coordinates)
output  points      float32 [N, K, 2]
# visibility logits (0 = outside the image, 1 = occluded, 2 = visible)
output  vis_logits  float32 [N, K, 3]

4. Setup

Prerequisites: Linux, Python 3.13 (pinned in .python-version), uv, and an NVIDIA GPU (torch built for CUDA 12.8; an 8 GB GPU is enough for smoke tests, full training targets a 96 GB class GPU). Dependencies are pinned with == in pyproject.toml and hash-locked in uv.lock.

git clone https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment.git
cd High-Angle_Robust_Fast_FaceAlignment
uv sync --frozen                                   # default: onnxruntime-gpu 1.22.0 (the validated toolchain)
uv sync --frozen --no-group ort --group tensorrt   # alternative: onnxruntime-gpu 1.26.0 for TensorRT BF16 (trt_bf16_enable)
# NOTE: `uv run` re-syncs the environment with the DEFAULT groups before running, i.e. a plain
# `uv run python ...` silently switches back to 1.22.0. Keep the same group flags on `uv run`
# (or use `uv run --no-sync` / `.venv/bin/python` after the manual sync):
uv run --no-group ort --group tensorrt \
python demo/demo_hrffa_onnx.py \
-v 0 \
-o output_dir \
-d tensorrt

uv run python -m unittest tests/test_model_arms.py

4.1 Weights (ckpts/, none of them are bundled)

| File | Purpose | Source /

High-Angle_Robust_Fast_FaceAlignment (PINTO0309/High-Angle_Robust_Fast_FaceAlignment)

Searching Database…

Что это?HRFFA — это конвейер обучения, дистилляции и развёртывания ONNX для выравнивания лица по кропам всей головы, который остаётся устойчивым в экстремальных позах головы (взгляд вверх/вниз, полный поворот на 360°, вид в профиль) и при этом достаточно лёгкий для инференса на CPU.
Он работает по тесно обрезанным лицам или по кропам всей головы?Он намеренно работает по кропам всей головы, а не по тесно обрезанным лицам: детекторы объектов находят головы намного надёжнее, чем лица, а дополнительный контекст головы и фона повышает точность в экстремальных позах.
Как запустить модель?Можно взять один из предобученных чекпоинтов ONNX/TFLite (например, vitt-256, hg0-256), перечисленных в таблице результатов, и запустить его напрямую либо использовать прилагаемые скрипты обучения/дистилляции, чтобы воспроизвести модели или переобучить их с нуля.
Можно ли сравнивать приведённые цифры WFLW/300W/COFW с другими работами?Нет — все тестовые выборки на реальных изображениях (WFLW test, 300W, COFW test) здесь включены в обучение, поэтому приведённые цифры показывают, насколько модели подогнаны под собственное обучающее распределение, а не способность к обобщению, и прямо не сравнимы со стандартными опубликованными бенчмарками.
Могут ли экспортированные модели работать полностью офлайн на CPU?Да — дистиллированные модели ONNX/TFLite рассчитаны на локальное выполнение на CPU без зависимости от сети и делают один инференс за 1.5-12 мс на обычном настольном процессоре.
В чём разница между моделью-учителем и дистиллированными моделями-учениками?Учитель — это DINOv3 ViT-L/16 на 320x320 с 308 млн параметров, занимающая на CPU около 520 мс, а ученики (ViT-T/16 и CNN PP-HGNetV2-B0, каждый в вариантах на входы 256 и 96) обучены онлайн-дистилляцией и работают за 1.5-12 мс при 1.6-9 млн параметров.
Почему лица выравниваются по кропу головы, а не по тесно обрезанному лицу?Потому что детекторы объектов находят целые головы гораздо надёжнее, чем лица, а подача всей головы с полосой фона даёт модели больше контекста — хотя из-за этого прямое сравнение с бенчмарками по кропам лиц становится некорректным.
Какие экстремальные позы и выходные лицевые точки поддерживает модель?Она справляется с замеренным тангажом за пределами ±85° (взгляд вверх/вниз), полным поворотом на 360° в плоскости изображения и рысканием до ±90° (вид в профиль), предсказывая 68, 98 или 29 лицевых точек плюс для каждой точки метку видимости из трёх классов (видима, перекрыта или вне изображения).
What are similar projects?RelateAnything, yolov3, YOLOX, yolov7, yolov5, NanoJev, SolarWM, pointdit
Creator
PINTO0309
Size
1 MB
Stars
195
Age
1mo
Commits · 90 Days