Replacing our face engine with an MIT-licensed stack — and beating it on accuracy

Until this week, the default face engine behind this API was the InspireFace SDK with its Megatron model pack — a ResNet-18 ArcFace trained on Glint360K. It is good. It is also an InsightFace research release with no commercial grant: shipping it in a paid product means negotiating an agreement with the InsightFace team, and our fallback path (SCRFD + ArcFace from the buffalo_l pack) carries exactly the same restriction.

For a cloud API that’s a licensing conversation. For the thing we actually sell hardest — a self-hosted single binary that a bank can run on its own infrastructure — it’s a blocker. You cannot hand a customer a binary whose core models they have no right to run commercially.

So the constraint was set upfront: find a stack that is permissively licensed end to end, and refuse to regress on accuracy. This post is the record of that swap — including the approaches we tried and rejected, because those are usually the useful part.

The recognition market has a strange shape: the strongest open checkpoints are mostly research-only, but a few genuinely permissive ones exist if you look past the leaderboards.

YuNet (OpenCV Zoo, face_detection_yunet_2023mar) is an MIT-licensed face detector that also emits the 5-point landmarks needed for alignment. AdaFace (mk-minchul), an IR-50 trained on WebFace4M, has its weights released by the authors under MIT. SFace (OpenCV Zoo) is Apache-2.0 and went in as a baseline. Our passive anti-spoof model, MiniFASNet, was already Apache-2.0 across all backends — recognition and detection were the gap.

Full license details, provenance notes and where each license text ships are in the repository’s THIRD_PARTY_LICENSES.md.

We benchmarked on the LFW View-2 protocol: 6,000 labeled pairs, half genuine, half impostor, scored with engine-side detection and alignment — the same protocol as our first real-data run on the production engine on 13 August. The shootout harness (scripts/clean_swap_bench.py, Python + onnxruntime) was parity-checked first: Megatron re-scored 3.23% EER on it versus the 3.26% our Rust eval binary had recorded, which is agreement well within run noise.

The shootout results, equal-error rate on the same 6,000 pairs:

LFW View-2, 6,000 pairs, engine-side detect + align, Python harness. Full sweep and reproduction command in EVAL.md (2026-08-30 entry).

The headline finding is the gap between row two and row four: same MIT model, same detector, 1.57 points of EER apart. Two cheap pipeline changes did it:

Run the detector on a 2× upscaled image. LFW originals are 250px; faces are small, and YuNet’s five landmarks get noticeably sharper when the detector sees a 2× cubic upscale. Better landmarks mean better alignment, and recognition accuracy lives and dies on alignment.

Flip test-time augmentation. Embed the face and its horizontal mirror, use both. Doubles embedding compute, costs nothing at the accuracy level, and buys a consistent chunk of the gap.

Just as instructive were the things that didn’t work:

MTCNN alignment — AdaFace’s own published eval path — made things worse. 6.34% EER and 39 outright detection failures on these 250px originals. The lesson: an upstream repo’s evaluation pipeline is tuned for its benchmark conditions, not yours. Do not “upgrade” your landmark source because a paper used it.

The bigger model added nothing. IR-101 trained on WebFace12M scored 3.30% against IR-50’s 3.33%, with a 0.994 score correlation between them — at that point you have hit the protocol floor (LFW has label noise on the order of dozens of pairs). We ship IR-50: half the size (174MB of ONNX) and about twice as fast.

Score fusion was tested and rejected. Averaging in SFace dragged EER to 5.17%; fusing IR-101 with IR-50 was flat. Fusion is only worth its complexity when the models disagree usefully. These didn’t.

The production engine is pure Rust, so the winning Python pipeline had to be ported: YuNet decoding in core::yunet, AdaFace inference through tract, and the detector’s letterbox upscale capped at 2.0× with cubic resize to match the Python behaviour exactly. Then we re-ran the full 6,000 pairs through the shipped binary:

LV__ENGINE__BACKEND=clean ./target/release/eval pairs_full.csv

EER 3.05% @ cosine 0.126   (6,000 pairs, 0 skipped)
FMR = 0 at every threshold >= 0.28
FNMR @ 0.32 = 3.67%
per-pair parity vs Python: median |Δcosine| = 0.002

3.05% EER, zero skipped pairs — numerically ahead of both the Python reference (3.33%) and the Megatron engine it replaces (3.23%). And the operational comparison is cleaner than the EER one: Megatron could not produce embeddings for 19 of the 6,000 pairs and was scored with its 19 hardest cases excluded. The clean stack scored every pair. In production, a detection failure is not an excluded row — it’s a user who cannot complete KYC.

The shipped operating point (score midpoint 0.30, match_threshold 55) lands at cosine ~0.317, which on this set means roughly 3.7% FNMR at zero measured false matches — strictly better than the old Megatron operating point, which sat near 15% FNMR after our August recalibration.

The next accuracy work is an India-representative evaluation set; per our benchmarks policy, numbers from it get published with the dataset and demographic breakdown attached or not at all.

Model licensing is invisible in a sales deck and very visible in procurement. If a vendor offers you an on-prem deployment, ask what license the face models inside it carry and who granted the commercial right to run them — it’s on our list of questions to ask any face-KYC vendor. As of v0.3.0, our answer is simple: the clean backend runs YuNet (MIT), AdaFace IR-50 (MIT) and MiniFASNet (Apache-2.0) on the MIT/Apache-2.0 tract runtime, license texts ship next to the model files, and the eval harness to check our accuracy claims ships in the same repository.

One honest footnote on provenance: AdaFace was trained on WebFace4M, a dataset distributed for research. The model authors released the resulting weights under MIT, and that grant governs the artifact we ship — this is industry-standard practice, and materially different from models whose own license forbids commercial use. We would rather state that distinction plainly than let you find it in due diligence.