The Masking Challenge
Face masks occlude approximately the lower half of the face, the nose and mouth region. For face recognition systems trained primarily on unmasked faces, this was a significant performance degradation: NIST testing in 2020 found that even the best algorithms failed to match 5% to 50% of masked photos, depending on mask type and coverage.
The primary issue was that many systems had learned to weight lower-face features significantly. The lower face, particularly mouth width and shape, chin configuration, and jaw profile, contains substantial identity information. Occluding this region removed information that many systems had relied on heavily.
How the Field Adapted
Two primary adaptations emerged: masked-face fine-tuning, augmenting training data with synthetic masks applied to face images, forcing the network to learn effective representations from upper-face features alone, and periocular recognition, focusing recognition on the eye and eyebrow region that remains visible behind masks.
Post-COVID face recognition models are significantly more robust to partial face occlusion than pre-pandemic models. The forced adaptation produced better-than-expected upper-face recognition capability: the eye and brow complex, it turns out, contains more identity information than the lower face at equivalent quality, and modern systems exploit this effectively.