Verification vs Identification
Benchmarks test one of two tasks, and the numbers mean different things:
- Verification (1:1). Are these two photos the same person? Think phone unlock or passport gates. Reported as accuracy, or as the true accept rate at a fixed false match rate.
- Identification (1:N). Who in this gallery is this person? Think searching a database. Reported as rank-1 accuracy or the miss rate.
A celebrity lookalike search is closest to identification, except there's no correct answer: you're not in the gallery, so the system returns the nearest neighbours rather than a match.
LFW: The Classic
Labeled Faces in the Wild was released by the University of Massachusetts Amherst in 2007. It has 13,233 images of 5,749 people collected from news photos. The standard test uses 6,000 pairs (3,000 same-person, 3,000 different-person) split into ten folds.
LFW drove a decade of progress. Early methods scored in the 70s and 80s. In 2014, Facebook's DeepFace reached 97.35%, close to human performance on the same tightly cropped faces (about 97.5%). Today the best models score above 99.8%, and the remaining errors are partly mislabelled pairs.
Its limits are well known: mostly front-facing, well-lit news photos, a heavy skew toward white men in the public eye, and no real challenge left for modern models. It's still useful as a sanity check, which is why Ollie reports 99.5% on it, next to its scores on the harder tests below.
Read more: VGGFace2 Explained: 3.3 Million Faces of 9,131 People
Harder Tests Built on the Same Idea
When LFW became too easy, researchers built tests that target specific weaknesses:
| Benchmark | Tests | Size | Top scores (approx.) |
|---|---|---|---|
| CFP-FP | Front vs profile photos | 7,000 images, 500 people | ~98 to 99% |
| AgeDB-30 | Photos about 30 years apart | 16,488 images, 568 people | ~98% |
| CALFW | LFW pairs with big age gaps | Based on LFW | ~96% |
| CPLFW | LFW pairs with big pose gaps | Based on LFW | ~93 to 95% |
These are quick to run and are reported in most face recognition papers alongside LFW. A model that scores 99.8% on LFW but much lower on CPLFW is struggling with turned heads.
Ollie's own 20-layer model scores 96.5% on AgeDB-30 (photos about 30 years apart), 95.2% on CALFW (big age gaps) and 88.7% on CPLFW (turned heads). Turned heads are its weakest spot, which is why front-facing photos give the best matches.
IJB-C: Realistic Templates
The IARPA Janus Benchmarks (IJB-A in 2015, IJB-B in 2017, IJB-C in 2018) were funded by the US intelligence research agency to reflect real investigative work. IJB-C has 3,531 people, 31,334 still images and 11,779 videos, with deliberately hard conditions: extreme poses, poor light, occlusion and low resolution.
Instead of comparing single photos, IJB-C compares "templates": all the images and video frames of a person from one source. And it reports performance at very strict thresholds, such as the true accept rate when only 1 in 10,000 or 1 in a million different-person pairs may be wrongly accepted. Top models score around 96% at the 1-in-10,000 level. That's a far tougher bar than LFW's overall accuracy.
Read more: Face Recognition Accuracy Across Demographics: What the Research Shows
Get new guides by email
NIST FRVT: The Industry Standard
The US National Institute of Standards and Technology runs ongoing evaluations of commercial face recognition algorithms, now called FRTE (Face Recognition Technology Evaluation), long known as FRVT. Companies submit their algorithms and NIST tests them privately on government datasets of millions of images: visa photos, mugshots, border crossing photos and more.
Two things make NIST's tests stand out. The data is never released, so nobody can tune their model to it. And NIST reports results by demographic group. Its 2019 report (NISTIR 8280) tested 189 algorithms and found that many had higher false match rates for some groups, including African and East Asian faces, women, children and older people, with large differences between vendors. The best algorithms showed much smaller gaps.
For more on that issue, see accuracy across demographics.
How to Read a Benchmark Claim
When an app or paper quotes a number, a few questions help:
- Which benchmark? 99% on LFW is ordinary today. 99% on IJB-C at a strict threshold is excellent.
- Was the test data kept out of training? Ollie removed every LFW identity from its training data before testing.
- At what threshold? Verification rates mean little without the false match rate they were measured at.
- Any demographic breakdown? An overall number can hide large differences between groups.
- Does it match the task? A model tuned for passport photos may do worse on selfies, and vice versa.