Data vs Architecture: What Matters More?
In the early years of deep learning, much research effort was spent on architecture design, inventing new network structures to squeeze more performance from fixed datasets. The emerging consensus from large-scale empirical studies is that data scale and diversity dominate architecture choice once model capacity is sufficient. A larger, more diverse dataset with a simple architecture typically outperforms a small, curated dataset with a complex architecture.
For face recognition specifically, the jump from the LFW-era datasets (thousands of images) to VGGFace2-scale datasets (millions of images across thousands of identities, multiple conditions per identity) produced dramatically larger accuracy gains than any single architectural innovation of the same period.
What Makes a Good Face Recognition Dataset?
Four properties define a high-quality face recognition training dataset: Scale, enough images per identity (>20 ideally) and enough identities (>5000) to learn a well-generalised notion of face similarity. Diversity of conditions, each identity photographed under varied lighting, angles, ages, and expressions, so the model learns condition-invariant representations. Demographic balance, representation of all demographic groups that will appear at deployment. Label accuracy, correct identity labels; mislabelled photos corrupt the training signal.
Well-known research datasets aim for exactly these properties. VGGFace2 has 9,131 identities and over 3.31 million images chosen for variety in pose, age and lighting. MS1MV2, the dataset Ollie's network was trained on, is larger still: about 5.8 million images of 85,742 people.
The Transfer Learning Shortcut
Training face recognition from scratch on millions of images is computationally expensive, requiring days of GPU compute. Transfer learning makes this tractable for most applications: start with a model pre-trained on a large dataset (like VGGFace2), then fine-tune the final layers on your specific task or with task-specific augmentation.
The pre-trained backbone has already learned general face representations that are useful across tasks, and fine-tuning adapts them to a new domain. Ollie took the harder route: its 20-layer network was trained from scratch on MS1MV2, with no pretrained weights, which took about 10 days on a single RTX 4060 Ti.