The Concept: Borrowed Knowledge
Transfer learning exploits the fact that neural networks trained on large datasets develop general, reusable representations. A network trained to recognise face identity must learn about edges, shapes, skin texture, facial geometry, and identity-relevant proportions. These representations are useful not just for the specific faces in the training set but for any face recognition task.
Rather than learning these representations from scratch, which requires enormous amounts of data and compute, transfer learning takes a pre-trained network and adapts it for a new task by fine-tuning on new data. The pre-trained network provides a rich starting point; the fine-tuning adjusts it for the specific target task.
How Fine-Tuning Works
Fine-tuning typically freezes the early layers of the pre-trained network (which encode general low-level features) while allowing the later layers (which encode task-specific high-level features) to update. This preserves the learned general representations while adapting the task-specific representations for the new domain.
The learning rate for fine-tuning is typically much lower than for initial training, a small adjustment to an already-useful representation rather than large gradient steps from random initialisation. Ollie's own network wasn't fine-tuned: it was trained from scratch on MS1MV2, so its early layers learned their edge and texture detectors from face photos directly.
Why It Works So Well
Transfer learning works because the representations learned by large-scale pre-trained models are genuinely general. The features a network learns to distinguish millions of face identities turn out to be exactly the features needed to match any new pair of faces, not because of task-specific engineering, but because the same structural information that discriminates between celebrities also discriminates between any two people.
This generalisation is what allows a model trained entirely on celebrities to work for ordinary user uploads. The model has never seen your face, but the representation it learned from millions of celebrity photos is rich enough to place your face accurately in the same space.