The Variability Problem in Face Matching
A face recognition system receives a stream of pixel values, not a face in any semantic sense. A slight change in lighting can alter hundreds of thousands of those values while the actual face remains unchanged. This is the core challenge: the input signal is highly sensitive to conditions that have nothing to do with identity. A photo taken in bright sunlight differs enormously at the pixel level from a photo taken indoors, even when the subject is identical.
Modern systems address this using deep neural networks trained on deliberately varied datasets, the same people photographed in many different lighting conditions, angles, and environments. This forces the network to learn features that survive variation, effectively separating surface appearance from structural identity. The result is much more robust than early systems, but not perfectly stable. Sufficiently different photos can still produce noticeably different top matches.
Lighting: The Biggest Source of Variation
Lighting is the single largest source of variation in face recognition results. Harsh directional light creates shadows that substantially alter the contrast map the network processes. A shadow across one side of the face can reduce apparent nose width, deepen apparent eye socket depth, and alter the apparent jaw angle. These changes shift the facial embedding toward celebrities with different geometric profiles.
Backlighting, a bright window or lamp behind you, is particularly problematic. It can reduce the face to a near-silhouette, eliminating the detail the recognition network depends on. Coloured lighting (neon signs, coloured gels, coloured walls reflecting onto your face) shifts skin tone values outside the distribution the network was trained on, further degrading embedding quality.
Camera Angle and Lens Distortion
Camera angle is the second major source of variation. A face rotated even 15–20 degrees from front-facing appears different in the network's early feature layers, because the projections of three-dimensional facial structures change. Nose width and prominence, ear visibility, apparent jaw width, and chin projection all shift substantially with pose.
The lens used also matters significantly. The front-facing (selfie) camera on most smartphones uses a wide-angle lens that creates perspective distortion at close range, making the nose appear 20–30% wider and the jaw narrower than in a normal portrait photo. This systematic distortion shifts the embedding toward celebrities with wider noses and narrower jaws. Photos taken with the rear camera at arm's length minimise this distortion and produce more stable embeddings.
Image Quality and Compression
Low-resolution images and heavy JPEG compression both degrade matching accuracy. A face that occupies fewer than 100 pixels of width in an image does not contain enough detail for reliable feature extraction. Compression artefacts, the blocky distortions introduced by high JPEG compression, add structured noise to the image that can resemble genuine facial features, shifting the embedding unpredictably.
Screenshots from video calls, images downloaded from social media (which apply heavy compression), and crops from large group photos are the most common sources of quality-related variation. Uploading the original, uncompressed file from your camera app produces the most reliable results.
How to Get Consistent Results
The conditions that produce the most stable facial embeddings are: even, diffuse lighting from the front; a front-facing pose with the camera at roughly eye level; rear camera shot from at least 50–60 cm distance; and sufficient image resolution (face at least 200 pixels wide). A photo taken near a window on a bright overcast day checks most of these boxes simultaneously.
If you upload multiple photos and receive different top matches, look at which conditions differed. Consistent results across different photos indicate strong geometric similarity to a particular celebrity. Shifting results often indicate that different photos are emphasising different features due to lighting or angle, each photo is pulling the embedding in a slightly different direction in the feature space.