How Ollie's Celebrity Lookalike AI Works: A Technical Overview

The Pipeline at a Glance

When you upload a photo to Ollie, it goes through four steps. Each has one job.

The whole thing takes a second or two, and most of that is uploading the photo. Knowing the steps also explains where results can go wrong, so it's worth going through them.

Step 1: Finding Your Face

Detection uses YuNet, an open-source face detector that ships with OpenCV. It scans the image for faces and, for each one, places five landmarks: the centre of each eye, the tip of the nose and the two corners of the mouth.

If there's more than one face, Ollie keeps the largest. That's usually the person who took or is the subject of the photo, but in a group shot it might not be you. If no face is found, usually because the photo is too dark, too small or the face is turned too far, you'll get an error rather than a guess.

Read more: How Does AI Facial Recognition Actually Work? A Plain-English Guide

Step 2: Lining It Up

Faces in photos come at all sizes and angles. The network needs them in the same position every time. Alignment uses the five landmarks to work out a transformation (rotate, scale and shift) that moves those points to fixed target positions, then crops a 112 by 112 pixel square.

After alignment, your eyes sit at the same pixel positions as the eyes in every training photo and every celebrity photo. This is why a tilted head barely matters: it's simply rotated back. A head turned to one side is harder, because the hidden side of the face can't be recovered. See the best angle for a face match.

Step 3: The Neural Network

This is the part we built ourselves. The aligned crop goes through a 20-layer convolutional neural network with the SphereFace architecture, which uses residual connections so that information flows easily through the layers. The last layer outputs 512 numbers, which are scaled to unit length. That list is your face embedding, sometimes called a facial fingerprint.

The network was written and trained from scratch in PyTorch, with no pretrained weights:

During training the network was never told what a nose or jaw is. It learned which patterns tell people apart because getting that right was the only way to reduce its error.

Read more: How Ollie's Celebrity Database Was Built

Step 4: Searching the Celebrities

Ollie's index holds over 5,000 celebrities and over 40,000 photos, all freely licensed from Wikimedia Commons, with 2 to 12 verified photos per person. Every one of those photos went through steps 1 to 3 ahead of time, so their embeddings are already stored. How the database was built is covered in how Ollie's celebrity database was built.

Your embedding is compared with all of them in one matrix multiplication, which takes milliseconds. Because every embedding has unit length, the comparison is cosine similarity: the closer two embeddings point in the same direction, the more alike the faces.

Each celebrity is scored by their single closest photo, so you see the exact photo that matched you best. The raw scores are rescaled into percentages that are easier to read. There are no hand-made adjustments for skin tone, gender or anything else in the score itself.

By default, results include everyone. You can limit them to men or women, using each celebrity's gender as recorded on Wikidata, and narrow results to actors, singers or footballers.

What Happens to Your Photo

Your photo is processed in memory on the model server and never written to disk or logged. Once your matches are returned, it's gone. For each search, Ollie stores only an account ID (or a hashed IP address for guests), the IP address and a timestamp, to enforce search limits. Those records are deleted after 90 days.

Where It Can Go Wrong

Knowing the pipeline makes the failure modes easy to predict:

The fixes are mostly about the photo. See six ways to get better results.