What Confidence Means in AI
In the context of face matching, a confidence score (or similarity score) represents the model's estimate of how similar two face embeddings are, not a direct statement about whether the match is 'correct' in any ground truth sense. A score of 95% means the embeddings are close together in the 512-dimensional space; it does not mean there is a 95% probability that this is the 'right' answer, because there may not be a single right answer.
For celebrity lookalike matching, there is no ground truth. You are looking for the closest match in a database, the result is definitionally the most similar face, not a correct or incorrect identification. In this context, the score is genuinely meaningful as a relative similarity measure.
Calibration: When Confidence and Accuracy Align
For applications where there is a ground truth, for example, two photos should show the same person or different people, calibration is the property that confidence scores accurately reflect actual accuracy. A well-calibrated model produces scores where faces scored at 80% similarity are correctly matched 80% of the time; faces at 90% are correctly matched 90% of the time. A poorly calibrated model might be confidently wrong, consistently reporting high similarity for mismatched pairs.
Ollie's percentages are not calibrated in this sense. They are distances between face fingerprints, rescaled so the differences between your top matches are easy to see. They are good for ranking, but an 80% doesn't mean an 80% chance of anything.
When Confident Results Are Surprising
A high confidence match that surprises you visually is telling you something: either the similarity is in dimensions not obvious to casual inspection (the embedding has found geometric relationships you are not noticing), or the photo conditions have shifted your embedding in a direction that produces an unexpected result. The response is different for each case.
Try uploading another photo under different conditions. If the same celebrity appears at high confidence across multiple photos, trust the result, the similarity is real even if not obvious. If the result changes significantly across photos, the original photo's conditions were driving the match more than your actual face geometry.