Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cast a wider net for face similarity #84

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

amacinho
Copy link

This is obviously not a pull-request. I also didn't have time to learn drizzle so it's a Frankestein code. However, I'm creating this PR as a concept of a proposal (!) to improve person merging.

Current problem: Only one face is chosen for the template (the person to be merged with) and one face for all other targets (people we pick to merge) to compare against. If I'm not mistaken, these faces are picked arbitrarily and the similarity between two such faces is a very noisy approximate of the actual similarity between the two people. Maybe the order Postgresql returns the faces from asset is not arbitrary and it returns some special face that is prototypical for the person, but in any case using just one face per person is very suboptimal.

Ideally, we could compute an all-pairwise similarity between all faces of the template person and all faces of the target people and compute a distributional measure. This is prohibitive, however, and there is no obvious choice for the similarity measure.

Just to experiment with it myself, I implemented (or rather, tried to implement) the following approach:

  1. Pick 10 random faces of the template person.
  2. Compare these 10 faces to all other faces.
  3. For each target person, compute the average similarity over all their faces (after filtering out faces with a similarity lower than 0.15)
  4. Pick top 12 target people based on average similarity.
  5. Use most similar faces of these people as recommendations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant