GitLab

One-sentence description

Add face recognition to photos

Motivation

Because it is really nice to be able to easily see all the photos taken of yourself.

Desired functionality

I think there are two ways to implement this:

1. A single 'album' that shows all the pictures taken of you in reverse chronological order (oldest last).
2. An option to only show photos of you in every photo album

My preference would be option 1. This is what I implemented as proof-of-concept.

Suggested implementation

Doing face recognition without GPU is really easy when using the face_recognition library. It is written in Python, has good documentation and an easy API. We would only need to change our docker images to include dlib somehow. An example of a Dockerfile to do this is in the PoC.

You might already have wondered how to do this without having to do an expensive operation every time you want to obtain the photos with your face.

In my PoC I indexed all faces on the 'large' image size using the API, which is a bit slow. But that is to be expected when you're loading images from a remote server. We can do the same, save the face encodings in a database and connect them to the photos they belong to. Then a user can upload images (no real need to save those, but it is possible) from which we will extract face encodings which we can compare using a SQL query to get similar faces. That should be a fast way to compare hundreds of encodings and option 1 is done!

Getting the face encodings from uploaded images is possible either when we're resizing and uploading or we can run a cronjob of some sorts to get those encodings.

If we really want to make this fancy we could include the face boxes (location of faces) in the images and add a possibility to tag people. That would assign the face encodings we extracted. But that's not easy and fun anymore probably.