How does Face Recognition work

Everyday Image Processing

We encounter image processing every day. Often, we are probably not even aware of it.

How many of you unlock your cell phone with your face? - Probably each of you. And probably for several years already.

But how many of you know what is actually behind face recognition and how it all works? Caught? - Don't worry, we got you!

As soon as you calibrate the face recognition in your phone, a mathematical model is created based on your face. Several factors play a role here. For example, the distance between eyes or ears, the depth of your eyes, and many others. Each software takes different anchor points into account. In total, there can be up to 80 different anchor points. When unlocking the phone or logging into apps, the face is then compared with the stored model. And if it matches, the phone can be unlocked. So far so good.

By the way, facial recognition has been around much longer than smart phones were even ready for the masses and facial recognition is used several times a day, even in private environments. In the early days, facial recognition was used to identify criminals, among other things. This was not always unproblematic, as the technology was still far from perfect and false faces were recognized. It was only with the further development of artificial intelligence and neural networks that false positives became much less frequent.

By now the technology behind face recognition in smart phones involves several components, including an infrared camera, a dot projector, and a flood illuminator. When a user looks at its device, the dot projector emits a pattern of infrared dots onto its face. The infrared camera then captures an image of this pattern and sends it to the chip, where it is processed by neural networks to create a 3D representation of the user's face. This 3D model is then converted into a mathematical representation called a depth map, which is a collection of data points that represent the contours and shape of the user's face. The depth map is then fed into a neural network, which is a type of artificial intelligence algorithm that is designed to learn from data. The neural network is trained using a large dataset of faces to recognize and classify different facial features, such as the anchor points mentioned before.

In industrial sectors such as security, retail or healthcare face regocnition is also already in line. Also in vehicles like cars, methods are being tested to detect microsleep at an early stage. While the underlying technology and mathematical models used in these industries may be similar to the one explained before, there might be some differences in how they are implemented and the level of accuracy required. It depends on the environment.