Recognizing human subjects under various shooting environments: image analysis technology that uses AI to identify faces
From unlocking smartphones to authenticating identities for access to buildings or making payment, facial recognition technology has come to be used for person identification. Technological advancements have improved its precision, such that it is increasingly adopted in a variety of fields.
Facial recognition does not require keys or inputting passcodes, and therefore does not bear the risk of being lost, stolen, or forgotten. Unlike fingerprint or finger vein biometric authentication, it can be conducted contactlessly, thus avoiding hygiene concerns. Furthermore, it does not require the use of additional devices, thus enabling easy proof of entry and exit. For example, when installed in walkthrough gates, it allows smooth ID authentication so that you can enter and exit through the gates without stopping in front of the camera.
Image showing face detection at a walkthrough gate
However, differences in imaging conditions such as brightness, face angle, lighting conditions such as backlight, as well as alterations to a person’s external features such as hairstyle changes, aging, or wearing masks or glasses can reduce the accuracy with which faces are detected and identified.
Examples of situations where facial recognition is difficult
Advantages of facial recognition
(1) Convenience—hands-free use; simple
(2) Contactless use; hygienic
(3) Only needs a camera (does not require special equipment)
(4) No risk of loss or theft
(1) Differences in shooting condition, such as backlight or face angle
(2) Altered hairstyle or aging-related facial feature changes
(3) Altered appearance due to glasses or face coverings
Facial recognition works by automatically making one-on-one comparisons between images of faces preregistered in a database and those of faces captured in the place or situation in which facial recognition is to be conducted. The two images are determined to be of the same person when the degree of similarity between them is over a certain threshold value.
Facial recognition technology has advanced tremendously due to deep learning, a type of AI technology. “Neural networks” make it possible for computers to replicate the functionality of the human brain. Deep learning uses Deep Neural Networks (DNN), which are neural networks with many more layers. In facial recognition that is conducted via deep learning, the computer “pre-learns” from a massive database of facial information and generates a DNN model, which it then uses to extract from and compare with the facial features in captured images. This enables it to achieve a level of precision that exceeds human performance. The massive input of teaching data in the form of facial images of people of different races and ages train the computer until images of the same person give feature values that are close matches, making it possible to recognize faces with a high degree of accuracy.
Facial Recognition Process
Canon’s facial recognition technology harnesses the image analysis technologies that it has cultivated through development of digital and network cameras. With its large-scale databases and unique high-precision, high-speed AI deep learning architecture, it is capable of highly accurate facial recognition even for low resolution images and those shot from angles that were challenging to detect using existing technology.
To train its DNN model, Canon built a massive, diverse database of face images that consisted not just of normal photographs but also high angle images from network cameras installed in high locations, as well as images captured in poor conditions that are blurred or carry image noise. The training also accounted for differences in shooting environment such as backlight and different face angles, as well as changes in external appearance such as hairstyle changes and aging. In addition, it also trained the DNN model so that face images of the same person would give similar feature values, and those of other people with similar facial features would give feature values with larger differences, hence ensuring a high recognition accuracy rate even for images for which identification is difficult. For cases in which subjects are wearing masks or sunglasses, Canon collected a massive amount of image data in-house and input them as training data, enabling highly accurate comparison and recognition even for partially obscured faces. These training efforts, together with the optimized combination of numerous algorithms, enabled the realization of reliable and accurate facial recognition in various environments including walkthrough gates.
The DNN is trained so that feature values for other people are set apart from those for the individual even if there are variances in the features.
Features of Canon’s facial recognition technology
(1) High accuracy even for face images shot from a steep overhead or side angles
(2) Works well on low-resolution, low-quality, and blurred images
(3) High-accuracy recognition even with masks or sunglasses
In 2021, Canon’s facial recognition technology ranked number 1 in Japan and among the top in the world for search accuracy※1 in four categories of the Face Recognition Vendor Test (FRVT) with the 1:N investigation scenario conducted by the National Institute of Standards and Technology (NIST). The scenario involved finding a specific person from among many other faces.
In particular, the error rate (the probability of misestimating a person as someone else) in the Visa Border category, which benchmarks with the Visa images (which uses full frontal face images captured under good lighting) and Border images (which includes face images shot at immigration gates that may not be full frontal nor under good lighting) was only 0.15% (number 1 in Japan, number 4 globally). This testified the reliability of the technology under diverse image capture environments.
The 1:N investigation scenario involves finding a specific person from within a massive number of faces in the dataset.
The FRVT is conducted throughout the year. Although its evaluation methodology is undisclosed, all participating vendors are fairly evaluated using NIST’s massive dataset consisting of face photos of 12 million people, making it an industry standard benchmarking test. Its 2021 test evaluated over 300 algorithms submitted by vendors from all over the world, including Russia and China. Canon’s excellent performance in these tests demonstrates the high accuracy of its facial recognition technology.
A test that uses visa images (face images for immigration documents) and border images (face images shot at immigration control gates). Visa images are full frontal face images shot under good lighting. Border images include non-full-frontal images and images shot under not ideal lighting condition.
A test that uses mugshot images (face images that are almost full frontal) and webcam images (face images shot on inexpensive cameras). Webcam images include non-full-frontal images, low contrast images, and low-resolution images.
A test that uses visa images (face images for immigration documents) and kiosk images (face images shot at traveler kiosk terminals). Kiosk images include images where the face is looking downward or partially out of frame.
A test that uses mugshot images (face images that are almost full frontal) and profile images (side view face images).
Categories in the NIST facial recognition benchmarking
FRVT where Canon was ranked No. 1 in Japan (Based on 1:N Investigation scenario)
Canon’s unique facial recognition technology is already being used in a variety of solutions. December 2021 marked the launch of its Facial Recognition Application for Milestone Systems’ XProtect software (available in Japan only). This software enables swift, high-precision facial recognition via network cameras installed on walls and ceilings, without the need to pause in front of special terminals. By linking up with Canon network cameras to automatically correct exposure based on facial brightness, the software makes seamless facial recognition possible even in backlight and times of the day where brightness varies.
With sufficient consideration to the issues of privacy and potential bias, Canon will continue to polish its unique, Japan-developed AI-based facial recognition technology to provide image analysis solutions that work in synergy with its digital and network cameras.
The diverse fields of application for facial recognition