Iman Haq and Dillon Murphy
Every day, we encounter numerous faces of diverse genders, ages, and races, and our ability to identify them accurately is crucial. However, human facial recognition can sometimes be prone to errors, making it challenging for individuals to recognize and distinguish important people in their lives. Fortunately, advancements in Artificial Intelligence (AI) have enabled machines to recognize faces as well. While Artificial Intelligence technology, a product of human ingenuity, can more effectively identify individuals based on their facial features (Fetzer, 1990), it also has the potential to introduce bias, leading to inaccuracies that disproportionately affect minority groups. Recognizing the limitations of both human and Artificial Intelligence facial recognition systems, this article aims to explore the causes and implications of facial recognition failures.
Human facial recognition is a complex cognitive process that involves several stages: perception, feature extraction, facial encoding, and memory retrieval. Specifically, the sequence begins with the visual detection of facial features such as the eyes, nose, mouth, and overall facial structure (Keil, 2009). Specific facial features are then extracted and include the size, shape, and color of different facial components. Next, the extracted facial features are encoded into unique representations in the brain. When we encounter a familiar face, we match the visually presented face to a stored representation and if there is a match, we recognize the person (Rossion et al., 2012). Once the face is recognized, we make decisions based on the recognized individual’s identity and the context in which the recognition takes place. This can include retrieving relevant memories, making social judgments, and guiding behavior and responses.
There are several brain regions involved in facial recognition, but the most active regions are the occipital face area and fusiform gyrus (Rossion et al., 2012). The occipital face area is responsible for lower-level visual recognition, or the fundamental level of visual processing where individual features of a face are detected and distinguished. The fusiform gyrus is responsible for higher-level visual recognition; it involves integrating and interpreting multiple features to form the holistic image of a face, which means objects are seen as whole rather than individual parts (Lopatina, et al., 2018). Damage to either of these regions can lead to impaired facial recognition ability, potentially leading to Prosopagnosia (a disorder characterized by the inability to recognize faces) and heightened anxiety in social situations where we need to recognize important people.
Similar to humans, Artificial Intelligence facial recognition technology employs a step-by-step approach to identifying faces, but rather than relying on cognition to form patterns and features of individuals, Artificial Intelligence relies on computational algorithms to learn patterns and features associated with different individuals. Specifically, Artificial Intelligence technology can identify most faces from digitized images via three main steps: detection, analysis, and recognition. In the detection phase, Artificial Intelligence machines begin by identifying significant facial features such as the mouth, nose, and eyes (similar to humans). Once these key features are detected, the image undergoes analysis where geometrical relationships between various facial attributes are recorded, including the eyes’ positioning, the distance between the forehead and chin, the shape of jawlines, and the size and contour of cheekbones (Edelman et al., 1992). This facial information is then transformed into a mathematical formula known as a “faceprint” (Wehrli et al., 2022). Everyone possesses a unique faceprint, and a match between a person’s faceprint and one stored in the program’s database indicates that the geometrical relationships between facial features in the person’s face closely resemble those in the database entry. However, if no faceprint match is found, the facial recognition system cannot identify the face, and further investigative measures, such as verifying identity through personal identification documents or conducting interviews, may be necessary.
Using these three steps, Artificial Intelligence facial recognition has demonstrated superior efficiency and accuracy compared with humans (Fetzer, 1990). However, errors can still occur, often stemming from inherent design flaws within the system itself (Wehrli et al., 2022). These errors, referred to as systematic misidentifications, are primarily attributed to the underrepresentation of certain demographic groups in the training dataset. For instance, facial identification systems misidentify Black or Asian faces up to 100 times more frequently than White faces (Edelman et al., 1992). This disparity in accuracy is a consequence of embedded discrimination within the algorithms, resulting in the systems being predominantly trained to recognize White faces more precisely than faces of Asian or Black individuals. This flaw could perpetuate and amplify racial biases and discrimination and also lead to disproportionate surveillance, biased decision-making, and potential violations of civil liberties for individuals from marginalized communities.
Artificial Intelligence can significantly enhance its ability to identify faces by augmenting the size and diversity of its training dataset. In facial recognition technology, the training dataset encompasses a broad collection of facial images that serve as the foundation for training the recognition algorithm to distinguish between facial features. Consequently, a more comprehensive and representative training dataset would contribute to improved accuracy and reduced bias in facial recognition. Looking forward, a more diverse training dataset would empower Artificial Intelligence systems to precisely detect and identify individuals, which could liberate human operators from the burden of such an important task.
Just as there can be biases in Artificial Intelligence facial recognition, humans may illustrate similar biases, potentially arising from a similar mechanism. Specifically, humans often demonstrate an own-race bias: the tendency for individuals to be more proficient at recognizing and distinguishing faces from their own racial or ethnic group compared to faces from other groups (Meissner & Brigham, 2001). There are two main accounts regarding the origin of this bias: the perceptual expertise account and the social cognitive mechanistic account. The perceptual expertise account suggests that the own-race bias occurs because individuals have more exposure and experience with faces from their own racial or ethnic group, leading to the development of perceptual expertise in recognizing and distinguishing those faces (similar to how bias arises in Artificial Intelligence systems). This account emphasizes the role of perceptual learning and the formation of detailed representations of own-race faces (Hills & Lewis, 2006). In contrast, the social cognitive mechanism account proposes that faces are processed based on group membership, categorizing them as either in-group or out-group members. This categorization influences the memory encoding processes, with individuals showing better recognition of faces from their own group compared to faces from other groups (Pauker et al., 2009; Sporer, 2001). Regardless of the mechanism, the own-race bias in human facial recognition could cause potential challenges in cross-racial interactions, eyewitness identifications, and criminal investigations.
While Artificial Intelligence can improve by using more diverse training data sets, we can reduce racial bias when recognizing faces by promoting diversity and inclusivity through education, cultural applications, and diverse social networks where people can enhance familiarity with individuals from different racial backgrounds. Additionally, increasing awareness about the own-race bias and its consequences can help individuals consciously challenge their biases and avoid grouping faces. Creating and implementing programs in schools that focus on improving cross-racial recognition skills and sensitivity can also be beneficial for reducing bias (Wehrli et al., 2022). For future generations, addressing systemic biases and discriminatory practices in society is crucial. By fostering inclusivity and raising awareness, we can work towards a more fair and unbiased recognition of faces across races.
In conclusion, facial recognition plays a crucial role in human interaction and society, allowing us to identify and remember individuals in our daily lives. The process of human facial recognition involves a sequential progression, beginning with visual perception and feature extraction, followed by facial encoding, memory retrieval, and culminating in the final stage of facial recognition. This process involves the activation of the fusiform and occipital face area and remains effective even when dealing with low-quality images (Keil, 2009). In contrast, Artificial Intelligence facial recognition technology offers a more systematic and structured approach, using detection, analysis, and recognition steps to identify faces with greater efficiency and accuracy. However, human and Artificial Intelligence systems have their limitations as both can be prone to introducing bias towards individuals who are unlike them or differ from the norm. These systemic errors highlight the urgent need for diverse and inclusive training datasets as well as the rigorous evaluation and regulation of facial recognition technologies to ensure fairness, accuracy, and ethical use of Artificial Intelligence facial recognition technology.
References
Edelman, S., Reisfeld, D., & Yeshurun, Y. (1992). Learning to recognize faces from examples. Lecture Notes in Computer Science, 588, 787-791.
Fetzer, J. H. (1990). What is artificial intelligence? In artificial intelligence: Its scope and limits. Studies in Cognitive Systems, 4, 3-4.
Hills, P. J., & Lewis, M. B. (2006). Reducing the own-race bias in face recognition by shifting attention. The Quarterly Journal of Experimental Psychology, 59, 996-1002.
Keil, S. M. (2009). ‘‘I Look in Your Eyes, Honey’’: Internal face features induce spatial frequency preference for human face processing. PLOS Computational Biology, 5, 1-14.
Lopatina, O. L., Komleva, Y. K., Gorina, Y. V., Higashida, H., & Salmina, A. B. (2018). Neurobiological aspects of face recognition: the role of oxytocin. Frontiers Behavioral Neuroscience, 28, 1-11.
Meissner, C. A., & Brigham, J. C. (2001). Thirty years of investigating the own-race bias in memory for faces: A meta-analytic review. Psychology, Public Policy, and Law, 7, 3-35.
Pauker, K., Weisbuch, M., Ambady, N., Sommers, S. R., Adams, R. B., Jr., & Ivcevic, Z. (2009). Not so black and white: Memory for ambiguous group members. Journal of Personality and Social Psychology, 96, 795-810.
Rossion, B., Hanseeuw, B., & Dricot, L. (2012). Defining face perception areas in the human brain: a large-scale factorial fMRI face localizer analysis. Brain and Cognition, 79, 138-157.
Sporer, S. L. (2001). Recognizing faces of other ethnic groups: An integration of theories. Psychology, Public Policy, and Law, 7, 36-97.
Wehrli, S., Hertweck, C., Amirian, M., Glüge, S., & Stadelmann, T. (2022). Bias, awareness, and ignorance in deep-learning-based face recognition. AI and Ethics, 2, 509-522.