Select a region
Facial recognition technologies show bias towards gender and skin colour and appear to favour white men, scientists have found.
Researchers from MIT’s Media Lab tested three facial commercial recognition systems and uncovered that error rates for dark-skinned women were at least 20% compared to just 0.8% for light-skinned men when it came to gender classification.
The team says its findings raise questions about how modern artificial intelligence networks, which are trained to perform tasks by looking for patterns in huge data sets, are evaluated.
The three systems – which include those from Microsoft, IBM and Chinese firm Megvii – are what the researchers describe as general purpose facial-analysis systems, ie, ones that could be used to match faces in different photos as well as assess characteristics such as gender, age and mood.
To test the commercial systems, the team built a data set of 1,270 faces and applied the three facial-analysis systems to their newly-constructed data set.
They found that across all three systems, the error rates for gender classification were consistently higher for females than they were for males, and for darker-skinned subjects than for lighter-skinned subjects.
For darker-skinned women the error rates were 20.8%, 34.5%, and 34.7% – and in two of the three systems, the error rates for the darkest-skinned women were 46.5% and 46.8%, the researchers said.
The researchers wrote in their paper: “Overall, male subjects were more accurately classified than female subjects replicating previous findings (Ngan et al, 2015), and lighter subjects were more accurately classified than darker individuals.
“An intersectional breakdown reveals that all classifiers performed worst on darker female subjects.”
Joy Buolamwini, lead author and researcher in the MIT Media Lab’s Civic Media, said all three companies appeared to have a relatively high accuracy overall, but added that she was surprised to see the technologies fail for over one in three women of colour.
She said: “In fact, as we tested for women with darker and darker skin, the chances of being correctly gendered came to a coin toss.”
She added that the machine-learning techniques that make gender classification possible are already being used in many other applications, emphasising that companies “should do better with commercially sold products”.
She said: “The same data-centric techniques that can be used to try to determine somebody’s gender are also used to identify a person when you’re looking for a criminal suspect or to unlock your phone.
“And it’s not just about computer vision. I’m really hopeful that this will spur more work into looking at [other] disparities.”
In a statement, Microsoft said the company has “already taken steps to improve the accuracy of our facial recognition technology”, while IBM responded saying it has “several ongoing projects to address dataset bias in facial analysis – including not only gender and skin type, but also bias related to age groups, different ethnicities, and factors such as pose, illumination, resolution, expression and decoration.”
Megvii, whose facial-analysis system Face++ was used in the research, did not immediately respond to request for comment.
The research is published in Proceedings of Machine Learning Research and will be presented at the Conference on Fairness, Accountability, and Transparency later this month.
