Artificial Intelligence Can Determine Racial Identity from Medical Images

A study shows that artificial intelligence (AI) has the capability to accurately predict someone’s race in radiographic images. This is something that human experts were not able to do. They couldn't predict self-reported race from these images.

AI's ability to guess race

The research suggests that information on race could be incorporated into image analysis, causing racial bias and disparities in medical settings.

The study was published in The Lancet.

The researchers realized that their study uncovered the possibility that AI could have a predisposition towards race. Although AI is used in medicine to diagnose illnesses with human-like reasoning and intelligence, the notion of a simulated machine having bias is concerning for researchers. They realize the pros and cons in creating AI that is so close to human intelligence. It can both transform health care, while also showing unintentional bias through its programming.

“AI has immense potential to revolutionize the diagnosis, treatment, and monitoring of numerous diseases and conditions and could dramatically shape the way that we approach health care,” said Dr. Judy Gichoya, first study author and NIBIB Data and Technology Advancement (DATA) National Service Scholar.

“However, for AI to truly benefit all patients, we need a better understanding of how these algorithms make their decisions to prevent unintended biases,” she also mentioned.Previous studies show AI does have bias

There have been previous revelations on AI having racial bias, such as this previous research on DeepAI that states the stereotypes AI could possibly learn, unfortunately. The report states that “Our experiments definitively show robots acting out toxic stereotypes with respect to gender, race, and scientifically-discredited physiognomy, at scale.” It mentioned that the AI recognizes men over women and white people over people of color. Not only that, but the research showed the AI also picked up stereotypes against both women and people of color as well. “The results showed that the robot has not only learned a general bias against recognizing women and people of color, but has also learned specific toxic stereotypes.”

This suggests that the idea of bias AI is not new or surprising but is troubling. Other past research, such as an analysis in Nature Medicine, has shown that AI implementation can be influenced by demographics, which includes race. It said that “there is growing concern that such AI systems may reflect and amplify human bias, and reduce the quality of their performance in historically under-served populations such as female patients, Black patients, or patients of low socioeconomic status.” This bias often leads to troublesome underdiagnosis by AI algorithms.

The study from The Lancet said there could be several factors that lead to bias, such as using data that doesn’t represent an entire patient population. Another factor that can lead to AI bias is artificial intelligence learning about traits that might be present in the population, like certain phenotypes, or characteristics, such as bone density.

The study

The research team wanted to see if AI models could determine race only from chest X-rays. They used three large datasets that included a large, diverse population and what they found was astounding. The AI could predict the race of a patient with precision based solely on the X-ray, something even the experts were not able to do.

The AI could also determine the race even when the images were “highly degraded or cropped to one-ninth of the original size, or when the resolution was modified to such an extent that the images were barely recognized as X-rays.”

Concerning factors

To prevent phenotype racial bias, the researchers used other non-chest X-ray datasets that included mammograms, chest computed tomography (CT) scans, and cervical spine radiographs. The AI was able to still determine the race of the person.

“Our results suggest that there are 'hidden signals' in medical images that lead the AI to predict race,” said Gichoya. “We need to accelerate our understanding of why these algorithms have this ability, so that the downstream applications of AI—such as building image-based algorithms to make predictions about health—are not potentially harmful for minority and underserved patient populations.”

Although the researchers were trying to determine how the AI made these correct projections, they were unable to do so. They couldn’t identify any specific factor that could explain the ability of AI to correctly predict self-reported race. In other words, they aren’t sure how AI determined the race, but had the ability to do so despite nothing physically giving the AI an aspect that could reflect one’s race.

Artificial intelligence can be beneficial to society, and in this case, medicine. However, before examining others and possibly making life-or-death diagnoses, AI must be examined itself to make sure racial bias is not programmed into its system.