These Algorithms Look at X-Rays--and Somehow Detect Your Race

[Nevilledog]

Link to tweet

Tweet text:
hypervisible dot pdf
@hypervisible
“The results have spurred new concerns that AI software can amplify inequality in health care, where studies show Black patients and other marginalized racial groups often receive inferior care compared to wealthy or white people.”

These Algorithms Look at X-Rays—and Somehow Detect Your Race
A study raises new concerns that AI will exacerbate disparities in health care. One issue? The study’s authors aren’t sure what cues are used by the algorithms.
wired.com
5:56 AM · Aug 5, 2021

https://www.wired.com/story/these-algorithms-look-x-rays-detect-your-race/

Millions of dollars are being spent to develop artificial intelligence software that reads x-rays and other medical scans in hopes it can spot things doctors look for but sometimes miss, such as lung cancers. A new study reports that these algorithms can also see something doctors don’t look for on such scans: a patient’s race.

The study authors and other medical AI experts say the results make it more crucial than ever to check that health algorithms perform fairly on people with different racial identities. Complicating that task: The authors themselves aren’t sure what cues the algorithms they created use to predict a person’s race.

Evidence that algorithms can read race from a person’s medical scans emerged from tests on five types of imagery used in radiology research, including chest and hand x-rays and mammograms. The images included patients who identified as Black, white, and Asian. For each type of scan, the researchers trained algorithms using images labeled with a patient’s self-reported race. Then they challenged the algorithms to predict the race of patients in different, unlabeled images.

Radiologists don’t generally consider a person’s racial identity—which is not a biological category—to be visible on scans that look beneath the skin. Yet the algorithms somehow proved capable of accurately detecting it for all three racial groups, and across different views of the body.

*snip*

[Hugh_Lebowski]

1. When you're dealing with computers and algorithms, there's no such thing as 'somehow' ...

This would be impossible if there were not physical realities being manifested for the algorithm to discern.

I'm very curious as to what the accuracy is. The actual statistics, confidence intervals, etc. I doubt it's anywhere near 100% accurate.

But if it is anywhere near 100% then it means that there are, in fact, at least some reliable physical differences between races. There's no other explanation, whether one is happy to hear something like that, or not.

[-misanthroptimist]

2. I generally agree with you, but with a couple of additions

1. There could be some sort of fraud involved. (Not claiming this, just citing it as a possibility)
2. There could be some sort of error in the coding that enhances the probability that the algorithm correctly identifies race. (Again, making no claim.)

But it's entirely possible that there are some subtle giveaways visible in x-rays.

[Hugh_Lebowski]

3. If I were to guess I'd speculate that it's just skin tone being picked up

You might think there's 0 difference and there might not be to the naked eye, but a bit-by-bit analysis of the image might make it discernable in ways your eye wouldn't pick up.

Second guess would be bone density differences.

[-misanthroptimist]

4. Those were my numbers 1 and 2 guesses as well

[Spider Jerusalem]

7. There are structural skeletal differences.

People of sub-Saharan African ancestry have denser bones than people of European or Asian ancestry, for instance.

[fescuerescue]

5. Seems like it would be easier to just look at the patients file

Then to run a AI to determine what is usually right at the top of your healthcare record. (age sex race etc)

Seems to be that if radiologist are determined to be racist, they already have all the info needed.

[mainer]

6. Unless they're looking at skull x-rays, I'm baffled by this

You can certainly make a good guess as to race by features of the skull and the teeth, or by skeletal changes from sickle cell anemia, but otherwise, I'm not sure what "racial" features these algorithms are going by.