'Automated Racism': Chinese Police Are Reportedly Using AI to Identify Minority Faces

Illustration for article titled 'Automated Racism': Chinese Police Are Reportedly Using AI to Identify Minority Faces
Photo: Getty

Facial recognition technology can target entire populations of a specific demographic, and in the wrong hands, can be used as a powerful tool for discrimination. In China, this isn’t a cautionary tale, it’s already happening.


According to a report from the New York Times published on Sunday, the Chinese government is using a facial recognition system to track Uighurs, the country’s Muslim minority. The technology reportedly targets this population based on their physical appearance.

According to government procurement documents obtained by the Times, beginning last year, nearly two dozen police departments in China wanted technology that could identify and track Uighur individuals. And the documents reportedly indicate that the interest in this type of tech has grown in the last two years. In Yongzhou, for instance, police wanted software that could “characterize and search whether or not someone is a Uighur.”

The technology is reportedly being deployed across the country, including in Hangzhou, Wenzhou, Fujian, and Sanmenxia. In Sanmenxia, police reportedly identified Uighur residents 500,000 times in just a month, starting February of this year. The database for this facial recognition system reportedly included tags for “rec_gender”, “rec_sunglasses” and “rec_uygur”. The latter tag was meant to indicate whether a camera identified a Uiguir, which reportedly happened 2,834 times and included images with the entry.

In China, the Uighur population is treated with suspicion and oppression by the Communist Party and is heavily surveilled by authorities. Many Uighurs have disappeared into internment camps, last year watchdogs estimated that up to a million people could be incarcerated in these specially designed prisons.

“I don’t think it’s overblown to treat this as an existential threat to democracy,” Jonathan Frankle, an A.I. researcher at the Massachusetts Institute of Technology, told the New York Times. “Once a country adopts a model in this heavy authoritarian mode, it’s using data to enforce thought and rules in a much more deep-seated fashion than might have been achievable 70 years ago in the Soviet Union. To that extent, this is an urgent crisis we are slowly sleepwalking our way into.”

The use of widespread surveillance tools is far from uncommon in China—authorities in the country already monitor the location of alternative energy vehicles, the real-time moods of students in class, and citizen’s “social credit” scores. And these barely factor in the existing nearly 200 million cameras in the country used for surveillance. And last year, cops in the country literally wore facial recognition systems on their face in the form of smart glasses. The stated purpose was to spot fugitives, but skeptics believed they would be misused to target activists and minorities.


This also isn’t the first time China has used facial recognition tech to keep tabs on the Uighur population. Early last year, a report revealed that law enforcement in Xinjiang had been using software to track Muslims in the region since 2017, and were notified whenever an individual matching that description was located over 300 meters from home or work, the Independent reported.

And the Times report published on Sunday reveals that authorities are becoming increasingly comfortable using this technology to racially profile individuals—the stated purpose is no longer an innocuous misdirection for an otherwise insidious use case. Cops are seeking out facial recognition systems that blatantly target the country’s ethnic minority with increasing demand, and it’s a disturbing normalization of an already troubling technology.


“A system like this is obviously well-suited to controlling people,” security expert Jim Harper, executive vice president of Competitive Enterprise Institute, told the Independent. “‘Papers, please’ was the symbol of living under tyranny in the past. Now, government officials don’t need to ask.”

[New York Times]




Xinjiang is littered with security cameras. I mean “cameras in every shop and dozens on every block of street”. The New York Times also had a photo essay that gives a good feeling, and every journalist who’s been approved to enter Xinjiang has brought up the same thing.


There are police roadblocks and checkpoints everywhere, residents are encouraged to spy on their neighbors, children on their parents, and anyone seen as “too religious” (which could literally be growing a beard or teaching the Qur’an to your children) end up disappearing, most likely for the notorious “work training centers” in Xinjiang. It’s hard to say, because it’s difficult even for Uighur activists to get information, let alone outside journalists.

Uighurs are the main target right now, the density of surveillance and policing in Xinjiang far exceeds most other parts of China (Tibet may be an exception) but these systems are being designed and tested with the likely goal to apply them everywhere. It’s easy to see how this all fits into the “social credit” system Beijing has designed. Notably while Uighurs have faced heavy repression due to fears of ethnic separatism and (exaggerated) fears of Islamist extremism the government has also begun targeting other minorities, particularly Muslims ones, as there have been reports of mosques being closed or demolished and activists arrested from the Hui minority (who are Muslim but otherwise very similar to Han Chinese).

Muslims may be an easy target at the moment, and Chinese money/power have kept the vast majority of Muslim countries or the international community from speaking on the Uighur detention camps, but it’s a safe bet some or all of these repressive measures will spread. This is the start of a new digital authoritarianism that can be far more repressive than any regime in the past. If it works in China, it will spread.