Facebook Researchers Teach AI to 'Re-Skin' People in Real Time

Screengrab: YouTube/Alp Guler
Screengrab: YouTube/Alp Guler

Facebook researchers have developed a new tool that maps 2D images onto humans in videos. The 2D images themselves look silly, but they’re proof of concept that AI can be trained to look past environmental noise and identify humans in complex scenes.

Facebook’s AI research team released a paper last week detailing the process. The team used a neural network that can analyze complicated scenes in video footage—such as people walking, dancing, playing basketball—and identify which pixels belong to humans. That’s step one.

Researchers then overlaid all human-belonging pixels with these Minecraft-esque skins. It seems silly, but the textures remain consistent in the videos, even as people kept moving and walking. The AI can map the textures onto humans, which is a feat considering how the people in the videos move unpredictably (imagine what dancing “looks” like to a computer), and wear clothing, which alters their shape.

Illustration for article titled Facebook Researchers Teach AI to Re-Skin People in Real Time

The team discusses gaming applications in the paper, particularly in the augmented reality space, which Facebook has expressed tons of interest in before. But, as The Next Web notes, law enforcement would be very interested in being able to analyze footage and instantly isolate and categorize people in crowds according to their movements. Body cam maker Axon (formerly known as Taser) has been working on similar object recognition tech since 2017. For now, at least, the takeaway remains the same: computers are able to “see” us better than ever.

[The Next Web]

Of course I have pages. I had pages five years ago. How anyone can believe I don’t defies belief.

Share This Story

Get our newsletter


C.M. Allen

This is also important technology for autonomous vehicles. The better they are at distinguishing what they can ‘see’ in the environment, the better they can properly respond. In this case, knowing a pedestrian is a pedestrian and not a sign post makes a world of difference.