Facebook researchers have developed a new tool that maps 2D images onto humans in videos. The 2D images themselves look silly, but they’re proof of concept that AI can be trained to look past environmental noise and identify humans in complex scenes.
Facebook’s AI research team released a paper last week detailing the process. The team used a neural network that can analyze complicated scenes in video footage—such as people walking, dancing, playing basketball—and identify which pixels belong to humans. That’s step one.
Researchers then overlaid all human-belonging pixels with these Minecraft-esque skins. It seems silly, but the textures remain consistent in the videos, even as people kept moving and walking. The AI can map the textures onto humans, which is a feat considering how the people in the videos move unpredictably (imagine what dancing “looks” like to a computer), and wear clothing, which alters their shape.
The team discusses gaming applications in the paper, particularly in the augmented reality space, which Facebook has expressed tons of interest in before. But, as The Next Web notes, law enforcement would be very interested in being able to analyze footage and instantly isolate and categorize people in crowds according to their movements. Body cam maker Axon (formerly known as Taser) has been working on similar object recognition tech since 2017. For now, at least, the takeaway remains the same: computers are able to “see” us better than ever.