An extra post-it note is all you really need to prevent someone from spying on you through a laptop’s built-in webcam. But Jason Mayes apparently likes to do things the hard way: He’s developed an AI-powered tool for browsers that can erase people from live webcam feeds in real-time but leave everything else in the shot.
Mayes is a Google web engineer who developed his Disappearing-People tool using Javascript and TensorFlow, which is Google’s free, open source software library that allows the terrifying potential of artificial intelligence and deep learning to be applied to less terrifying applications. In this case, the neural network works to determine what the static background imagery of a video is in order to develop a clean plate—a version without any humans moving around in the frame—without necessarily requiring the feed to be free of people to start with.
The neural network used in this instance is trained to recognize people, and using that knowledge it can not only generate a clean image of a webcam feed’s background, but it can then actively erase people as they walk into frame and move around, in real-time, while allowing live footage of everything else happening in the background to remain.
Mayes has created test versions of the tool that you can access and try yourself in a browser through his personal GitHub repository. The results aren’t 100 percent perfect just yet (you can still see quite a few artifacts popping up here and there in the sample video he shared where he walks into frame), but as the neural network powering this tool continues to improve, so will the results.
Eventually, it could be used to erase people (tourists) from live webcams set up in popular destination spots around the world, but it could be even more useful for personal use. Yes, you can cover your laptop’s front-facing webcam for maximum privacy, but are you really going to put a piece of tape on your smartphone or tablet’s camera? Theoretically, at least, this tool could potentially ensure you’re never caught on camera if you don’t want to be. The real-time performance is probably the most impressive part of this demonstration. As we’ve recently seen with behind-the-scenes looks at how shows like The Mandalorian were filmed, Hollywood is always looking for easier and cheaper ways to create impossible worlds and fantastical creatures, and a tool like this could be used to automatically erase puppeteers and crew as scenes are being shot, reducing the amount of visual effects work needed in post-production.