Google's Autoflip Can Intelligently Crop Videos on the Fly to Fit Any Aspect Ratio

Google has released an open-source tool, Autoflip, that could make bad cropping a thing of the past by intelligently reframing video to correctly fit alternate aspect ratios.

Advertisement

In a blog post, Google’s AI team wrote that footage shot for television and desktop computers normally comes in a 16:9 or 4:3 format, but with mobile devices now outpacing TV in terms of video consumption, the footage is often displayed in a way that looks odd to the end-user. Fixing this problem typically requires “video curators to manually identify salient contents on each frame, track their transitions from frame-to-frame, and adjust crop regions accordingly throughout the video,” soaking up time and effort that could be better spent on other work.

Autoflip aims to fix that with a framework that applies video stabilizer-esque techniques to keep the camera focused on what’s important in the footage. Using “ML-enabled object detection and tracking technologies to intelligently understand video content” built on the MediaPipe framework, Google’s team wrote, it’s able to adjust the frame of a video on the fly.

Advertisement

What’s more, Autoflip automatically adjusts between scenes by identifying “changes in the composition that signify scene changes in order to isolate scenes for processing,” according to the company. Finally, it analyzes each scene to determine whether it should use a static frame or tracking mode.

Illustration for article titled Googles Autoflip Can Intelligently Crop Videos on the Fly to Fit Any Aspect Ratio
Graphic: Google AI

This is pretty neat and offers obvious advantages over static cropping of videos, though it’s probably better suited to things like news footage and Snapchat videos than movies and TV shows (where being able to view an entire shot is more important).

Advertisement

For a more technical explanation of how all this works, the Google AI team explained the various technologies involved in its blog post. The project’s source code is also available to view on Github, along with instructions on how to take it for a spin.

"... An upperclassman who had been researching terrorist groups online." - Washington Post

Share This Story

Get our newsletter

DISCUSSION

Tebow Kneeled First

Great! I can’t wait for all of the conversions of my favorite movies and tv shows so I can finally view them on my phone in the most natural way!

*cracks open new bottle of cough syrup and starts chugging*