Using machine learning to swap out one image for another isn’t new, but ask these systems to do something a bit more complicated than a basic faceswap, and you can end up with underwhelming or laughable results. Now, researchers say they have developed a technique that produces far more convincing images for complex tasks, such as swapping out skirts for pants, cups for bottles, or a herd of sheep for a mess of giraffes.
Researchers at the Korea Advanced Institute of Science and Technology and the Pohang University of Science and Technology say their technique, InstaGan, stands out for its ability to tackle changes in shape and altering multiple image elements at once. The method, based on generative adversarial networks (GANs), processes instance information, such as the ability to identify objects and boundaries in images. In their paper, the researchers write that they think they are the first to develop such a technique on more complex images “to the best of our knowledge.”
The results are pretty impressive. Women in photos without simple or plain backdrops have their pants swapped out for skirts without any discernible changes to the background. But it’s not completely flawless: the legs are a little blurry in the output.
The images showing how their system swaps out several sheep for giraffes are more impressive. Both the background and the animals look realistic, especially compared to other systems trying to do the same thing. The researchers made a point to contrast their technique with previous methods, which yield some sheep-giraffe hybrids or seemingly headless sheep.
The researchers point out a number of use cases for their improved technique. The clothing examples, for instance, could be used by people interested in comparing different items they might want to buy. They also note that this research can apply to neural machine translation—the process of using AI to predict what words might come next in a sentence—and video generation.
Of course, there are also the more nefarious ways this tech can be used. With any system that can convincingly generate fake content, there’s the concern that it will be used to intentionally and harmfully misinform its viewers. Such tools already exist—and are being exploited—but this one will likely be used to help further the work of other well-intended researchers. And, on its own, I’m hard-pressed to imagine how switching out a sheep for a giraffe might spark any meaningful chaos.