This New Deep Learning Tool Shows You the Most Memorable Parts of Your Photos

Illustration for article titled This New Deep Learning Tool Shows You the Most Memorable Parts of Your Photos

Bummed that your latest cat pic didn’t get more traction on Instagram? Wondering how to make people remember your company’s logo first and foremost? A clever algorithm developed by MIT computer scientists may be able to help with a new online tool.


Meet MemNet, a new program that uses techniques from the deep learning field of artificial intelligence research to determine how memorable our visual media are. This is how it works. Feed MemNet any image, and it’ll generate a heat map predicting which parts are going to lodge themselves in a human observer’s brain. Ultimately, the researchers behind the algorithm would like to turn it into an app that uses memorability prediction to tweak images, rendering them utterly unforgettable.

The latest version of MemNet is available online. Being an amateur cat photographer myself, I decided to give this a try. Apparently, the most memorable part of Mr. Tango Tangerine’s face is his left ear:

Illustration for article titled This New Deep Learning Tool Shows You the Most Memorable Parts of Your Photos

Deep learning uses systems that mimic neural networks to teach computers new skills, or train them to search for complex patterns. Typically, this involves asking an algorithm to correlate datasets without any prior human guidance. The computer does so by performing a vast number of iterative computations, and its abilities improve as it processes more data.

These types of techniques are the engine behind numerous innovations from the last decade, including Apple’s Siri and Google’s auto-complete.

In this case, computer scientists fed MemNet tens of thousands of pictures from several datasets. All the pictures had been given a “memorability score” based on how well human subjects remembered them in online experiments. After training, the algorithm was pitted against actual people, and asked to predict how memorable a group of never-before-seen images were. It performed 30 percent better than any existing algorithms, and within a few percentage points of humans.

In other words, MemNet is about as good as people at judging how memorable an image is. It’s not hard to imagine MemNet’s abilities eventually surpassing our own, given enough training—and this could translate into some seriously clever image editing tools.


If you find the notion of companies using software to ensure that their logos burn a permanent mark in your grey matter somewhat unsettling, bear in mind that this technology has all sorts of non-dystopian applications. It could be used to develop better teaching resources or digital assistant devices that help you remember things you want to remember. Perhaps MemNet will even shed new light on the fundamental nature of human memory, which, in turn, could lead to better tools for improving it.

Sure, advertisers probably stand to benefit. But can the marketing world really get much more absurd than it has this week?


[MIT News]

Follow the author @themadstone

Top: For each image, the MemNet algorithm produces a heat map identifying the most memorable regions. Image Credit: CSAIL


Share This Story

Get our newsletter


I wonder if this struggles with animals or action. It has to be one of the two, because I have a picture of a Cheetah in full sprint that I took in South Africa, and it’s easily been the picture that most people stop and gawk at of all of my shots from the trip. LaMem rated its memorability as “Low,” Which none of my other photos got.