The Colour Of That Dress Poses The Same Problem As Colour on Mars

Human wetware is astonishingly good at pattern recognition and interpreting complex, noisy data, but it's also painfully buggy. Today, the internet came to a screeching halt debating a dress's colour, but the same optical illusion is responsible for how uncertain we are about interpreting colours on Mars.

When we send robotic explorers to Mars, we equip them with colour calibration targets of known optical properties. Yet, even with those calibrated instruments, the view of the same rock in "true" colour can change dramatically just based on the brightness from the time of day.

This leads to an almost philosophical problem: even raw images don't tell us what we'd actually perceive in different lighting conditions. The problem gets even weirder when you try to compensate for lighting that we're familiar with instead of the lighting as it actually is on the planet. Here's Mount Sharp in true colour, and then rebalanced to blue skies to look like it would under more familiar Earth-normal lighting:

Mount Sharp in natural Martian lighting. Image credit: NASA/JPL


Mount Sharp rebalanced to Earth lighting. Image credit: NASA/JPL

This illusion hit the earliest observers, too — it's why Percival Lowell thought he saw canals, and why that myth persisted right up until planning the first up-close mission to the planet.

Like finally seeing that dratted dress as black and blue, trying to imagine how we'd see the planet if we had real, human eyes there is not as simple as it seems like it should be!


The "What Color Is This Goddamn Dress?" Debate Explained By Science

A rift is tearing the internet apart. And it's the question of what color this dress is. Society has been split into the white/golds and the blue/blacks. Thankfully some people have stepped up to explain, scientifically, what color this goddamn dress is.

If you've somehow missed it, it started on tumblr when Swiked posted this dress and the question "guys please help me - is this dress white and gold, or blue and black? Me and my friends can't agree and we are freaking the fuck out":

For the record, I see white and gold in shadow. But others, including other members of the io9 staff, insist they see blue and black. And some people have seen it switch before their eyes, causing no end of consternation.

The dress is apparently this Roman Originals dress, which settles the debate about what it is in real life, although not why the picture is fooling us:


The lovely people over at Deadspin conducted a photoshop analysis, which determined that, at least, the photo appears to be blue and brown. And, inverted, it appears to be even more white and gold:

Until you tilt your screen, then the thing changes color:



An optical illusion, that's what. The explanation behind it boils down to the neuroscience of perception, and it's a problem researchers have wrestled with before. Behold, the checkerboard illusion:


Squares A and B are, in fact, the same color – but our brains perceive them as different. Back in 2010, Alasdair Willkins explained why, here on io9:

So how does it work? A lot of it has to do with the shadows cast by the big green cylinder. As Adelson [creator of the illusion] explains, the brain has to assess how much light is coming off the surface of each square on the board. This is known as the luminance of each square, and the brain also has to figure out how much of the luminance (or lack thereof) is caused by the color of the square and how much is created by the shadows. So the brain has to figure out where the shadows are and then compensate for them, and that's where we start running into trouble.

Adelson explains some of these processes:

"The first trick is based on local contrast. In shadow or not, a check that is lighter than its neighboring checks is probably lighter than average, and vice versa. In the figure, the light check in shadow is surrounded by darker checks. Thus, even though the check is physically dark, it is light when compared to its neighbors. The dark checks outside the shadow, conversely, are surrounded by lighter checks, so they look dark by comparison.

A second trick is based on the fact that shadows often have soft edges, while paint boundaries (like the checks) often have sharp edges. The visual system tends to ignore gradual changes in light level, so that it can determine the color of the surfaces without being misled by shadows. In this figure, the shadow looks like a shadow, both because it is fuzzy and because the shadow casting object is visible."

Part of the problem is where the squares are located. Most people know enough about checker or chessboards to know that a square will be the opposite color of all the adjacent squares. That means two squares that are next to each other, or separated by an even number of squares, should be different colors. Since squares A and B are two squares away, they logically should be different colors, and our brains really seem to want to think that as well.

But don't worry, says Professor Adelson. All this really means is that your brain is functioning the way it's supposed to:

"As with many so-called illusions, this effect really demonstrates the success rather than the failure of the visual system. The visual system is not very good at being a physical light meter, but that is not its purpose. The important task is to break the image information down into meaningful components, and thereby perceive the nature of the objects in view."


How this works in relation to the dress photo was best described by Wired:

Light enters the eye through the lens—different wavelengths corresponding to different colors. The light hits the retina in the back of the eye where pigments fire up neural connections to the visual cortex, the part of the brain that processes those signals into an image. Critically, though, that first burst of light is made of whatever wavelengths are illuminating the world, reflecting off whatever you're looking at. Without you having to worry about it, your brain figures out what color light is bouncing off the thing your eyes are looking at, and essentially subtracts that color from the "real" color of the object. "Our visual system is supposed to throw away information about the illuminant and extract information about the actual reflectance," says Jay Neitz, a neuroscientist at the University of Washington. "But I've studied individual differences in color vision for 30 years, and this is one of the biggest individual differences I've ever seen." (Neitz sees white-and-gold.)

Usually that system works just fine. This image, though, hits some kind of perceptual boundary. That might be because of how people are wired. Human beings evolved to see in daylight, but daylight changes color. That chromatic axis varies from the pinkish red of dawn, up through the blue-white of noontime, and then back down to reddish twilight. "What's happening here is your visual system is looking at this thing, and you're trying to discount the chromatic bias of the daylight axis," says Bevil Conway, a neuroscientist who studies color and vision at Wellesley College. "So people either discount the blue side, in which case they end up seeing white and gold, or discount the gold side, in which case they end up with blue and black." (Conway sees blue and orange, somehow.)


Context is screwing with us pretty hard with this dress. Our eyes are getting scrambled trying to decode reflection data and lens flare. But what if we try looking at just a segment without context, to give our eyes a break? From Wired again:

The point is, your brain tries to interpolate a kind of color context for the image, and then spits out an answer for the color of the dress. Even Neitz, with his weird white-and-gold thing, admits that the dress is probably blue. "I actually printed the picture out," he says. "Then I cut a little piece out and looked at it, and completely out of context it's about halfway in between, not this dark blue color. My brain attributes the blue to the illuminant. Other people attribute it to the dress."

[...] So when context varies, so will people's visual perception. "Most people will see the blue on the white background as blue," Conway says. "But on the black background some might see it as white." He even speculated, perhaps jokingly, that the white-gold prejudice favors the idea of seeing the dress under strong daylight. "I bet night owls are more likely to see it as blue-black," Conway says.


The upshot? When you look at this photo, your brain works to make sense of what it's seeing. When perceived luminance and color are weighed against eachother, different brains will assign more weight to one than the other, resulting in a marked difference in perception. The dress may be blue and black in person, but, in this photo at least, its color is in the brain of the beholder.

Additional reporting by Mika McKinnon