At Yale University there is a robot that can make you hear what it wants you to hear. It does so by grabbing the corners of your mouth with string. But it’s not the threat of pain that messes up your ears. It’s the “head-had” effect.
A lot of factors influence what we hear. Only one of them involves the actual sound wave in our ears. Show someone a certain image and they’ll be absolutely certain they heard a sound that was never made. We see this clearly when we look at the McGurk effect – see a person mouth the word “vah” and we’ll hear the word “bah” as “vah,” even if we know we’re being tricked.
But it’s not just our eyes that push our ears around. A robot with control over our face can do exactly the same thing. Doctor David Ostry, at Yale University, fashioned a robot that did little more than pull at strings attached to the corner of a person’s mouth with sticky pads. Sometimes it pulled the lips up, into a kind of smile. Sometimes it pulled the corners of the mouth down, so the person looked, just a bit, like a tragedy mask.
While this was happening, the person heard a sound. While their lips were being pulled up into a semi-smile, they heard the sound as the word “head.” While their lips were being pulled down, they heard the sound as the word “had.” Why? Say “head,” and then say “had,” and pay attention to how your lips move for both.
Not only do other people’s faces make us hear certain sounds, our own faces do the same, even when they’re manipulated.
Image: Ildar Sagdejev
