If you get carried away with setting up helpful reminders, eventually you’ll get tired of your smart speaker’s endless nagging and will start ignoring it. But what if instead of Siri or Alexa reminding you to water the plants or that your Pop Tarts are ready, it sounded like your ficus or your toaster was actually talking and trying to grab your attention?
Smart speakers have evolved from simple devices that can play music and report the weather to central hubs that interface with all of the other smart devices installed around your home. Everything from smart microwaves to smart washing machines can not only be remotely controlled through voice commands, but they’ve also been given voices of their own through smart speakers that can provide audible reminders for when a load of clothes are clean, or when your popcorn is popped.
That can create a disconnect, however, when an alert for a given device, object, or appliance is coming from the other side of your house, and it may be less impactful or memorable as a result. Maybe you’d be more inclined to water a plant if it sounded like the plant itself was pleading with you for a drink. But instead of slapping speakers on every last item in your house, researchers from Carnegie Mellon University’s School of Computer Science’s Smash Lab have developed a prototype of an even-smarter smart speaker that can project its voice like a ventriloquist, using some clever tricks with sound waves to make it sound like other objects in your home are speaking to you.
Smash Lab’s Digital Ventriloquism prototype works much differently from a human ventriloquist. While a traditional ventriloquist tricks your brain into thinking a dummy is talking by holding the dummy close and not moving his lips while he makes noise, the digital ventriloquist uses moving servos, a webcam for visual targeting, and a two-dimensional array of ultrasonic transducers.
The transducers emit a highly directional ultrasonic signal at around 40 kHz, which is well beyond the threshold of human hearing. As a result, there’s no audible sound as the signal travels toward an object, but when those sound waves eventually collide and bounce off their target, they demodulate and produce frequencies that fall within our range of hearing. Thus: Your plant is talking to you.
Even though the sounds all originate from the smart speaker, the resulting effect isn’t technically an illusion; the audible messages actually emanate directly from a given object. However a person is situated in a given space in relation to the object, they should always be able to discern exactly where it originated from. In a study conducted by the researchers that involved five different objects in three unique environments, test subjects were able to correctly identify where the sound was coming from 92 percent of the time, and repeat the message they heard 100 percent of the time.
In its current form the Digital Ventriloquism prototype definitely looks the part, and the use of servos and a limited size ultrasonic array means it takes a few seconds for it to reposition itself and target a new object. That would be problematic in a real-world setting, so the researchers are looking ahead with a design that looks more like the smart speakers you can buy from companies like Amazon and Apple. The design is completely surrounded with an array of transducers, so it can instantly transmit sounds in any direction. Its abilities are limited to objects in the same room as the speaker itself, which means you’d need quite a few of these installed around your home to give a voice to every last object. But once you did, you could spend your days talking to your possessions, and they’d start talking back.
Update, April 21, 3:23 p.m. ET: The research team reached out to Gizmodo asking to clarify that the paper’s lead author, Yasha Iravantchi, is currently a PhD student at the University of Michigan, and that the project itself was a joint effort between CMU’s Smash Lab and Future Interfaces Group.