Amazon has managed to make Alexa the brand people think of when it comes to digital voice assistants, but the Google Assistant is the smartest of the bunch according to a new study.
Digital marketing agency Stone Temple quizzed the top AI assistants by throwing about 5,000 questions at them to see which was capable of providing the most information. Google Assistant came out on top, edging out Alexa, Microsoft’s Cortana, and Apple’s Siri.
All of the personal assistants were served up the same sets of questions and were graded on how many questions they attempted to answer and how many they got right. Every one of the disembodied voices hit on 80 percent or more of the responses that they provided, with the Google Assistant setting the bar in both categories—though the assistant worked better on a smartphone than it did on the Google Home smart speaker, which is weird but sure.
The study also showed all these assistants are getting smarter over time. Alexa upped its game significantly since it was put through the wringer last year, more than doubling the number of questions it attempted to answer. That trend will probably continue as these things continue to learn more from users lobbing millions of questions at them every day.
If we’re being real though, all of these assistants still kind of suck. While they mostly hit on the questions they bother to answer, most of them don’t even take a crack on a good chunk of requests thrown at them. Only Google Assistant tried to answer more than 75 percent of the questions. Cortana took on nearly two out of three, Alexa answered just over half of the questions asked, and Siri could only be bothered to respond to about 40 percent. That’s pretty bad!
Most people who interact with virtual assistants seem to realize these shortcomings pretty quickly, as well. According to a report from Alpine.AI, voice assistants have a pretty terrible retention rate, with just three percent of people continuing to regularly use the services two weeks after first interacting with them.
That failure to retain users is in part attributable to the fact that these assistants come up empty on a significant number of questions and requestions, and in part that they just aren’t all that convenient or time-saving in most use cases.
It wasn’t until recently that Alexa could handle more than one command at once, which meant you would have to feed the assistant one request, wait for response or confirmation that it was completed, then wake it up and request something else. Now you can jam a couple requests into the same sentence. But if Alexa mishears one part, you might find yourself undoing the command and trying it again in its entirety. Having a machine do things for you is novel, but it’s not making tasks noticeably easier.
It can feel weird interacting with voice assistants, too. Creative Strategies reported a vast majority of people are afraid to communicate with virtual assistants when other people are around. Just six percent are willing to talk to them while in public. Even in private, there are barriers. If you’re talking to a smart speaker with an accent, good luck getting it to understand you. Support for languages outside of English can be lacking.
Virtual assistants are not without their benefits. For people with visual impairments or limited motor skills, voice assistants can certainly make life easier. AbilityNet suggests the services may also have benefits or people with learning disabilities or dyslexia. (The same can’t be said for those who suffer from voice disorders, who may find digital assistants to be unresponsive and therefore uninclusive.)
Alexa, Google Assistant, Siri, and Cortana will all likely continue to get incrementally better over time. Maybe one day they will actually prove to live up to the potential that they have promised. Until then, they are creepy, always-listening novelty services that are using you to collect data. But at least you can find out what the weather is going to be like without looking at your phone!