More and more people are using smart speakers everyday. But how are they really using them? Tawfiq Ammari, a doctoral candidate at the University of Michigan, in conjunction with researchers at Mozilla and Yahoo, published a paper which sheds some light on the question. To do this, he gathered surveys and user logs from 170 Amazon Alexa and Google Home users, and interviewed another 19 users, to analyze their daily use of voice assistants.
Users of both Google Home and Alexa devices can access a log showing all the interactions they’ve had with their device. Our 170 users gave us a copy of this log after removing any personal information, which meant we could understand what people were really using their devices for, rather than just what they remembered using their devices for when asked later. Together, these logs contained around 259,164 commands.
We collected 193,665 commands on Amazon Alexa which were issued between May 2015 and August 2017, a period of 851 days. On average, the datasets for our 82 Amazon Alexa users span 210 days. On the days when they used their VA, Alexa users issued, on average,18.2 commands per day. We collected 65,499 commands on Google Home between September 2016 and July 2017, a period of 293 days. On average, the datasets for each of the 88 Google Home users spans 110 days. On days when they used their VA,Google Home users issued, on average, 23.2 commands per day with a median of 10.0 commands per day.
For both Amazon Alexa and Google Home, the top three command categories were listening to music, hands-free search, and controlling IoT devices. The most prevalent command for Amazon Alexa was listening to music, while Google Home was used most for hands-free search. We also found a lot of items in the logs reflecting that both devices didn’t often understand queries, or mis-heard other conversation as commands — that’s 17% in the case of Google Home and 11% in the case of Alexa, although those aren’t quite comparable because of the way that each device logs errors.
People used their smart speakers for all sorts of searches. For example, some of our respondents use VAs to convert measurement units while cooking. Others used their VAs to look up trivia with friends. Users also searched for an artist who sang a particular song, or looked for a music album under a specific genre (e.g., classical music).
The third largest category was controlling Internet of Things (IoT) devices, making up about 10% of the Google Home commands and about 17% of the Alexa commands. These were most frequently turning smart lights on and off, although also included controlling smart thermostats and changing light colors. Users told us in interviews that they were frustrated by some of the aspects of IoT control. For example, Brad told us that he was frustrated that when he asked the smart speaker in his kitchen to “turn the light off,” it wouldn’t work. He had to tell it to “turn the kitchen light off”.
We also found a long list of particular uses of smart speakers: asking for jokes, weather reports, and setting timers or alarms, for example. One thing we found interesting was that on both platforms there were nearly twice as many requests to turn the volume down than requests to turn the volume up, which suggests that default volume levels may be set too high for most homes.
Despite their use of voice assistants, our interviewees had some real concerns about their voice assistants. Both Amazon Alexa and Google Home provide user logs where users can view their voice commands. They both also provide a feature to “mute” their VAs. While most of our survey respondents were aware of the user history logs (~70%), more than a quarter of our respondents did not know that they could delete entries in their logs and only a small minority (~11%) had viewed or deleted entries in their logs.
Users also worried about whether their voice assistant was “listening all the time.” This was particularly contentious when family members and friends became “secondary users” of the voice assistant just by being in the same physical space. For example, Harriet told us that her “in-laws were mortified that someone could hack in and see what I’m doing, but what are they going to learn?”
Other users were worried about how their data was being processed on cloud services and shared with third party apps. John noted that he was concerned about how VAs “reach out to…third party services” when for example asking about the weather. He was concerned that he knew very little about what information is sent to third party services and how these data are stored.
While Mozilla has no plans to make a smart speaker, we do think it’s important to share our research as part of our mission to ensure that the Internet is a public resource, open and accessible to all. As more people install voice assistants in their homes, designers, engineers, and policy makers need to grapple with issues of usability and privacy. We take an advocacy stance, arguing that as personal assistance become part of people’s daily experiences, we have the responsibility to study their use, and make design and policy recommendations that incorporate users’ needs and address their concerns.