Findings from the Marketplace User Research

Bill Selman

I posted last month about a user research study we conducted in Portland, Oregon to gather a qualitative understanding of users’ current behaviors and relationship to the App Store associated with their current mobile OS. More specifically, we wanted to find out more about:

  • How and why users make decisions about installing and/or purchasing apps
  • Users’ understanding of privacy and how they believe their personal information is used by an App Store and smartphone apps.

I wanted to share a few qualitative findings from our larger pool of information gathered. Incidentally, we also conducted a prototype test with the same users for a Mozilla App Marketplace that will be an essential part of the forthcoming Firefox OS. While we are not ready to share those findings, we did learn a great deal from our participants that will contribute to the user experience of the final Mozilla Marketplace.

A quick aside for methodology… We interviewed nine smartphone users who were between the ages of 18-35. Four were female and five were male. Four of the users owned Android phones and used the Google Play marketplace. The other five owned Apple iPhones and used the Apple App Store. Four users who have limited data plans with their service providers and download three or fewer apps per month. Five users who have unlimited data plans and download more than three apps per month. The sessions were conducted in the Portland offices of Mozilla on July 12-13, 2012.

App Information
There is a lot of information that app stores present to users about apps. We wanted to know which pieces of information about an app users most commonly employed to make a decision about installing an app regardless of the App Store used. The three most frequently reported items were:

  1. Rating
  2. Screenshot
  3. Review

If you have ever used an app store, these results should not come as a huge surprise. The reasons users provided for each of them were informative however.

In many cases, rating and reviews are used by users to assess the quality of an app. “Quality” in this case has a few meanings depending on the participant. For some users, “quality” refers to how useful they ultimately find an app. So, for example, if it is a restaurant location tool they use often, they feel it should have a better rating. As an extension, “quality” is whether or not an app matches their expectations of what the app should accomplish based on participants’ expectations of functionality, etc.

Android users used ratings (and reviews) in a couple of specific ways. As there are many types of Android devices (and varying levels of compatibility with specific apps), participants reported using reviews and ratings to see if other users who own the same devices are able to use an app successfully. Also, Android users reported that some apps in the marketplace were “shady,” misused their information, or were untrustworthy. They looked to peer reviews to point out ways in which apps behaved in ways that were unexpected or untrustworthy.

As expected, screenshots were useful to participants to get a visual idea of what they were downloading looked like. However, users also reported a couple of additional ways they used screenshots. For example, some participants said that they used screenshots to assess the user experience and usability of an app. As one participant said, “If I can’t make sense of a screenshot, how would I be able to use the app on my phone?” One participant said that he used the design of an app as displayed in a screenshot to further assess the quality of an app. For him, an app that was better designed looked “more professional” and he anticipated that it would be better quality because the developer “put more time into it.”

Free vs. Paid
We explored app price in a slightly different manner. Of all of our participants, only three reported ever paying for an app (of those, two were Apple iPhone users). Users reported many reasons for not purchasing an app. Some lumped the idea of app purchases into their overall phone plan budget. Many felt that there were free apps out there that “basically did the same thing as a pay app” as one user said. Some users did not purchase apps as a matter of principle.

Of those users who did purchase apps, it was because they wanted a specific feature that was unavailable elsewhere. In the case of the Android user who paid for apps, she paid for apps she felt were useful (she cited a battery management and Craigslist app as examples) and did so because she wanted to support what she perceived to be the continued quality and development of the apps.

From the perspective of app selection, users do look at price as a factor in installing an app. Namely, if they do not pay for apps as a matter of practice, then a pay app will be excluded as a possible selection. Where possible, these users filter apps according to pay or free apps and generally only look at free apps.

Trust and Privacy
Users trust a central authority to vet the trustworthiness of apps in terms of making unauthorized purchases or misusing their personal information. Those participants who used an app store that did not have a review process for apps spoke of their concern about installing apps that might have nefarious purposes. They believe there were “shady” apps that could steal their financial information or make unauthorized purchases. Further, these participants said that they believed these apps exist because “there isn’t a review process.” In contrast, participants who used an app store that had a review process did not mention apps or app developers being able to violate their trust or data.

Users are overwhelmed by the privacy implications of how their personal information is used by apps and an apps marketplace. Further, they feel a sense of resignation about their ability to control their data. When asked about privacy policies for apps, many users said privacy policies were written in “legalese” (a term four participants used) and were designed for app marketplaces and app developers more than consumers.

Wrap Up
These qualitative findings definitely challenged some of our ideas about how users approach an app marketplace. Further, we gathered some perspectives that we had not yet considered. In the future, we plan to explore some specific directions in greater depth, especially those involving privacy and managing personal information.