{"id":4960,"date":"2011-08-15T11:32:11","date_gmt":"2011-08-15T18:32:11","guid":{"rendered":"http:\/\/blog.mozilla.org\/metrics\/?p=4960"},"modified":"2019-09-18T12:05:34","modified_gmt":"2019-09-18T19:05:34","slug":"text-mining-users-definitions-of-browsing-privacy","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/metrics\/2011\/08\/15\/text-mining-users-definitions-of-browsing-privacy\/","title":{"rendered":"Text mining users’ definitions of browsing privacy"},"content":{"rendered":"

One issue that\u2019s been on everyone\u2019s mind lately is privacy.\u00a0 Privacy is extremely<\/span><\/a> important<\/span><\/a><\/span> to us at Mozilla, but it isn’t exactly clear how Firefox users define privacy.\u00a0 For example, what do Firefox users consider to be essential privacy issues?\u00a0 What features of a browsing experience lead users to consider a browser to be more or less private?<\/span><\/p>\n

\n

In order to answer\u00a0 these questions, we asked users to give us their definitions of privacy, specifically\u00a0privacy<\/em>\u00a0while browsing<\/em><\/strong>,\u00a0in order to answer these questions.\u00a0 The assumption was that users will have different definitions, but that there will be enough similarities between groups of responses that we could identify \u201cthemes\u201d amongst the responses. By text mining\u00a0user\u00a0responses to an open-ended survey question asking for definitions of browsing privacy,\u00a0 we were able to identify themes directly from the users’ mouths:<\/span><\/p>\n

    \n
  1. Regarding\u00a0 privacy issues, people know that tracking and browser history are\u00a0 different issues, validating the need for browser features that address\u00a0 these issues independently (“private browsing” and “do not track”)<\/span><\/li>\n
  2. People’s definition of personal information vary, but we can group people\u00a0 according to the different ways they refer to personal information (this leads to a natural follow-up question; what makes some information more personal than others?)<\/span><\/li>\n
  3. Previous focus group research, contracted by Mozilla, showed that users are aware that spam indicates a\u00a0 security risk, but\u00a0what didn’t come out of the focus group research was that users also\u00a0also consider spam to be an invasion of their privacy (a follow-up question, what do users define as \u201cspam?\u201d\u00a0 Do they consider targeted ads to be spam?)
    \n<\/span><\/li>\n
  4. There are users who don’t distinguish privacy and security from each other<\/span><\/li>\n<\/ol>\n

    Some previous research on browsing and privacy<\/span><\/h2>\n

    We\u00a0 knew from our own focus group research that users are concerned about viruses, theft of their personal information and passwords, that a\u00a0 website might misuse their information, that someone may track their\u00a0 online \u201cfootprint\u201d, or that their browser history is visible to others.\u00a0\u00a0 Users view things like targeted ads, spam, browser crashes, popups, and\u00a0 windows imploring them to install updates as security risks.<\/span><\/p>\n

    But it’s difficult to broadly generalize\u00a0findings from focus groups.\u00a0 One group may or may not have the same concerns as the general population.\u00a0 The quality of the discussion moderator, or some unique combination of participants,\u00a0 the moderator, and\/or the setting can also influence the findings you get from focus groups.<\/span><\/p>\n

    One way of validating the representativeness of focus group research is to use surveys.\u00a0 But while surveys may increase the representativeness of your findings, they are not as flexible as focus groups.\u00a0 You have to give survey respondents their answer options up front.\u00a0 Therefore, by providing the options that a respondent can endorse, you are limiting their voice.<\/span><\/p>\n

    A typical\u00a0 way to approach this problem in surveys is to use open-ended survey questions.\u00a0 In the pre-data mining days, we would have to manually code\u00a0 each of these survey responses: a first pass of all responses to get an idea of respondent \u201cthemes\u201d or \u201ctopics\u201d and a second pass to code each\u00a0 response according to those themes.\u00a0 This approach is costly in terms of time and effort, plus it also suffers from the problem of reproducibility; unless themes are extremely obvious, different coders might not classify a response as part of the same theme.\u00a0 But with modern text mining methods, we can simulate this coding process much more quickly and reproducibly.<\/span><\/p>\n

    Text mining open-ended survey questions<\/strong><\/span><\/h2>\n

    Because text mining is growing in popularity\u00a0primarily due to its computational feasibility\u00a0, it\u2019s important to review the\u00a0 methods in some detail.\u00a0 Text mining, as with any machine learning-based approach, isn\u2019t magic.\u00a0 There are a number of caveats to make about\u00a0the\u00a0text mining\u00a0approach used. First, the clustering algorithm I chose to use requires an arbitrary\u00a0and a priori\u00a0decision regarding the number of clusters.\u00a0 I looked at 4 to 8 clusters and decided that 6 provided the best trade-off between themes expressed and redundancy.\u00a0 Second, there is a random component to\u00a0 clustering, meaning that one clustering of the same set of data may not produce the exact same results as another clustering. Theoretically,\u00a0 there shouldn’t be tremendous differences between the themes expressed in one clustering over another, but it’s important to keep these details in mind.<\/span><\/p>\n

    The general idea of text mining is to assume that you can represent documents as “bags of words”, that bags of words can be represented\u00a0or coded\u00a0quantitatively, and that the quantitative representation of text can be projected into\u00a0a multi-dimensional space. For example, I can represent survey respondents in two dimensions, where each point is a respondent’s answer.\u00a0 Points that are tightly clustered together mean that these responses are theoretically very similar with respect to lexical content\u00a0(e.g., commonality of words).<\/span><\/p>\n

    \"\"<\/span><\/a><\/span><\/div>\n

    I\u00a0 also calculated a score that identifies the relative frequency of each word in a cluster, which is reflected in the size of the word on each\u00a0 cluster\u2019s graph.\u00a0 In essence, the larger the word, the more it \u201cdefines\u201d\u00a0 the cluster\u00a0(i.e.\u00a0its location and shape in the space).<\/span><\/p>\n

    \"\"<\/span><\/a><\/span><\/p>\n

    Higher resolution .pdf files of these graphs can be found here<\/span><\/a><\/span> and here<\/span><\/a><\/span>.<\/span><\/p>\n

    Cluster summaries<\/strong><\/span><\/h2>\n
    \n