Categories: The Glass Room

How to Lie with Analytics

The following guest post, “How to Lie with Analytics” was written by Ted Byfield for Mozilla’s Internet Citizen blog.

Learn more about the history of data analytics and visualization at The Glass Room through December 18, 2016. Ted will be hosting a session and guided tour on December 11 at 3 p.m.

analytics constellation

How to lie with analytics

There’s a particular kind of image that’s associated with “big data” — you’ve probably seen it. Let’s call it the luminous city. The background — maybe a city, a nation, or the entire earth — is stark, often dark. Against that backdrop we see thousands of radiant trajectories fanning out and spanning across empty expanses, intertwined tendrils dispersing and converging. Aaron Koblin’s “Flight Patterns” (2011) is one famous example; Paul Butler’s “Visualizing Friendships” (2010) for Facebook is another. There are many more.

Images like this are supposed to show how our lives coincide — in cars and on planes, on the internet or social networks, whatever. What they don’t show is everyone and everything that isn’t so “connected.” The poorer neighborhoods, industrial and mixed-use areas, the people, places, and things forgotten in one way or another by “innovation.” They disappear into the background as if they never even existed. In this way, visualizations can often help us to forget many of those gritty, obstinate, and inconvenient worlds.

When people talk about visualization, they often emphasize how it can “surface” important patterns and correlations. But surfacing one thing means submerging everything else — in other words, forgetting it. Hiding isn’t a bug of visualization, it’s a feature. It’s not just fanciful visualizations that do it: their poor cousins, spreadsheets, do it too — with much more impact.

My point isn’t to condemn visualization, of course — that would be ridiculous. Every serious decision made anywhere in the world now — in government, business, manufacturing, construction, science, education, civil society, and the military — is fundamentally shaped by visualization. That is why we need to think more seriously about how visualization works — and also how it doesn’t work. But the questions we ask shouldn’t just be functional — about whether what we hope to discover or communicate is clear, effective, persuasive, or elegant. We also need to ask about the unintended effects: what disappeared into the background?

Lies, damn lies, and analytics

So what does this have to do with analytics? And how do you lie with them, anyway?

Along with the adage that “there are three kinds of lies: lies, damned lies, and statistics,” the phrase How to Lie with Statistics is one of the main references for what many people associate with statistics: lies. The phrase was the title of a 1954 book by Darrell Huff — the first popular book to tackle statistics. To this day, people often note its breezy style and describe it as current; but emphasizing its style obscures key ways in which the book is very dated.

The goalposts have moved in the last sixty-odd years. When Huff wrote it, data wasn’t an everyday fact at work, home, and everywhere between. It was still mostly hidden away in bean-counting back offices and an occasional feature of print journalism. Now, though, data is front and center — for example, in “data-driven” journalism, where the data is the story. For the outlets that publish those stories, analytics play a decisive role in what kinds of stories are developed, presented, and promoted. The same is true, in different ways, everywhere else data appears: it describes the questions we ask. As in the “luminous cities” example I began with, we should ask ourselves what kinds of stories are not developed and what kinds of data aren’t collected.

Huff’s book wasn’t really a how-to manual for lying with data, it was a primer in how not to be lied to. And that problem remains just as real decades later: how not to be lied to. The challenge now is to recognize where we can think critically and practically about the many ways that statistics and analytics shape and distort our own — and others’ — lives.

So…how do you lie with analytics?

The first step is to remember that analytics, however sophisticated, is an evolutionary step in, not a revolutionary break from, statistics. True, analytics also involves some important distinctions about how data is processed and analyzed. But the field of statistics has changed dramatically over the last two to three centuries, and on a basic level analytics is just another step. And if we can lie with statistics, we can lie with analytics.

Most people will never command what Mark Surman, executive director of the Mozilla Foundation, called “data empires.” And — hopefully — they won’t break or hack into data centers. Nor can most people immerse themselves in the esoteric world of governing algorithms. These aren’t viable options. So what is viable? What can you do?

I’d argue that that one answer is right in front of you, in the steady parade of visual presentations of data and statistics you see every day. It’s no longer just a question of the informed consumer casting a skeptical eye on how a handful of facts and figures are used in front-page stories, though. Instead, it begins with a simple set of questions: What else is in the background? What’s in all those empty, “negative” spaces between, behind, and around the statistics we see? And, if or when you’re called upon to act on those images of data, you can factor in the gritty, obstinate, and inconvenient worlds they often hide.


Ted Byfield is a retired artist, frequent editor, escaped urbanist, governance hobbyist, perpetual collaborator, and recovering academic. He’s moderated nettime for a really long time and, more recently,co-founded the Open Syllabus Project. He’s currently writing a history of what people imagine information looks like, which isn’t at all what you’d imagine.

Join Ted at The Glass Room on Sunday, Dec. 11 at 3 p.m. to learn more about “How to lie with analytics.”

51 comments on “How to Lie with Analytics”

  1. Tho wrote on

    Hi Ted, nice article 🙂
    Ever encountered TrackMeNot http://cs.nyu.edu/trackmenot/ (firefox addon)?
    It’s an attempt to lie to search engines by passing fake queries,
    yet shown not powerful (https://arxiv.org/abs/1211.0320 https://zenodo.org/record/59558 ).
    There may be a bit of discussion around your article here: https://www.facebook.com/groups/499463776745202/permalink/1385042341520670/
    Cheers,

  2. Jc wrote on

    Have no idea what you are talking about. You need examples and explanations

    1. Pedro Aquino wrote on

      I agree. I could not make head nor tail of it. Re the space behind the visualization: it is black because it contains no data! ! You’re going to have to do better.

      1. J. P. DeMeritt wrote on

        Actually, the space appears black because it contains none of the data that the analyst thought relevant. But there is data there.

        There can be many reasons the black — or blank — space isn’t populated with any visible data. For example, the algorithms used may be based on particular assumptions about how the world works that overlook what’s happening in the “black” spaces. Another possibility has to do with scale: if your theory covers the flows of information at the global level, what’s happening in neighborhoods won’t be visible. And, of course, the data themselves are always suspect: how do you, as a consumer of someone else’s analytics, know that the data collected and analyzed actually relate to the questions you want answers for? The data are always analogs for behaviors we’re interested in understanding.

        It’ll be interesting to see how Ted addresses this issue tomorrow.

    2. particolor wrote on

      You saved me reading ALL of it !:-) I was lost on the first Paragraph 🙂 🙂

    3. Minecraftyfyer wrote on

      exactly the same

  3. Rob wrote on

    Huh?

  4. Wayne Stewart wrote on

    Are these presentations available after initial broadcast? How are they accessible?

  5. Tulio Cimerilli, MD wrote on

    Think that Thomas Robert Malthus (UK, 1766-1834) was right, after all. His Theory applies today in our world -Reclaiming Malthus, in Google-, but we got aware too late…

  6. Bill DAVIS wrote on

    The assumption that only bad and lowly things are missing from the grid is invalid.The real nightmare comes from the idea that there is only this “grid world”,and all else is bad. The “winners” ,on the grid, are the 1984 nightmare world we are becoming.

    1. Garrick wrote on

      exactly on point. The police state is here and it is run by a blindfolded dispensation of state mandated moral-speak.

      1. roby wrote on

        u r right on garrick…..and moral that are not so moral!!!!!!! just a robbery to individual rights and a dispensary of rules and regulations to lower the individual way of thinking as we were a bunch of sheep !!!!!!

  7. Michael Lunse wrote on

    As someone who spent a number of years in mining and producing metrics in a large corporation, myself and my team realized that many people:
    – use and focus only on the metrics they like
    – manipulate data reporting to tell the story they want told
    – misinterpret or don’t understand what the data is telling them
    From this background we would create special metrics, which may or may not always be published, to:
    – weed out the manipulators
    – provide a more balanced picture for the focusers in the crowd
    – provide additional information to help the dazed and confused

    Mark Twains’s, and others, views on statistics (and you can include analytics and data in that) are as true now as when the phrase was first uttered.

    1. Michael wrote on

      Just people doing what they do but using analytics to get what they want. Sort of like using drones to carry drugs over the border.

  8. michael scott wrote on

    The demands that are made on my time and my purse by various internet, let’s call them “utilities”, such as Mozilla and Google are great, but the services they provide are incredibly valuable to me, they are also worth paying for and I would much rather pay a regular annual fee for them than the current state of affairs, where I get bugged for contributions, as if they were some kind of charity. The downside of the current system is that I know that a small percentage of users pay for access to them, and the vast majority use them and try to avoid paying for them. Facebook too is corrupted.
    Analytics probably got Trump elected, and that we will all pay for sooner or later.
    Here’s what I suggest:
    Charge me a regular annual fee, and guarantee me freedom from hidden adware and PUPs and from incessant begging letters.

    1. PayZaLee wrote on

      I only see this happen once a year, maybe I’m not remembering correctly. I send a payment when I 1st get asked, What I would like to see is, once I have sent a payment I simply don’t get the request (reminder) anymore until the following year. I would think if you actually had to pay for the service, human nature would dictate the service would fade away eventually since there are also free services offered with the appearance of basically the same service. Is that not where Netscape (Mozilla) was born from in the first place, an alternative from Microsoft’s IE and the control and limitations it had.

  9. Javier Dorantes wrote on

    Es escalofriante darme cuenta que mis sospechas sobre el tema de que lo hermoso en la estadística es que se vea hermoso aunque no que se cuente la verdad.

  10. Suzanne Wood wrote on

    Thank you for this timely article. The responsibility for self-determining the underlying facts, or background behind statistics, or story content, whether on Facebook, Twitter, other internet venues, subscription media is often missing and simply becomes heresay.

  11. W. Townsend wrote on

    Cyber Monday:
    I was and am not the Buyer nor the Product. I keep myself and family far from the frenzied crowds of “Sheeple” and ignore the the din of advertising supplied by corporations who fear they will be otherwise unable sell their undesirable, shabby, imported, unneeeded products.
    Thank you,
    W. Townsend

  12. Garry Jones wrote on

    Red
    Enjoyed reading your article “How to lie with analytics”

  13. Allen Grace wrote on

    Thought Provoking. The dead spaces are never intended but get lost because of our desire to find some light on our interest, but that allows it to be manipulated if not careful.

  14. j wrote on

    touche!

  15. Doug Meier wrote on

    Great read challenging our assumptions about relying on data analytics

  16. M. Reel wrote on

    Prolix

  17. KenB wrote on

    True statement… but… We are governed more by the bubbles we live in. If in the conservative bubble, I can look at analytics behind global warming, climate change, and find a snowball to present in the Senate, showing what analytics are hiding in the background. Everything can be taken to the ludicrous in a world that wants a one page answer to the meaning of life.

  18. Thad Guidry wrote on

    Best article I have read ALL YEAR.

  19. Old Hat wrote on

    I agree about being willing to pay an annual fee (not too much, please, I live on $40,000 pa, which doesnt go far when you have 40 in family to donate to at Christmas, plus charities).
    Life without constant nagging advertising is very pleasant; too many people nowadays have too much stuff anyway – the older I get, the more delight I find in de-stuffing my life.

    1. roby wrote on

      howdy there old hat i wish i had your income!!!!!!!!!!!!! but all in all GOOD FOR YOU MAN!!!!!!!

    2. Sheila Mennear wrote on

      Hi Old Hat. I am in Canada and actually live on about $20,000 and am considered well off. Good for you to have that much. However, 40 in family means some of it is spread pretty thin. This is a good article and meant for us to sit up and take notice which we all seem to be doing. I actually prefer the little requests at the end of the year. I’ve sent my money and know the requests are no longer aimed at me so I just carry on.

  20. G Schwei wrote on

    Analytics / Statistics do not lie – people use them to lie.

    1. John wrote on

      Now this is true.

  21. jojodawildboy wrote on

    dont get this story ty it in inglis with examples !nice use of LARGE words and complicaed sentences! SO whats ur point????

  22. Mary Ulsh wrote on

    When it comes to truth, this world does not have integrity. This world is guided by a person that has power to cause things to seem like truth exists. The power he has is beyond human strength. Read Revelation 12:12, 1 John 5:19 He started way back before 4026 BCE.

  23. Mary Ulsh wrote on

    What I said is very true. Have the highest authority, that what was said is absolutely true.

  24. Glyn Rogers wrote on

    Why do people who have enough always want more and don’t stop to enjoy what they have already before it is too late. Man made things are not worth having, the beauty of nature which includes humans is all around if you only stop open your eyes clear your brain of all the rubbish and look.

    1. roby wrote on

      u r right on glyn….exactly my thinking ……i do have enough and i enjoy life to the point where i dont even watch tv !!! i dont need all that rubbish !!!!!! HAVE A MERRY X-MAS !!!!!!!

    2. Lancelot wrote on

      I surely shouldn’t comment , I get in trouble lol, but you intrique me, why?? First point: because they are insecure and seek validation and are “driven” to seek peer group approval but their peers are as inadequate as themselves and so they need reiterative “fixes”; the “Never Ending Story”. Second point: I am a man, I make things. I make tools to make tools, some of them are beautiful, don’t diss’ the quest. I walk in the NZ jungle at night without a light, I surely know nature and “the way” of creation. I set myself at rest before I move. I Listen, silent, in the night. can You do that that true too? 😉

    3. cf wrote on

      If there was a “like” button, this is where I would be clicking it right about now!!!!! Well written!

    4. PayZaLee wrote on

      It’s not about money it’s about power. Why would a billionaire need another billion? There is nothing he/she can’t buy with their billions except more people’s adulation. The only real power comes from other people not things!

  25. john g farrell wrote on

    very good and true the truth always wins in the end.

    1. Richard wrote on

      Wrong, the truth does not win. Rather the winner defines what is true.

  26. Garrick wrote on

    The concept of “the grid” exists and is governed by a police state which is in turn run by a blindfolded dispensation of state mandated moral-speak. The Analytics described above are used to prop-up the fundamental lie that is perpetuated on a daily basis, namely that man is capable of governing himself.

  27. bobert wrote on

    Excellent! Very easy to extend the concepts presented beyond the cyberworld and into everyday interactions.

    For example., reading some of the replies to your article. It would be easy to make conclusions about the intelligence of some of the posters. But not knowing anything about what was not revealed by them would make these conclusions bogus. ?

  28. Russ Greenway wrote on

    “Facts are truth” are things that we choose to believe at that particular moment in time. As Maynard-Keynes said, ” when the facts change- I change my mind”. The internet encourages us to live in the “now” time frame. We rarely reflect on things and discuss them – a bit like me posting this! Much of the information on the internet is flak. It’s there to obscure and deflect attention from the real goings on in our world. How do I know this you may ask? Well -I read it on the internet! We have a big thing now over “populism”. David Cameron, the UK Prime Minister has just blamed it for him losing the Brexit vote. But, Donald Trump has used it to get into the White House. Maybe it’s the way that we are treated by the number crunchers’ who can accurately predict our intentions that get us annoyed. That others actions and intentions are able to be predicted – but not us because we are not sheep but they are. If you use the internet then Google et al will have plenty of information on your on line activity to pigeon hole you for the benefit the number crunchers. Just like this now!

  29. Lance Humphries wrote on

    When 15 years old, 1966, I was reading my 6-year-older sisters statistics text books to her, 3rd year BSc, as she balled wool preparatory to knitting. She struggled to understand the terminology and methodology while I struggled to grasp how something so simplistic and one dimensional could possibly be a 3rd year paper. Wtf, it’s just a side issue of true maths, my later major. Statistics is like a black and white photo of a tree or flower, it may give you a glimpse but not define the plant. It’s a static clip of a dynamic, and when you get into the quantum “emergence” of life it’s utterly useless. but bean-counters cling to it like a life saver. Statistics do not lie, but they are no more than the box that a high-tech computer comes in: the “lie by omission” is in the heads of those of truncated vision; unfortunately the majority. If a ship sails the Tasman Sea to Australia in 3 weeks then 3 ships will sail it in 1 week. The “implied lie” is in every part of advertising, “Get more…!” More than what, where are we starting from? “The best…” Best of a bad bunch, best of a bad situation?? It’s still bad; ‘best’ is bogus! You all live and breath and eat this crap and cling together in co-dependent clots, you cower from Reality. “I don’t know why she swallowed the fly…” So what point in debating analytics when even those who pursue it merely conceive its function but evade perceiving the true picture. Kodachrome! I suggest you look into “The calculus of Life.” True discipline, I kid you not!

  30. Clark in Arizona wrote on

    Evolution has caused the human eye (more properly “eye-brain system”) to be more attracted by light. As someone who has been “into” computers since a few weeks after Neil Armstrong took his “one small step” (on the Moon), it seems to me that it would be easy to do an “inverted bright city” — i.e., one that shows all white and starts darkening the “data lines” — to (literally) highlight where the hidden stuff is.

  31. W J wrote on

    J’ai travailler toute ma carriere avec des statistiques et je sais qu’on peu leurs faire dire que l’on veux. Analytique, Statistique, c’est du pareil au même.

    I work all my career with statistics and I know that we can make them say that we want. Analytic, Statistical, it is all the same ting.

  32. Ivan wrote on

    Ted thanks, this is a very thought provoking article and an issue most people are not really aware of or taking seriously yet. The implications and potential threat to ‘Truth’ that this sort of information ‘moulding’ and ‘selectivity’ poses is profound indeed. The ‘Truth’, for me, exists in the reason for asking the question in the first place; but perhaps before that event occurs, we must inevitably find ourselves facing some sort of ‘inconvenience’ or ‘existential crisis’.

  33. Jack Spence wrote on

    Thanks for the article. I really hadn’t given much thought about the deliberate masking of data by adjusting the selectivity parameters, to favor a “conclusive” result.

    The mainstream media seems to be learning how to use selective statistics in a crude way, to alter the minds of their viewers. We should remain vigilant, not accepting each presentation (or newscast) as an undisputed truth, regardless of the source.

  34. Steve Kugelman wrote on

    I guess the old saw still applies; “figures never lie, but liars always figure.

  35. Robert St. Louis wrote on

    Hi Ted:
    Good article, but I have one small ask: in the future, please remember that the word data is the plural form of the word datum. Thus, when you speak about the data, you need to refer to them, not it. I found this an amusing irony in your piece!