Oy, not another how to spot fake news article. Yup, afraid so. But — this one comes with a twist. It’s all about STATISTICS, which, it turns out, are even easier to fake than headlines. And more insidious because, more often than not, statistics are presented to be persuasive no matter what source they’re coming from, fake news or not.
So, does that mean don’t trust statistics? Well, yes … and no. At least, don’t trust the statistics as they are presented in the media. Statistics in the news or touted by politicians sometimes can be shown out of context, manipulated to present only one point of view, or taken from flawed research. The only way to determine the truth behind the statistics is to look at the original research. Ask the usual questions: who, what, when, how and why. Be specific. Did the research involve a representative sample? Who sponsored it? For what reason? When did it take place? How were the questions framed? Were the right questions asked? How was the data analyzed?
3 WAYS TO SPOT A BAD STATISTIC
In the TED Talk below, Mona Chalabi shows how easily statistics in polls can be misinterpreted and presents three questions we should ask ourselves when examining statistics reported in the news:
- Can you see uncertainty? Data visualizations create an illusion of certainty. In reality, polls cannot accurately predict the future.
- Can I see myself in the data? When research points to an “average,” it does not mean that will be your experience. In fact, it probably won’t be. By definition, there have to be numbers above and below the average, right?
- How was the data collected? This goes back to who, what, when, how, why. Without knowing the answers, we cannot know if the research is valid.
1 IN 3 AMERICANS WOULD MOVE TO AN ALIEN PLANET TO ESCAPE U.S. POLITICS
WHO, WHEN, WHY
Let’s apply our new-found critical thinking skills to this Huffington Post article: 1 In 3 Americans Would Move To An Alien Planet To Escape U.S. Politics. Right off the bat, we see the only source noted is Survey Monkey, which is a tool, not an author. But, when we expand the survey data, we see that Huffington Post is in the URL (https://docs.google.com/viewerng/viewer?url=http://big.assets.huffingtonpost.com/NewPlanetspollresultsSheet1.pdf), so we can assume it created the survey, most likely for entertainment purposes. We also find a date: February 23, 2017.
Next, we probably can assume it’s an “opt in” survey, very likely posted on the HP site, which means the respondents probably are Huffington Post readers, inclined to take surveys, express political opinions, and interested in space travel, or a combination of all four factors. So, that tells us it’s not a very scientific survey and probably is not using a representative sample. We also see there is no margin of error noted, another indication that it’s not a scientific survey.
Finally, if we compare the title of the article to the data, we see that it’s a little misleading. One in three sounds like a lot, right? It’s anchoring us to think a lot of people would rather leave Earth than deal with current politics. But the data reminds us that 71% would rather NOT leave Earth — quite a different story. We also see that only one question in the entire survey focuses on the political environment; the rest ask about space exploration and new planets.
Undoubtedly, this is a tongue-in-cheek for- entertainment-only survey, but it is in the news and most likely has been shared thousands of times. Indeed, a quick search on Google brings up a good number of hits. But by applying the who, what, when, how, why test, we’ve quickly eliminated it as a reliable source of information and can dismiss it as just some entertaining reading.
WHERE TO FIND MORE INFORMATION
A Field Guide to Lies by Daniel J. Levitin, 2016
The Half-Life of Facts by Samuel Arbesman, 2012
Situations Matter: Understanding How Context Transforms Your World by Sam Sommers, 2011
LINKS TO MORE WEBSITES