We’re bombarded by claims using data every day. In BC during COVID we are currently talking about our vaccination rates in terms of “eligible” people. In some ways this makes sense, but when we talk about vaccination rates for other diseases, do we use the same term? Are “eligible” people the same group in both circumstances?
Asking questions about data is what The Data Detective is intended to help readers do[^1]. Before we get into the meat of the review, I want to talk about an interesting concept Harford introduces called manufactured doubt.
Manufactured doubt is when anyone or any organization presents information that gets people more concerned about topics that don’t have anything to do with the people or organization[^2]. Harford cites cigarette companies in the 50’s funding a bunch of other studies on climate change and other things with the purpose of pointing to something that wasn’t smoking people should think about.
We see this today as well, but it’s called fake news or misinformation or alternative facts. It’s whole goal is to muddy the waters with information that probably doesn’t matter so that knowing what to believe is hard and thus people will discount much valid information along with the misinformation.
If you want to read more about this idea then The Death of Expertise covers it as well.
With that concept out of the way, let’s dig into some of the questions Harford says we need to be asking when presented with data in it’s various forms.
We Dismiss Data we Don’t Like
One big trap we all fall into is looking for data that confirms what we believe or refutes information we don’t believe[^3]. If you’re a coffee drinker, you don’t want to hear any information that suggests your habit is one that’s bad for you.
The one big thing Harford tells readers to do is to stop and think about what the data you’re encountering makes you feel. Then ask yourself if what your reading is true, and if the truth of the claim matters to you. This can help filter out the manufactured doubt that comes our way[^4].
What is Your Personal Experience?
One thing I stick to is the idea that I don’t argue about zebra’s. Living outside Vancouver Canada, zebra’s are a very rare thing. When I ride my bicycle around I see lots of horses and I suppose at some point I’ll see a zebra, but it’s so highly unlikely that it’s essentially impossible.
The thing is the news only talks about zebra’s[^5]. In fact news isn’t about news it’s about eyeballs and entertainment[^6]. News used to tell you what you needed to know, now it tells you what will keep your attention.
In light of this, look at the experience of those around you. While major events like terrorism are scary, almost none of us know anyone that’s been the victim of such an event. We’re all in far more danger daily from driving cars, yet we are more likely to be scared of a terrorist event.
What Aren’t They Telling You
On this site I’ve talked about the jam study, which showed that giving you less choice meant more purchases. If you put too many types of jams in front of someone they were less likely to purchase. You’ll find this study talked about in many places like The Harvard Review.
Did you know that when you look at all the studies like this the net result averages out to 0[^7]. Yup, in this context the number of choices makes no difference. Don’t just take the published statistics, because journals are predisposed to publish novel information that draws attention. Then when a follow up study shows that the counterintuitive point is in fact not true but our basic assumption was correct, good luck getting that published.
Who is Missing?
One important thing to know about any study is what group of people was used in the study. Many studies end up using College students, because they’re done by faculty and it’s easy to pay students a tiny bit for them to participate. Unfortunately this has a selection effect though since people that attend college are more likely to be similar[^8].
Probably worse is the number of studies that exclude 50% of the population, women. Invisible Women is an entire book about how women are invisible in almost every part of life. From the standard sizes of things, based on a man’s proportions, to transit design which is done mainly by men and discounts the needs of female riders. It’s an enlightening and depressing read I highly recommend.
Getting Started with Zettelkasten
If you’ve been wondering about what Zettelkasten is and how to start organizing your notes with this excellent system then this course is for you. I’ll walk you through how I use this system to develop my reading research for posts like this one here. You can also become a member to get all my courses.
$99 USD (30-day guarantee)
When you hear any published claims, make sure you investigate who was surveyed or who participated. Is that group of people an accurate cross-section of those that should have been included?
Beware Data Visualization
Infographics are very popular today, but just because something is in a pretty graphic doesn’t mean it’s true[^9]. It’s very easy for a pretty design to distract you from the fact that the data in question is terrible. Harford shows an infographic about deaths in the Gulf War. By simply changing the title and the colour of the bars the same data changes from evoking anger, to relief that deaths are going down.
The Golden Rule
The biggest thing that Harford would have readers do is to stay curious. Curiosity helps stop people from rolling into the trap of polarization[^10]. When confronted with someone you don’t agree with, and who can speak reasonably, stop and have a conversation. Interrogate what they think and why.
Out of this discussion, there are two things that can happen. The deeper you dig, the more the person you’re talking with may realize they don’t understand their position. This can help them question the ideas they hold to. Alternatively, it can help you understand an idea better and possibly change your mind.
If you fear having your mind changed, then it’s highly likely you don’t understand something as deeply as you think. This is the reason that I often start with reading I’m most likely to disagree with when interrogating a hard topic.
Should You Read The Data Detective by Tim Harford?
Yes you should read this. I didn’t cover all the rules, just the ones I found most interesting so there is much more to get out of this book than I’ve provided.
Alongside it I’d recommend:
Purchase The Data Detective on Amazon
[^1] Page 9
[^2] Page 12
[^3] Page 24
[^4] Page 42
[^5] Page 56
[^6] The Death of Expertise Page 141
[^7] Page 107
[^8] Page 137
[^9] Page 216
[^10] Page 268