The Data Detective - Tim Harford

In The Data Detective, Tim Harford takes a good look at how statistics are presented to us in various mediums working to help readers develop the tools needed to evaluate the claims being put forward in society.

If you're going to follow one rule from the book, be curious. Don't just take claims at face value, ask yourself how the conclusions were arrived at? What were the rules of the questions asked to get the outcome described?

The book is a great read and does a good job at equipping readers to question the facts around them so that they do come to good conclusions about how the world operates.

Read my longer review here.

• Purchase on Bookshop.org - support local bookstores

• Purchase on Amazon

**Purpose**
- to teach us how to evaluate statistical claims for ourselves Page 9

- [[manufactured doubt]] Page 12

### Rule One: Search Your Feelings

- [[Johannes Vermeer]] Page 20
- we find ways to dismiss information we don’t like to hear. Like more coffee increases the risks of cysts in women’s breasts. Coffee drinking women don’t want to hear that. [[Disconfirmation Bias]] Page 24
- we also like to find information that agrees with our beliefs which is [[Confirmation Bias]] Page 24
- [[the ostrich effect]] Page 25
- we can combat all these affects listed above if we simply notice the emotions that new information brings to the table. Then we can mitigate emotional responses that are outsized for the information gathered. Page 25
- [[motivated reasoning]] Page 28
- being educated about something doesn’t guard you from any [[cognitive bias]] in fact sometimes it’s a detriment and gets you to notice fine details that can lead you astray Page 33
- [[partisanship]] Page 34
- [[biased assimilation]] Page 35
- One of the best checks to easy mistakes based on how you feel is to take a minute and ask if the truth matters and if what you’re reading is true. That helps filter out most of the easy [[manufactured doubt|fake news]] that would cross our paths. Page 42
- are we straining to make a decision with the facts at hand that suits our world view?

### Rule Two: Ponder Your Personal Experience

- [[Tags/anti-vacination]] Page 53
- [[naive realism]] Page 54
- the news talks about [[Tags/zebratic arguments]] (zebras) and if we follow it, our information about what’s common in the world is skewed by all the zebras. Page 56
- understanding just the statistics is incomplete. It could lead us to know that people survive on $27 a day without really understanding what that means to their quality of life and how different it is from our quality of life in NA. Page 64.
- [[worms eye view]] vs [[birds eye view]] means understanding the big picture and understanding what it’s like on the ground to the people experiencing the ideas you’re looking at. Page 64

### Rule Three: Avoid Premature Enumeration

- A large portion of the reason that the US has such a high infant mortality rate is how early in a pregnancy the US determines it’s a miscarriage vs a live baby that died shortly after Page 66
- when we normalize the weeks between countries the infant mortality rates are almost the same
- before we dive into the numbers of the statistics we need to understand what is being measured and what definition is being used as a measure Page 69
- [[the curse of knowledge]] Page 71
- the more dramatic a headline the bigger the chance you need to dig deeper into the claims to understand exactly what happened before you can have any hope of knowing the truth Page 77
- [[net wealth]] is a great way to measure riches, but a bad way to measure [[Tags/poverty]] Page 79
- A subsistence farmer may have a net wealth of $100 because they own their cow, while a Jr Doctor has a negative net wealth when they graduate. One of them is going to be entirely fine while the other struggles and net wealth doesn’t tell that story


### Rule Four: Step Back and Enjoy the View

- In 1965 two scientists found that what counts as news depends on the frequency that readers check for news. If media knows that we check every day or every few hours they tend towards the most sensational headlines to grab us. Page 89
- a newspaper that was published every 25 years wouldn’t mention almost anything that a daily does. A weekly or monthly paper is likely to skip over lots of things that seemed important daily, but actually had no real importance
- [[Gini coefficient]] Page 92
- make sure you step back and get the context and perspective on the claims being made. Is that a big number? Is the difference highlighted actually notable? Page 95
- unfortunately media outlets rarely do this for us because they want eyeballs with sensational claims

### Rule Five: Get the Backstory

- [[jam study 2020070627]] Page 106
- when you look at all the studies that look at the effect of more choices on taking action the net result averages out to 0. The famous [[jam study 2020070627]] is one of the outliers (of which there are a few) that got published and had a large effect. Many papers failed to find anyone to publish them because the effect was so small, or didn’t exist. Page 107 ^67777a
- most books people read are bestsellers, and yet most books are not bestsellers Page 109
- in fact, most books never get finished
- [[Abraham Wald]] Page 110
- [[survivorship bias]] Page 110
- [[HARK]] Page 118
- many high profile studies failed to stand up to replication Page 122
- often called [[replication crisis]]
- In summary, don’t just take the published fantastic results. Look at more studies, ask about the stuff that didn’t get published. Many times once you dig deeper you find that the fantastic was merely the outlier and reality wasn’t as clear cut

### Rule Six: Ask Who is Missing

- [[N = All]]
- as in, who were not in the experiments? Many only have College students. Many exclude women (see [[Invisible Women - Criado Perez]] for more on women missing in science) Page 137
- most places collect income based on household which assumes pooled resources and would hide abusive relationships where money is power and punishment. Page 141
- This missing people is also why a win was predicted for [[Hillary Clinton]] Over [[Tags/Donald Trump]]. Turns out it was simply harder to get Trump supporters to respond and no one out in the leg work. Page 147
- Or think about [[Tags/ai assistant]] data trained on white faces and English. This excludes a large portion of people while we fool ourselves into thinking our data is complete. Page 151

### Rule Seven: Demand Transparency When the Computer Says No

- so far the chapter has been about how algorithms work and are trained. It’s shown that Google Flu trends sort of turned out to be a winter predictor, and talked about another one that after looking at skin cancer and regular cancer photos said rulers predicted skin cancer because rulers were in the picture.
- cites one researching saying that the data assessing a teacher’s performance based on kids results is so noisy that it’s pointless to assess teachers based on student improvement tests. Page 164
- what if a teacher finds a way to game the system one year. The teacher the next year is in trouble because the paper geniuses need to show improvement next year and they’re ranked far better than they should be
- [[current offence bias]] Page 169
- the problem with [[algorithm|algorithms]] is lack of transparency in how they operate. Companies like [[Google]] want to keep that information in their black box but it means we have a hard time reckoning with any type of bias that has crept into the system because we can never see it until it’s grossly obvious. Many people have already fallen as quiet victims to the bias. Page 171
- the difference between [[scientific method]] and [[alchemy]] is that one was practiced in the open and under debate, and the later was performed in secrecy and rarely shared with others. Page 174
- there is no value in turning lead into gold if anyone can do it so of course it was a secret
- this secrecy left people thinking that others had done it, currently or in the past, and they just needed to keep muddling along to figure it out themselves
- [[algorithm|algorithms]] are similar to [[alchemy]] in they way they’re kept secret because of their value. This also means that we can’t share knowledge and evaluate the [[efficacy]] of them in the public realm. PAge 175
- [[Tags/criminal]] Page 178
- just like people [[algorithm|algorithms]] are neither trustworthy or untrustworthy. Think of Sheldon, don’t lend him money because you’re never getting it back but he’ll be there at 2am to help with something or let you stay at his house without end. I’d trust him with my kids. Page 179
- but I can know that because I have the information about his behaviour and it’s not in a black box like most algorithms are

### Rule 8: Don’t Take Statistical Bedrock for Granted

- independent statistical agencies like the [[Congressional Budget Office]] are great for looking at ideas and weighing wether they’ll really work based on the numbers and ignoring the politics of it. Page 187
- I wonder if [[Stats Can]] does something similar in Canada?
- perhaps one of the best indications of the independence of an organization is the attempts to subvert or discredit it by the people in power. PAge 190
- [[Stats Can]] has pushed back on [[Stephen Harper]] and [[Justin Trudeau]] when they’ve made steps that might reduce the independence of it’s numbers. Page 196
- [[politics should not rest on conjecture 230620210533]] Page 204
- [[reliable statistics can be used to hold government accountable 230620210541]] Page 206

### Rule Nine: Remember That Misinformation Can Be Beautiful, Too

- [[Florence Nightingale]] Page 213
- [[data visualization]] Page 216
- never let the fancy design of an [[data visualization|infographic]] distract you from the possibility the data might be wrong. Page 216
- decoration on data is often used to hide the fact that the data may not be on solid footing. Page 221
- also be wary of no references for the data, then no one can follow up on it
- ask yourself, what emotion is the visualization trying to evoke? Like the [[Gulf War]] death toll graph which was pointing down and red and titled “Iraq’s bloody toll”. When coloured blue, flipped and titled “Iraq: Deaths on the decline” it evokes a totally different emotion with the same information. Page 232


### Rule Ten: Keep an Open Mind

- [[The Great Depression]] Page 243
- even if you’re the smartest one in the room, it isn’t easy to chang your mind. Page 244
- [[Confirmation Bias]] Page 246
- specifically when Millikan’s measurement for the weight of an atom was too low, it set the standard by which others measured, thus something “far too high” was seen as a gross error and it took a gradual increasing to arrive at the actual weight decades later. He framed the starting point and people’s confirmation bias did the rest.
- [[Philip Tetlock]] Page 249
- [[superforecasters]] Page 252
- see the few pages before as well to get the whole story
- [[Tags/John Maynard Keynes]] Page 255
- Unlike [[Irving Fisher]], [[Tags/John Maynard Keynes]] didn’t have a perfect track record, so when [[The Great Depression]] hit he was ready to change tactics because he had done so before, or at least realized his limitations. Contrarily, Fisher, had always seen success so doubled down on his ideas and led himself to financial ruin. Page 259
- if that’s not [[Confirmation Bias]] then…
- making public pronouncements on an idea makes it harder to change your mind, you often get dumber as you double down on a bad idea. Page 260, 261

### The Golden Rule: Be Curious

- curiosity in a person breaks the pattern of [[polarized|polarization]] that people fall into. Page 268
- the more curious people are the more likely they are to converge closer to each other on hot topics because they rely on the evidence and are willing to change their minds in the face of new information
- they are less likely to fall into [[Confirmation Bias]] because they don’t filter out information they don’t want to hear
- [[information gap theory of curiosity]] Page 271
- [[illusion of explanatory depth]] Page 272
- to communicate well you need to spark a person’s [[curiosity]] or they won’t pay attention. Page 278