In the lecture we discussed the notion of causality. We contrasted the causality mechanism that you might be most interested in (your story) with the causality mechanism(s) that might be driving your data (the data story). For good data-analysis you need to be aware of both: have a clear idea of what you want to analyse or explore as well as of all the things that could be going on in your data. To practise this I ask you to go through a series of examples. These are suggestive causal relationships typically found on the basis of some correlation in one dataset or another. In each case:
- Can you think of a mechanism that would support the causal relationship suggested?
- Can you also think of at least one reason of why the relationship observed in the data might have been the consequence of confounding factors in the world but not considered by the analyst (or even reverse causality)? As a consequence of which the effect suggested might be either non-existent or biased?
- In which direction would the bias from your suggested confound go?
For each case I give you a suggestion as example when clicking the answer tab. You might also come across some interesting causal claims that are made e.g. in the news. If so you are very welcome to share those on the course forum.
Exercise 1.1
Police body worn cameras increase violence against the police?
(For the full story, visit: https://www.govtech.com/public-safety/study-finds-negative-effects-of-police-worn-body-cameras.html)
We would typically expect the opposite; i.e. if people are on camera it is more likely that they can be prosecuted and punished for attacking a police officer. But as I suggested: a key skill of an applied data analyst is to think of imaginative stories. So here we can test that skill: Maybe there are people who like a reputation as misfits. Maybe that’s a badge of honour in their peer group. Now what could be better for those people than being known and seen on camera for attacking a police officer? Another story could be as follows: perhaps having a camera changes the behaviour of the officers. They now feel more confident - indeed perhaps overconfident - to engage with people that are more prone to attack officers.
On the other hand: Maybe the story of the data behind these results is another one. Perhaps the introduction of cameras let violence against police officers to be better measured. Maybe police officers previously did not report all violence against them as doing so would require them to substantiate their claims which is harder if you don’t have a video record. With that available reporting went up but not actual incidence of violence. This “data story” would imply an upward bias on the actual impact of cameras on violence.
Exercise 1.2
Guacamole makes you more succesful at online dating?
The headline suggests that there is a positive causal effect from mentioning Guacamole to receiving responses in online dating. A driver for this is could be that Guacamole is a signal for a healthy lifestyle and health and fitness are qualities that are desired when looking for a romantic partner (The story of the article)
The story of the data could be another:
Perhaps younger people are more likely to be into guacamole. But younger people can also be expected to get more dating responses. This would imply an upward bias of the Guacamole->Dating success effect.
A taste for guacamole could be more prevalent among people living in cities. People are more likely to respond to people that are close by. Hence, people in cities will have more people close by and therefore receive more responses. Again this implies an upward bias.
Exercise 1.3
Going to the Museum makes you live longer?
(For the full story, visit: https://www.nytimes.com/2019/12/22/us/arts-health-effects-ucl-study.html)
It’s conceivable that visiting a museum calms you down, allows time for reflection and leas to insights that help you to be more healthy and less stressed and as a consequence makes you live longer.
Equally, museum visits could simply be conflated with other more clear-cut factors that make you live longer. Income and education would be clear candidates. That said, the study quoted in the article has already accounted for these. But there are other factors that are not easily controlled for. For instance, having a stressful job with little spare time it probably not good for both finding time to go the museum as well as for your life expectancy. This mechanism would bias the museum -> life expectancy effect upward (stressful job is negatively correlated with both museum visits and life expectancy).
Exercise 1.4
Eating chocolate helps you win a Nobel?
A New York doctor wanted to investigate the effect of flavanols, compounds found in chocolate but also tea and red wine, on cognitive ability. He used the number of Nobels as a convenient proxy for his outcome variable and country-level chocolate consumption data as his explanatory variable and got a highly significant positive result. Before you rush to the shop for your bar of Cadbury’s though, think of what else might be driving the result. It might well be that we have an omitted variable here - wealth - which drives that positive relationship. A richer country (like Switzerland, which has 26 Nobel winners) will have more resources to invest in research and its affluent citizens might be more likely to be able to treat themselves frequently to chocolate. It might also be that people who study (and are therefore more likely to get a Nobel, of course), need a sugar fix more often and snack more. In both of these cases we’d have an upwards bias as wealth and time spent studying are positively correlated with both chocolate consumption and the number of Nobel prizes.
Exercise 1.5
Sex makes you rich?
Institute for the Study of Labor based in Bonn, Germany, published a study examining the “The Effect of Sexual Activity on Wages” in their discussion paper series in 2013. Sure, it might be the case that having sex triggers a rush of hormones, which then make you more productive and give you an edge in the workplace. But it is perhaps more plausible (and this is in fact what the original study claims) that sex is another indicator of health, which is our omitted variable. In this case, we would have an upwards bias.
Another possibility is that there could be a measurement error. The data comes from a survey and you should always ask yourself whether it might be the case that respondents could systematically misrepresent certain information. Other studies have found that when it comes to sex, men often over-report how much of it they’re having while women under-report. If the sample was not gender-balanced we may get upwards or downwards bias, depending on which gender prevails.
Exercise 1.6
Marriage makes you happy?
And they lived happily ever after…or did they? This study from a 2006 issue of the Journal of Socio-Economics actually does not make any grand claims at the outset but instead sets out to investigate which way the causal story is going: is it that happier people get married or that marriage actually makes people happy? Reverse causality is a tricky nut to crack. This study finds evidence that happier singles opt more frequently for marriage and that benefits of marriage vary widely among couples – not surprisingly an equal division of labour at home is an important factor driving happiness in marriage.
Exercise 1.7
Sleeping with a night light as a kid makes you blind later?
Short-sightedness is a growing problem globally. Researchers at the Children’s Hospital of Philadelphia thought they found the culprit: night-lights. The data, at face value, seemed to have suggested that kids who sleep with night lights develop myopia later in life. According to the researchers, the story was that the light makes the eye develop abnormally, which then affects focus. There could be a more obvious explanation, however, which is that myopia is hereditary – myopic parents are more likely to install night lights in the house, including their children’s bedrooms, but the light itself has nothing to do with their kids becoming short-sighted. This is, again, a case of upwards bias as myopia in parents is positively correlated with the presence of night lights in the house and with myopia in their children.