We all know that correlation does not equal causation. Just because something occurs with something else, doesn’t mean that one caused the other. If you do a dance and then it rains, that’s not enough evidence that the dance caused it to rain. Even if it rains almost every time you dance, it could be that something else is causing both the rain and your dancing. Perhaps a drop in barometric pressure causes your joints to hurt and you dance to loosen them up while the same drop in pressure causes rain. It’s a silly example. But you get the point. (Check out this hilarious website which shows other spurious correlations, such as the one between cheese consumption and death-by-bedsheets.)
Nonprofits (and everyone else) often make erroneous claims based on correlation. We might conclude, based on our data, that participation in our employment training leads to higher wages over time. Well, maybe. But perhaps employment in our city is on the rise and affecting everyone, not just participants in our program. Or maybe our program tends to attract participants who are quite motivated to find jobs and would do just as well without the program.
Correlation is necessary but not sufficient to prove causation. Indeed causation is a very high bar to reach. You must have three conditions: 1) correlation: two factors co-occur, 2) precedence: the supposed cause comes before the supposed effect in time, and 3) no plausible alternatives. This third condition is the trickiest. It involves ruling out other causes of the observed effect. So you can see why even carefully designed studies can rarely produce incontrovertible evidence of causation. (For more on establishing cause and effect, read this.)
Short of hiring researchers to design and conduct rigorous (and usually expensive) studies of your work, you can at least consider plausible alternatives. When you observe something good or bad happening in your organization, consider possible causes both within and outside of your organization’s control. If possible, try altering just one factor, collect data over time, chart it, and see if a trend changes. Explore rather than assume.
See other data tips in this series for more information on how to effectively visualize and make good use of your organization's data.
Photo by Ariel Lustre on Unsplash