We all live, by necessity, in our own little worlds. We interact with our colleagues and our participants, clients, or audiences way more often that we interact with other folks. This way of life has both its comforts and dangers. One of these dangers may have never occurred to you. Academic types call it “selecting on the dependent variable.” Catchy name, right? It sounds more complex than it is. It simply means paying attention only to cases in which some phenomenon is observed and ignoring cases in which it is not observed.
Here’s an example from my own life. For years I suffered from back pain. I saw all kinds of physicians, and they all agreed that that three ruptured discs in my spine were causing the pain. The ruptured discs were plain enough to see on an MRI, so I accepted the diagnosis. Then, after 10 years of unsuccessful treatments, I learned of many reports in the medical literature of ruptured discs in patients with no history of back pain. The ruptured discs were discovered on CT or MRI studies conducted to investigate other parts of the body. If ruptured discs caused back pain, there should be a clear correlation between the observation of ruptured discs and back pain. We have all heard that correlation does not, on its own, imply causation. But I encountered physicians who were willing to label something as a cause (ruptured discs) without even a strong correlation with the effect (back pain)! It wasn’t until I started looking at other possible causes (stress/anger) that I solved the problem and began to feel better.
My doctors were drawing conclusions based on what they observed in their own worlds: patients with back pain and the MRIs of these patients’ spines. They weren’t looking at the spines of patients without pain. If they had, they might have drawn a different conclusion. If doctors, who know more about science than most of us, can make this sort of mistake, the rest of us should also beware.
Think about the ideas you develop based on just the participants, clients, and audiences your program or organization serves. Then widen the circle. Does your idea hold up when you consider similar participants, clients, and audiences in other programs or organizations? For example, you might think: participants who drop out of this program live far away from the program site. Perhaps. But are there similar programs (within your organization or at other organizations) that participants are willing to travel to? If so, then you might consider what other aspects of the program or characteristics of the participants might lead to drop out.
To see past data tips, click HERE.
Let’s talk about YOUR data!
Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.
