Data visualization has its pitfalls and landmines. Sometimes charts, graphs, and maps can have harmful consequences, intended or not. So I’m offering up another tip in a series of 60-Second Data Tips that point out pitfalls and landmines to avoid. This time, it’s about hiding variability.
If we see a bar chart like the one below showing median income of Medicare beneficiaries by demographic group, we may make the following mistakes in interpreting it:
Assume that all or most of those in the higher income groups are earning more than all or most of those in lower income groups rather than assume that there may be more variation in income within groups than between groups.
Assume that the higher median income of a given group suggests that those in that group are more capable, smarter, skilled, etc. than those in lower median income groups rather than assume that those in the lower group faced challenges (e.g. racism, lack of education, etc.) that resulted in the lower median income. (This is called the fundamental attribution error.*)
Generalize our error-ridden conclusions about the people represented in the chart to all of those in certain racial, age, gender, or other demographic groups.
Source: Kaiser Family Foundation
Eli Holder and Cindy Xiong conducted studies in which they showed participants charts that emphasized within group variability as well as those that did not like the chart shown above. And they found that participants were less likely to make the mistakes listed above when shown charts that emphasize within group variability such as the one below which shows that there is a lot of variability in scores within schools that one can’t appreciate by only looking at averages (represented by the black horizontal bars.)
For a great ten-minute summary of the research, check out this video. What to do to avoid this pitfall? The video suggests that we find ways to represent variability within groups as shown below:
To see past data tips, click HERE.
*When visualizing data, we should consider how humans think including heuristics (i.e. mental shortcuts) and biases. Some of them have particular relevance to data visualization such as the fundamental attribution error, which is our tendency to relate behavior to a person’s character and personality rather than to the person’s context. So when someone cuts you off in traffic, if you write them off as a jerk rather than someone who is late for work or distracted, that’s the fundamental attribution error.
Let’s talk about YOUR data!
Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.
