Data Viz Resources You Should Know: Visual Capitalist

Here’s a new addition to my highly-curated resources list: Visual Capitalist. I occasionally write a 60-second data tip describing a particular resource, including why I think it’s cool. And I link each of these tips to a resources list on my website.

What is it?

Visual Capitalist is an on-line publisher of data visualizations. Their aim is to make the world’s data more discoverable. As the name suggests, the visualizations primarily focus on markets, technology, energy, and the global economy but also on education, healthcare, and the environment using public sources of data including government entities (i.e. U.S. Census data, FRED, etc.), intergovernmental organizations (IMF, World Bank, etc.), or established thought leaders in their respective sectors (think tanks, non-profits, corporations).

Who’s it for?

Anyone who wants to understand or show information on markets, technology. energy, healthcare, education or the global economy. They allow much of their original visual content to be used by others. In many use cases, this can be done for free. (See more info on using their content.)

Who’s behind it?

Visual Capitalist is independently owned, with its founder, Jeff Desjardins, as a majority shareholder.

Why I think it’s cool

Their visualizations can be quite engaging with high production values. You can share them on social media for free. Sign up for a free subscription on any page of their website to get one in your email box each day.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Behold the Heated Bar Chart

I’ve come across this useful chart a number of times and have yet to find a name for it. So I’m giving it one: the heated bar chart. If a bar chart and a heat map had a baby, it might look like this and be even more powerful than its useful parents. This heated bar chart shows months along the horizontal axis and days of the week along the vertical axis. Darker and longer bars on the two bar charts and darker cells on the heat map show when there has been the highest participation in hours. This allows us to examine patterns in participation. We can see, for example, that the relatively high participation in May is being driven by participation on Mondays during that month but that Monday was not a particularly high participation day during other months. We can also see that, overall, participation was lower during the fall months, regardless of day of the week.

This powerful combo chart can be used with many different types of data fields. For example, you might want to create one showing the number of participants from different gender identity and age groups to see if certain gender groups within certain age brackets are not well represented among your participants.

I made this chart with Tableau Public, the free version of Tableau. But this type of chart can be created with any number of applications. For those of you with at least a little Tableau know-how, check out the steps I took in the comic strip below.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Pop Quiz: Guess What This Chart Shows

Reposted from September 2022

Go ahead and make a guess from the options below. Then scroll down to see how your response compares with others’ and what the answer is!

Keep scrolling!

The answer: The decline in child poverty in the U.S.


As reported in The New York Times, “the sharp retreat of child poverty represents major progress and has drawn surprisingly little notice, even among policy experts.” Read the article (and view the detailed line chart) to learn more about the role of government aid in lifting children and families out of poverty.

I share this chart with you—in this way—for a couple of reasons:

1) It’s an engagement strategy you can use. Rather than present a list of stats to your audience, you can engage them in your data by first quizzing them on an interesting, fun, or counterintuitive finding from your data.

2) Bad new bias. Bad news is more likely to be reported than good news, possibly because bad news sells, according to this article citing various research. Perhaps because of that bias, we may be more likely to assume a chart is telling a negative story. This chart is a reminder of the importance of taking a broader view to gain a more balanced understanding of an issue.

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Ideas You Should Steal From This Viz (Installment 13)

“All creative work builds on what came before.” —Austin Kleon in Steal Like An Artist.

Today I offer up another steal-worthy interactive viz that I came across in the Tableau Public Gallery. Scroll down to see what you should steal from it.

Source: Nir Smilga on Tableau Public

Here’s what I suggest you steal from this viz:

  • Small Multiples. To highlight each country and to allow for easy comparisons across countries, Smilga created one small chart per country and placed them alongside each other, aka a “small multiples chart.”

  • Gray comparison trends. While Smilga highlights the trend for the featured country in each chart using color, the trends for other countries are also in each chart but in a light gray. This allows us to easily compare the trend for the featured country to that of others in general.

  • Color distinguishes trend types. Recent downward trends are highlighted in red, and recent upward trends are highlighted in blue.

  • Choose your view. Smilga allows the viewer to customize the view by selecting the number of columns, time period, and countries shown.

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

A Common Problem with Survey Data - And How to Avoid It

Sampling bias occurs when some members of the group you are trying to understand are less likely to be included in your data than others. Survey data is especially vulnerable to sampling bias. The data you collect is often not representative of the whole group that received the survey but rather the subgroup that was willing to complete the survey — as illlustrated in the “sketchplanation” above. So here are some quick tips to avoid this type of bias in your survey data:

  • Clearly define the group and related subgroups that you want to understand. Consider what might be necessary to collect sufficient data from all of the subgroups.

  • Follow up with those who don’t respond to the survey to understand why they didn’t respond. Did you ask the wrong questions or target the wrong audience? Apply these insights next time you are planning a survey.

  • Make your survey brief and easy to understand.

  • Finally, don’t overinterpret your survey date. Assess which types of respondents were the least likely to respond and interpret accordingly.

For a fuller explanation of types of sampling bias and strategies to avoid it, check out this SurveyMonkey article.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Create Data Stories That Actually Engage People

Here’s a sneak preview of a workshop I’m doing on September 26th. I’m sharing the first few slides below. Click on the right to advance through them.

Presentation Preview by Amelia Kohm

To tell a story in a way that humans can understand and get behind, it helps to understand humans’ powers and challenges when it comes to consuming data. Then we can make better charts, maps, and graphs (aka data visualizations) and present them in a way that humans can absorb.

I hope you can join me on September 26th, 8:00 am - 9:00 am PT | 11:00 am - 12:00 pm ET. Click HERE to register.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Activate Board Member Fundraising With Visuals

Reposted from January 2023

I recently came across this excellent article by The Fundraising Authority. As promised in the title, it provides a “Simple, Step-by-Step Process for Getting Your Board to Refer New Prospects to Your Non-Profit.” In a nutshell, here are the four steps:

  1. Explain how referrals work and assure your board members that no one they refer will be asked for money until they indicate a desire to get involved.

  2. Show board members how many people they actually know through a mind mapping exercise.

  3. Ask board members for referrals usually in person.

  4. Bring referral success stories back to board meetings on a regular basis.

My tip is to enhance steps 2 and 4 with visuals.

Visuals for Step 2: For the mind map, the point is for board members to brainstorm all the people they know by considering people in different categories of their lives. You can use Canva whiteboards (or a similar tool) to create a mind map that the board member (pictured in the middle) can use to add the names of people in each category on virtual post-it notes.

Visuals for Step 4: The article claims that “this is a key step. Nothing will convince your board members to bring you more referrals than hearing from other board members that have done it successfully.” You can visualize the donors whom various board members brought in using tools like Flourish to show their networks, as in this example. Scroll over the circles to interact with it and learn more. Some board members brought in donors who, in turn, brought in other donors. To make something similar, select one of the network graph templates on Flourish and fill in the data needed. (See snapshots of the data I added for the visual below.)

Links data

(used to show who is connected to whom)

Points data

(used to show groups by color, size points according to amount of donations, and add images for board members)

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Sure Data Confuses You. You Are An Early Human.

Reposted from August 2022

I recently read this article which suggests that, if we avoid a large catastrophe, we may be living at the early beginnings of human history. That is sort of mind-boggling. But even if we are not early humans, we are certainly at the beginning of our journey with data.

We aren’t so good at processing words and numbers and making sense of them. Think about the last time you looked at a spreadsheet and got the gist of it in a few seconds. By contrast, we can get the gist of a photo in less than a few seconds. That’s because our brains have evolved over millions of years to process visual information — color, shape, size, placement — at lightening speed. Indeed, our survival depended on it. Think about detecting predators in the tall grass at a distance.

Processing words and numbers is a quite recent activity for humans, as the timeline below shows. Perhaps, as our brains evolve, we will be able to discern a spreadsheet at a glance. But, until then, we should consider visualizing our data by translating words and numbers into color, shape, size, and placement in the form of charts, maps, and graphs.

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Ideas You Should Steal From This Viz (Installment 12)

“All creative work builds on what came before.” —Austin Kleon in Steal Like An Artist.

Today I offer up another steal-worthy interactive viz that I came across in the Tableau Public Gallery.

Here’s what I suggest you steal from this viz:

  • Dot Matrix Chart Type. Each dot in a dot matrix chart is colored to represent a category and sized to represent the magnitude of the category. Thus the chart provides an overview of the distribution and proportions of each category in the data set. By clicking through the ban categories at the top (classrooms, libraries, etc,), we get a quick sense of the magnitude and origin of book ban challenges.

  • Explanation/Directions. The text on the left tells us everything we need to know to extract meaning from the visualization without loading us down with unnecessary details.

  • Details on Demand. For those who want more information, details are available on demand by scrolling over the individual circles and the info button.

Source: Gbolahan Adebayo on Tableau Public

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Another Data Viz Resource You Should Know: USAFacts

Here’s a new addition to my highly-curated resources list: USAFacts. I occasionally write a 60-second data tip describing a particular resource, including why I think it’s cool. And I link each of these tips to a resources list on my website.

What is it?

USAFacts is dedicated to making government data more accessible. The idea is to help people understand where their tax dollars are going and to help those working on issues of concern in the philanthropic, nonprofit, and public sectors to easily access information to inform their decisions.

Who’s it for?

The general public, policymakers, philanthropists, nonprofit managers, and researchers, among others.

Who’s behind it?

After he retired from tech and began to focus on philanthropy, former Microsoft CEO Steve Ballmer wanted to understand what the government spends on programs to help people and the outcomes of those programs. However, unlike businesses, US governments are not mandated to compile reports on their expenditures. He then hired data analysts to compile this data, and this was the origin of USAFacts, a not-for-profit, nonpartisan civic initiative which is solely funded by Ballmer.

Why I think it’s cool

It’s free and the data is shared under a Creative Commons license. They only ask that you credit USAFacts when using their curated material. Most of their data is visualized and all of it is well-documented. Charts can be easily downloaded in an image format or embedded into your website using an embed code like the chart below.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

This Is A Teenager: Showing The People Behind The Data

My tip this week is to check out This Is A Teenager by Alvin Chang at The Pudding. It’s a visual essay that traces the paths of hundreds of teenagers, starting in 1997, to see how their childhood experiences relate to their life outcomes. You can view it in video form (see below) or as a scrolling visualization HERE. The actual teenagers behind the data* don’t get lost in aggregates represented by bars, lines, and circles. Instead, we see them as individuals whose lives tend to follow the paths of other individuals with similar childhood experiences.

* The data is from the National Longitudinal Survey of Youth.

Source: The Pudding

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Create More Diverse, Equitable, and Inclusive Data Visualizations

Reposted from February 2022

When we visualize information, we make a series of decisions which affect the way that viewers process the information in our charts, maps, and graphs. Sometimes they don’t feel like decisions at all. We go with the default settings in the application we are using. Or we just do something the way it’s usually done. But a more diverse, equitable, and inclusive approach to presenting and visualizing data requires us to make those decisions more consciously and deliberately. Jonathan Schwabish and Alice Feng of the Urban Institute provide some helpful tips, based on the Urban Institute’s own style guide, which you can apply the next time you present data.

Here is a summarized version of Schwabish and Feng’s article.* And here is my 60-second version of their recommendations:

  • Use people-first language in titles, text, and labels associated with charts, maps, and graphs. For example, use “people with disabilities” rather than “disabled people.” Also the Urban Institute does not refer strictly to skin color. For example, they refer to “Black people” not “Blacks.”

  • Order and present groups purposefully. The first group shown in a table or the first bar in a chart can affect how readers perceive the relationship or hierarchy among groups. For example, if the first group is “Men,” then it may appear that men are the default group against which other groups should be compared. One way to prevent viewers from making certain comparisons is to display groups in side-by-side charts (aka “small multiples” charts) rather than on a single chart. In general, make ordering and grouping decisions to promote certain comparisons and prevent others.

  • Point to missing groups. If certain groups are missing from the data, explain why in text boxes or footnotes. Also add information on groups included in “Other” categories and consider providing a more specific label than “Other” which can have an exclusionary connotation.

  • Do not use color palettes that reinforce gender or racial stereotypes. This one may seem obvious, but it bears repeating. Also, the Urban Institute’s color palette is accessible to people with certain color vision deficiencies, and the contrast between those colors and white and black text meet basic accessibility guidelines.

  • Depict a variety of races and genders when using icons and avoid icons that make inappropriate depictions of people or communities or reinforce stereotypes such as showing traditionally feminine icons to depict nurses or traditionally masculine icons to depict bosses.

  • Find ways to show the people behind the data. Data visualizations are, by definition, abstractions of larger realities. But in the process of abstracting, we may obscure the lived experiences of the real people whom the data represent. Visualizations can remind viewers about the individuals behind the data by, for example, depicting them as individual circles rather than aggregating them in a single bar.

* The full paper has been published as an OSF Preprint and can be accessed here.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Data Viz Resources You Should Know: Data.gov

Here’s a new addition to my highly-curated resources list: Data.gov. I occasionally write a 60-second data tip describing a particular resource, including why I think it’s cool. And I link each of these tips to a resources list on my website.

What is it?

Data.gov is the United States government’s open data site. Open data is data that can be freely used, re-used, and redistributed by anyone - subject only, at most, to the requirement to attribute and sharealike. Data.gov is designed to “unleash the power of government open data to inform decisions by the public and policymakers, drive innovation and economic activity, achieve agency missions, and strengthen the foundation of an open and transparent government.”

Who’s it for?

It’s for the general public.

Who’s behind it?

The U.S. government. More specifically, The U.S. General Services Administration, working with the Office of Management and Budget and other agency partners, launched Data.gov in 2009. Government agencies compile metadata such as title, description, keywords, and links for accessing their datasets, and the Data.gov catalog automatically “harvests” that metadata to populate a continually updated catalog.

Why I think it’s cool

Unlike many other open data catalogs, you can find and download data quickly and visualize it. You can begin by searching for keywords in the search box. And there are helpful filters to narrow the results by, for example, topic categories, location, and agency. This is a great place to find data to show the need for your organization’s services and the problems you and your colleagues are working to address.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Presenting Data Fast and Slow

Reposted from May 2022

Sometimes we approach the challenge of sharing data with others as if we were trying to con a pet into taking a pill. We think that our audience is too busy, disinterested, or distracted to focus on the data. So we wrap it in something that attracts their attention and feed it to them as quickly as possible. The problem with this approach is that it may get the data into their brains—momentarily—but it won’t stay there long. See where the pill ends up in this video.

If we want others to LEARN from the data — which involves not only retaining it but also drawing knowledge from it and applying that knowledge in the future — then we need a different approach. Daniel Kahneman’s Thinking Fast and Slow can help us.

First a little background on how the brain works, according to the evidence Kahneman presents. For learning to happen, information first must get past System 1 of our brains. This is where fast thinking happens. System 1 is the harried gate keeper, madly processing all of the information that comes in through our senses, pitching most of it, keeping only what is deemed necessary. But making it through the gate is only half the battle. Once in, information confronts System 2. This is the part of the brain that allows for conscious thought or slow thinking. The problem is that System 2 is lazy. Conscious thought is hard, and System 2 is always looking for an excuse to avoid it. However, if System 2 engages with information, the resulting knowledge can find its way to long-term memory and learning happens.

So the challenge when presenting data is to make it past System 1 AND engage System 2. Let’s consider a series of vizes from Harvard Business Review (HBR) that I think meets both parts of this challenge. Yes, it’s an example from the for-profit world, but could easily work with nonprofit data. See snapshot of the first viz in the series below .

How to get data past System 1

Getting data past the System 1 fast-thinking gate keeper is all about grabbing attention. We process images much more quickly than words and numbers, so images are a great foot-in-the-door. The HBR viz does it with bright colors and a cool-looking, somewhat unusual chart. There’s plenty of information out there about how to attract attention, including the use of images with:

  • Stand out colors and textures

  • Human faces (we are wired to focus on them)

  • Novelty (images that are unusual in size, placement, etc.)

Data visualizations can use color as well as images to draw attention. But getting past System 1 is not nearly enough. For learning to happen, the viz also has to engage System 2.

How to engage System 2

System 2 is smart but lazy. So we need to pique its interest. The HBR viz starts with a title that poses a question. When confronted with an interesting question, we may be more likely to stick around for an answer. Then the viz leads you through the answer in a visually engaging way (see interactive version of the viz HERE). These are two great ways to slow down and engage the brain with data. Here’s a list of ways to engage System 2:

  • Ask a question in the title as the HBR viz does—questions beg answers.

  • Make it personal. We may be more likely to engage with data when we have a personal connection with it. This New York Times viz, for example, allows you to enter in your county to see what the barriers to COVID vaccination are in your area.

  • Highlight a surprising finding. Many of us love the counterintuitive and the creative. If you draw attention to something new that the data suggests, you may have a better chance at hooking System 2. For example, this viz from The Economist shows that China emits far less greenhouse gas per person than Western countries at the same stage of economic development. Or check out this viz by Dimiter Toshkov showing that small countries can be big players in development and good governance.

  • Hand draw it. There is some evidence that making information harder to consume, for example by presenting it with harder-to-read fonts, makes the brain slow down and engage in effortful and analytic processing. Although the jury is still out on this, I do find myself more likely to engage in hand-drawn vizes like two of the winners of the World Data Visualization Prize in 2019. Perhaps it’s simply the novelty of hand-drawn charts that engages me. Anyway, it’s something you might consider, and all you need is a pen and paper.

  • Walk them through it. A great way to slow down your viewers is to set the pace by walking them through the data as HBR does in the example. I love how HBR presents what the data might look like if our assumptions were confirmed followed by what it actually looks like.

Sources: Veritasium, Visual Content Space, MIT News, Springer Link,

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Ideas You Should Steal From This Viz (Installment 11)

“Every artist gets asked the question: ‘Where do you get your ideas?’ The honest artist answers, ‘I steal them.’ . . . What a good artist understands is that nothing comes from nowhere. All creative work builds on what came before.” —Austin Kleon in Steal Like An Artist.

Today I offer up another steal-worthy interactive viz that I came across in the Tableau Public Gallery.

Source: Kizley Benedict on Tableau Public

Here’s what I suggest you steal from this viz:

  • Beeswarm Chart. The beeswarm chart at the top allows you to easily compare several countries and to see the overall distribution along the Gender Inequality Index among large and small countries. For more on beeswarm charts, see this tip.

  • Highlight a Country. For users who want to know about a particular country, the dashboard provides a search tool which highlights the selected country.

  • Overall Then Zoom In. After getting a sense of the overall distribution from the beeswarm chart, the user can zoom in and make comparisons among and within regions with the maps along the bottom of the dashboard.

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Don't Measure Impact . . . Wait, What?

Reposted from September 2022

Ghost of Christmas Yet To Come in The Chrismas Carol by Charles Dickens, Illustration by J. Leech, Source: Flickr

Most organizations should not waste time and money on impact evaluations. Measuring impact is difficult and expensive. It’s difficult because you need a good counterfactual. A counterfactual is what Dickens’s Ghost of Christmas Yet to Come shows Ebenezer Scrooge: what would happen if you did not change anything. The impact of an intervention or program is the difference between what happened and what would have happened without the intervention. Since, in the real world, you can’t observe the same group of beneficiaries with and without the intervention (as we do when we watch The Christmas Carol), you need a good proxy for the would-have-been condition. The best proxy is a group of potential beneficiaries that were randomly selected from a larger group of potential beneficiaries. These folks do not get the intervention. Then you can compare those who did and did not receive the intervention over time to estimate the impact of the intervention. This is called a randomized control trial or RCT.

Of course, withholding an intervention from potential beneficiaries can be a difficult and morally-questionable pursuit. And tracking a large group of beneficiaries and non-beneficiaries over time is expensive. This usually requires a team of skilled data collectors and analysts. Non-randomly-selected comparison groups are not nearly as good because they may differ from the intervention group in known or unknown ways. So it’s difficult to determine if the outcomes observed are due to the intervention itself or to pre-existing biases or characteristics. This costly and challenging process is further complicated by the need to start with a well-established intervention, one that has already worked out the kinks.

Due to the many challenges of measuring impact, most organizations should not waste time and money on impact evaluations. Instead, they should consider interventions that already have a strong research base, ideally because they have been rigorously tested with RCTs. (Check out: Where to Search for Evidence of Effective Programs.)

In a Stanford Social Innovation Review article, Mary Kay Gugerty and Dean Karlan suggest that, before beginning a new program, organizations ask: “What do other evaluations say about it? How applicable is the context under which those studies were done, and how similar is the intervention? Study the literature to see if there is anything that suggests your approach might be effective.”

Rather than assessing impact, your limited resources are better spent assessing implementation. You can do this by collecting data that shows whether what you planned is actually happening. If you can pinpoint where the problems are, you are in a better position to make fixes, alter plans, refine processes.  Many organizations make their plans using a logic model (aka theory of change). A logic model is a flow chart with inputs and outputs. The best logic models draw on past impact evaluations to determine what inputs are most likely to lead to what outputs. And organizations can easily assess progress to date by plugging their logic models into real time data. Interested? Read more about “living logic models” HERE.

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

It's Love Data Week!

What’s that?

Love Data Week is an international celebration of data, taking place every year during the week of Valentine's Day. Nonprofit organizations, universities, government agencies, corporations and individuals are encouraged to host and participate in data-related events and activities.

What’s in it for me?

Lots of free online events, many of which are relevant to nonprofit work such as workshops on data visualization, infographics, data resources, data privacy, etc. See a full list of events HERE.

Where can I learn more?

HERE.

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How and Why to Visualize Variability

Every dataset includes variability. The people and things we measure differ from one another in many ways. And visualizing data always involves some decisions about how much of that variability to show. There are tradeoffs:

  1. If you show too much variability, you obscure patterns and trends. To understand anything with data, we usually need to reduce its complexity. We can’t extract meaning from a table full of numbers and letters. So we summarize the data through such activities as grouping people, concepts, and time periods; calculating averages; or organizing individuals or groups in a rank order. This process allows us to detect patterns and trends within the data. Patterns and trends become even more apparent when we visualize the data in the form charts, maps, and graphs by assigning visual cues such as color, size, and shape to groups and values. However, too many colors, sizes, and shapes make discerning the patterns and trends more difficult.

  2. If you show too little variability, you obscure reality*. Overly simplified visualizations do not show just how complex and messy the data actually is. And the viewer may mistake the simplified, summarized version of the data as reality.

You can find a great example of problem #2 in Eli Holder’s article Divisive Dataviz: How Political Data Journalism Divides Our Democracy. He describes the danger of red and blue political maps in the U.S. in this way: “there’s no such thing as a “red state” or a “blue state.” Consider Texas, which is often called a “red” state. In the 2020 presidential election, more Texans voted for Joe Biden (5.26 million) than every other “blue” state, except for California. Even New York, a Democratic stronghold, had roughly 20,000 fewer Biden voters than Texas. . . . While popular election maps accurately reflect the ‘winner-take-all’ dynamic of the electoral college, they create the misimpression that state electorates are monolithic blocks of only-Republicans or only-Democrats.”

And the misimpressions such maps engender have real-world consequences. Holder describes an experiment in which people were shown either dichotomous maps or continuous maps (see examples below). Those shown dichotomous maps were more likely than those shown continuous maps to feel that their state was dominated by one party and thus that their votes mattered less because the election outcome was a foregone conclusion.

So when deciding how many shades of gray or circle sizes to show, consider how much summarization is needed to make patterns and trends perceptible without misleading the viewer with an oversimplified view of the data. Take, for example, these three versions of a map. They each show the same CDC chronic illness survey data with a “diverging color palette” in which blue states ranked high on health indexes; orange states ranked low; and gray states were in the middle. The maps differ in the degree of variability shown. Which map allows you to see corridors of good and bad health without oversimplifying the matter?

*More specifically, the full reality of the data. This 60-second data tip doesn’t get into the nature of reality in general!

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Balance Your Information Diet

Here’s a question for you. And don’t go Googling. Just make your best guess.

Have the number of people experiencing homelessness in the U.S. increased or decreased since 2007?

Whatever your answer, you likely drew on your own personal experience as well as images and information from the media when guessing at the answer. Perhaps you drew on some statistics too. But, unless you have expertise in this area, probably not. Stick with me for a minute, and I’ll not only provide an answer to the question but also some insight into how we consume information.

Personal experience, media, and statistics affect how we understand any issue, and there are limits to each of these inputs. So we would do well to understand those limits before acting on our understanding by voting, donating, or making decisions about programs that our organizations operate. Max Roser’s article in Our World in Data (The limits of our personal experience and the value of statistics) walks us through some of those limitations:

Personal Experience

“The world is large, and we can experience only very little of it personally,” Roser notes. “For every person you know, there are ten million people you do not know.” Even the most social and well-traveled among us can have only a limited understanding of the world through personal experience. I, for example, do not know anyone personally who has been unhoused, and most of my interactions with people in this situation occur on the street when someone asks me for money. This experience provides no information about the breadth of the problem or the range of experiences with this issue over time.

Media

“This fact is so obvious that it is easy to miss how important it is: everything you hear about anyone who is more than a few dozen meters away, you know through some form of media,” Roser points out. “The news reports on the unusual things that happen on a particular day, but the things that happen every day never get mentioned. This gives us a biased and incomplete picture of the world; we are inundated with detailed news on terrorism but hardly ever hear of everyday tragedies like the fact that 16,000 children die every single day.” If I recently heard a story about a city clearing homeless encampments, I may assess the problem as larger, and if I haven’t heard about anything on the issue in awhile, I may assess it as smaller.

Statistics

“The collection and production of good statistics is a major challenge,” writes Roser. “Data might be unrepresentative in some ways, it might be mismeasured, and some data might be missing entirely.” But, unlike personal experience and the media, it provides a way of assessing the full range of an issue. So it’s important to add statistics, along with personal experience and the media, to our information diet.

To add some statistics to your understanding of homelessness, the number of people experiencing homelessness in the U.S. decreased from about 650,000 in 2007 to about 580,000 (about 18 of every 10,000 people) in 2022 according to The 2022 Annual Homelessness Assessment Report to Congress.

We should not discount personal experience, the media, or statistics because of their limitations. But we should appreciate their limitations when forming opinions and taking actions based on them. As Roser notes: “Each way of learning about the world has its value. It’s about how we bring them together: the in-depth understanding that only personal interaction can give us, the focus on the powerful and unusual that the news offers, and the statistical view that gives us the opportunity to see everyone.” As described in many tips in this blog, well-designed charts make data/statistics more accessible to everyone and thus allow everyone to see everyone.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Best Data Viz of 2023

Looking for a fun (if somewhat geeky) study/work break? Check out these best-of lists for 2023:

New York Times

Visual Capitalist

538/ABC News

FlowingData

To see past data tips, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.