How To Recognize and Reform Vanity Metrics

Vanity metrics are like cheap, trendy sunglasses. They may help you to look cool, briefly, but they don’t last long and do little to improve your eyesight. You’ve seen vanity metrics, even if you haven’t used this term to describe them. They are those flashy statistics (sometimes called “big ass numbers”) and charts showing how many services an organization has provided or people they’ve served or some other seemingly impressive stat. The problem is that these metrics don’t help you to better understand your current work and improve it. In this tip, I’ll give you some quick advice on recognizing and reforming vanity metrics.

How To Recognize A Vanity Metric

This Tableau article suggests three questions to ask to identify a vanity metric. I’ve put a nonprofit spin on each:

  • What decision can we make with the metric? If the metric can’t help you to make a decision, it’s probably a vanity metric. For example, does knowing how many meals you delivered help you to decide who, what, where, when, or how to deliver meals in the future? Or do you need a more specific metric such as the gap between need and service provision for various subgroups of clients?

  • What can we do to intentionally reproduce the result? Did some random event produce the big number? For example, did you see a bump in the number of participants last year because another organization, providing a similar service, closed down? If you cannot consistently reproduce the same result next year, this isn’t a helpful metric.

  • Is the data a real reflection of the truth? Let’s face it. There are always ways to misrepresent the truth. You can tell the world that attendance at all of your programs last year totaled 3,237. Sounds good, but that’s probably a “duplicated number” and can be misleading depending on what you want to understand or broadcast to the world. Some people likely attended more than one program. So the total number of individuals who participated in any program could be much lower. The central question to ask yourself when considering a metric is whether or not it will help your organization achieve its goals. If your goal is to reach more folks, this metric is not helpful.

How To Reform A Vanity Metric

  • Provide context. The metrics that are worthy of your attention and your stakeholders’ attention are those that are directly related to your goals. You may have overall goals for all of your participants, clients, audiences, services, programs, etc. But you also might have specific goals for subsets of those groups and for specific time periods. Present your metrics in relation to the goals. And compare metrics for subgroups to each other to see where you are making progress and where you are not.

  • Use more than one statistic. Sometimes what you want to improve cannot be measured with just one metric. For example, if you aim to improve the diversity of your staff, you may want to look at a set of metrics together including number, tenure, and seniority of staff by race/ethnicity, gender identification, age, etc.

Sources: Moving Beyond Vanity Metrics, Stanford Social Innovation Review and Vanity Metrics: Definition, How To Identify Them, And Examples, Tableau.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Why You Should Know About Span Charts

Tired of bar and pie charts but not sure what your other options may be? Meet the simple and friendly span chart.

This is a new addition to a series of tips on different chart types. In each tip, l give you need-to-know information in a format akin to the “Drug Facts” on the back of medication boxes: active ingredients (what the chart is), uses (when to use it), and warnings (what to look out for when creating the chart). The idea is to fill up your toolbox with a variety of tools for making sense of data. And the span chart is a simple tool you can put to good use!

Active Ingredients (What is a span chart?)

A span chart shows the range between a minimum value and a maximum value. Check out the example below which displays the salaries of full-time employees of the City of Chicago in various departments. The chart on top shows the range between the lowest and highest salaries in each department, and when you click on a bar, the chart below shows the salaries of individual employees in that department (which you can see by scrolling over the circles). So we can see, for example, that although the public library department has a wide range of salaries, the large majority of employees earn less than $100K per year.

Uses

Span charts provide the extreme values. So if you want your viewer to appreciate the range of values and compare the range of different subgroups, as the example above does, it can be quite effective. In addition to salary ranges, a nonprofit organization might use a span chart to show the range between:

  1. The largest and smallest donation amounts per person by year or by subgroup.

  2. The highest and lowest grade point average of students in a tutoring program by semester or by subgroup.

  3. The most and least days of participation among adults in a job training program by month or by subgroup.

Here are instructions for creating a span chart with Tableau and Excel.

Warnings

Span charts do not show the values in between the minimum and maximum or the average value. So you have no sense of the distribution of data points. Are the values evenly distributed or are most at the high or low end? If understanding the distribution is important, you can pair a span chart with a chart that provides more information on the in-between values, as the example above does. Other chart types which show distribution include: histograms, scatter charts, and box plots.

Fun Fact

Span charts go by a variety of names including range bar/column graph, floating bar graph, difference graph, and high-low graph.

To see past data tips, including those about other chart types, scroll down or click HERE.

Sources: The Data Visualisation Catalogue

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Make Big Numbers Tangible

We’ve talked about the problem with big numbers before. Most recently, we considered the difficulty humans have digesting large numbers and how “perspectives” — simple sentences that relate a large number to something more familiar to us — can help us to understand, assess, and recall numbers. (For more on this, check out the data tip.)

I’m returning to the big number problem today and offering up some new tips for dealing with them. The inspiration for these tips came from the data-driven documentaries of Neil Halloran, specifically his first documentary called The Fallen of World War II. If you have a few more minutes to spare after reading this 60-second tip (and are not among the 13 million + who have viewed it already), I highly recommend that you check it out. It’s 18 minutes long, but the techniques listed below all appear in the first 7 minutes.

Halloran uses the following techniques to make larger numbers understandable. And you don’t need to be a filmmaker to use them. You can apply them to simple data presentations on websites, reports, and PowerPoints.

  1. Use shapes or icons (rather than bars) to represent one or more people, programs, etc. Halloran uses a human figure shape to represent 1,000 people.

  2. Show an aggregate and then break it down by subgroups and time periods. Halloran shows aggregates, such as the total number of U.S. soldiers who died and then, using animation, redistributes the human figures to show how many soldiers died in the European and Pacific theaters and then how many died over time. The animation is cool but not necessary. You can do the same thing with a series of static images. See example below.

  3. Juxtapose photos and charts. To keep the discussion from becoming too abstract, Halloran reminds the audience what actual soldiers (rather than icons) look like by incorporating photos into his presentation. Again, animation is not necessary. Static photos can be placed along side charts.

  4. Walk audience through the data. To give the audience a sense of scale, the video progresses from smaller to larger numbers. Halloran first walks us through casualty stats for the U.S. and European countries. These numbers seem quite high so by the time Russian stats are shown, we are blown away.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Improve Your Organization’s Time Line

Here’s a simple data viz idea. Next time you make a time line showing your organization’s milestones, size those milestone markers (usually circles) according to some key measure. Voila! You are not only showing what happened but also your progress along the way.

The data for such a viz is super simple. Something like this:

Screen Shot 2021-08-24 at 9.01.48 AM.png

I connected the data shown above to Tableau Public (the free version of Tableau) to create the time line below. Vertical time lines not only suggest an upward progression but also work better on phone screens.

Dashboard 1 (3) (1).png

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Make Data Dashboards More Comfortable

I’m always looking for ways to make data more comfortable, particularly for non-data people. Sometimes that means creating simple charts that everyone can understand. Sometimes that means limiting the number of charts presented. And today, I offer up another tip: show your data in a familiar environment. For many of us, paper, file folders, paper clips, and post-it notes are familiar and, dare I say, even comforting.

Give this old-school-meets-new-school data dashboard a spin and see what you think. It shows real-time data in an interactive format. And if you’d like some help creating a similar dashboard for your organization, feel free to contact me.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Create More Diverse, Equitable, and Inclusive Data Visualizations

When we visualize information, we make a series of decisions which affect the way that viewers process the information in our charts, maps, and graphs. Sometimes they don’t feel like decisions at all. We go with the default settings in the application we are using. Or we just do something the way it’s usually done. But a more diverse, equitable, and inclusive approach to presenting and visualizing data requires us to make those decisions more consciously and deliberately. Jonathan Schwabish and Alice Feng of the Urban Institute provide some helpful tips, based on the Urban Institute’s own style guide, which you can apply the next time you present data.

Here is a summarized version of Schwabish and Feng’s article.* And here is my 60-second version of their recommendations:

  • Use people-first language in titles, text, and labels associated with charts, maps, and graphs. For example, use “people with disabilities” rather than “disabled people.” Also Urban does not refer strictly to skin color. For example, they refer to “Black people” not “Blacks.”

  • Order and present groups purposefully. The first group shown in a table or the first bar in a graph can affect how readers perceive the relationship or hierarchy among groups. For example, if the first group is “Men,” then it may appear that men are the default group against which other groups should be compared. One way to prevent viewers from making certain comparisons is to display groups in side-by-side charts (aka “small multiples” charts) rather than on a single chart. In general, make ordering and grouping decisions to promote certain comparisons and prevent others.

  • Point to missing groups. If certain groups are missing from the data, explain why in text boxes or footnotes. Also add information on groups included in “Other” categories and consider providing a more specific label than “Other” which can have an exclusionary connotation.

  • Do not use color palettes that reinforce gender or racial stereotypes. This one may seem obvious, but it bears repeating. Also, Urban’s color palette is accessible to people with certain color vision deficiencies, and the contrast between those colors and white and black text meet basic accessibility guidelines.

  • Depict a variety of races and genders when using icons and avoid icons that make inappropriate depictions of people or communities or reinforce stereotypes such as showing traditionally feminine icons to depict nurses or traditionally masculine icons to depict bosses.

  • Find ways to show the people behind the data. Data visualizations are, by definition, abstractions of larger realities. But in the process of abstracting, we may obscure the lived experiences of the real people whom the data represent. Visualizations can remind viewers about the individuals behind the data by, for example, depicting them as individual circles rather than aggregating them in a single bar.

* The full paper has been published as an OSF Preprint and can be accessed here.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How To Present Diversity Data (or What To Steal From This Diversity Scorecard)

Today’s tip is to take inspiration from Chantilly Jaggernauth’s excellent diversity scoreboard displayed below. It shows diversity among employees in a company but can easily be applied to staff or participants in a nonprofit organization.

I suggest you steal the following ideas from Chantilly:

  • Metric Definitions. In a Tableau Conference session, Chantilly shares the pros and cons of the four metrics in the dashboard. See image of the slide below. None of the metrics are perfect. But together they provide an understanding of where an organization is in its diversity efforts. These definitions are not incorporated in the dashboard itself but could be added through a link or in a tooltip (scroll over) feature.*

  • Views of Diversity. The dashboard provides three views of diversity: overall, gender, and people of color (POC). By providing side-by-side charts with these three views, the dashboard allows users to see variations that overall diversity charts obscure.

  • Color Coding. Each type of diversity has its own color, which makes the comparison among overall, gender, and POC easy, even when you scroll down and can no longer see the column headers. Also the comparison groups (non-diversity, male, and non-POC) are represented by the same colors in lighter shades. This approach makes the dashboard easier to understand. Assigning three additional colors for the comparison groups could be confusing and require a color legend.

  • Simple Charts. These are all charts we all know how to read. So the scorecard is accessible immediately to anyone, even if they are not familiar with the data or the organization.

  • Also, note that the dashboard and the slide use different terms for two of the metrics.

Source: HR Diversity Scorecard on Tableau Public by Lovelytics

Image above from Tableau Conference session called “Next Gen Analytics for Your New Normal” on 11/10/21.


Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People: The Cheat Sheet

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #10)

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #9)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #8)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #7)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #6)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #5)

Source: Tuomi, Ilkka (2000). "Data is more than knowledge". Journal of Management Information Systems. 6 (3): 103–117. doi:10.1080/07421222.1999.11518258.

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #4)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

How Data Viz Can Save Your Thanksgiving

Your next data challenge may involve turkey. And I’m here to help. This week we take a break from nonprofit data and consider Thanksgiving data.

Thanksgiving involves many more dishes than you would normally serve in one meal. If you are in charge this year, and you have a medium to small oven and fridge, you have to be strategic. When should you cook, chill, and reheat each dish to make the most of your time and limited oven/fridge space?

I give you my Thanksgiving Game Plan Gantt Chart (partially pictured below) originally shared in tip #87. It has become a Thanksgiving tradition here at Data Viz for Nonprofits, and works like a charm. I made it in Google Sheets. Nothing fancy, but it does the trick. Feel free to adapt it to your recipes or perhaps your next fundraising event!

Happy Turkey/Tofurky Day. And stay tuned. Next week we will return to the series of data tip comic strips on essential data facts for non-data people.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

Featured
044-CcCS-bIy_ss.jpg
Hi, I'm Amelia.

10 Essential Data Facts For Non-Data People (Fact #3)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #2)

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

10 Essential Data Facts For Non-Data People (Fact #1)

Sources: OECD Glossary of Statistical Terms. OECD. 2008. p. 119. ISBN 978-92-64-025561. and Tuomi, Ilkka (2000). "Data is more than knowledge". Journal of Management Information Systems. 6 (3): 103–117. doi:10.1080/07421222.1999.11518258.

Sources: OECD Glossary of Statistical Terms. OECD. 2008. p. 119. ISBN 978-92-64-025561. and Tuomi, Ilkka (2000). "Data is more than knowledge". Journal of Management Information Systems. 6 (3): 103–117. doi:10.1080/07421222.1999.11518258.

I’m posting comic strips explaining ten essential data facts for non-data people over time. Check out other data facts which have been posted so far by scrolling down or clicking HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

When Is Nothing Something You Should Show?

Reposted from July 13, 2020

Reposted from July 13, 2020

Data visualization is all about making what’s invisible (or not-immediately-perceptible) in the outside world visible and clear. You can’t see the range and strength of your clients’ feelings about the programs you offer, but you can see them on a chart. You can’t see air pollution across thousands of miles, but you can see it on a map. In both of these examples, data visualizations show the presence of something. Can they also show the absence of something? And when is nothing something you really should show?

This cartoon from the start of the pandemic makes the absence of something — COVID cases averted by individual actions — perceptible with gray lines and dots. It got me thinking about how we visualize the absence of things in charts, maps, and graphs.

See animated version of the cartoon here.

A common problem in almost any endeavor involving data is “missing data.” This is data that was not collected because, for example, a respondent skipped a survey question or someone did not fill in a data field in a database. And often missing data is eliminated from charts, maps, and graphs. We don’t show what we don’t know. But that can be a mistake, especially when the majority of data is missing such as in this pie chart. By showing the amount of unknown and missing values, it emphasizes the need for better data collection so that we can understand which groups are most affected.

source: WBUR

These maps (from March 2020) stress the absence of certain policies rather than the presence of them.

source: Politico

The colors gray and white often are used to signify the absence of something. But this chart uses green to draw your attention to the times when NONE of Britain’s power was generated by coal, presumably helping to make for a greener environment.

source: The Guardian

Zero points on axes also help to show the absence of something. This chart emphasizes the point on the X-axis representing no bias with a red line.

source: The Economist

Consider what absences may be instructive to your staff, board members, funders, clients, or participants. Perhaps it’s the absence of data or the absence of revenue or the absence of problems following an intervention or the absence of essential services in a community. Remember showing nothing can be just as enlightening as showing something.

To see past data tips, including those about other chart types, click HERE.

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.