Reposted from March 2022
Vanity metrics are like cheap, trendy sunglasses. They may help you to look cool, briefly, but they don’t last long and do little to improve your eyesight. You’ve seen vanity metrics, even if you haven’t used this term to describe them. They are those flashy statistics (sometimes called “big ass numbers”) and charts showing how many services an organization has provided or people they’ve served or some other seemingly impressive stat. The problem is that these metrics don’t help you to better understand your current work and improve it. In this tip, I’ll give you some quick advice on recognizing and reforming vanity metrics.
How To Recognize A Vanity Metric
This Tableau article suggests three questions to ask to identify a vanity metric. I’ve put a nonprofit spin on each:
What decision can we make with the metric? If the metric can’t help you to make a decision, it’s probably a vanity metric. For example, does knowing how many meals you delivered help you to decide who, what, where, when, or how to deliver meals in the future? Or do you need a more specific metric such as the gap between need and service provision for various subgroups of clients?
What can we do to intentionally reproduce the result? Did some random event produce the big number? For example, did you see a bump in the number of participants last year because another organization, providing a similar service, closed down? If you cannot consistently reproduce the same result next year, this isn’t a helpful metric.
Is the data a real reflection of the truth? Let’s face it. There are always ways to misrepresent the truth. You can tell the world that attendance at all of your programs last year totaled 3,237. Sounds good, but that’s probably a “duplicated number” and can be misleading depending on what you want to understand or broadcast to the world. Some people likely attended more than one program. So the total number of individuals who participated in any program could be much lower. The central question to ask yourself when considering a metric is whether or not it will help your organization achieve its goals. If your goal is to reach more folks, this metric is not helpful.
How To Reform A Vanity Metric
Provide context. The metrics that are worthy of your attention and your stakeholders’ attention are those that are directly related to your goals. You may have overall goals for all of your participants, clients, audiences, services, programs, etc. But you also might have specific goals for subsets of those groups and for specific time periods. Present your metrics in relation to the goals. And compare metrics for subgroups to each other to see where you are making progress and where you are not.
Use more than one statistic. Sometimes what you want to improve cannot be measured with just one metric. For example, if you aim to improve the diversity of your staff, you may want to look at a set of metrics together including number, tenure, and seniority of staff by race/ethnicity, gender identification, age, etc.
Sources: Moving Beyond Vanity Metrics, Stanford Social Innovation Review and Vanity Metrics: Definition, How To Identify Them, And Examples, Tableau.
Let’s talk about YOUR data!
Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.
