Recently, a client I’m building a data dashboard for asked me: “Our survey response rate was 30 percent. Isn’t that really good?”
I could see where she was coming from. Given how hard it is to get people to respond to surveys these days, 30 percent may feel strong—and it may be higher than what your organization typically sees.
But imagine she asked this instead: “I just bought a used car for $15,000. Isn’t that really good?” You’d probably say, “Well, it depends on the quality of the car.” If it’s a junker that can’t make it to the end of the block, it’s not a good deal.
Survey data is no different. A “good” response rate depends on the quality of the data. Rather than asking “Is my response rate good?” a more useful question is: “How representative is my survey data?” Your survey results might reflect what your clients, volunteers, or participants think and experience overall. On the other hand, they might mainly reflect the views of a small group of highly engaged, eager survey-takers.
To assess representativeness, compare your survey respondents to the larger group they’re meant to represent. Ask questions like:
Do respondents reflect the broader group’s demographics?
Are different programs, locations, or levels of participation represented?
Are newer participants responding at the same rate as long-time ones?
Even a high response rate can be misleading if certain subgroups are underrepresented.
If your respondents don’t look like the larger group, here’s what you can do:
1. Talk to staff and clients.
Gather insight on why certain groups may be less likely to respond. Barriers might include time constraints, language, survey length, digital access, or lack of trust in how data will be used.
2. Adjust how you collect data from underrepresented groups.
Based on what you learn, consider strategies to collect additional survey responses such as:
Offering the survey in multiple languages
Providing paper, text-based, or in-person survey options
Shortening the survey or breaking it into sections
Using trusted staff or community partners to encourage participation
Offering small incentives or emphasizing how feedback will be used
3. Be transparent about limitations.
Whenever you present survey results in data dashboards, presentations, your website, or social media, clarify who responded and who didn’t. For example:
“These results reflect primarily long-term program participants; newer clients were underrepresented.”
4. Apply lessons to the next survey.
Use what you learned to improve both response rates and data quality next time.
