I Used Free ChatGPT To Analyze Survey Data. Here's What I Learned.

1. You collect this type of survey data.

Here’s a familiar scenario. You survey your clients, participants, donors, volunteers, etc. and you include some “Other, please specify” options or other “open-ended” questions to better understand respondents’ opinions, experiences, etc.

2. But you don’t know what to do with it.

You collect your survey data but don’t have the time and/or analytical skills to deal with this qualitative data.* Maybe you create one of those horrible word clouds or, even more likely, you just analyze the quantitative data and ignore the qualitative data.

If you had the time and know-how, you might have “coded” the data in order to analyze it. This involves assigning themes to each open-ended survey response in Excel (or the like) or perhaps using one of these free tools.

*Quantitative data is numerical, countable, or measurable. Qualitative data is interpretation-based, descriptive, and relating to language.

3. But what about AI?

You’ve heard that it’s supposed to make tedious, repetitive tasks much easier, and coding survey responses certainly qualifies as both. Could you use the free version of ChatGPT to get this job done? I shared your curiosity and gave it a try. Bottom line: It helped to identify themes to use as codes but it didn’t do all the work for me. For a little longer description of my experience, keep reading.

4. Prepare for AI.

I watched this YouTube Video based on this article to learn how to craft a prompt that would likely get me what I wanted. I also found free survey data on the City of Chicago Data Portal to use for my experiment. The survey asked 43rd Ward residents about “other priorities” for their ward. I thought I could just upload the CSV data file to Chat, but it turns out you need the paid version for that. So I ended up pasting in the survey answers after entering the prompt. Also note that I used publicly available data. You should think twice about entering any type of sensitive data into Chat.

5. Craft the prompt.

Here it is. Yes, it’s long and yes, I say “please,” although I’m not sure if that affected the results!

6. Here’s what happened.

I first tried pasting in the prompt plus the data but that was too long for Chat. So I had to feed the data (all 23 pages) in batches of about 3 pages at a time and despite entreaties to Chat to update the charts based on ALL of the data I shared so far, it only gave me charts for the last batch I had entered, and I had to combine them in Excel. At first I was impressed with the almost instant tables, but I felt my AI assistant wasn’t quite listening to my instructions or just not understanding them. Still, I did develop this list of themes and Chat did code each survey response according to these themes, but I would not feel comfortable relying on these results and would want to possibly combine some of these themes and read through all the responses to see if I agree with the coding.

To see past data tips, including those about other chart types, scroll down or click HERE.

---

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

---

Without Data Viz, You Can Get It All Wrong

The aim of today’s tip is to remind you of the importance of visualizing data. Without charts, maps, and graphs, we can get it all wrong. We base our understanding on a few stories in the media, on the experience of someone we know, or on what we hear most often about the topic rather than on what is actually happening. More on this in a moment. First, please take this short pop quiz and then scroll down.

“Stories about individual people are much more engaging – our minds like these stories – but they cannot be representative for how the world has changed,” writes Max Roser. “To achieve a representation of how the world has changed at large you have to tell many, many stories all at once. . . .“

Roser made the series of charts below to tell all of those stories in a way that we can understand. It shows the number of people out of 100 with various experiences over the course of 200 years. It’s worth checking out Roser’s whole article entitled “The short history of global living conditions and why it matters that we know it.”

I’ll leave you with one more quote from Roser: “The result of a media – and education system – that fails to present quantitative information on long-run developments is that the huge majority of people is very ignorant about global development and has little hope that progress against serious problems is even possible. Even the decline of global extreme poverty – by any standard one of the most important developments in our lifetime – is only known by a small fraction of the population of the UK (10%) or the US (5%). “

Source: Our World in Data

---

Let’s talk about YOUR data!

Got the feeling that you and your colleagues would use your data more effectively if you could see it better? Data Viz for Nonprofits (DVN) can help you get the ball rolling with an interactive data dashboard and beautiful charts, maps, and graphs for your next presentation, report, proposal, or webpage. Through a short-term consultation, we can help you to clarify the questions you want to answer and goals you want to track. DVN then visualizes your data to address those questions and track those goals.

---

What Data Do You Absorb (And What Eludes You)?

There is a certain breed of nonprofit staff who roll their eyes at the mention of “evidence-based practices” or “KPIs” or other data jargon. I myself have experienced mild nausea when listening to someone try to quantify what seems unquantifiable: what a child feels after learning to paint or what a homeless adult feels upon acquiring an apartment.

But I’m here to implore, beseech, even beg you to not write off data. Why? Because of how our brains work.

What we perceive is based not just on what we actually observe but also on what we expect to observe. This is how it works. First, the brain evaluates which of a variety of probable events are actually occurring. Then it uses this information, along with signals from the outside world (aka data), to decide what it is perceiving.  And here's the surprising news: there are far more signals coming from within the brain that affect our perception than data signals from the outside.

These inner brain signals or expectations can distort our understanding of a situation. Thus data are quite important to confirming or negating our expectations.  But you have to pay attention to data to make that happen. And that’s where things get tricky.

The esteemed philosopher and psychologist, William James, noted in The Principles of Psychology (1890): “Millions of items of the outward order are present to my senses which never properly enter into my experience. Why? Because they have no interest for me. My experience is what I agree to attend to. Only those items which I notice shape my mind .” 

So how can we see what we are not attending to, the stuff that eludes us? The answer is to think like a scientist. Rather than operating on assumption or instinct, form hypotheses about how your programs and services work and then gather data to test them. You might be surprised.

(And, if you missed it, check out last week's data tip on how data visualization can help correct misperceptions.)

See other data tips in this series for more information on how to effectively visualize and make good use of your organization's data.

Can Data Viz Unite Us?

Progress in organizations — and in all of human history — starts with the concession that we might be wrong. As Yuval Noah Harari suggests in Sapiens: A Brief History of Humankind, the scientific revolution was the point in history when “humankind admits its ignorance and (as a result) begins to acquire unprecedented power.” 

That’s what I said in Data Tip #23, and I’m sticking with it. However, it’s way easier to say than to practice. Particularly in these partisan times.

I recently soaked in a series of well-executed yet oh-so-depressing data vizes in this article from the Pew Research Center. You’ve seen these types of charts before. The liberals are a blue iceberg, the conservatives a red iceberg, and they are drifting apart at an alarming rate. “We” seem to be living in a different reality than “them.”

Shortly after reading the Pew article, I climbed out of my dark hole long enough to happen upon this article from the Washington Post. It describes a series of experiments in which data displayed in charts significantly reduced the misperceptions of subjects, both liberal and conservative, on important poltical issues. And, get this, charts (bar graphs, line graphs) had much more impact than the same information presented in text — perhaps because we process visual information much more efficiently than we process words and numbers.

Okay, it was just a few experiments. But still. Let’s not totally curb our enthusiasm. This is promising. This suggests that there MIGHT be a way to bring all of us back to a somewhat similar reality. And data visualization might help get us there.

In the meantime, when different factions of our boards do not agree or when we are looking for a way to convince a reticent funder to support our work, we should remember the power of a humble chart, map, or graph.

See other data tips in this series for more information on how to effectively visualize and make good use of your organization's data.

Photo by Kristina Litvjak on Unsplash

Go With The Flo (Florence Nightingale)

Like many nonprofit staff today, Florence Nightingale probably wasn’t a numbers person at the outset. She became a nurse to serve others. Yet, she soon realized she could provide care more effectively with the help of data. Working with a statistician named William Farr, Nightingale analyzed mortality rates during the Crimean War. She and Farr discovered that most of the soldiers who died in the conflict perished not in combat but as a result of “preventable diseases” caused by bad hygiene.

Nightingale’s solution? She invented the polar area chart, a variant of the pie chart, meant “to affect thro’ the Eyes what we fail to convey to the public through their word-proof ears.” Each pie represented a twelve-month period of the war, with each slice showing the number of deaths per month, growing outward if the number increased, and color-coded to show the causes of death (blue: preventable, red: wounds, black: other). Clearly seeing the importance of hygiene, the Queen and Parliament quickly set up a sanitary commission and, as a result, mortality rates fell.

See other data tips in this series for more information on how to effectively visualize and make good use of your organization's data. 

Also, see the Smithsonian's excellent article The Surprising History of the Infographic for more on the history of data visualization.

Image Source: Smithsonian

(This data tip originally appeared on Philanthropy News Digest’s PHILANTOPIC blog.)

Make Your Data Visible

Nonprofits are lousy with data. But, like secretive hoarders, we are reluctant to admit how little data we actually use. We may pay lip service to “evidence-based practices” or “data-driven strategies”. But, if pressed, many of us admit that we care about the people and the programs and glaze over at the site of a spreadsheet.

Indeed, we are not wired well for processing data in spreadsheets.

Our visual system has evolved, over millions of years, to process images essentially in parallel. We don’t read the Mona Lisa from top to bottom and from left to right. We take it all in together and understand, almost instantly, that this is a picture of a woman in front of a landscape, sporting a dark dress and an inscrutable smile. Words and numbers, which only appeared within the last few thousand years, require our visual system to scan individual characters one at a time and piece them together to create meaning.

Data is encoded in words and numbers making it difficult for us to extract the stories they can tell. However, if we use visual elements (like bars, pie slices, and sloping lines) to encode the data, the story can come into focus more quickly. 

See other data tips in this series for more information on how to effectively visualize and make good use of your organization's data.

Image Sources: thinglink.com and perfect-cleaning.info

(This data tip originally appeared on Philanthropy News Digest’s PHILANTOPIC blog.)