Creating surveys people will actually answer
As I looked out the window, a handful of stars was already visible up in the dark blue horizon of the late afternoon sky. My back was craving for a stretch after the long sequence of online meetings I had had — along with four friends, I was interviewing leaders of popular culture groups as part of a project that would later become a fascinating app. Just when I was about to sign off, a friend of mine told me: “I really liked the way you conducted the interviews”. At first, it only seemed like a nice compliment, but that was my motivation for a profound reflection on how we collect information.
You know it just as much as I do: most people hate answering surveys. Whether it’s work-related feedback, app reviews, or experience assessments, the process of providing information is often stressful and excruciating. However, that should be no surprise: considering that the user experience of the respondents is hardly ever thought about when designing evaluations, how could it be different?
The consequence is a valley of extremes: people either collect data by causing their users distress or simply don’t collect it. And in the social impact sector, the situation is critical: a surprising 80% of social businesses in Brazil don’t measure their impact at all (Pipe Social). The 20% that is left, on the other hand, often do it at the expense of their beneficiaries’ patience.
The problem with that? Besides being terribly inefficient, the way we currently collect data may also be damaging to our own success. Many of the users who could provide us precious feedback don’t even take our surveys (and the ones who do often answer them as fast as they can just to get it over with). That’s a disastrous way of missing valuable insights — and for social initiatives, of putting themselves closer to the cliff of failure. Having that said, the question arises:
How to create meaningful surveys?
The essence of designing a good survey relies on the experience of collecting data itself. People who do it often forget about the human perspective of how that process feels. It is precisely the humanity of our survey that makes all the difference in the results we obtain.
If you’re doing scientific research, some of the following ideas may not fit your needs. If you’re crafting surveys for any other reason, however, like creating apps or evaluating social impact projects, I’ve got good news! Here are a few tips on how to make them better.
1. Make it decision-driven
The data you can get is different from the data you need. As if by impulse, people usually go for the first instead of the latter. That results in terribly long surveys, and most of the data collected ends up having no actual use. How can you make sure your survey goes straight to the point?
After taking Acumen’s Lean Data course, I started following their piece of advice: make your survey decision-driven. Reflect about what you want to validate, and pre-articulate the decisions you’re going to make depending on the results you will get. This helps you keep your questions relevant and actionable, so always think: what changes will you implement depending on the information you are getting with this question? If the answer is “none”, cut that one out.
You may remember a project I mentioned in this article that was created to connect public school students to volunteer mentors through WhatsApp. After a while, the number of active students decreased, and the need to make changes in the methodology became clear. Had they decided to create a survey, they could’ve asked, for example, how easy it was for students to access WhatsApp, pre-articulating the decision to stay on the platform or move to a new one. Another possibility was asking what motivated students to enrol, pre-articulating the decisions of seeking more qualified mentors or starting to record video classes instead. Once the results were ready, the entire team would have known what to do beforehand and would have implemented changes more effectively.
I know that gathering all the data you’re capable of is tempting, but for the sake of your users, please don’t.
2. Divide and conquer
Once, after participating in a hackathon, I was asked to fill a survey about my experience during the event. There were questions about the platform we used, the mentors that helped us, the case provided by the sponsor company… In short, there were questions about everything, and that had two immediate consequences:
- a 14-pages-long survey and;
- a feeling of exhaustion before I had even filled half of it.
I wasn’t the only one feeling like that, and as a result, most people probably closed the survey and never looked at it again. The organization that promoted the hackathon missed a chance to acquire relevant insights simply because their survey scared users away. But what if they had split their questions among users instead of asking everything to everyone?
At first, such a strategy might feel counterintuitive, as it gives the impression of having fewer results in the end to analyze. But in fact, what might happen is the exact opposite: since you’re asking less, answering won’t feel like a burden, and more users might get to the end of your survey.
Of course, this doesn’t apply to all surveys. But in case yours is aimed at evaluating a series of different aspects, try splitting your questions into groups and then asking users to answer one set of questions only. Besides avoiding user frustration, you can acquire more (and better) answers in the end.
3. Spark curiosity
People tend to say love is a powerful feeling, but curiosity has a remarkable motivational effect which is usually unheard of. You’ve probably done lots of things just because you were curious, and your users can happily go through your entire survey for the same reason as well. That is a simple but compelling effect. Besides, a bonus of sparking curiosity is the good mood that comes along: When people answer questions because they are curious, they tend to express their feelings more, which translates into better-quality answers and results.
One way of sparking curiosity I personally like is to say, at the beginning of a survey, that the user is going to take a specific amount of time to complete it. But by saying “specific,” I mean it. I remember a small experiment when, after not getting many answers on a survey, a group of friends and I added the following phrase at the beginning: this survey is going to take you 2 minutes and 37 seconds. Indeed, it took most users around two and three minutes (thus, we didn’t cause them any frustration), but the “37 seconds” actually made the number of answers we were getting increase.
4. Make your target audience feel comfortable
Thinking about the environment in which your users will see themselves while answering your survey may positively influence its success. If you’re interviewing people, will you simply approach them as they walk in the street, or will you do it in a place where there’s shade and comfort? At what time of the day will you speak to them, and what impact that might have on their schedule? If you’re doing a written survey, will it be on paper or online, and how convenient is the channel you’re choosing for your target audience? How pleasant is the layout and flow of the website that’s hosting your questionnaire for your users? Are they taking the survey after an exhausting activity or before a pleasant moment? If not well thought, these are factors that may decrease the quality of the answers you collect.
During my time at Enactus UFPE, I worked on recycling projects a couple of times and, coincidently, I had the chance of surveying the same people twice and compare the difference. At first, we did a standard interview, not considering our survey’s user experience at all. We had been working with a recycling cooperative for months and had to collect some critical data with them. The team and I arrived at the cooperative around 10:00 a.m., in the middle of their workday, armed with traditional questionnaires and formal inquiries. At first, nothing seemed wrong, as we collected all the information we needed.
One year later, I had the chance of conducting another interview with the same people. This time, however, everything was different. First, we chose to meet them on a Saturday afternoon, not to disturb their workday. Second, instead of starting the survey right away, we spent some time simply having a nice talk so that everyone (both us and them) would feel more comfortable. Our questions were phrased differently this time and asked naturally in the middle of the conversation too.
The difference between the interviews was crystal clear. Like before, we also got all the information we needed, but we were surprised by the innumerous insights we could spot this time. It turns out that, feeling more comfortable, respondents went further in their answers, not only answering our immediate questions but also sharing their perspectives about the subjects.
5. Make sure you’re speaking like a human
A vital component of the feeling your users get comes from the language you use. For some reason, surveys usually display a cold or formal language, which does not generate empathy, creates a dull atmosphere, and may even compromise understanding. The key here is to understand your target audience and communicate like you would in an actual conversation. Who is your survey targeting? What language do they speak? What kind of vocabulary do they use? How answering your questions is going to feel for them? Do your best to keep it simple, ethical, and enjoyable. Using plain language is an easy way of avoiding brain pain for your users!
When I’m conducting interviews, I do my best to make it a conversation, inserting the questions I have naturally throughout the talk. This means changing from a one-way data extraction mindset to a two-way data exchange one — which is how human relations actually are. Working with written surveys, there are similar tricks that may help: you can rephrase your questions as if you were in the flow of a conversation, for example.
Instead of asking questions such as:
- Please evaluate X.
- Which option describes you the most: A or B?
- Select the option that reflects your opinions more accurately: C or D?
Try rephrasing your questionnaire for something like:
- First, we’d like to ask you about X.
- Interesting to know! However, that brings up another question: do you think A or B?
- I see what you mean. And in this case, would you rather pick C or D?
Don’t act like if you were a robot. Just like the person you’re interviewing, you are also a human being, so speak like one!
One last thing.
Don’t think this means you won’t have to keep it rigorous: although we want users to enjoy the experience, this doesn’t mean our professionalism while creating the survey is any lower. Avoiding bias, for example, like questions that may lead to a specific kind of answer, is always important. Crafting specific questions, instead of broad ones, likewise.
Considering the trauma caused by poorly designed surveys, it is no surprise that fewer people are willing to answer them. To change that, we must put ourselves in the shoes of our users, considering the human perspective that has always existed in the millennial process of asking questions around.
