Good research starts with good questions

Researchers always struggle when it comes to writing down the questions they need to ask their participants. Sure, this gets easier over time and with experience, but the act of writing an interview guide or test plan never gets “easy.” At the end of the day, we are all human and we are susceptible to our own weaknesses and limitations.

The deck is stacked against us when you start to consider social, personal, professional, and sometimes logistical factors that can inhibit our ability to have a conversation with someone else. Predicting all these factors before research even starts is no small feat. This in turn makes writing down lines of inquiry that will result in good, useful data seem daunting. But you have to start somewhere and iterate as you learn what questions work and which fall flat. To help you with this, first we need to discuss what role questions fulfill when you’re conducting any type of research.

The role of questions in research

It’s hard to conduct research when you don’t know what question needs to be answered. Every research effort starts with you needing to know why something happens, what people do in certain circumstances, and how they perform key tasks. To answer these questions, we must find people to talk to and phrase our questions effectively to get to the heart of the matter. Otherwise, we would be making wild guesses and shooting in the dark. While that’s often tempting, this degree of freedom leads to failure and your product never seeing the light of day.

How good questions go wrong

We can’t tell you how many times we’ve written down a question and thought, “This is it! This will get us some awesome information from people,” only to have it fall flat during a session. This happens to all researchers and it will happen to you. And that’s OK! Bad questions can be mitigated through the planning phase if you know what makes a question go bad. The following factors can lead to misinformed or poor research results.

Leading questions

It’s easy to get caught up in the excitement of research. This can trick you into asking questions that give participants a clue, or directly point them, to the type of answer you’re looking for. These are called leading questions, and they can hinder your research session and the data collected. An example of a leading question would be asking, “How do you use Outlook to communicate your work status?” A better alternative would be “How do you communicate your work status?” The second question allows more responses than leading the participant to describe a specific use of email. Research participants want to be helpful and want to provide value to your team. Since they are primed to help, if you ask a question that implies the type of answer you want, they are more likely to give you that answer, even if it doesn’t really apply to them.

Shallow questions

One golden rule of research is never ask yes/no questions. When creating questions for an upcoming research effort, you’ll find avoiding these questions is hard. Yes/no questions are harmful because they give participants an easy out. The question “Do you use Yammer for team discussions?” can quickly be answered and dismissed. Participants don’t have to think deeply to respond, and they are giving you confirmation that may or may not be useful. A better question is “How do you communicate with your team throughout the day?”

Personal bias

We all have our own beliefs about how products work, or how they should work. These biases can sneak into our questions. The best approach, then, is to remove yourself from the actual research. While strict practice may suggest not conducting the research, we recommend developing questions from the point of view of the product, the customer, or even stakeholders of the product. The less “you” there is in the interview, the better the information that you collect will be. This results in questions more like “Tell me about your experience with your accounting software” than “I know I always struggle with invoices; what challenges do you have with your software?”

Unconscious bias

Our brains make tons of decisions every day, many of which we aren’t aware of. These can be influenced by social norms, personal history, past experiences, or expectations. These biases are the hardest to catch. Unconscious biases fail to recognize that others’ perception of a situation is not the same as our own. To avoid this, dig deeper no matter how uncomfortable that might make you feel. For instance, gender bias exists within the workplace because most people aren’t aware that the bias exists at all. Asking “Where do you guys go to unwind after work?” has implicit gender biases, whereas “Where does your team go after work?” is more neutral.

Knowing when to break the rules

If you’re just starting to build out your research skills, it’s important to avoid the aforementioned factors. However, once you get a few studies under your belt, you’ll find you can use leading questions and shallow questions in strategic ways. You can even use a participant’s personal and unconscious bias to drive to a deeper conversation about how people might use a product.

Leading

These are best used when you suspect the response will be opposite to the hints you provide in your questioning. You can use leading questions to help build trust with a participant and to validate a previous comment they made that maybe wasn’t totally clear.

Example: How much do your friends and family appreciate photo albums when you make one for them?

Shallow

When you start a research session, sometimes participants aren’t yet comfortable and they need to get used to talking with you and answering your questions. Shallow questions give participants that opportunity and can help ease them into the activity so you can get to the good stuff.

Example: How many times do you log in to Facebook in a day?

Personal bias

There is something to be said about being a good devil’s advocate—someone who can take the opposite view in a conversation to spark additional thought or comments. You can use your personal thoughts and opinions to get to deeper conversation by giving the participant something to disagree with.

Example: Do you think the Cubs actually have a chance at the World Series this year?

Years of practice and failure to master

The only way to practice research is by finding people to talk to. The first few studies you run won’t be the best, and that’s OK! You will learn something after each session, even if every question you ask isn’t the best version of that question. The goal is to improve your line of questioning and to find ways to hold a meaningful conversation with someone rather than treat research like a verbal questionnaire.

We have both had our fair share of failing during our years as researchers. In the early days, we asked overly leading questions and missed important areas of discussion because we didn’t know what we were looking for. But thanks to mentors providing feedback and guidance, we eventually overcame these failings. We still make some mistakes today, and you will too, but as long as you have a consistent feedback loop in place, you’ll continue to improve and eventually master the art of research.