Dismantling assumptions with user research
5 questions for Steve Portigal: User research war stories, mishaps and patterns, and challenging your assumptions.
Inner Circle. (source: James Lee on Flickr)
I recently asked Steve Portigal, user researcher, and author of Doorbells, Danger, Dead Batteries, and Interviewing Users, to discuss what the unpredictable nature of user research can teach us, establishing rapport, and what happens when user researchers find themselves in dicey scenarios. At the O’Reilly Design Conference, Steve will be presenting a session, user research war stories.
You’re presenting a session on user research war stories. Why is user research so hard to do well?
I talked about this a bit in Interviewing Users—that the assumption we can just use our social defaults because it’s just “talking to people” holds us back from being better at user research. We have to unlearn a lot of patterns (e.g., sharing about yourself) in order to get to a very different outcome (a good session versus a good hangout). In looking at war stories, I’m digging further into the challenges we face in doing research, and hopefully not stating the obvious, but research with other people will have massive elements of unpredictability in it. That means we learn what we didn’t know we didn’t know (and would never otherwise have thought to ask about), but it also means that our attempts to plan and control the process are somewhat foolish (and yet, someone who does research without planning is obviously a fool). There’s an element here of temperament, or worldview, that isn’t so natural for everyone. In some of the stories I’ve gathered, people do everything right, and yet things still go wrong. That’s not a welcome truth.
Learn faster. Dig deeper. See farther.
You plan to cover some of the mishaps and patterns that emerged. Can you share a few here?
When people first hear about this topic, they gravitate immediately to problems with participants, but there’s a lot more. People find themselves in situations that may or may not be dangerous. In one story, the researcher is taken for a high-speed drug-fueled drive around Miami, where his gay client is bullied by the participant into accepting a private dance. Researchers stumble into emotionally fraught contexts they weren’t expecting, where an innocent question about a product feature leads to a reflection about a loss. Researchers sometimes have to deal with ethical conflicts with what they observe, where they have a responsibility to their participant, but also to their employer or client. In one story, the researcher is put in touch with the participant courtesy of that participant’s boss, but sees that the participant is using an online account from his past employer, likely a fireable offense. In another story, the participant shows the researcher their BitTorrent server and archive of pirated files.
Should all members of an organization learn how to do user research?
There are levels of “doing user research.” I think it’s great when as many people as possible can get out to see how real people are living their lives. Not everyone needs to be able to plan, conduct, analyze, etc., user research. But being in the second chair position in user research (and I’m not talking about sitting behind a two-way mirror) takes some practice. I think everyone should be given the opportunity to get a little better at that, which is probably more a skill in listening deeply rather than asking questions.
What lessons or reminders have you discovered throughout your career for how to effectively conduct user research in the field?
There are a couple of things that I keep re-learning over and over again. One is about how much effort it takes to establish a significant level of rapport. You can complete the tasks of an interview, cover all the topics, but not necessarily be at that point where something extra is happening. I feel like I work so hard to get to that other place, and I often get there, but even in those cases, I’ll be surprised how much time, patience, and focus it takes. I like to walk away feeling I’ve made a connection because that energizes and inspires me and helps me carry what I’ve learned forward. The other area is around judgement (and this is certainly a theme that many of the war stories reflect). I personally find it thrilling to have my judgement proven wrong, at least in the context of user research. As we all do, I have unspoken expectations for people based on their age, or their economic status, or where they live, or how they present themselves. And very frequently, I discover that I have those expectations (well, let’s call them biases) when the person I’m talking to happens to disprove them. There’s a huge amount of rapid judgement we use just to get by in this world, and when research gives me the chance to learn something new by dismantling my assumptions, that’s one of the best feelings. But it doesn’t stop me from girding myself with those assumptions, so I keep revisiting this every time I’m out in the field. The key, I think, is not “don’t judge” but to be available when those signals that challenge your assumptions appear.
You’re speaking at the O’Reilly Design Conference in March. What sessions are you interested in attending?
There are so many to choose from! Auughhh! Okay, for starters, Catherine Courage [Adventures in startup land], Leah Buley, Brandon Schauer [Wielding the soft (and hard) science of service design], and Peter Morville [The secret to strategic design:Mastering Planning] never fail to inspire me. I’m a fan of both Aarron Walter [Hard learned lessons in leading design] and Sara Khoury [Adventures in startup land], but I’ve never seen either of them give a presentation.