Guide to User Testing Interviews & Obtaining Valuable Insights

At Highland, we believe that user testing doesn’t have to be wildly expensive or time-consuming.

In her book Just Enough Research, Design Consultant Erika Hall reminds UX researchers that they can gather valuable data using simple, easy-to-implement research methods — regardless of team size or budget. They just need to have the courage to talk to their users.

Both Hall and the design thinking experts at Neilsen Norman Group agree that talking to a small number of users can make a meaningful difference when it comes to design. In their groundbreaking article “Why You Only Need to Test with 5 Users”, Neilsen Norman Group recommended that researchers only needed to interview 5 users to gather over 80% of their research insights. After that, “you are wasting your time by observing the same findings repeatedly but not learning much new.” It is also important to ensure you select the right demographics for your user testing, else the qualitative data that you collect and post-test analysis may not benefit your product iteration.

So if you’re ready to start talking to (just enough) users, here are a few tips to help you get started. If you don't ask the right questions to your target audience then the functionality, overall experience and user interface may fall wide of the mark and the overall impression of your product may be lower than expected. I’ll follow these tips with a step-by-step script that you can use to conduct your user interviews in the testing session. These are particularly geared towards product testing, but hopefully you’ll find they can improve any user interviews you do, whether it be for e-commerce journey path usability testing, general UX design research or understanding how users interact with mobile apps.

Get in the Right Mindset

1. The goal is to learn from the user, not prove a product concept.
2. If they hate it, great. Now you have a better sense of what they actually want. Iterate on the idea and try again.
3. If they love it, fabulous. Still, iterate and try again.
4. As the facilitator your job is to get them talking and listen attentively to their words and tone.
5. Forget your normal conversational style. You should talk very, very little.
6. Be mindful of your body language—lean in, make eye contact, nod your head.
7. Follow your script (see below) but be curious. Always be listening well.

Ask Good Questions to Listen Well

1. Testers should ask well-crafted open-ended questions. Avoid “yes/no” questions.
2. Avoid specific questions or leading questions. Open-ended questions force them to use their words for things. I.e. “What do you think about that?” not “Do you think this is helpful?”
3. Prompt, prompt, prompt for more thoughts with follow up questions: Tell me more… How so… What do you mean?
4. Don’t let them leave it at “that’s nice / I like that”—prompt them to be specific.
5. Keep it conversational. Connect your questions to their prior comments, i.e. “You mentioned that this was confusing. Tell me how you imagine using this?
6. Don’t assume you know anything. Ask questions you think you know the answer to.

A Basic User Interview Script Structure

When it comes to pre-test, never start user research without crafting a thoughtful script (sometimes called a field guide). And never write a script until you’re clear on your learning goals for your research. For example are you trying to understand the overall user experience, or understand how test participants handle a specific task or function/ These can help form the types of questions that you need to include. As a serving suggestion, here is a simple sample script:

Introduction

[Warm intro] Thanks so much for meeting with me today! We’re in the really early stages of exploring a new product concept that I’m curious to get your feedback on.

[Clarify aims] I’ll start by asking a few background questions. Then, I’ll show you what we have and I’d like you to simply explore and narrate your thoughts out loud for me.

[Consent] Before we dive in, do I have your permission to record? (use phone/computer)

Warm-Up Questions

• Tell me a little more about you… [3–4 questions relevant to learning goals — examples below]
• What kind of work do you do? What do you like to do when not working?
• How do you normally __________? What tools have you used to help __________?
• What do you ultimately want them to do for you?

Product Feedback

• Would you be willing to look at our prototype now?
• I’d like you to simply explore and narrate your thoughts out loud — what you’re noticing, if you like something or not, or if you’re confused. Your frank, candid feedback is most helpful.
• Let’s imagine ________________. [Set the stage with a nudge/scenario about the product]
• [Show them the prototype] Explore and narrate your thoughts for me.

Prompt throughout with questions like:

• Tell me more… How so?
• What is this? What is this for?
• What are you noticing? What do you think of that?
• What would you expect that to do?

Concluding Thoughts

“Bonus” questions can be added here, like:

• Overall, how would you describe this to a friend?
• On a scale of 1 to 5, how likely would you be to pay for/use this? Why that number?
• If you had a magic wand, how would you make this more valuable to you?

Thank you! That’s all I have for you; do you have any lingering thoughts for me?

When to do user testing

Always, always start with user testing before building and launching a new product idea. Most new products fail due to a lack of customer understanding. User testing can be useful to help create user-friendly products, and can help justify a redesign when pain points are identified since real users have gone through the testing process and screening questions. Testing your product design concept with users first can help you get off to the right start.