Article

Advice for CX & UX Researchers: Frame Questions with Intention

If you hate answering general, open-ended questions (like How are you? How was your day? How are things going?) you’re not alone. There’s at least one person in the world like you: I hate them, too.

Questions like these are the reason why I despise networking and would never voluntarily go to a cocktail party or reception. And yet, I didn’t really understand why these questions felt so painful to me until recently, when I was conducting research for a client project at Highland.

I suddenly realized — I was asking interviewees the types of questions that I hate to answer!

This realization helped me observe my questions and consider how I might change the way I approach interviews. In practice, not only did I gain comfort in asking difficult questions; it helped me better connect with interviewees, leading to better outcomes for the client I was working with and for the people they serve.

We were hired by a national nonprofit to help them better understand the needs of their staff members across the country. Our first objective was to examine the tools and processes they used, as well as their relationship with, and perception of, organizational leadership.

This organization wanted to do a brave thing: they wanted to understand how their staff across the country truly felt about organizational leadership. They wanted to be proactive about understanding the concerns and frustrations of local staff members because they knew organizational misalignment could hold them back from making true progress on a larger digital strategy.

Based on some initial conversations with CEOs at regional offices, I had a strong feeling that there were some frustrations simmering beneath the surface. If those feelings of frustration were there, I had to find a way to uncover them. Otherwise, the existing problems would not be addressed and the staff experience would not improve.

When I started to write an interview field guide to prepare for this research, I struggled to balance the open-ended nature of my initial questions with the specificity that would be required to get the answers we needed. I wanted to uncover the authentic experiences of staff — not just the politically correct stuff and positive feelings. I needed to hear the stories that were harder to tell, but I couldn’t come up with a way to ask about that directly without asking a leading question.

For example, I couldn’t say something like: “Tell me about the last time you felt frustrated by organizational leadership.” I didn’t want to deter staff from trusting me by making an assumption about their negative feelings. And I also didn’t want to lead the participant into thinking this was my sole interest, and consequently pressure them into making something up.

If I couldn’t ask explicitly about negative feelings and experiences, I had to find another path to get to these types of answers that seemed less direct and more open-ended.

At first, I ended up with questions like:

“What are your thoughts about organizational leadership?”

“What are your feelings?”

“What have past experiences been like?”

It didn’t feel quite right. But I wanted to get started with the research, so I jumped into a couple of interviews and tested out the field guide.

In response to questions about organizational leadership, I mostly heard uninteresting, humdrum responses. No one said anything negative about leadership, but they didn’t really say anything positive either. I was doubting my questions — I felt pretty stuck on how I could ask them differently while maintaining their open-ended nature.

But then, right when I needed it most, I had an eye-opening interview with a participant that I would typically characterize as a nightmare interviewee. We were only about fifteen minutes into the interview when the participant started talking about what he wanted to get off his chest at the time. He launched into a tirade about the national office and its leadership.

Instead of taking back the reins and redirecting him, I sat back, put my field guide aside, and just listened. He ranted for the next forty minutes, and when he paused, I prodded with clarifying questions.

I noticed that he kept repeating a phrase over and over again:

“Leadership doesn’t care about ______________.”

At the end of the hour, I thanked him sincerely for his time. Upon reflection, I realized that by following this interviewee’s train of thought, I could ask interview participants to make some assumptions about what organizational leadership “cares about.” I was hopeful that this could engender some more specific answers without asking closed or leading questions.

I went back to the field guide, got rid of the other questions that I had deemed too general, and replaced them with the following questions:

1. What do you think leadership cares about or values most?

2. Follow up: What makes you say that?

3. What do you wish leadership cared about more?

4. Follow up: Why?

The interviews that followed were much more fruitful than the first few. Participants opened up immediately and talked about the values they perceived from leadership’s past actions and behaviors. From the interview findings, we uncovered that the staff believed the focus and values of the national office to be really different from their local concerns. As a result, they felt frustrated and were distrustful of some of the leaderships’ goals for the organization.

My hypothesis for why these questions were so much more effective is due to the same phenomenon that makes it hard for us to answer questions like: how are you? Or how was your day? There are so many tiny moments in a day to sift through and then quickly synthesize in order to answer that question honestly. The question is too broad.

On the other hand, a question that is framed intentionally tells a respondent what to focus on, and provides a starting point for seeking memories that could help form an answer.

If you’re sick of hearing questions like how are you or how was your day, then start experimenting with questions that have more intentional framing, such as: What was your favorite part of your day?

The revelations we uncovered from these interviews led us to conclude that redesigning the organization’s digital strategy without first addressing the issues related to staff mistrust would be a mistake. When we showed our findings to the organization’s leadership, one person said with both relief and exasperation, “For the first time we’re finally hearing the truth from staff.”

To which another leader responded: “Well, for the first time we asked the right questions.”

Download “The Essential Guide to Launching a Digital Product for Experts & Expert Firms”

Let's Talk
Tell us about the opportunity you're pursuing, and we'll follow up in one business day. If you prefer, you can email ask@highlandsolutions.com.