From Problem to Solution. Faster. - Part 2: Solution Interviews

In Part 1 of this post, we looked at how problem interviews can be a useful technique to understand the problem you’re trying to solve and whether or not it’s worth solving. In this second part, we’ll look at where to go from there: exploring and validating valuable solutions for your customer. Again, I’ll be using the example of how this technique impacted development around our WYSIWYG editor.

Solution Interviews: A technique for early and continuous feedback

Having discovered and understood our problem to be story articulation (giving form to a story through text, images and other formatting) using our wiki editor, we set out to explore the solution space by conducting a series of collaborative sketching (design charrettes) sessions with stakeholders. This resulted in 17 concepts, from which we identified 10 patterns. Far too many from which to choose. Which one was “right”?

Enter Solution Interviews

To validate which solution concept was most effective at resolving our problem of story articulation, we conducted a series of solution interviews. To be clear, a solution interview is not user testing or usability testing. A solution interview is a method to validate your assumption that what you’re building will solve their problem. It’s focused on the solution and its efficacy at solving the problem at hand.

Unlike problem interviews, where your users are doing most of the showing, solution interviews require you take the stand to show your proposed solution. However, you still mostly watch and listen. But again, it’s all about how.

Tips for a Successful Solution Interview

(Hint: it’s not just presenting a solution, but how you present it)

The solution interviews we conducted helped us both identify the most effective solution and provide early and continuous feedback through multiple iterations. Here’s what we learned about conducting a successful solution interview.

1. Do your Homework: Define Scenarios & Success Criteria Beforehand
   Again, preparation is essential. Before conducting your solution interview, define interview scenarios and success criteria beforehand. This will help you understand whether or not your solution is valid. If you’re testing multiple solutions for the same problem, make sure to use the same scenario and success criteria as a control. For example, when we were testing out solutions to more easily insert inline images, we had the following scenario, which we gave to the user, and success criteria, which is not shared but used to assess the outcome of the interview:

   Scenario: You’re reviewing a story and realize that a mock-up would help developers understand the written content and how it should look within your product. Please update your card to include mock-up.
   Success Criteria (not given to the user): The user is able to easily insert desired image with no surprises or discomfort. She intuitively knows how to move it within the card. (Pay attention to whether or not the user attempts to use D&D functionality.) Make sure the user understands the scenario before you leave and she begins.

2. Start as Early as Possible (and Know when to Move On)
   When do you know you are ready to start solution interviews? As soon as you have an idea for the solution. Test it out with paper prototype and, if it resonates with your user, start building higher-fi prototypes and eventually, workable software. You don’t have to present a full solution to conduct a fruitful solution interview. In fact, focusing your interview on a particular aspect of your solution often provides more focused and useful feedback. Lastly, don’t make the mistake of conducting too many solution interviews on the same functionality. This can end up wasting both your and your user’s time. After about three to five interviews, you should have a good idea about whether or not your solution has been (in)validated. If you can start predicting responses or you’re not learning anything new, it’s time to move on.
3. Make Sure They're Comfortable
   You don’t want your user to be unable to carry out the scenario because of any issues other than flaws of the solution itself. Ensure the user is familiar with the computing equipment (hardware and software) you’re using for the interview. Details are important! From mouse vs. trackpad (including direction and orientation) to browser preference, you want to make the interview environment as similar to their personal one as possible. Ideally, use their own machine.
4. Record
   It’s best to have a record of the interview so you can go back and review it with your team and display it on monitors as an information radiator. It’s also helpful to observe the recording to see how the user’s behavior differs from verbal feedback she gives, as well as to identify pain points she experienced while completing the scenario, but may not have been aware of. Make sure to ask your users if they’re okay being recorded beforehand. Good tools for recording include Silverback and Screenflow. Even digital meeting tools such as GoTo work if you have to conduct the interview remotely.
5. Provide a Quiet, Distraction-Free Space
   If possible, try to reserve a room for the solution interview. After the user confirms understanding of the scenario, make sure to leave the room, but make known you’ll be right outside should she have a question. This helps people loosen up and become more comfortable carrying out the scenario. It also allows you to keep passersby quiet so as not to disturb your user. We observed how differently people behaved when they were able to conduct the interview alone. When we sat with them, they had to both complete the task and felt the need to keep us engaged. Without the pressure of observers, they would pause and think more naturally, revealing more about their experience using the proposed solution to solve their problem.
6. Exit Chat
   After a reasonable about of time, check in on your user. (Make sure to knock before entering.) Ask something along the lines of, “How’d it go?” to get initial qualitative feedback about their experience. “Can you show me” is another good question to get verbal commentary about what they did and what has been recorded. However, don’t ask them to show you the whole thing (since you hopefully have a recording), only what stands out.

Final Thoughts

Jumping to building solutions is a nearly universal reaction, which often leads to building something--and perhaps something even great--that fails to solve the problem at hand. However, returning to understand the problem, and quickly validating multiple solutions in continuous feedback cycles, can help you build the right thing, faster.

For the Mingle team, using an interviewing methodology of problem and solution interviews better ensured we designed the most appropriate solution, faster by helping us understand the problem from our customers’ point of view and paint a picture of what a “day in their life” looked and felt like. With this deep understanding we were able to confidently attack their most painful problems and generate multiple solutions, validating the most successful both quickly and iteratively.

The point is, in the spirit Einstein, understand the problem and the solution will emerge.