Making user testing inclusive, quick, efficient, and actionable through unmoderated user research.
In today's competitive technological landscape, continuous testing and refinement have become the cornerstones of successful product launches. User experience research has evolved into an indispensable tool for product leaders, serving as a crucial step in validating design decisions, and ensuring that products will engage and satisfy users – and drive more revenue. However, user research can be costly, time-consuming, and difficult to manage, especially when conducting moderated research and trying to get users to come to a lab environment.
In light of these challenges, unmoderated methodologies have emerged as a compelling solution. These approaches are sought after by mobile app owners and product managers for their agility, cost-effectiveness, user inclusivity, and the unique insights they produce.
Top 5 reasons to use unmoderated research
- Scalability: Unmoderated research can accommodate a larger number of participants, enabling access to a more extensive and diverse pool of users.
- Cost-Effectiveness: The reduced costs are attributed to the absence of a moderator and the ability to conduct testing remotely, eliminating the need for in-person facilities and travel expenses.
- Reduced Bias: The absence of moderators in unmoderated testing can lead to more honest and unbiased feedback from participants.
- User Inclusivity: Unmoderated testing allows the inclusion of users with different work schedules, transportation options, and accessibility needs, ensuring that the product is designed to be inclusive and user-friendly for all.
- Agility & Continuous Improvement: Unmoderated research can be executed swiftly, providing quick feedback, and it can be seamlessly integrated into the product development cycle, enabling continuous product refinement.
In addition to these key advantages, unmoderated usability research can also be conducted longitudinally, allowing app owners to track user behavior and preferences over time.
“It is safe for the LGBTQ community, that is good!”
A user explains why she was interested in legal protections available in the neighborhood she was looking at in a real estate app.
—Refilwe, Gen Z, Maryland
In the video, we observe a user browsing a real estate app. They express particular appreciation for the "legal protection" section, specifically highlighting the app's recognition of LGBTQ+ safe spaces. This unsolicited feedback underscores the significance of considering many users' backgrounds in app design to create more inclusive and meaningful user experiences.
The user's feedback also exemplifies the power of unmoderated testing. Unlike moderated research, where time constraints often limit the number of questions researchers can ask, unmoderated approaches allow participants to volunteer information organically.
The growing power of unmoderated user experience research over traditional methods
Traditional usability research usually involves a skilled moderator, and is often the go-to tool for many product and research professionals. While this method is effective in many ways, it comes with time constraints, high costs, and the potential for moderator bias. These factors have paved the way for the rise of unmoderated usability testing, which offers a more flexible, scalable, and unbiased alternative.
Unmoderated usability research empowers participants to interact with a product independently and at their convenience. In task-based testing scenarios, participants are provided with a set of tasks to attempt and complete. They record their experiences and feedback through screen recordings and follow-up surveys. This method allows access to a broader pool of participants, including those from different geographical locations, professions, and time zones.
Another unmoderated method is the diary study, where users are asked to interact with a product as they normally would, then record issues that arise or provide general feedback. This allows product owners to get authentic, unprompted feedback from users.
“I am 47, these apps are not made for older people!”
A user explains challenges she faces using real estate apps as someone who is older, and has work-from-home needs
—Melissa, Gen X, Virginia
This user's observation shows how the failure to consider the needs of different age groups and users with varying visual abilities can create barriers to a seamless user experience. Accessibility is often looked at as a "nice to have" for a small subset of disabled users, but Microsoft's design team makes the case that what are perceived as accessibility "constraints" can actually open up new possibilities:
"Designing for people with permanent disabilities can seem like a significant constraint, but the resulting designs can actually benefit a much larger number of people. For example, curb cuts in sidewalks were first created to make it safer and easier for people in wheelchairs to cross the street. But curb cuts also help people with a wide range of circumstances, from kids riding bicycles, to parents pushing strollers, to workers hauling heavy equipment."
– Microsoft Inclusive Design Toolkit Manual
Challenges to navigate when conducting unmoderated usability research
Despite its numerous advantages, unmoderated research presents a few challenges. One common obstacle is the lack of real-time guidance, which can lead to misunderstandings of task instructions and difficulties in navigating the testing process. Additionally, ensuring the quality and reliability of data can be challenging without a moderator present to clarify responses and identify potential errors.
When your research is well designed and executed, you can get insightful feedback like the example below.
“I worry if 1+ bedrooms includes 2+ as well!”
A user offers one positive and one negative feedback on a real estate app’s filters.
—Zharia, Millennial, California
This user is navigating a real estate app for the first time, revealing two valuable observations. First, when attempting to filter houses based on the number of bedrooms, the app's labeling system posed a challenge. The options presented included 1+, 2+, 3+, etc. and left the user uncertain whether selecting "1+" would also display results for "2+".
Second, while this user was exploring neighborhood demographic information, she expressed a preference for living in a neighborhood with an older population, even though she is a Millennial. Her desire for a quieter neighborhood, free from loud music and late-night disturbances, contradicts the assumption that people are typically drawn to neighborhoods with people their own age.
How to start or enhance your unmoderated user research
When planning for remote usability research, reliable tools are essential for capturing comprehensive data and generating actionable insights. Pulse Labs offers a trusted and user-friendly platform for designing, conducting, and analyzing unmoderated usability research. This includes screen recording, task tracking, data analytics, and more. We're happy to talk to you about your research needs and see how we can help optimize your unmoderated research approach.
Want to learn more about the tools we use for our research and how they can help you power your product design and user experience?
Stay tuned for more insights from Pulse Labs IQ™ and sign up for our newsletter.
