Pulse Labs Insights is your source for understanding and improving the digital user experience - written by and for those working in all aspects of UX Research. We empower UX Researchers, Research Ops teams, Product Managers, Designers, and Innovators with the knowledge they need to build and launch better products.

  • Insights: Timely, digestible takeaways and analysis on trends in UX, AI, and emerging technology.
  • Best Practices: Expert tips and strategic guidance for applying user-centric principles to your product development process.
  • Proprietary Research: Deep-dive research and recommendations- powered by our rigorous two-part methodology outlined below.

Proprietary Research Methodology
Pulse Labs combines in-the-moment user experiences with survey responses to understand how users interact with and perceive different AI platforms. Participants complete two parts:

  • Part 1: Self-Directed (SD) Exercise
  • Part 2- Side by Side (SBS) Comparison

Part 1 – Self-Directed (SD) Exercise
Participants are invited via the Pulse Labs Power Portal and assigned one of five platforms: ChatGPT-4, ChatGPT-3.5, Bard, Bing, or Claude.

  • Users get 2–3 minutes to try one or more prompts in the platform’s native web experience.
  • They record a video and answer 6–7 questions on metrics such as satisfaction, NPS, choice drivers, query type, and theme.
  • Upon completion, they move to Part 2.

Part 2 – Side-by-Side Comparison
Participants review blind responses from a random pair of platforms and indicate which they prefer.

  • Each sees five sets of questions across different themes.
  • Order and options are randomized to reduce bias, and each platform appears equally often.
  • After each choice, participants explain their preference via survey questions.

Preference Score CalculationBased on the choice a user makes, a preference score is calculated for each platform. This is calculated as a percentage:

  • (No. of times a platform is preferred)/ (No. of times a platform is shown) x 100
  • A formula is used to calculate the preference score for each of the four platforms. The scores are then analyzed by demographics, technographics, and type of question to look for trends and patterns.
  • Significance Tests are carried out on the data at a confidence level of 95%.