Where AI becomes first choice over search
Search is so dominant that it’s stunning to think it could be replaced by something else. But if you can only start in one place, where would you start? We asked that question of 227 US consumers familiar with AI to see which they would use first in nine
AIQ Report
Most common user goals: does AI satisfy?
To better understand the AI opportunities that are lower-hanging fruit, it is important to understand what consumers are asking and what types of prompts come naturally to them. Our research shows that 3 out of 4 people (75%) begin with prompts that fall into three types: * Factual (34%) ex. “Ho…
AIQ Report
AI platforms already have very different DNA
As part of our research, we ask users to do a side-by-side comparison of responses to the same prompt from different platforms. After users pick their preferred response, we ask them why they chose it. After reviewing 1,135 reasons why users prefer certain responses on certain platforms, we color
AIQ Report
AI Tube: Watch where platforms win, or lose
As part of AIQ’s effort to compile a comprehensive overview of users’ real-time experiences across various AI platforms, this selection of videos captures real prompts and user reactions. Our AIQ database of thousands of videos is growing monthly. We’ll periodically pull a few to compare and contras…
AIQ Report
Claude Enters the Race, Overtaking Bing In Preference
Claude recently released its platform to the public. AIQ™ results shows a strong initial showing: * Claude receives a 45% preference rate when consumers rate responses to a variety of prompts in our blind side-by-side comparison of 5 leading AI platforms. * Claude overtakes Bing which is preferre…
AIQ Report
People Willing To Give Data to AI Platforms for Personalization
Consumers show a willingness to provide AI platforms with their personal data because they value the personalized results and they anticipate better results. * 77% of users expressed their readiness to share personal information believing it enables them to enjoy a more tailored AI experience *…
AIQ Report
Claude Solid on Factual, Trails on Emotional
When compared to other platforms based on 5 different types of prompts, we can see some strengths and weaknesses emerge for Claude. * Claude rates better than AIQ average on “Factual” prompts (23%) * Example: “In which year did the Apollo 11 moon landing take place?” * Claude rates a bit behind…
AIQ Report
AI: A Useful Travel Agent
Travel and trip planning is seen as a natural use case for AI platforms. In the planning stage, consumers are making large considered purchases and need to compare and weigh different options with a number of variables. Consumers see high potential in AI’s ability to alleviate trip planning anxiet…
AIQ Report
GPT-4 stays on top but Bard gains
GPT-4 still ranks highest in overall user preference score, confirmed in the latest testing of over 500 responses during the last two weeks. But Bard is steadily closing that gap and not far behind GPT-4. Overall, GPT-4 responses have the leading preference score among all platforms tested * GPT-…
AIQ Report
When branding removed, GPT fans often prefer Bard responses
If you remove the brand, and people didn’t know the platform behind the response, would that impact perceived quality and preference ratings? Using the two pronged approach in AIQ™ we examine this and find some differences. * Among GPT-4 “fans” who rate GPT-4 highly in branded testing, those same…
AIQ Report
Only 6% believe AI can replace human creativity and innovation
As the impact of AI and its implications ripple through society, bigger questions are being raised, including in Hollywood, where the entertainment industry relies on human creativity and innovation. The current writer/actor strike has brought this to the forefront: what is the role of AI in creativ…
AIQ Report
21% of all AI queries are asking for advice
1 out of 5 (21%) of all self-directed user queries relate to “advice.” The types of advice users seek range from dating tips, managing difficult bosses, health, to outfits for a theme party. At right are a sample of user experiences capturing “advice” prompts with video and audio.
AIQ Report
Is GPT-4 worth the extra cost?
Open AI has a freemium model whereby GPT-3.5 is free and GPT-4 (“Plus”) is not. Understandably, there are questions about whether GPT-4 is worth the cost. Is it better? How much better? One simple answer is: GPT-4 is 8% better. Why? The perceived quality of the response. Putting other
AIQ Report
GPT-4 leads major AI platforms in side-by-side comparison
We gathered 4,200 responses from the four leading AI platforms and had 400+ users pick which ones they preferred. The emerging winner so far is GPT-4, but it’s a close race. GPT-4 leads the way, with responses preferred 55% of the time. That’s a significant advantage over Bing
AIQ Report
When GPT-4 is not preferred. Why? TMI
While GPT-4 does very well overall in the comparison rating, there was a consensus among those who DIDN’T prefer GPT-4 responses as to why: GPT-4 is not preferred when there is “TMI” or “too much information.” Implication: Striking the balance of “just enough” information - but not too much -
AIQ Report
Hallucinations: Consumers don’t see but industry worries
Hallucinations - an industry term used to describe the situation where an AI platform generates responses that are inaccurate or don’t make sense but are presented as factual - is an area where platforms and the public may not see the same thing. * Only 5% of user interactions rated are
AIQ Report
People think they see AI in many places
While AI in its current consumer form is relatively new, many people feel they have seen it already and in many places beyond the platforms offering LLM capabilities. * Many users (22%) state that their first interaction with AI was via assistants, smartphones, and apps that have been on the marke…
AIQ Report
Top topics: travel and trip planning videos
In AIQ, one component of the research design that we call “self-directed” is to let users decide what they want to do on an assigned AI platform. While the topics and interests they chose vary, overall we see certain topics and use cases emerge as the most popular. The top
AIQ Report
Each AI has unique strengths and weaknesses
In blinded side-by-side testing, users select their preferred AI response to a series of queries and describe why. As a result, we see how the four platforms compare across hundreds of users and various question types. * Bard stood out for responses that were viewed as most complete, accurate, and…
AIQ Report
Younger users want more human-like AI
Across all users, the top three reasons for choosing a preferred AI response in our blinded tests are: 1. Feels accurate 2. Feels relevant 3. Feels complete In addition, among younger users, the attribute “feels human like” increases in importance. This video of a young user asking Bard for a
AIQ Report
User questions for AI vary by age
In our testing, the top three types of question users submitted to the AI platforms were in the areas of: 1. Learning & Education 2. Technology 3. Media & Entertainment Looking at differences by generation, we find: * Millennials more interested in Money, Finance, Insurance * Gen Z more inte…
AIQ Report
Microsoft users prefer GPT over Bing
Microsoft’s investment in OpenAI has resulted in ChatGPT-4 powering Bing. However, looking at Microsoft users (defined by Windows PC or MS Tablet users), when presented with blinded responses from different platforms to the same prompts, they preferred GPT-4 and 3.5 over Bing’s responses. The % i…
AIQ Report
AITube: Watch what happens on each platform
Our study captures hundreds of videos of actual user experiences with the four AI platforms. A unique and powerful aspect of AIQ™️ is the ability to see these interactions in the moment. Two videos were chosen for each AI platform that demonstrate meaningful user insights. See what users see and
AIQ Report
What is AIQ?
AIQ™ measures the performance of the leading AI platforms from the perspective of your most important audience: users. Each week, our report captures insights from hundreds of real user interactions with AI. This allows you to always have an accurate, up-to-date read on developments in this rapidly evolving space.
