Over 477k votes and Claude 3 Opus has surpassed GPT-4

This result sparks the question: Is the era of GPT dominance coming to an end? To answer this, let’s have a quick look at the data from Semrush as of February 2024:

Visits: ChatGPT garnered 2.36 billion visits, marking an increase from 1.97 billion in December. In comparison, Claude attracted 17.23 million visits, up from 11.98 million in December.
Global Rank: ChatGPT stands at 21, while Claude is at 5850.
Average Visit Duration: ChatGPT users spend an average of 14.17 minutes, whereas Claude users stay for 20.31 minutes.

It’s still early to declare ChatGPT dominance is over, especially with GPT-5 on the horizon.

It is interesting to see Claude’s higher average visit duration. this could perceived as more engaging and enjoyable, and users are exploring beyond their initial queries. However, it’s important to consider various factors that could influence this metric. from a more cautious perspective, there are concerns that Claude may have longer response times or may not fully meet user expectations, necessitating multiple follow-up queries for refinement. This could indicate areas where the platform might benefit from further optimisation to streamline user interactions and enhance satisfaction.

From a product management perspective, setting key results or KPIs requires critical evaluation to determine their effectiveness as success criteria. Visit duration might be a great indicator and an obvious one to track. It’s crucial to interpret these numbers correctly, as the increment might be a symptom of underlying issues. It is always best to combine and compare with other guardrail metrics like system health like response time, and timeout rates; and other engagement metrics like idle time, and active time, to name a few.

The user scenario is crucial. I was once assigned the task of increasing the time spent on an event app. However, I objected with a proposal to reduce the time spent instead. Consider the typical user at an event; their primary JTBD involves locating information swiftly and navigating to their desired destination with minimal delay. In such scenarios, the longer a user needs to engage with the app to find what they’re looking for, the more it reflects a lack in the app’s efficiency rather than success. An optimal user experience, especially in an event setting, is measured by how quickly and effortlessly the app facilitates the user’s needs, underscoring the importance of designing with user intentions and context in mind.

Another interesting observation is Claude’s mobile access ratio of 6:4 compared to ChatGPT’s 5:5. This ratio highlights the growing importance of mobile accessibility, which could significantly impact Claude’s potential to outperform ChatGPT in the mobile domain.

You can check out the leaderboard and compare side-by-side here: chat.lmsys.org to find out which works best for you.