OpenAI's ChatGPT Indexing Levels the Playing Field for Small Websites, New Data Reveals

New data from French SEO consultancy Resoneo shows that OpenAI in-house search index, dubbed 'labrador', treats both licensed and unlicensed websites equally when serving up information in ChatGPT. This finding challenges earlier assumptions that smaller, non-partnered sites might be overlooked by the AI giant, suggesting a more democratic approach to content discovery within the platform. The analysis of over 1,200 ChatGPT responses captured in July revealed no difference in how pages were formatted, their length, or freshness, regardless of whether a licensing deal was in place. This revelation arrives amidst ongoing industry debate about how AI companies utilize web content and the potential for a 'zero-click internet,' where users get answers directly from AI models, bypassing original websites. While some AI crawlers like GPTBot, primarily used for training, have faced widespread blocking by publishers worried about content usage without compensation, this data on 'labrador' indicates a separate, more equitable indexing process for OpenAI search functionality. Importantly, for free ChatGPT accounts, 'labrador' frequently handled queries for factual answers and local businesses, although news results were often scraped from Google. The implications are significant for content creators and webmasters navigating the evolving landscape of AI-powered search. While blocking training bots like GPTBot might protect intellectual property, allowing OpenAI search-specific crawler, OAI-SearchBot, could be crucial for maintaining visibility and potentially driving referral traffic from ChatGPT massive user base, which boasts 900 million weekly active users. Publishers should consider how to optimize their sites for both traditional and AI search indices, balancing content protection with the opportunity for AI-driven discovery and engagement.