Sitemap: https://gizmodo.com/sitemap_index.xml Sitemap: https://gizmodo.com/sitemap-news.xml Sitemap: https://gizmodo.com/download/downloads.xml.gz User-agent: * Disallow: /stats/ Disallow: /api/ Disallow: /ajax/ Disallow: /embed/ Disallow: /setbucket* Disallow: /game/score/* Disallow: /game/summary/* Disallow: /gateway/* Disallow: /showcase/* Disallow: /search$ Disallow: /search? Disallow: /?s= Disallow: /search/ Disallow: /wp/wp-admin/ Allow: /wp/wp-admin/admin-ajax.php # Allow traditional search indexing User-agent: Googlebot User-agent: Bingbot Allow: / # Allow AI search and agent use User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-User User-agent: Claude-SearchBot User-agent: PerplexityBot Allow: / # Disallow AI training data collection User-agent: AI2Bot User-agent: Amazonbot User-agent: anthropic-ai User-agent: AndiBot User-agent: Applebot-Extended User-agent: Bytespider User-agent: CCBot User-agent: ClaudeBot User-agent: Cohere-ai User-agent: Diffbot User-agent: FacebookBot User-agent: FirecrawlAgent User-agent: Google-Extended User-agent: Google-CloudVertexBot User-agent: GPTBot User-agent: ImagesiftBot User-agent: Kangaroo Bot User-agent: Meta-ExternalAgent User-agent: MistralAI-User User-agent: Omgilibot User-agent: PanguBot User-agent: Timpibot User-agent: Webzio-Extended User-agent: YouBot Disallow: / # Disallow site scrapers User-agent: HTTrack User-agent: Nutch User-agent: Scrapy Disallow: /