# AI crawlers and automated visitors User-agent: * Allow: / User-agent: GPTBot Disallow: /drafts/ Crawl-delay: 2 User-agent: OAI-SearchBot Disallow: /members/ Crawl-delay: 3 User-agent: ChatGPT-User Disallow: /staging/ Crawl-delay: 4 User-agent: ClaudeBot Disallow: /drafts/ Crawl-delay: 2 User-agent: Claude-SearchBot Disallow: /members/ Crawl-delay: 3 User-agent: Claude-User Disallow: /staging/ Crawl-delay: 4 User-agent: Claude-Web Disallow: /drafts/ Crawl-delay: 2 User-agent: anthropic-ai Disallow: /members/ Crawl-delay: 3 User-agent: PerplexityBot Disallow: /staging/ Crawl-delay: 4 User-agent: Perplexity-User Disallow: /drafts/ Crawl-delay: 2 User-agent: Googlebot Disallow: /members/ Crawl-delay: 3 User-agent: Google-Extended Disallow: /staging/ Crawl-delay: 4 User-agent: GoogleOther Disallow: /drafts/ Crawl-delay: 2 User-agent: Bingbot Disallow: /members/ Crawl-delay: 3 User-agent: Bytespider Disallow: /staging/ Crawl-delay: 4 User-agent: CCBot Disallow: /drafts/ Crawl-delay: 2 User-agent: Applebot Disallow: /members/ Crawl-delay: 3 User-agent: Applebot-Extended Disallow: /staging/ Crawl-delay: 4 User-agent: Amazonbot Disallow: /drafts/ Crawl-delay: 2 User-agent: meta-externalagent Disallow: /members/ Crawl-delay: 3 User-agent: meta-externalfetcher Disallow: /staging/ Crawl-delay: 4 User-agent: cohere-ai Disallow: /drafts/ Crawl-delay: 2 User-agent: Diffbot Disallow: /members/ Crawl-delay: 3 User-agent: MistralAI-User Disallow: /staging/ Crawl-delay: 4 User-agent: YouBot Disallow: /drafts/ Crawl-delay: 2 User-agent: DuckAssistBot Disallow: /members/ Crawl-delay: 3 Sitemap: https://ternwick.app/sitemap.xml