# ===================================================== # Atmostfear Entertainment – robots.txt # Purpose: Optimize crawlability, SEO visibility, and content protection # Last updated: 2025-04-05 # ===================================================== # =========================== # General Rules (User-agent: *) # =========================== User-agent: * Disallow: /wp-admin/ Disallow: /wp-login.php Disallow: /xmlrpc.php Disallow: /wp-content/plugins/ Disallow: /comments/feed/ Disallow: /trackback/ Disallow: /feed/ Disallow: */feed/ Disallow: /?s= Disallow: /?search= Disallow: /?fluentcrm= Allow: /wp-admin/admin-ajax.php Allow: /wp-content/uploads/ # =========================== # Allow Search Engines and Social Media Bots # =========================== User-agent: Googlebot Allow: / Allow: /wp-content/uploads/ User-agent: Googlebot-Mobile Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-News Allow: / User-agent: Googlebot-Video Allow: / User-agent: Google-Extended Disallow: / # Blocking Bard/Vertex AI from using content for training User-agent: Mediapartners-Google Allow: / User-agent: Bingbot Allow: / User-agent: BingPreview Allow: / User-agent: Applebot Allow: / User-agent: Yandex Allow: / User-agent: Baiduspider Allow: / User-agent: Baiduspider-image Allow: / User-agent: Baiduspider-video Allow: / User-agent: FacebookBot Allow: / User-agent: Twitterbot Allow: / User-agent: ia_archiver Allow: / # =========================== # Block AI Content Scrapers & LLM Training Bots # =========================== #User-agent: GPTBot #Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: AnthropicAI Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Perplexity-ai Disallow: / User-agent: Common Crawl Disallow: / User-agent: commoncrawl Disallow: / User-agent: Bytespider Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Meta-AI Disallow: / User-agent: MetaAI Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: Gemini Disallow: / User-agent: DeepSeek Disallow: / User-agent: Hugging Face Disallow: / User-agent: OAI-SearchBot Disallow: / #User-agent: OpenAI #Disallow: / User-agent: Open AI Disallow: / User-agent: AI21 Labs Disallow: / User-agent: AI SEO Crawler Disallow: / User-agent: AI Writer Disallow: / User-agent: CrawlQ AI Disallow: / User-agent: WebChatGPT Disallow: / # =========================== # Block Competitive SEO Tools (Optional) # =========================== User-agent: MJ12bot Disallow: / Crawl-delay: 10 User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: dotbot Disallow: / User-agent: SEOkicks Disallow: / User-agent: rogerbot Disallow: / # =========================== # Sitemap Index # =========================== Sitemap: https://www.atmostfear-entertainment.com/sitemap_index.xml Sitemap: https://www.atmostfear-entertainment.com/news-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/post-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/page-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/category-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/product-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/product_brand-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/product_cat-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/video-sitemap.xml Sitemap: https://www.atmostfear-entertainment.com/locations.kml