Free tools
Robots.txt Examples
Robots.txt file content for francebleu.fr.
Robot.txt file for: francebleu.fr
User-agent: * Sitemap: https://www.francebleu.fr/sitemap.xml Sitemap: https://www.francebleu.fr/sitemap-news.xml # Player Embed Disallow: /player/export Disallow: /player/embed Disallow: /player/evenementiel Disallow: /9j/ # Widgets Disallow: /widgets Disallow: /search-widget-results Disallow: /bleustatic-production Disallow: /cruiser-production Disallow: /get-block-meteo Disallow: /cruiser-static-production Disallow: /embed/elections # FTV Disallow: /tv/ # Preview Disallow: /preview/ # Other Disallow: /newsletter/html Disallow: /recherche? Disallow: /sms Disallow: /grid Disallow: /musique/c-est-quoi-cette-chanson/list Disallow: /amp/ User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: Google-Extended Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: Linguee Bot Disallow: / User-agent: Amazonbot Disallow: / User-agent: Bytespider Disallow: / User-agent: ClaudeBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: FacebookBot Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: PerplexityBot Disallow: /