Free tools
Robots.txt Examples
Robots.txt file content for journaldesfemmes.fr.
Robot.txt file for: journaldesfemmes.fr
# www.journaldesfemmes.com User-agent: * Disallow: /recherche/ Disallow: /*/recherche/ Allow: /nom-de-famille/recherche/ Disallow: /agenda/ Disallow: /*f_impression Disallow: /*?print Disallow: /*f_imprimer Disallow: /*f_zone=impression Disallow: /*f_zone=printer Disallow: */imprimer/ Disallow: /*xhr Disallow: /api/ Disallow: /prenoms/*.swf Disallow: /prenoms/avis/ Disallow: /prenoms/avis_depose/ Disallow: /prenoms/cgi/fiche/fiche.php Disallow: /prenoms/cgi_flash/ Disallow: /prenoms/detail-copain/ Disallow: /prenoms/prenom/*/19*/19*/ Disallow: /prenoms/prenom/*/19*/20*/ Disallow: /prenoms/prenom/*/20*/19*/ Disallow: /prenoms/prenom/*/20*/20*/ Disallow: /prenoms/prenom/*/*-2/ Disallow: /prenoms/recherche Disallow: /nom-de-famille/*.swf Disallow: /nom-de-famille/cgi_flash/ Disallow: /nom-de-famille/detail-copain/ Disallow: /*/temoignage/reagir/ Disallow: /maman/maternite/*, Disallow: /maman/maternite/recherche Disallow: /maman/ecole/recherche Disallow: /maman/creches/recherche User-agent: Mediapartners-Google* Disallow: User-agent: Pinterestbot Disallow: / Sitemap: http://www.journaldesfemmes.com/sitemap/ # Block https://opensiteexplorer.org/dotbot User-agent: dotbot Disallow: / # Block http://ahrefs.com/robot/ User-agent: AhrefsBot Disallow: / # Block https://dataforseo.com/dataforseo-bot User-agent: DataForSeoBot Disallow: / # Block https://www.semrush.com/bot/ User-agent: SemrushBot Disallow: /