Free tools
Robots.txt Examples
Robots.txt file content for wordpress.com.
Robot.txt file for: wordpress.com
# If you are regularly crawling WordPress.com sites, please use our firehose to receive real-time push updates instead. # Please see https://developer.wordpress.com/docs/firehose/ for more details. Sitemap: https://wordpress.com/sitemap.xml Sitemap: https://wordpress.com/news-sitemap.xml Sitemap: https://wordpress.com/go/sitemap.xml Sitemap: https://wordpress.com/blog/sitemap.xml Sitemap: https://wordpress.com/support/sitemap.xml User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Disallow: /typo/ Disallow: /read/ Disallow: /*/read/ Allow: /log-in/ Allow: /log-in$ Disallow: /log-in? Disallow: /abuse/?* Disallow: /abuse?* Disallow: /plugins/?s= Disallow: /*/plugins/?s= Disallow: /*?aff= Disallow: /*&aff= Disallow: /*/?like_comment= Disallow: /wp-login.php Disallow: /wp-signup.php Disallow: /press-this.php Disallow: /remote-login.php Disallow: /activate/ Disallow: /cgi-bin/ Disallow: /mshots/v1/ Disallow: /next/ Disallow: /public.api/ # This file was generated on Fri, 29 Nov 2024 16:01:07 +0000