Free tools
Robots.txt Examples
Robots.txt file content for stylecraze.com.
Robot.txt file for: stylecraze.com
User-agent: * # applies to all robots Disallow: /inputfile1.txt Disallow: /wp-dp.php Disallow: /fork/stylecraze.html Disallow: /feed-stylecrazev44.php Disallow: /tag/ Disallow: /banners/ Disallow: /staging/ Disallow: /activity/ Disallow: /affiliates/ Disallow: /questions/tags/ Disallow: /disscussion-forums/ Disallow: /filter-products Disallow: /xn/ Disallow: /pages/ Disallow: /page/ Disallow: /photo/ Disallow: /video/ Disallow: /blog/ Disallow: /profiles/ Disallow: /forum/ Disallow: /article/ Disallow: /groups/ Disallow: /comment-subscriptions Disallow: /questions/ Disallow: /photos/ Disallow: /videos/ Disallow: /members/ Disallow: /brands/ Disallow: /add-blog-post/ Disallow: /create/ Disallow: /upload-video/ Disallow: /shop/* Disallow: /blogs/ Disallow: */view/slideshow/ Disallow: /wp-admin/ Disallow: /venus* Disallow: /fb-posts/ Disallow: /links/ Disallow: /mobile* Disallow: /sponsored/ Disallow: /search-results/ Disallow: /fblivevid.html Disallow: /members* Disallow: /v.lkqd.net/* Disallow: /*?pid=* Disallow: /*aul_class Disallow: /wp-login.php* Disallow: /giveways/* Disallow: /nootropics/* Disallow: /slideshow/* Disallow: /exclude-homepageads/* Disallow: /sc-admin-login/* Disallow: /errors/error403.htm Disallow: /pn/* Disallow: /find/* Disallow: /search/* Disallow: /trending/ Disallow: /collaboration/ Disallow: *.pdf Disallow: *.ttf Disallow: *.woff2 Disallow: *.js Disallow: */page/* Disallow: /feedback-form/* Disallow: /performance-test/* User-Agent: MJ12bot Disallow: User-agent: ia_archiver Disallow: / User-agent: Boomtrain-Content-Bot* Disallow: Allow: /* Sitemap: https://www.stylecraze.com/sitemap_index.xml