Free tools

Robots.txt Examples

Robots.txt file content for nyc.gov.

Robot.txt file for: nyc.gov

      User-agent: *
Disallow: /html/misc/