Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

 

Robots.txt Generator tool is a free online tool that helps you to easily create your robots.txt file.

 

This tool allows you to specify which directories and files you want search engine crawlers to index and follow, and which ones you would like to keep hidden from their view.

 

It is easy to use – simply enter the URL of the website you want to create a robots.txt file

 

This tool can help you create a robots.txt file for your website in minutes, without any coding knowledge required.

 

Robots.txt is a file that tells search engine crawlers what to do with the content on your site. It is important to have a robots.txt file because it helps you control how search engines crawl and index your site.