Robots.txt Generator

Search Engine Optimization

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Robots.txt is a file that can be set in the root organizer of your site to assist with looking through motors file your site all the more properly. Web indexes, for example, Google use site crawlers, or robots that survey all the substance on your site. There might be portions of your site that you don't maintain that they should creep to remember for client indexed lists, for example, administrator page. You can add these pages to the document to be unequivocally overlooked. Robots.txt records use something many refer to as the Robots Exclusion Protocol. This site will effectively produce the record for you with contributions of pages to be avoided.