Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

{p} {t}What is {h}Robots.txt Generator{/h}? {/t} {b}

Now, Create 'robots.txt' report at your root directory. copy above text and paste into the textdocument.

Robots.txt Generator generates a document this is very a great deal contrary of the sitemap which suggests the pages to be blanketed, consequently, robots.txt syntax is of top notch significance for any website. every time a search engine crawls any website, it constantly first looks for the robots.txt file that is located on the area root level. while recognized, crawler will read the file, after whichidentify the files and directories that can be blocked.{/b} {/p}