THE DEFINITIVE GUIDE TO ROBOT

If you wish to block your site from search results, use another strategy such as password security or noindex. Should your Online page is blocked that has a robots.txt file, its URL can nonetheless look in search engine results, although the search end result will likely not have a description. Image documents, movie files, PDFs, and also other non