You want that some of your Web pages should not be crawled. Which one of the following options will you use to accomplish the task?
Use HTML NO Crawl tag in the Web page not to be crawled
Place the name of restricted Web pages in the private.txt file
Place the name of restricted Web pages in the robotes.txt file
Enable the SSL
Submit