Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.
What is Robot.txt?
Robot.txt or Robot Text is the first thing search engines look for when they crawl your website. Once they find the robots.txt, it will check the list of files and directories.
Help Us Improve Our Services