robots.txt

What is a Robots.txt File?

A robots.txt file is a simple text file that you can create to tell search engines how you want to…