Robots.txt
Definition
A file used by websites to instruct search engine crawlers which pages or sections of the site they are allowed or disallowed to index, helping control which content appears in search results.
Translation →
Translation
The digital bouncer that tells search engines, “You can come in here, but stay out of the VIP rooms,” making sure they don’t snoop around where they’re not welcome.