Fix Your robots.txt or Your Site Disappears from Google
January 2026
TL;DR
Your site will be removed from Google search results if you don't have a robots.txt file or the Googlebot site crawler can't access it.
Here's the video from Google Support that covers it:
Wait, what?
Adam Coster ran into a weird issue with site traffic and posted about it in the Shop Talk Show discord. Traffic incoming from Google looked like this:
The issues seemed to be that Google wouldn't index the site without a robots.txt file.
My first reaction: No fucking way.
I can't...
Read more at alanwsmith.com