top | item 45015402

(no title)

gabeio | 6 months ago

> It's also a page that's never visited by humans.

Never is a strong word. I have definitely visited robots.txt of various websites for a variety of random reasons.

  - remembering the format
  - seeing what they might have tried to "hide"
  - using it like a site's directory
  - testing if the website is working if their main dashboard/index is offline

discuss

order

sdenton4|6 months ago

Are you sure you are human?

gspencley|6 months ago

Yes. I have checked many checkboxes that say "Verify You Are a Human" and they have always confirmed that I am.

In fairness, however, my daughters ask me that question all the time and it is possible that the verification checkboxes are lying to me as part of some grand conspiracy to make me think I am a human when I am not.