top | item 30596566

(no title)

rezbull | 4 years ago

“…treat the absence of one as a lack of consent. ” - do you have a source for this?

Their documentation states otherwise https://developers.google.com/search/docs/advanced/robots/ro...

discuss

order

otterley|4 years ago

You are correct; my mistake. Nevertheless S3 returns a 403 (unauthorized) response for robots.txt by default which causes Google not to index it.