[IvanAchlaqullah]

At this rate, robots.txt will be ignored by bots if it's not already.

Just like ads company completely ignoring Do Not Track because it's always set to deny / opt-out by the browser. It's already been fixed, but the damages is already done.

https://en.wikipedia.org/wiki/Do_Not_Track#Internet_Explorer...

[whalesalad]

It’s ignored by many for years. SEM Rush and ironically Bing are an egregious violators of this in my experience.