[csydas]

You are correct it's poor and sloppy, but it's not "just" that. It's a lack of concern over the effects of their poor/sloppy crawler implementation.

The poor implementation is not really relevant, it's companies deciding they own the internet and can take whatever they want, let everyone else deal with the consequences. The companies do not care what the impact of their ai non-sense is..

[fuzztester]

https://news.ycombinator.com/item?id=43429532

[Forbo]

A capitalist externalizing costs?! Why, no, never! /s