What is really mind blowing is that, if understood correctly, bots would be used to check the availability of a product, that sounds so a "hacky" method, like "seriously people are doing that in 2025".
Yeah, their list of recommendations could use another point: expose the public data in a simple, structured way.
I'm working right now on an inventory management system for a clinic which would really benefit from pulling the prices and availability from a very specialised online shop. I wish I could just get a large, fully cached status of all items in a json/CSV/whatever format. But they're extremely not interested, so I'm scraping the html from 50 separate categories instead. They'll get a few daily bot hits and neither of us will be happy about it.
If people are scraping data that you're not selling, they're not going to stop - just make it trivially accessible instead in a way that doesn't waste resources and destroy metrics.
The counterpoint is 'Why hand your competitors data on a silver plate'?
Sure you might be willing to build the bot to scrape it... but some other competitors won't go to this effort so it still means a bit of information asymmetry and stops some of your competitors poaching customers / employing various marketing tactics to exploit short term shortages or pricing charges etc.
I wonder if LLM agents will know to go for apis and data or if they'll keep naively scraping in the future. A lot of traffic could come down to "find me x product online" chats eventually
viraptor|4 months ago
I'm working right now on an inventory management system for a clinic which would really benefit from pulling the prices and availability from a very specialised online shop. I wish I could just get a large, fully cached status of all items in a json/CSV/whatever format. But they're extremely not interested, so I'm scraping the html from 50 separate categories instead. They'll get a few daily bot hits and neither of us will be happy about it.
If people are scraping data that you're not selling, they're not going to stop - just make it trivially accessible instead in a way that doesn't waste resources and destroy metrics.
Closi|4 months ago
Sure you might be willing to build the bot to scrape it... but some other competitors won't go to this effort so it still means a bit of information asymmetry and stops some of your competitors poaching customers / employing various marketing tactics to exploit short term shortages or pricing charges etc.
nemomarx|4 months ago
mschuster91|4 months ago
bdcravens|4 months ago