[tuhgdetzhh]

I still think it is just a matter of time until scrapers catch up. There are more and more scrapers that spin up an full blown chromium.

[kstrauser]

It seems inevitable, but in the mean time, that's vastly more expensive than running curl in a loop. In fact, it may be expensive enough that it cuts bot traffic down to a level I no longer care about defending against. Like GoogleBot had been crawling my stuff for years without breaking the site. If every bot were like that, I wouldn't care.

[raw_anon_1111]

Serious question, in 2026 you can actually have a successful crawler with just curl? I just had to create one for a customer - for their own site - and nothing would have worked without using Chromium.

[solid_fuel]

Cool, if they're running full blown chromium maybe the next step can be mining bitcoin on any pages served to bots.

[hxtk]

Even that functions as a sort of proof of work, requiring a commitment of compute resources that is table stakes for individual users but multiplies the cost of making millions of requests.

[gruez]

AFAIK you can bypass it with curl because there's an explicit whitelist for it, no need for a headful browser.

[cantalopes]

Well it's a race, just like security. And as long as anubis is in the front, all looks bright