top | item 45359888

(no title)

fibers | 5 months ago

you have to feed it multiple arguments with rate limiting and long wait times. i am not sure if there have been recent updates other than the js interpreter but ive had to spin up a docker instance of a browser to feed it session cookies as well.

discuss

order

m_ke|5 months ago

Yeah we had to roll through a bunch of proxy servers on top of all the other tricks you mentioned to reliably download at a decent pace

trenchpilgrim|5 months ago

What are your thoughts on the load scrapers are putting on website operators?