top | item 37032008

(no title)

dmitrysergeyev | 2 years ago

They could in theory combat it by comparing results with a second crawler that uses a different User Agent.

discuss

order

mdaniel|2 years ago

If they were going to the amount of energy to do a 2nd crawl using a different user agent, then why bother advertising the user agent at all and just feed it the Chrome one like every other home-grown spider does