top | item 46324362

(no title)

drbscl | 2 months ago

> So fuzzycanary also checks user agents

I wouldn't be so surprised if they often fake user agents to be honest. Sure, it 'll stop the "more honest" ones (but then, actual honest scrapers would respect robots.txt)

Cool idea though!

discuss

order

No comments yet.