top | item 43478145

(no title)

ANarrativeApe | 11 months ago

Excuse my ignorance, but is it time to update the open source licenses in the light of this behavior? If so, what should the evolved license wording be?

I appreciate that this could be easily circumvented by a 'bad actor', but it would make this abuse overt...

discuss

order

johnnyanmac|11 months ago

From my little understanding, we have a sort of agreement in place with an item called robot.txt that's more or less a hanshake with such scrapers. Of course, the issue is these scrapers are blatantly ignoring robots.txt

A license can help as well, but what's a license without enforcement? These companies are simply treating the courts as a cost to do business.

CaptainFever|11 months ago

Close, robots.txt was originally for web crawlers, to reduce accidental denial-of-service attacks. It had nothing to do with the scraping (i.e. downloading content and parsing the HTML tags in a programmatic manner).