(no title)
foob | 2 years ago
I know that web scrapers carry some negative connotations, but keep in mind that search engines like Google couldn't possibly exist without web scraping. A world where you can't block ads or scrape content for indexing is a world where only a few preordained companies have the ability to build search engines. Proposals like Web Environment Integrity (WEI) accomplish two goals for Google: they make ad blocking more difficult, and they kick down the ladder to prevent new innovative search engines from emerging. There are already many websites which only allow-list Google's IPs for indexing, and I think we should be very hesitant about anything that could further entrench their monopoly on search even if we support content creators being compensated through ads.
Bu9818|2 years ago