[Ian_Kerins]

Some web scraping can be unethical, say for example if you are scraping a site solely to mirror their content and add zero value to the original content owner.

However, there are a lot of web scraping use cases which are beneficial to the site being scraped and actually add value. Two examples:

- Google: Ahrefs & SEMRush scrape Google so they can provide SEO analytics to companies looking to grow their companies. Googles keyword analytics aren't great, so Google has effectively outsourced providing a good analytics tool to Ahrefs & SEMRush who products increase the value of the Google SERPs ecosystem.

- Amazon + Other E-Commerce: Amazon wants brands and 3rd party stores to list products on their site, and the companies scraping Amazon to provide product placement tools to their users make it easier and more profitable to list products on Amazon. Leading to more and more companies listing products on Amazon.

[megous]

> Some web scraping can be unethical, say for example if you are scraping a site solely to mirror their content and add zero value to the original content owner.

Archiving is unethical?

[Ian_Kerins]

Good point, wouldn't say archiving is unethical at all...I was thinking more along the lines of someone scraping a entire segment of a websites data and reproducing it 1 for 1 on their own site with zero value add.

I think we can't make broad statements saying that web scraping is ethical or unethical, it isn't that black or white. It really depends on what is being scraped, how is it being used, and the intention of the scraper.