DantesTravel's comments

DantesTravel | 3 years ago | on: Ask HN: Have you ever used anti detect browsers for web scraping?

The browser in most case has a client for windows

DantesTravel | 3 years ago | on: Ask HN: Have you ever used anti detect browsers for web scraping?

Some posts are good, more technical, others not. But in my opinion is worth reading.

DantesTravel | 3 years ago | on: Ask HN: Have you ever used anti detect browsers for web scraping?

Yes, that’s what I’ve done up to now. When forced to use Playwright, I’ve noticed too that Webkit is less detected, but depends from website to website. I tried the solution described on the substack, fundamentally the gologin browser, based on chromium, opens a port on your local machine and Playwright connects to that browser, automating the crawling.

DantesTravel | 3 years ago | on: Ask HN: Have you ever used anti detect browsers for web scraping?

In the substack post are mentioned several browsers, so I would say no. I didn’t even mention the Substack so no one can say I’m promoting it.

DantesTravel | 3 years ago | on: Ask HN: Have you ever used anti detect browsers for web scraping?

I didn’t mention because someone would think I’m promoting it: it’s called The Web Scraping Club.

DantesTravel | 3 years ago | on: Ask HN: How do I scrape my tweets without using Twitter's own tool?

I've recently discovered on a substack about web scraping the existence of anti detect browsers. Using Selenium or Playwright you can automate these browsers, each can have hundreds of isolated profiles, with different fingerprints and cookies. If you add also a proxy layer, you should be ok.