top | item 39240991

(no title)

pragmar | 2 years ago

Congratulations on shipping, the site looks great. I've been working on a desktop app in the same space (website health/administration). It's been a similarly long journey. I've found marketing to be an uphill battle. It's a crowded marketplace, which in many ways indicates a healthy ecosystem, but difficult to crack nonetheless. Having a focused audience (e.g. bloggers) will help there. If you ever want to discuss marketing or talk shop, definitely reach out (ben at hn-username dot com)... maybe we could both learn something.

discuss

order

brunosutic|2 years ago

Quick question: how do you handle bot protection (eg Cloudflare) with your app?

pragmar|2 years ago

I'm not sure I've the best antibot solution, but it's handled through some crawler options exposed to the user. Out of the box, I use a (fast) http crawler with my app's user-agent. It is not at all resilient to antibot. I direct users who are encountering issues to first try a user-agent override, and if that doesn't work, to next enable javascript crawling (think headless chrome), which is slower and heavier, but clears up a lot of issues. I don't have a strategy for aggressive antibot (captcha/etc.) other than to tell the customer to dial it back on their website.

Edit: I've seen antibot SAAS providers, which claim to provide workarounds at a cost. You route traffic through their network, and they have teams that are constantly tweaking things to keep the requests working, much like scrapers adapting to website redesigns. It would be a treadmill to do on your own. There more info on this at https://substack.thewebscraping.club/ In my case, selling single-user perpetual licenses, it doesn't make sense.