(no title)
thebricklayr | 2 years ago
At the moment, Umami's scraping is done server-side. I'd really like to speed it up, so I'm going to start working on a 100% client scraper soon (for the native apps & browser extensions; the web version will always have cross-domain browser restrictions though).
samldev|2 years ago
thebricklayr|2 years ago
scary-size|2 years ago
thebricklayr|2 years ago
Yep, the client just sends the recipe URL and then the server fetches the HTML. Agreed, it would be better to have the client send the HTML for paywalled sites, which is another reason I just want to do it all on the client.
> I spent quite some time on ingredient labelling (what’s a unit, quantity etc.)
I can relate to you there. It was a long process of trial and error for me to get right, and there are still plenty of edge cases left to handle. Long-term I think AI + NLP will make this kind of thing easier, but for me it wasn't fast, reliable, cheap, or portable enough to run in an iOS app in real time quite yet.