[kanzure]

Instead of directly scraping with GPT-4o, what you could do is have GPT-4o write a script for a simple web scraper and then use a prompt-loop when something breaks or goes wrong.

I have the same opinion about a man and his animals crossing a river on a boat. Instead of spending tokens on trying to solve a word problem, have it create a constraint solver and then run that. Same thing.