You built a cool product. I'm actually one of the founders of https://medisearch.io which is similar to what you are building. I think the long-tail problem that you describe can be solved in other ways than with live APIs and you may find other problems with using live APIs.
amber_raza|2 months ago
You are definitely right that Live APIs come with their own headaches (mostly latency and rate limits).
For now, I chose this path to avoid the infrastructure overhead of maintaining a massive fresh index as a solo dev. However, I suspect that as usage grows, I will have to move toward a hybrid model where I cache or index the 'head' of the query distribution to improve performance.
Always great to meet others tackling this space. I’d love to swap notes sometime if you are open to it.