[suresk]

GAR - Generation-Augmented Retrieval?

I've actually had some success with getting ChatGPT to create Redshift queries based on user text and then I can run them and render results, which has some interesting use-cases.

Max calls out the biggest problem with using something like ChatGPT in a search flow - it is way too slow. I've talked to a lot of people wondering if we can just shove a catalog at ChatGPT and have it magically do a really good job of search, and token limits + latency are two pretty hard stops there (plus I think it would be generally a worse experience in many cases).

What I'm trying to look at now is how LLMs can be used to make documents better suited for search by pulling out useful metadata, summarizing related content, etc. Things that can be done at index time instead of search time, so the latency requirements are less of an issue.