[falsenapkin]

I'm not familiar with the summarization or NLP space really but I remember ~2011-2015 I signed up for a couple of daily email services that summarized a number of news articles and the summaries were fantastic. I don't even remember what they were called, they eventually sold out with ads and worse formatting/summaries to make money I guess. I often use them as an example of 1) why LLMs are a bit old news for the summary use case and 2) how various LLM use cases will probably also be ruined because for a lot of people tools like that seem novel and useful but all I can see is onboarding to more advertising.

So to someone who is actually knowledgeable in this space, are LLMs really that much better than whatever we had 10 years ago? Is this tech the key to some features we truly didn't have before?

[akiselev]

Not an NLP expert but the biggest difference in my experience is guided focus, so to speak. When summarizing something huge like the US Code, for example, you can tell the LLM to focus on specific topics and anything adjacent to them so that it ignores irrelevant details (which is usually >99.9% of the text in my use case). The word relationships encoded in the LLM are really good at identifying important adjacent topics and entities.

LLMs are also really good at the harder NLP problems like coreference resolution, dependency parsing, and relations which makes a huge difference when using recursive summarization on complex documents where something like "the Commisioner" might be defined at the beginning and used throughout a 100,000 token document. When instructed, the LLM can track the definitions in memory itself and even modify it live by calling OpenAI functions.

[falsenapkin]

Interesting so maybe not my trivial "summarize an article" example but clearly the upper bound on what's possible is higher and more interesting.

[dontupvoteme]

Might I ask how you use OpenAI's function calling here? That's the one bit of their functionality I haven't really explored.