[mattyyeung]

Can quotations be hallucinated? Or are you using something like "deterministic quoting"[1]?

Disclosure: author on that work.

[1] https://mattyyeung.github.io/deterministic-quoting

[keefle]

That's really cool, do you think this might be the basis for potential natural language navigation? (when going over a document, instead of having to search by keyword or regex, one can search for more complicated concepts using English)

If not, what extra work is needed to bring it to that level?

[mattyyeung]

I think you could get a pretty good solution for that using RAG and some tricks with prompt engineering and semantic chunking. With google's very-long-context models (Gemini) you may also have good results simply with some prompt engineering. Preprocessing steps like asking the LLM to summarise themes of each section can be helpful too (in RAG, this info would go in the 'metadata' stored with each chunk, presented to the LLM with each chunk).

A key engineering challenge will be speed ... when you're navigating a document you want a fast response time.

[brokensegue]

the quote cannot be hallucinated but we use a different approach. your work seems interesting though.

[mattyyeung]

I would love to learn more, where would you recommend I look?