top | item 41597383

(no title)

HeralFacker | 1 year ago

What external checks are included to verify the chatbot output?

discuss

order

SoulAuctioneer|1 year ago

Wherever possible, the chatbot output is deterministic, in that to answer a query, we're realtime generating and running code or SQL against your data. Our LLM orchestrates that, and finally evaluates whether the output correctly and adequately answers the question.

We also extensively use synthetic data and examples to guide and constrain our models.

Another way we're ensuring good-quality output is to ensure good-quality _input_ -- by enriching the detail and specificity of the user's question, and asking the user to disambiguate when we determine the question is too broad.

HeralFacker|1 year ago

None of that is external verification. You're using the generating tool to do the verification, leaving the door open for opaque errors.