[RobAley]

> The hallucination issue can be worked around by providing that demonstrates the agent's working (i.e. what tools they called with what parameters).

And this is (in my opinion) an intractable problem - You can get the AI to list the tools/parameters it used, but then you can't be sure that it hasn't just hallucinated parts of that list as well, unless you both understand that they were the right tools and right parameters to use, and run them yourself to verify the output. And at that point you might as well just have done it yourself in the first place.

I.e. if you can't trust the AI, you can't trust the AI to tell you why you should trust the AI.

[MoreQARespect]

It's intractable unless the problem space demands 100% correctness at all times.

When Im using observability apps I dont demand correctness, I'm very happy if the LLM came up with 3 hypotheses about what happened and I could discard 2 of them by reading its working.