top | item 39886418

(no title)

kwinkunks | 1 year ago

I've been disappointed by my few experiments with Langchain's agent tooling. Things I have experienced:

- The pythonrepl or llm-math agent not being used when it should be and the agent returning a wrong or approximate answer.

- The wikipedia and webbrowsed agents doing spurious research in an attempt to answer a question I did not ask (hallucinating a question, essentially).

- Agents getting stuck in a loop of asking the same question over and over until they time out.

- The model not believing an answer it gets from an agent (eg using a Python function to get today's date and not believing the answer because "The date is in the future").

When you layer all this on top of the usual challenges of writing prompts (plus, with Python function, writing the docstring so the agent knows when to call it), wrong answers, hallucination, etc, etc, I'm unconvinced. But maybe I'm doing it wrong!

discuss

order

No comments yet.