[CityOfThrowaway]

They tested one specific agent implementation that they themselves made, and made sweeping claims about LLM agents.

[jrflowers]

This makes sense. The CRM company made a CRM agent to do CRM tasks and it did poorly. The lesson to be learned here is that attempting to leverage institutional knowledge to make a language model do something useful is a mistake, when the obvious solution for LLM agents is to simply make them more gooder, which must be trivial since I can picture them being very good in my mind.