[gkapur]

Very cool! Function calling seems to be a new paradigm that is really taking off.

Does anyone know how the LLM vendors are actually implementing function calling? Is it just a thoughtful prompt and a loop where they parse outputs and check if it corresponds to the arguments of the function? Or something else?

[bediashpreet]

Thanks, yes we're seeing a lot of use cases being unlocked by function calling.

To answer you first question, function calling models are trained to detect when a function should to be called (depending on the input) and to respond with JSON that adheres to the function signature.

A thoughtful prompt and a loop where they parse outputs and check if it corresponds to the arguments of the function is the ReAct model of building agents, which I've found starts to deteriorate in quality after about 5 tools.

Whereas with function calling, because the LLM is trained to detect/call functions it can work very well up-to 30-40 functions -- after which we need to start using techniques like delegation.