[throwthrowrow]

Question #1 was on the model's ability to handle arithmetic. The answer to question seems to be unrelated, at least to me: "you can enable or disable tool use in most APIs".

The original question still stands: do recent LLMs have an inherent knowledge of arithmetic, or do they have to offload the calculation to some other non-LLM system?

[ACCount37]

The knowledge was never the bottleneck for that, not since the days of GPT-3. The ability to execute on it was.

Which includes, among other things, the underappreciated metacognitive skill of "being able to decide when to do math quick and dirty, in one forward pass, and when to write it out explicitly and solve it step by step".

Today's frontier LLMs can do that. A lot of training for "reasoning" is just training for "execute on your knowledge reliably". They usually can solve math problems with no tool calls. But they will tool call for more complex math when given an option to.

[Terr_]

Some nice charts here [0], which IMO means LLMs are getting very good at guessing answers to certain arithmetic operations, but they don't actually perform it in a logical fashion.

[0] https://www.mindprison.cc/p/why-llms-dont-ask-for-calculator...