[a5withtrrs]

Not being able to do simple math is a surprisingly huge bug IMO

Me: 5000 * 5000 * 5000 ChatGPT: The result of 5000 x 5000 x 5000 is 125,000,000,000,000. Me: 5000 * 5000 ChatGPT: The result of 5000 x 5000 is 25,000,000. Me: 5000 * 25000000 ChatGPT: The result of 25,000,000 x 5000 is 125,000,000,000.

Integrations with services like wolfram alpha would be nice as well (although shouldn't be necessary to solve basic multiplication)

[p-e-w]

It's not a "bug" in the traditional sense, but a consequence of how language models work. Probability distributions over token chains are very powerful tools for generating human-like written text, but they perform poorly at symbolic inference. Humans have the same limitations, which is why math is a separate topic taught in schools that focuses not on knowledge but on algorithms. Understanding natural language syntax and semantics doesn't necessarily enable you to calculate.

[a5withtrrs]

Perhaps I should have added how I noticed in the first place. I gave it a block of code with a few nested loops and asked it how many times would it run. For a small number 5 x 5 x 5, it calculated correctly. But for 5000 x 5000 x 5000 it gave the wrong answer.

I'd like to see an integration into wolfram alpha or something. That would be amazing.

[zemo]

if you ask it a question and it gives you a confident answer that's 100% wrong that's a terrible experience and a non-working product. "it's not a bug in the traditional sense" is technically correct but missing the point entirely.

[Chico75]

and that's after the recent improvements

[city17]

I noticed the announcement in the latest update that it was supposedly better at math, but the first simple calculations I gave it were all wrong. Not sure how it was improved at all.