[logrot]

But surely if it's artificial intelligence then it'd know its limits and would respond appropriately? Oracle use no problem?

It is it because it's actually shit but it's the best thing we've seen yet and everyone is just in denial?

[mewpmewp2]

People constantly misevaluate their own limits though. Why should AI not be allowed to do that?

[Jensson]

Professionals don't constantly misevaluate their limits, if the AI is to replace a professional it has to know its limits.

[threeseed]

It depends on who you mean.

Most normal people look at AI like ChatGPT as an amazing tool and have used it effectively as a replacement for Google, Grammarly etc. And for them it's fine because any mistakes are localised to them.

The problem are those building products on LLMs e.g. Legal, Customer Service who are knowingly misrepresenting the capabilities of what it can do to companies who don't know any better. And I would argue this is fraudulent and where we will see most of the problems.