[thorax]

> It returns factually incorrect data, and it returns code with subtle but important errors if you ask it anything that's not regurgitated a thousand times in the training dataset.

To be fair, that's what pretty much every person does. The bar does seem pretty high if we need more than that (especially if not specifically trained on a topic). It's not a universally perfect expert servant, but I've been exploring the code generation of GPT4 in detail (i.e. via the 'cataclysm' module I just posted about). In 1 minute it can write functions as good as the average developer intern most of the time.

We're keeping score in a weird way if we're responding quickly with it needing to "code without subtle but important errors". Because that's the majority of human developers, too. I've been writing code for 30 years, and if you put a gun to my head, I would still have subtle but important flaws in every first typing of any complex generated code.

I'm not saying you're bashing it, by the way, I get your point, but I do worry a bit when the first response is citing that the SOTA models get things wrong in 0-shot situations without full context. That's describing all of us.

[taberiand]

I always say, if some nontrivial code compiles and runs on the first attempt, then you just haven't found the bugs yet.

GPT-4 is a fantastic collaboration tool for senior developers, who know what they want in detail and can review, verify and apply the output it generates.

Just yesterday I needed to write some detailed bash scripts. I'm no Linux guru but I know what I want and that was enough - in minutes I had a solid script that did everything I needed and wanted, something that would have taken much longer to hunt down through Google. And then I asked it about SQL, C#, AWS, Terraform, Rust and on and on and everything was high quality.

The only way I could have gained similar results without ChatGPT would have been to post all my questions to the dev slack channel and engage in hours long discussions with my colleagues.

[corbulo]

The standard should be higher for machines. Historically, they're inorganic tools.

If a hammer was only as dense as bone, would it be a good hammer?

[stale2002]

> would it be a good hammer?

Yes, it is possible that such a hammer would still be very useful, if that was the only available option.

Imperfect hammers can still be very very useful, when compared to having no hammers.

[mdp2021]

> that's what pretty much every person does

But the people who are relevant to the matter work to bring that beyond a minimum.

> and if you put a gun to my head

Like we do with bullshitters? ;)