[alexeiz]

I had a related episode at work when my coworker asked me why his seemingly trivial 10 line piece of code was misbehaving inexplicably. It turned out he had two variables `file_name` and `filename` and used one in place of another. I asked him how he ended up with such code, he said he used copilot to create it. Using code from a generative AI without understanding what it does is never a good idea.

[delusional]

We hired a new guy at work. In one of his first tasks he had chosen to write some bash, and it was pure nonsense. I mean it contained things like:

if [ -z "${Var}+x" ]

Where I can see what the author was trying to do, but the code is just wrong.

I dont mind people not knowing stuff, especially when it's essentially Bash trivia. But what broke my heart was when I pointed out the problem, linked to the documentation, but recieved the response "I dont know what it means, I just used copilot" followed by him just removing the code.

What a waste of a learning opportunity.

[falcor84]

I agree that it's a waste of a learning opportunity, but from my experience it is still often rational.

There were many times in my career when I had what I expected to be a one-off issue that I needed a quick solution for and I would look for a quick and simple fix with a tool I'm unfamiliar with. I'd say that 70% of the time the thing "just works" well enough after testing, 10% of the time it doesn't quite work but I feel it's a promising approach and I'm motivated to learn more in order to get it to work, and in the remaining 20% of the time I discover that it's just significantly more complex than I thought it would be, and prefer to abandon the approach in favor of something else; I never regretted the latter.

I obviously lose a lot of learning opportunities this way, but I'm also sure I saved myself from going down many very deep rabbit holes. For example, I accepted that I'm not going to try and master sed&awk - if I see it doesn't work with a simple invocation, I drop into Python.

[woctordho]

Everyone working with shell scripts should know shellcheck

[OJFord]

And of a salary...

[steelframe]

Wait until a manager who's evaluating a technical decision you're making copies and pastes ChatGPT's "analysis" of your proposal and asks you to respond to it.

[rwmj]

And you didn't sack him?

[imtringued]

https://stackoverflow.com/a/13864829

It is not nonsense. You use that expression if you want to check if a variable exists or not (as opposed to being set to an empty string) which is an extremely common problem.

[Tainnor]

And any decent IDE will highlight a variable that is declared but unused. We already have "artificial intelligence" in the form of IDEs, linters, compilers, etc. but some people apparently think we should just throw it all away now that we have LLMs.

[tomrod]

No need for quotes, the best AI integrations are the ones you see as just part of the tech stack like spell check and linters.

[sa-code]

Unless you're using Python and said variable was meant to be reassigned, but you used a different name instead. E.g. file_name = 1 filename = 2

[frumper]

I knew a guy that made a good living as a freelance web developer decades ago. He would pretty much just copy and paste code from tutorials or stack overflow and had no real idea how anything worked. Using code without understanding it is never a good idea, it doesn’t need to be from AI for that to be true.

[f6v]

Or maybe you’re just exaggerating. I’ve done my fair share of copy pasting and it never worked to just do it without understanding what’s going on.

I think the problem with “AI” code is that many people have almost a religions belief. There’re weirdos on internet who say that AGI is couple years away. And by extension current AI models are seen as something incapable of making a mistake when writing code.

[jazz9k]

I knew someone similar. They would just get free templates and sell them as a website to customers, with almost no changes, aside from logos and text. Most had no Javascript or css and looked terrible, even by 2005 standards.

His clients were usually older small business owners that just wanted a web presence. His rate was $5000/site.

Within a few years, business dried up and he had to do something completely different.

He also hosted his own smtp server for clients.It was an old server on his cable modem in a dusty garage. I helped him prevent spoofing/relaying a few times, but he kept tinkering with the settings and it would happen all over again.

[Laakeri]

But he made a good living out of it, so in the end it was a good idea?

[falcor84]

>a guy that made a good living ... never a good idea

Arguably the term for a bad idea that works is "good idea"

[Cthulhu_]

This is a known issue from like the 2000s where there were so many bad PHP tutorials, a lot of SQL injection and XSS etc came from those.

[JKCalhoun]

At least AI comments their code.

[yawnxyz]

Claude gave me something similar, except these were both used, and somehow global variables, and it got confused about when to use which one.

Asking it to refactor / fix it made it worse bc it'd get confused, and merge them into a single variable — the problem was they had slightly different uses, which broke everything

I had to step through the code line by line to fix it.

Using Claude's still faster for me, as it'd probably take a week for me to write the code in the first place.

BUT there's a lot of traps like this hidden everywhere probably, and those will rear their ugly heads at some point. Wish there was a good test generation tool to go with the code generation tool...

[danenania]

One thing I've found in doing a lot of coding with LLMs is that you're often better off updating the initial prompt and starting fresh rather than asking for fixes.

Having mistakes in context seems to 'contaminate' the results and you keep getting more problems even when you're specifically asking for a fix.

It does make some sense as LLMs are generally known to respond much better to positive examples than negative examples. If an LLM sees the wrong way, it can't help being influenced by it, even if your prompt says very sternly not to do it that way. So you're usually better off re-framing what you want in positive terms.

I actually built an AI coding tool to help enable the workflow of backing up and re-prompting: https://github.com/plandex-ai/plandex

[davidthewatson]

I'd refer you to a comment I made a few weeks ago on an HN post, to the same effect, which drew the further comment from gwern here:

https://news.ycombinator.com/item?id=40922090

LSS: metaprogramming tests is not trivial but straightforward, given that you can see the code, the AST, and associated metadata, such as generating test input. I've done it myself, more than a decade ago.

I've referred to this as a mix of literate programming (noting the traps you referred to and the anachronistic quality of them relative to both the generated tests and their generated tested code) wrapped up in human-computer sensemaking given the fact that what the AI sees is often at best a lack in its symbolic representation that is imaginary, not real; thus, requiring iterative correction to hit its user's target, just like a real test team interacting with a dev team.

In my estimation, it's actually harder to explain than it is to do.

[Glyptodon]

At least for me stupid bugs like this turn out to be some of the most time wasting to debug, no AI involved. Like accidentally have something quoted somewhere, or add an 's' to a variable by accident and I may not even correctly process what the error message is reporting at first. Always feel a bit silly after.

[Noumenon72]

These kinds of problems are often avoidable by linters or asking ChatGPT what is wrong, though I was just tearing my hair wondering why TSC_COMPILE_ERROR wasn't skipping TypeScript because I spelled it TSX_COMPILE_ERROR in my environment variable.

[arcticfox]

This type of bug is trivial for GPT to fix though. It was born for this. Sometimes it does generate real footguns but this sounds like an example from an earlier generation of generative AI.

[ben_w]

> Using code from a generative AI without understanding what it does is never a good idea.

True, but the anecdote doesn't prove the point.

It's easy to miss that kind of difference even if you wrote the code yourself.

[latexr]

> It's easy to miss that kind of difference even if you wrote the code yourself.

The developer in the story had no idea what the code did, hence they would not have written it themselves, making it impossible for them to “miss” anything.

[morgango]

Interestingly, a great application for GenAI is to copy and paste code and ask it, "Why is this not working?". It works even better if you give it the specific error you are getting (and it is a well understood system).

[bckr]

> Using code from a generative AI without understanding what it does is never a good idea.

Yes.

AI as a faster way to type: Great!

AI as a way to discover capabilities: OK.

Faster way to think and solve problems: Actively harmful.

[berniedurfee]

I burned a couple hours debugging some generated code only to finally realize copilot was referencing a variable as ‘variableO1’.

Artificial Incompetence indeed!

[planb]

In my experience this is exactly the kind of mistake an AI would not make.

[mooreds]

> Using code from a generative AI without understanding what it does is never a good idea.

Hear hear!

I feel like genAI is turning devs from authors to editors. Anyone who thinks the latter is lesser than the former has not performed both functions. Editing properly, to elevate the meaning of the author, is a worthy and difficult endeavor.

[EVa5I7bHFq9mnYK]

Sounds like javascript "code". A normal language with proper type system would not allow that.

[CalRobert]

I don’t see how typing relates to this.

[FanaHOVA]

Copilot wouldn't make a typo. He just made that up and / or broke the code himself.

[creesch]

LOL, are you kidding me? LLMs including the gpt family copilot uses are very good at sneaking in these sorts of thing.

[nope1000]

I don't know about copilot but I've seen typos from ChatGPT (although it was english, not code)

[fragmede]

would someone invent that and bother the author with that? I mean I suppose it's possible, but that seems like such a waste of time to me that I find that more unlikely. and while it's a typo, it's not fleinaem or something that's totally wrong, just a choice in breaking up the word filename. having written file handling code, the various permutations of filename and path and dirname get to be a bit much sometimes.

[boredhedgehog]

What likely happened is that he asked the AI two separate questions and fused the answers himself.

[brigadier132]

You are getting downvoted but you are right, a typo in a variable that already exists in a file like this is not the failure mode for LLMs. The failure mode is logic bugs, making up methods / functions.