[sasaf5]

Early 2023, when everyone started using chat GPT for coding, I thought it would be a big boost because it enabled us to quickly employ a large number of people in a project regardless of language or framework.

An year into the project I am forced to revise my opinion. When browsing my code-base I often stumble in abstruse niche solutions for problems that should not have existed. It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy.

Having an oracle that knows all answers is useless if you don't know what to ask.

[AlwaysRock]

> It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy.

Isnt this what code reviews are for? I catch a decent amount of code that looks AI generated. Typically, some very foreign pattern or syntax that this engineers never used nor is common in the codebase. Or something weirdly obtuse that could be refactored and shows a lack of understanding.

Normally I ask something like, "Interesting approach! Is there a reason to do it this way over (mention a similar pattern in our codebase)?" or if it's egregious, I might ask, "Can you explain this to me?".

This feel similar to early career engineers copy pasting stack overflow code. Now its just faster and easier for them to do. It's still fairly easy to spot though.

[steve1977]

> Isnt this what code reviews are for?

Wouldn't you need to have people with a proper understanding of the programming language and framework to do the code reviews?

[vouaobrasil]

> Isnt this what code reviews are for?

There is no substitute to doing something correctly in the first place. The problem is that in the real world, deadlines and lack of time will always cause the default solution to be accepted a small percentage of time even when it is not ideal. The increasing creep of AI will only exacerbate that and most technophiles will default into thinking of a new and improved AI tool to help with the problem, until it will be AI tools all the way down.

No thanks.

[ungreased0675]

>Isnt this what code reviews are for?

A foundational concept of quality control is to not rely on inspection to catch production defects. Why not? It diffuses responsibility, lets more problems get to the customer and is less efficient than doing it correctly to start with.

[arrowsmith]

> It's still fairly easy to spot though.

Or maybe it's far more common than you realise, and you're only spotting the obvious ones.

[bcrosby95]

It's harder to read code than write it. It's even harder to read code than use an LLM to write it for you.

[adastra22]

Not all organizations can afford code reviews.

[debarshri]

I recently was stumbled upon a code change done by a colleague who just followed the suggestion as is from Copilot, where it recommend `func (string, string) (error, struct)` whereas everywhere in the code we use `func (string, string) (struct, error)`

When I asked him what prompted him to do that, he said copilot suggest it so I just followed. I wonder if you could hijack copilot's results and inject malicious code as many end users does not understand lot of the niche code it generates sometimes, you could manipulate them to add the malicious code to the org's codebase.

[vintermann]

The insidious thing is that quite possibly, it "feels" there should be a couple of bugs like that because all the codebases it was trained on had a few.

It might even take the context of the typos in your code comments, and conclude "yeah, this easy to miss subtle error feels right about here".

[ben_w]

It's definitely possible to inject malicious code that humans don't spot, there was a whole competition dedicated to humans doing this in C well before LLMs: https://en.wikipedia.org/wiki/Underhanded_C_Contest

Now I'm wondering, can you put in a comment which the LLM will pay attention to such that it generates subtle back-doors? And can this comment be such that humans looking at the code don't realise this behaviour will be due to the comment?

[AshamedCaptain]

Almost two decades ago, I saw a junior colleague (this was Java) try to add "static" to almost all of one specific class' methods and members, i.e. making them global instead of attached to each class instance . Obviously this completely broke the software, albeit it did build. When questioned during review, my colleague just shrugged and said "Because Eclipse suggested to do so".

Apparently, if you tried to access a class member without specifying a class instance, one of Eclipse's "auto-fix-it" suggestions was to make all members of that class static, and he just followed that suggestion blindly.

[stouset]

> A year into the project I am forced to revise my opinion. When browsing my code-base I often stumble in abstruse niche solutions for problems that should not have existed.

This is a widespread problem regardless of AI. Hence the myriad Stack Overflow users who are frustrated after asking insane questions and getting pushback, who then dig their heels in after being told the entire approach they’re using to solve a problem is bonkers and they’re going to run into endless problems continuing down the path their on.

Not that people aren’t on too fine a hair trigger for that kind of response. But the sensitivity of that reaction is a learned defense mechanism for the sheer volume of it.

[nottorp]

> Hence the myriad Stack Overflow users who are frustrated after asking insane questions and getting pushback, who then dig their heels in after being told the entire approach they’re using to solve a problem is bonkers and they’re going to run into endless problems continuing down the path their on.

The problem is, SO can't tell someone who asks an insane question from someone who asks the same question but has constraints that make it sane. *

So in time, sane people during unusual stuff stop asking questions and you're left with homework.

* For example, "we can't afford to refactor the whole codebase because some architecture astronaut on SO says so" is a constraint.

Or another nice one is "this is not and will never be a project that will handle google-like volumes of data".

[jrockway]

Remembering more than 4096 tokens is my superpower.

[ein0p]

How ‘bout remembering a million tokens? I’m not feeling too confident about that. Basically my only moat, if there is one, is that I’m able to rely on higher level cognition which LLMs don’t yet have, rather than just on associative memory alone.

[ben_w]

I may remember more than 4096 tokens, but I probably only pay attention to 7 of them at any given moment: https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus...

[pwdisswordfishc]

No, you don’t. Seven, plus or minus two at best.

[jongjong]

>> Having an oracle that knows all answers is useless if you don't know what to ask.

That is a great point. The issue of not asking the right questions has been around as far as I can remember but I guess it wasn't seen as the bottleneck because people were so focused on solving problems by any means possible that they never had to think about solving problems in a simple way. We're still very far from that though and in some ways we have taken steps back. I hope AI will help to shift human focus towards code architecture because that's something that has been severely neglected. Most complex projects I've seen are severely over-engineered... They are complex but they should not have grown to hundreds of thousands of lines of code; had people asked the right questions, focused on the right problems and chosen the right trade-offs, they would have been under 10K lines and way more efficient, interoperable and reliable.

I should note though, that my experience with coding with AI is that it often makes mistakes for complex algorithms, or it implements them in an inefficient way and I almost always have to change them. I get a lot of benefit from asking questions about APIs and to verify my assumptions or if I need a suggestion about possible approaches to do something.

[osigurdson]

Agree. I use AI less now for coding than I did a year ago. I still use it a lot but it is a better teacher than a doer.

[crazygringo]

How do you know it was "AI-fuelled"? And what makes it a "frenzy"?

People have been committing terrible code to projects for decades now, long before AI.

The solution is a code review process that works, and accountability if experienced employees are approving commits without properly reviewing them.

AI shouldn't have anything to do with it. Bad code shouldn't be passing review period, no matter if it was AI-assisted or not. And if your org doesn't do code review, then that's the actual problem.

[eastbound]

> and accountability if experienced employees are approving commits without properly reviewing them.

You’re putting the entire responsibility on senior employees. So we need much more of them. In fact, we don’t need juniors, because we can generate all possible code combinations. After all, it’s the responsibility of the seniors to select which one is correct.

It’s like how hiring was made crap by the “One-click apply” on LinkedIn and all other platforms. Sure it’s easy for thousands of people to apply. Fact is, we offer quite a good job with high salary, and were looking for 5 people. We’ve spent a full year selecting them, because we’ve receive hundreds of irrelevant applications, probably some AI-generated.

It’s no use to flood a filter with crap, hoping that the filter will do better work because it has a lot of input.

[Incipient]

"give me a function to return the top 10 users" - copy paste "give me another to email a template to a list of users" - copy paste

AI makes it much easier to push out bad code, fast...in a "frenzied" way one could say.

[xanderlewis]

This is almost a special case of the very similar (and common) argument that ‘AI-generated misinformation isn’t bad; we’ve always had misinformation’.

The answer is also the same.

Volume. AI makes it trivially easy to generate vast amounts of it that don’t betray their lack of coherence easily. As with much AI content, it creates arbitrary amounts of work for humans to have to sift through in order to know it’s right. And it gives confidence to those who don’t know very much to then start polluting the informationsphere with endless amounts of codswallop.

[taneq]

> Having an oracle that knows all answers is useless if you don't know what to ask.

Honestly I find this to be the biggest advantage to using a coding LLM. It's like a more interactive debugging duck. By the time I've described my problem in sufficient detail for the LLM to generate a useful answer, I've solved it.

[ramones13]

I’ve had a project I’ve been doing for ~6 months learning python through Copilot+ChatGPT. It feels like any other project that accrues technical debt, but that debt is just a lot weirder. It was rough at first, but refactoring has gotten a lot better recently with bigger context sizes.

[gtirloni]

If you need Copilot to code in Python, have you really learned Python?

[antfarm]

Learning to be a good programmer is as much about learning how to avoid technical dept as it is about learning to use a programming language. It may take a while until an AI assistant will be able to help with that.

[1-6]

I think humans are excellent at finding fault. It’s easier to deconstruct a problem than it is to reconstruct one from scratch.

Having an oracle that knows how to put a framework of events together (even wit errors) is much better than asking a human to do it from scratch.

[shufflerofrocks]

>Having an oracle that knows all answers is useless if you don't know what to ask.

This sentence summarizes the issue with the current AI debacle, along with the whole "just copy/pase code from stackoverflow and earn top bucks" meme that was going around in 2010s.

You're not gonna be a valuable dev if you're just write wrong code faster. Not only does chatgpt/copilot give haphazard code half of the time, it approaches seemingly random syntax and format. Even if LLMs are polished, you're gonna need stand software engineering knowledge to know what's right and wrong.

[greenie_beans]

i started to not like coding once ai became involved

[forgetfreeman]

Coding just keeps getting grosser and weirder decade over decade as new layers of abstraction and complexity get piled on. Just about everyone who does this shit for a living eventually hits a break point where the headassery du jour becomes too much to turn a blind eye to.

[codegladiator]

"genie granting 3 wishes gone wrong" is making a come back

[joshstrange]

LLMs are no excuse for bad code reviews or developers who don’t know what it’s spitting out.

In my code reviews the person who wrote the code needs to explain to me what they changed and why. If they can’t then we are going to have a problem. If you don’t understand the code that an LLM spits out you don’t use it, it’s that simple. If you use it and can’t explain it, well… we are going to have to have some discussions and if it keeps happening you’re going to need to find other employment.

The exact same thing has been happening for pretty much the entire time we’ve had internet. Stack Overflow being the primary example now but there were plenty of other resources before SO. People have always been able to copy/paste code they don’t understand and shove it into a codebase. LLMs make that easier, no doubt, but the core issue has always been there and we, as an industry, have had decades to come up with defenses to this. Code review being the best tool in our toolbox IMHO.

[chx]

> Having an oracle that knows all answers is useless if you don't know what to ask.

But that's not what these LLM systems are. https://hachyderm.io/@inthehands/112006855076082650

> You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to.

> Alas, that does not remotely resemble how people are pitching this technology.

It is exactly what happened to you: it wrote bullshit. Plausible bullshit but bullshit nonetheless.

[teeray]

It’s also just not as good at the task anymore. It frequently gets lazy and gives you an outline with a bunch of vague pseudocode. Compare to when GPT-4 was slower at producing output, but all of that output was solid, detailed work. Some of the magic that made you say “wow” feels like it’s been enshittified out of it.

[beeboobaa3]

I sometimes try the free chatgpt when I run into a problem and it's just hilarious how terrible it is. Loves to go around in circles with the same made up solution that has no basis in reality, using functions in libraries that would be great if they actually existed.

[no-mana]

I noticed that like since a week ago? Output faster, but not impressive. Now I just skip to stack overflow or docs. The output is also giving error a lot more, as if the libraries on which the example is based off was old. Sometimes it's really trivial task just to save time, and it's just not of any help. Still helpful when you want to start something new, it just doesn't scale that well.

[chillfox]

Can’t say I have noticed any drop in quality at all. I do have a fairly detailed set of custom instructions set.

[romeros]

It is difficult to get a man to understand something when his salary depends on his not understanding it. ~ U. Sinclair

A significant problem is the subconscious defense mechanism or bias that compels us to conclude that AI has various shortcomings, asserting the ongoing need for status quo.

The capabilities of GPT-3.x in early 2023 pale in comparison to today's AI, and it will continue to evolve and improve.

[signatoremo]

I’m surprised to see this comment was downvoted heavily. The quote is very popular around here, to the point where there is an Ask HN if it’s the most quoted quote:

https://news.ycombinator.com/item?id=27771186

Yet people don’t like it in this thread. Does it touch a nerve?

[raincole]

> Having an oracle that knows all answers is useless if you don't know what to ask.

You just need to ask it what to ask. /s

[loceng]

You need to pay 80% of the efficiency improvement for a higher-tiered curated set of code output from only senior-experienced developers. /s