top | item 44247291

(no title)

zOneLetter | 8 months ago

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's code lol.

discuss

order

vmg12|8 months ago

Here are the cases where it helps me (I promise this isn't ai generated even though im using a list...)

- Formulaic code. It basically obviates the need for macros / code gen. The downside is that they are slower and you can't just update the macro and re-generate. The upside is it works for code that is slightly formulaic but has some slight differences across implementations that make macros impossible to use.

- Using apis I am familiar with but don't have memorized. It saves me the effort of doing the google search and scouring the docs. I use typed languages so if it hallucinates the type checker will catch it and I'll need to manually test and set up automated tests anyway so there are plenty of steps where I can catch it if it's doing something really wrong.

- Planning: I think this is actually a very under rated part of llms. If I need to make changes across 10+ files, it really helps to have the llm go through all the files and plan out the changes I'll need to make in a markdown doc. Sometimes the plan is good enough that with a few small tweaks I can tell the llm to just do it but even when it gets some things wrong it's useful for me to follow it partially while tweaking what it got wrong.

Edit: Also, one thing I really like about llm generated code is that it maintains the style / naming conventions of the code in the project. When I'm tired I often stop caring about that kind of thing.

xmprt|8 months ago

> Using apis I am familiar with but don't have memorized

I think you have to be careful here even with a typed language. For example, I generated some Go code recently which execed a shell command and got the output. The generated code used CombinedOutput which is easier to used but doesn't do proper error handling. Everything ran fine until I tested a few error cases and then realized the problem. In other times I asked the agent to write tests cases too and while it scaffolded code to handle error cases, it didn't actually write any tests cases to exercise that - so if you were only doing a cursory review, you would think it was properly tested when in reality it wasn't.

mlinhares|8 months ago

The downside for formulaic code kinda makes the whole thing useless from my perspective, I can't imagining a case where that works.

Maybe a good case, that i've used a lot, is using "spreadsheet inputs" and teaching the LLM to produce test cases/code based on the spreadsheet data (that I received from elsewhere). The data doesn't change and the tests won't change either so the LLM definitely helps, but this isn't code i'll ever touch again.

felipeerias|8 months ago

Planning is indeed a very underrated use case.

One of my most productive uses of LLMs was when designing a pipeline from server-side data to the user-facing UI that displays it.

I was able to define the JSON structure and content, the parsing, the internal representation, and the UI that the user sees, simultaneously. It was very powerful to tweak something at either end and see that change propagate forwards and backwards. I was able to hone in on a good solution much faster that it would have been the case otherwise.

j1436go|8 months ago

As a personal anecdote I've tried to create Shell scripts for the testing of a public HTTP API that had pretty good documentation and in both cases the requests did not work. In one case it even hallucinated an endpoint.

owl_vision|8 months ago

plus 1 for using agents for api refresher and discovery. i also use regular search to find possible alternatives and about 3-4 out of 10 normal search wins.

Discovering private api using an agent is super useful.

dataviz1000|8 months ago

I am beginning to love working like this. Plan a design for code. Explain to the LLM the steps to arrive to a solution. Work on reading, understanding, fixing, planing, ect. while the LLM is working on the next section of code. We are working in parallel.

Think of it like being a cook in a restaurant. The order comes in. The cook plans the steps to complete the task of preparing all the elements for a dish. The cook sears the steak and puts it in the broiler. The cook doesn't stop and wait for the steak to finish before continuing. Rather the cook works on other problems and tasks before returning to observe the steak. If the steak isn't finished the cook will return it to the broiler for more cooking. Otherwise the cook will finish the process of plating the steak with sides and garnishes.

The LLM is like the oven, a tool. Maybe grating cheese with a food processor is a better analogy. You could grate the cheese by hand or put the cheese into the food processor port in order to clean up, grab other items from the refrigerator, plan the steps for the next food item to prepare. This is the better analogy because grating cheese could be done by hand and maybe does have a better quality but if it is going into a sauce the grain quality doesn't matter so several minutes are saved by using a food processor which frees up the cook's time while working.

Professional cooks multitask using tools in parallel. Maybe coding will move away from being a linear task writing one line of code at a time.

collingreen|8 months ago

I like your take and the metaphors are good at helping demonstrate by example.

One caveat I wonder about is how this kind of constant context switching combines with the need to think deeply (and defensively with non humans). My gut says I'd struggle at also being the brain at the end of the day instead of just the director/conductor.

I've actively paired with multiple people at once before because of a time crunch (and with a really solid team). It was, to this day, the most fun AND productive "I" have ever been and what you're pitching aligns somewhat with that. HOWEVER, the two people who were driving the keyboards were substantially better engineers than me (and faster thinkers) so the burden of "is this right" was not on me in the way it is when using LLMs.

I don't have any answers here - I see the vision you're pitching and it's a very very powerful one I hope is or becomes possible for me without it just becoming a way to burn out faster by being responsible for the deep understanding without the time to grok it.

divan|8 months ago

On one codebase I work with, there are often tasks that involve changing multiple files in a relatively predictable way. Like there is little creativity/challenge, but a lot of typing in multiple parts/files. Tasks like these used to take 3-4 hours complete before just because I had to physically open all these files, find right places to modify, type the code etc. With AI agent I just describe the task, and it does the job 99% correct, reducing the time from 3-4 hours to 3-4 minutes.

throwawayscrapd|8 months ago

Did you ever consider refactoring the code so that you don't have to do shotgun surgery every time you make this kind of change?

majormajor|8 months ago

Amusingly, cursor took 5 minutes trying to figure out how to do what a simple global find/replace did for me in 30 seconds after I got tired of waiting for it's attempt just last night on a simple predictable lots-of-files change.

A 60x speedup is way more than I've seen even in its best case for things like that.

gyomu|8 months ago

So you went from being able to handle at most 10 or so of these tasks you often get per week, to >500/week. Did you reap any workplace benefits from this insane boost in productivity?

com2kid|8 months ago

I used to spend time writing regex's do to this for me, now LLMs solve it in less time than it takes me to debug my one off regex!

bgwalter|8 months ago

Some people cannot do anything without a tool. These people are early adopters and power users, who then evangelize their latest discovery.

GitHub's value proposition was that mediocre coders can appear productive in the maze of PRs, reviews, green squares, todo lists etc.

LLMs again give mediocre coders the appearance of being productive by juggling non-essential tools and agents (which their managers also love).

danielbln|8 months ago

What is an essential tool? IDE? Editor? Pencil? Can I scratch my code into a French cave wall if I want to be a senior developer?

osigurdson|8 months ago

I felt the same way until recently (like last Friday recently). While tools like Windsurf / Cursor have some utility, most of the time I am just waiting around for them while I get to read and correct the output. Essentially, I'm helping out with the training while paying to use the tool. However, now that Codex is available in ChatGPT plus, I appreciate that asynchronous flow very much. Especially for making small improvements , fixing minor bugs, etc. This has obvious value imo. What I like to do is queue up 5 - 10 tasks and the. focus on hard problems while it is working away. Then when I need a break I review / merge those PRs.

buffalobuffalo|8 months ago

I kinda consider it a P!=nP type thing. If I need to write a simple function, it will almost always take me more time to implement it than it will to verify if an implementation of it suits my needs. There are exceptions, but overall when coding with LLMs this seems to hold true. Asking the LLM to write the function then checking it's work is a time saver.

worldsayshi|8 months ago

I think this perspective is kinda key. Shifting attention towards more and better ways to verify code can probably lead to improved quality instead of degraded.

moritonal|8 months ago

I see it as basically Cunningham's Law. It's easier to see the LLM's attempt a solution and how it's wrong than to write a perfectly correct solution first time.

a_tartaruga|8 months ago

Came here to post this it is precisely right.

KronisLV|8 months ago

> I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc.

Friction.

A lot of people are bad at getting started (like writer's block, just with code), whereas if you're given a solution for a problem, then you can tweak it, refactor it and alter it in other ways for your needs, without getting too caught up in your head about how to write the thing in the first place. Same with how many of my colleagues have expressed that getting started on a new project from 0 is difficult, because you also need to setup the toolchain and bootstrap a whole app/service/project, very similar to also introducing a new abstraction/mechanism in an existing codebase.

Plus, with LLMs being able to process a lot of data quickly, assuming you have enough context size and money/resources to use that, it can run through your codebase in more detail and notice things that you might now, like: "Oh hey, there are already two audit mechanisms in the codebase in classes Foo and Bar, we might extract the common logic and..." that you'd miss on your own.

marvstazar|8 months ago

As a senior developer you already spend a significant amount of time planning new feature implementations and reviewing other people's code (PRs). I find that this skill transitions quite nicely to working with coding agents.

munificent|8 months ago

I don't disagree but... wouldn't you rather be working with actual people?

Spending the whole day chatting with AI agents sounds like a worst-of-both-worlds scenarios. I have to bring all of my complex, subtle soft skills into play which are difficult and tiring to use, and in the end none of that went towards actually fostering real relationships with real people.

At the end of the day, are you gonna have a beer with your agents and tell them, "Wow, we really knocked it out of the park today?"

Spending all day talking to virtual coworkers is literally the loneliest experience I can imagine, infinitely worse than actually coding in solitude the entire day.

majormajor|8 months ago

You will hit two problems in this "only hire virtual juniors" thing:

* the wall of how much you can review in one day without your quality slipping now that there's far less variation in your day

* the long-term planning difficulties around future changes when you are now the only human responsible for 5-20x more code surface area

* the operational burden of keeping all that running

The tools might get good enough that you only need 5 engineers to do what used to be 10-20. But the product folks aren't gonna stop wanting you to keep churning out the changes, and the last 2 years of evolution of these models doesn't seem like it's on a trajectory to cut that down to 1 (or 0) without unforeseen breakthroughs.

aqme28|8 months ago

Yeah was going to make the same point.

> I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc.

What they're saying is that they never have coworkers.

bob1029|8 months ago

My most productive use of LLMs has been to stub out individual methods and have them fill in the implementations. I use a prompt like:

  public T MyMethod<T>(/*args*/) /*type constraints*/
  {
    //TODO: Implement this method using the following requirements:
    //1 ...
    //2 ...
    //...
  }
Anything beyond this and I can't keep track of which rabbit is doing what anymore.

grogenaut|8 months ago

I'm categorizing my expenses. I asked the code AI to do 20 at a time, and suggest categories for all of them in an 800 line file. I then walked the diff by hand correcting things. I then asked it to double check my work. It did this in a 2 column cav mapping.

It could do this in code. I didn't have to type anywhere near as much and 1.5 sets of eyes were on it. It did a pretty accurate job and the followup pass was better.

This is just an example I had time to type before my morning shower

silverlake|8 months ago

You’re clinging to an old model of work. Today an LLM converted my docker compose infrastructure to Kubernetes, using operators and helm charts as needed. It did in 10 minutes what would take me several days to learn and cobble together a bad solution. I review every small update and correct it when needed. It is so much more productive. I’m driving a tractor while you are pulling an ox cart.

ofjcihen|8 months ago

“ It did in 10 minutes what would take me several days to learn and cobble together a bad solution.”

Another way to look at this is you’re outsourcing your understanding to something that ultimately doesn’t think.

This means 2 things: your solution could be severely suboptimal in multiple areas such as security and two because you didn’t bother understanding it yourself you’ll never be able to identify that.

You might think “that’s fine, the LLM can fix it”. The issue with that is when you don’t know enough to know something needs to be fixed.

So maybe instead of carts and oxen this is more akin to grandpa taking his computer to Best Buy to have them fix it for him?

gyomu|8 months ago

> I’m driving a tractor while you are pulling an ox cart.

Or you’re assembling prefab plywood homes while they’re building marble mansions. It’s easy to pick metaphors that fit your preferred narrative :)

munificent|8 months ago

> would take me several days to learn ... correct it when needed.

If you haven't learned how all this stuff works, how are you able to be confident in your corrections?

> I’m driving a tractor while you are pulling an ox cart.

Are you sure you haven't just duct taped a jet engine to your ox cart?

12345hn6789|8 months ago

How did you verify this works correctly, and as intended, in 10 minutes if it would have taken you 2 days to do it yourself?

zombiwoof|8 months ago

But you would have learned something if you invested the time. Now when your infra blows up you have no idea what to fix and will go fishing into the LLM lake to find how to fix it

valcron1000|8 months ago

> It did in 10 minutes what would take me several days to learn

> I review every small update and correct it when needed

How can you review something that you don't know? How do you know this is the right/correct result beyond "it looks like it works"?

opto|8 months ago

If it would have taken you days to learn about the topic well enough to write a bad implementation, how can you have any confidence you can evaluate, let alone "correct", one written by an LLM?

You just hope you are on a tractor.

ithkuil|8 months ago

I think this fits squarely with the idea that LLM today is a great learning tool; learning through practice has always been a proven way to learn but a difficult method to learn from fixed material like books.

LLM is a teacher that can help you learn by doing the work you want to be doing and not some fake exercise.

The more you learn though, the more you review the code produced by the LLM and the more you'll notice that you are still able to reason better than an LLM and after your familiarity with an area exceeds the capabilities of the LLM the interaction with the LLM will bring diminishing returns and possibly the cost of babysitting that eager junior developer assistant may become larger than the benefits.

But that's not a problem, for all areas you master there will be hundreds of other areas you haven't mastered yet or ever will and for those things the LLM we have already today are of immediate help.

All this without even having to enter the topic of how coding assistants will improve in the future.

TL;DR

Use a tool when it helps. Don't use it when it doesn't. It pays to learn to use a tool so you know when it helps and when it doesn't. Just like every other tool

greenhat76|8 months ago

This is such an arrogant take.

rgbrenner|8 months ago

if you work on a team most code you see isn’t yours.. ai code review is really no different than reviewing a pr… except you can edit the output easier and maybe get the author to fix it immediately

amrocha|8 months ago

Reviewing code is harder than writing code. I know staff engineers that can’t review code. I don’t know where this confidence that you’ll be able to catch all the AI mistakes comes from.

j-wang|8 months ago

I was about to say exactly this—it's not really that different from managing a bunch of junior programmers. You outline, they implement, and then you need to review certain things carefully to make sure they didn't do crazy things.

But yes, these juniors take minutes versus days or weeks to turn stuff around.

addaon|8 months ago

> if you work on a team most code you see isn’t yours.. ai code review is really no different than reviewing a pr… except you can edit the output easier and maybe get the author to fix it immediately

And you can't ask "why" about a decision you don't understand (or at least, not with the expectation that the answer holds any particular causal relationship with the actual reason)... so it's like reviewing a PR with no trust possible, no opportunity to learn or to teach, and no possibility for insight that will lead to a better code base in the future. So, the exact opposite of reviewing a PR.

gejose|8 months ago

Just to draw a parallel (not to insult this line of thinking in any way): “ Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to _compile_ your code and then reading it, understand it, fixing it, etc”

At a certain point you won’t have to read and understand every line of code it writes, you can trust that a “module” you ask it to build works exactly like you’d think it would, with a clearly defined interface to the rest of your handwritten code.

addaon|8 months ago

> At a certain point you won’t have to read and understand every line of code it writes, you can trust that a “module” you ask it to build works exactly like you’d think it would, with a clearly defined interface to the rest of your handwritten code.

"A certain point" is bearing a lot of load in this sentence... you're speculating about super-human capabilities (given that even human code can't be trusted, and we have code review processes, and other processes, to partially mitigate that risk). My impression was that the post you were replying to was discussing the current state of the art, not some dimly-sensed future.

gigel82|8 months ago

I think there are 2 types of software engineering jobs: the ones where you work on a single large product for a long time, maintaining it and adding features, and the ones that spit out small projects that they never care for again.

The latter category is totally enamored with LLMs, and I can see the appeal: they don't care at all about the quality or maintainability of the project after it's signed off on. As long as it satisfies most of the requirements, the llm slop / spaghetti is the client's problem now.

The former category (like me, and maybe you) see less value from the LLMs. Although I've started seeing PRs from more junior members that are very obviously written by AI (usually huge chunks of changes that appear well structured but as soon as you take a closer look you realize the "cheerleader effect"... it's all AI slop, duplicated code, flat-out wrong with tests modified to pass and so on) I still fail to get any value from them in my own work. But we're slowly getting there, and I presume in the future we'll have much more componentized code precisely for AIs to better digest the individual pieces.

esafak|8 months ago

Give it more than the minimal context so it can emulate the project's style. The recent async agents should be good at this.

ar_lan|8 months ago

> I just don't like reading other people's code lol.

Do you work for yourself, or for a (larger than 1 developer) company? You mention you only code for your own tools, so I am guessing yourself?

I don't necessarily like reading other people's code either, but across a distributed team, it's necessary - and sometimes I'm also inspired when I learn something new from someone else. I'm just curious if you've run into any roadblocks with this mindset, or if it's just preference?

HPsquared|8 months ago

The LLM has a much larger "working vocabulary" (so to speak) than I. It's more fluent.

It's easier to read a language you're not super comfortable with, than it is to write it.

satvikpendem|8 months ago

Fast prototyping for code I'll throw away anyway. Sometimes I just want to get something to work as a proof of concept then I'll figure out how to productionize it later.

mewpmewp2|8 months ago

It is just faster and less effort. I can't write code as quickly as the LLM can. It is all in my head, but I can't spit it out as quickly. I just see LLMs as getting what is in my head quickly out there. I have learned to prompt it in such a way that I know what to expect, I know its weakspots and strengths. I could predict what it is going output, so it is not that difficult to understand.

andhuman|8 months ago

Yes, the eureka moment with LLMs is when they started outputting the things I was beginning to type. Not just words but sentences, whole functions and even unit tests. The result is the same as I would have typed it, just a lot faster.

hintymad|8 months ago

> I still don't understand the benefit of relying on someone/something else to write your code and then reading it

Maybe the key is this: our brains are great at spotting patterns, but not so great at remembering every little detail. And a lot of coding involves boilerplate—stuff that’s hard to describe precisely but can be generated anyway. Even if we like to think our work is all unique and creative, the truth is, a lot of it is repetitive and statistically has a limited number of sound variations. It’s like code that could be part of a library, but hasn’t been abstracted yet. That’s where AI comes in: it’s really good at generating that kind of code.

It’s kind of like NP problems: finding a solution may take exponentially longer, but checking one takes only polynomial time. Similarly, AI gives us a fast draft that may take a human much longer to write, and we review it quickly. The result? We get more done, faster.

amrocha|8 months ago

Copy and paste gives us a fast draft of repetitive code. That’s never been the bottle neck.

The bottle neck is in the architecture and the details. Which is exactly what AI gets wrong, and which is why any engineer who respects his craft sees this snake oil for what it is.

resonious|8 months ago

It's an intentional (hopefully) tradeoff between development speed and deep understanding. By hiring someone or using an agent, you are getting increased speed for decreased understanding. Part of choosing whether or not to use an agent should include an analysis of how much benefit you get from a deep understanding of the subsystem you're currently working on. If it's something that can afford defects, you bet I'll get an agent to do a quick-n-dirty job.

unshavedyak|8 months ago

> I just don't like reading other people's code lol.

I agree entirely and generally avoided LLMs because they couldn't be trusted. However a few days ago i said screw it and purchased Claude Max just to try and learn how i can use LLMs to my advantage.

So far i avoid it for things where they're vague, complex, etc. The effort i have to go through to explain it exceeds my own in writing it.

However for a bunch of things that are small, stupid, wastes of time - i find it has been very helpful. Old projects that need to migrate API versions, helper tools i've wanted but have been too lazy to write, etc. Low risk things that i'm too tired to do at the end of the day.

I have also found it a nice way to get movement on projects where i'm too tired to progress on after work. Eg mostly decision fatigue, but blank spaces seem to be the most difficult for me when i'm already tired. Planning through the work with the LLM has been a pretty interesting way to work around my mental blocks, even if i don't let it do the work.

This planning model is something i had already done with other LLMs, but Claude Code specifically has helped a lot in making it easier to just talk about my code, rather than having to supply details to the LLM/etc.

It's been far from perfect of course, but i'm using this mostly to learn the bounds and try to find ways to have it be useful. Tricks and tools especially, eg for Claude adding the right "memory" adjustments to my preferred style, behaviors (testing, formatting, etc) has helped a lot.

I'm a skeptic here, but so far i've been quite happy. Though i'm mostly going through low level fruit atm, i'm curious if 20 days from now i'll still want to renew the $100/m subscription.

stirfish|8 months ago

I use it almost like an RSI mitigation device, for tasks I can do (and do well) but don't want to do anymore. I don't want to write another little 20-line script to format some data, so I'll have the machine do it for me.

I'll also use it to create basic DAOs from schemas, things like that.

esafak|8 months ago

If you give a precise enough spec, it's effectively your code, with the remaining difference being inconsequential. And in my experience, it is often better, drawing from a wider pool of idioms.

mgraczyk|8 months ago

When you write code, you have to spend time on ALL of the code, no matter how simple or obvious it is.

When you read code, you can allocate your time to the parts that are more complex or important.

jdalton|8 months ago

No different than most practices now. PM write a ticket, dev codes it, PRs it, then someone else reviews it. Not a bad practice. Sometimes a fresh set of eyes really helps.

pianopatrick|8 months ago

I am not too familiar with software development inside large organizations as I work for myself - are there any of those steps the AI cannot do well? I mean it seems to me that if the AI is as good as humans at text based tasks you could have an entire software development process with no humans. I.e. user feedback or error messages go to a first LLM that writes a ticket. That ticket goes to a second LLM that writes code. That code goes to a 3rd LLM that reviews the code. That code goes through various automated tests in a CI / CD pipeline to catch issues. If no tests fail the updated software is deployed.

You could insert sanity checks by humans at various points but are any of these tasks outside the capabilities of an LLM?