This article is lumps design and implementation together. In my experience, LLMs are really quite bad at designing anything interesting. They are sort of tolerable at implementation — they’re remarkably persistent (compared to humans, anyway), they will tirelessly use whatever framework, good or bad, you throw at them, and they will produce code that is quite painful to look at. And they’ll say that they’re architecting things and hope you’re impressed.
(Author here) I meant "design" as in designing physical objects — and all our "programming" is "design" in this definition, because "manufacturing" is done by compilers and bundlers.
And I wouldn't write this article 3 months ago. Since then the quality of the output jumped significantly, it is now possible to put the agent into a proper harness (plan/edit/review/test) and the output is good — and if it's not, you discard it and try again, or point out a detail for the next cycle of improvements.
Yes, this requires a lot of forethought to set up, but it works.
I'm not talking only about "web things", I'm working on a project that involves engineering calculations and a lot of optimization of hot paths, both CPU and GPU.
>LLMs are really quite bad at designing anything interesting
Let’s be honest, how many devs are actually creating something interesting/unique at their work?
Most of the time, our job is just picking the right combination of well-known patterns to make the best possible trade-offs while fulfilling the requirements.
I was wondering if our goal is to leverage them to think about interfaces a bit, like a slightly accelerated modeling phase and then let them loose on the implementation (and maybe later let them loose on local optimization tricks)
I was going to say - with agents the only part I actually have to do is design. Well, and testing. But they don’t really do design work so much as architecture selection if you provide a design.
This is a great take. It applies the SaaS is dead theory at a lower level (libraries are dead) but has it a much more nuanced view.
Yeh even if LLMs are 10x better than today you probably still don't want to implement cryptography from scratch, but use a library.
I also like the 3d printing analogy. We will see how good LLMs get, but I will say that a lot of AI coded tools today have the same feeling as 3d printed hardware.
If no engineer was involved the software is cheap and breaks under pressure because no one considered the edge cases. It looks good on the surface but if you use it for something serious it does break.
The engineer might still use an LLM/3d printer but where necessary he'll use a metal connection (write code by hand or at least tightly guide the LLM) to make the product sturdy.
Gotta love a world in which a tool which has ingested "all the world's libraries" is now trotted out as a solution to replace those libraries.
You know what would happen if all the people who handwrote and maintained those libraries revoked their code from the training datasets and forbid their use by the models?
:clown face emoji:
This LLM-maxxing is always a myopic one-way argument. The LLMs steal logic from the humans who invent it, then people claim those humans are no longer required. Yet, in the end, it's humans all the way down. It's never not.
> You know what would happen if all the people who handwrote and maintained those libraries revoked their code from the training datasets and forbid their use by the models?
The MCP servers combined with agentic search solved this possibility, just this year superseding RAG methods but all techniques have their place. I don't see much of a future for RAG though, given its computational intensity.
Long story short, training and fine tuning is no longer necessary for an LLM to understand the latest libraries, and therefore the "permission" to train would not even be something applicable to debate
it's a fast moving field, best not to have a strong opinion about anything
> What matters less is the mechanical knowledge of how to express the solution in code. The LLM generates code, not understanding.
I think it's the opposite -- if you have a good way to design your software (e.g., conceptual and modular), LLM will generate the understanding as well. Design does not only mean code architecture, it also means how you express the concepts in it to a user. If software isn't really understood by humans, I doubt LLMs will be able to generate working code for it anyway, so we get a design problem to solve.
LLM's are only as good as they are because we have such amazing incredible open source software everywhere. Because their job is to look at the types of really good libraries that have decades of direct and indirect wisdom poured into them, and then to be a little glue.
Yes the LLM can go make you alternatives, and it will be mostly fine-ish in many cases. But LLMs are not about pure endless frivolous frontiersing. They deeply reward and they are trained on what the settlers and town planners have done (referencing Wardley here).
And they will be far better at using those good robust well built tools (which they have latently built-in to their models some!) than they will be at re-learning and fine-tuning for your bespoke weird hodgepodge solution.
Cheap design is cheap now. Sure. But good design will be ever more important. Model's ability, their capacity, is a function of what material they can work with, and I can't for the life of me imagine shorting yourself with cheap design like proposed here. The LLM's are very good, but but honing in on good design is hard, period, and I think that judgement and character is something the next orders of magnitude of parameters is still not going to close the gap on.
I think the main point is that "reinventing the wheel" has become cheap, not software design itself.
For example, when a designer sends me the SVG icons he created, I no longer need to push back against just using a library. Instead, I can just give these icons to Claude Code and ask it to "Make like react-icons," and an hour later, my issue is solved with minimal input from me. The LLM can use all available data, since the problem is not new.
But many software problems challenge LLMs, especially with features lacking public training data, and creating solutions for these issues is certainly not cheap.
I really like the jig analogy. But that the software jig is no different from production software, I don't buy. Saying that the bytes are the same, is like saying that the atoms of a jig is the same as the atoms in the final product.
GenAI generally makes software cheaper. But there is a huge difference in how much. Prototypes and jigs may be 90% cheaper (just making up numbers here), while for production software it may be closer to 10%.
AI is taking over Senior Devs' Work is the same as IKEA is taking over carpenter's moat - no, no, and again no way.
AI lets you do some impressive stuff, I really enjoy using it. No doubt about that.
But app development, the full Software Delivery Life Cycle - boy, is AI bad. And I mean in a very extreme way.
I talked to a carpenter yesterday about IKEA. He said, people call him to integrate their IKEA stuff, especially the expensive stuff.
And AI is the same.
Configuration Handling: Works on my machine, impressive SaaS app, fast, cool, PostgreSQL etc.
And then there is the final moment: Docker, Live Server - and BOOM! deployment failed.
If you ever happen to debug and fix certain infrastructure and therefore deployment fails - you wish you were doing COBOL or x86/M68000 assembly code like it is 1987 all over again - if you happen to be a very seasoned Senior Dev with a lot of war stories to share.
If you are some vibe coder or consulting MBA - good luck.
AI fails so bad at doing certain things consistently well - and it costs company dearly.
Firing up a Landing Page in React using some Tailwind + ShadCN UI - oh well...
Software Design, Solution Architecture - the hard things are getting harder, not cheaper.
IKEA is great - for certain use cases. It made carpenter's work only more valuable. They thrived because of IKEA, they didn't suffer. In fact, there is more work for them to do. Is their business still hard, of course, but difficult in a different way (talent).
And all doomer's talking about the dev apocalypse - if AI takes over software development, who is in trouble then? Computer Science, software development? Or any and every job market out there?
Think twice. Develop and deploy ten considerably complex SaaS apps using AI and tell me how it went.
Access to information got cheaper. A fool with a tool is still a fool.
It's not cheap to develop new large AI models even when one has data. This is where the most interesting software development is. The limitation is because the required hardware is very expensive to buy and also to rent.
We are maintaining and extending a bunch (15 around) large ERP/data type projects which are over 20 years old with vibe coding. We have a very strict system to keep the LLMs in bounds, keeping to standard etc and we are feeling we are close to not having to read the code; we have over 2 months of not having to touch anything after review. I designed most of those projects 20-30 years ago; all have the same design principles which are well documented (by me) so the LLM just knows what to find where and what to do with it. These are large 'living' projects (many updates over decades).
This seems like an uncharitable take. Personally I would refrain from putting a 20 year barrier around the qualification, that seems a little harsh.
TFA's take makes sense in a certain context. Getting a high-quality design which is flexible in desirable ways is now easier than ever. As the human asking an LLM for the design, maybe you shouldn't be claiming to have "designed" it, though.
The problem is that if you don’t understand the code you’re taking a risk shipping that code.
There’s a reason why most vibe coded apps I’ve seen leak keys and have basic security flaws all over the place.
If you don’t know what you’re doing and you’re generating code at scale that you can’t manage you’re going to have a bad time.
The models are trained on all the slop we had to ship under time pressure and swore we’d fix later, etc. They’re not going to autocomplete the good code. They’re going to autocomplete the most common denominator code.
I don’t agree that design is cheap. Maybe for line-of-business software that doesn’t matter much.
amluto|19 days ago
klodolph|19 days ago
dottedmag|18 days ago
And I wouldn't write this article 3 months ago. Since then the quality of the output jumped significantly, it is now possible to put the agent into a proper harness (plan/edit/review/test) and the output is good — and if it's not, you discard it and try again, or point out a detail for the next cycle of improvements.
Yes, this requires a lot of forethought to set up, but it works.
I'm not talking only about "web things", I'm working on a project that involves engineering calculations and a lot of optimization of hot paths, both CPU and GPU.
ekropotin|19 days ago
Let’s be honest, how many devs are actually creating something interesting/unique at their work?
Most of the time, our job is just picking the right combination of well-known patterns to make the best possible trade-offs while fulfilling the requirements.
agumonkey|19 days ago
SequoiaHope|19 days ago
unknown|19 days ago
[deleted]
jascha_eng|19 days ago
Yeh even if LLMs are 10x better than today you probably still don't want to implement cryptography from scratch, but use a library.
I also like the 3d printing analogy. We will see how good LLMs get, but I will say that a lot of AI coded tools today have the same feeling as 3d printed hardware. If no engineer was involved the software is cheap and breaks under pressure because no one considered the edge cases. It looks good on the surface but if you use it for something serious it does break.
The engineer might still use an LLM/3d printer but where necessary he'll use a metal connection (write code by hand or at least tightly guide the LLM) to make the product sturdy.
lelanthran|19 days ago
That's LLMs extending C and C++ Undefined Behaviour to every project regardless of language.
-------------------
EDIT: I tried articulating it in a blog post in a sleep-deprived frenzy of writing on Sunday - https://www.lelanthran.com/chap14/content.html
mikert89|19 days ago
jaredcwhite|19 days ago
You know what would happen if all the people who handwrote and maintained those libraries revoked their code from the training datasets and forbid their use by the models?
:clown face emoji:
This LLM-maxxing is always a myopic one-way argument. The LLMs steal logic from the humans who invent it, then people claim those humans are no longer required. Yet, in the end, it's humans all the way down. It's never not.
yieldcrv|19 days ago
The MCP servers combined with agentic search solved this possibility, just this year superseding RAG methods but all techniques have their place. I don't see much of a future for RAG though, given its computational intensity.
Long story short, training and fine tuning is no longer necessary for an LLM to understand the latest libraries, and therefore the "permission" to train would not even be something applicable to debate
it's a fast moving field, best not to have a strong opinion about anything
feastingonslop|19 days ago
barishnamazov|19 days ago
I think it's the opposite -- if you have a good way to design your software (e.g., conceptual and modular), LLM will generate the understanding as well. Design does not only mean code architecture, it also means how you express the concepts in it to a user. If software isn't really understood by humans, I doubt LLMs will be able to generate working code for it anyway, so we get a design problem to solve.
jauntywundrkind|19 days ago
LLM's are only as good as they are because we have such amazing incredible open source software everywhere. Because their job is to look at the types of really good libraries that have decades of direct and indirect wisdom poured into them, and then to be a little glue.
Yes the LLM can go make you alternatives, and it will be mostly fine-ish in many cases. But LLMs are not about pure endless frivolous frontiersing. They deeply reward and they are trained on what the settlers and town planners have done (referencing Wardley here).
And they will be far better at using those good robust well built tools (which they have latently built-in to their models some!) than they will be at re-learning and fine-tuning for your bespoke weird hodgepodge solution.
Cheap design is cheap now. Sure. But good design will be ever more important. Model's ability, their capacity, is a function of what material they can work with, and I can't for the life of me imagine shorting yourself with cheap design like proposed here. The LLM's are very good, but but honing in on good design is hard, period, and I think that judgement and character is something the next orders of magnitude of parameters is still not going to close the gap on.
jampa|19 days ago
For example, when a designer sends me the SVG icons he created, I no longer need to push back against just using a library. Instead, I can just give these icons to Claude Code and ask it to "Make like react-icons," and an hour later, my issue is solved with minimal input from me. The LLM can use all available data, since the problem is not new.
But many software problems challenge LLMs, especially with features lacking public training data, and creating solutions for these issues is certainly not cheap.
BuildItBusk|19 days ago
GenAI generally makes software cheaper. But there is a huge difference in how much. Prototypes and jigs may be 90% cheaper (just making up numbers here), while for production software it may be closer to 10%.
_the_inflator|19 days ago
AI is taking over Senior Devs' Work is the same as IKEA is taking over carpenter's moat - no, no, and again no way.
AI lets you do some impressive stuff, I really enjoy using it. No doubt about that.
But app development, the full Software Delivery Life Cycle - boy, is AI bad. And I mean in a very extreme way.
I talked to a carpenter yesterday about IKEA. He said, people call him to integrate their IKEA stuff, especially the expensive stuff.
And AI is the same.
Configuration Handling: Works on my machine, impressive SaaS app, fast, cool, PostgreSQL etc.
And then there is the final moment: Docker, Live Server - and BOOM! deployment failed.
If you ever happen to debug and fix certain infrastructure and therefore deployment fails - you wish you were doing COBOL or x86/M68000 assembly code like it is 1987 all over again - if you happen to be a very seasoned Senior Dev with a lot of war stories to share.
If you are some vibe coder or consulting MBA - good luck.
AI fails so bad at doing certain things consistently well - and it costs company dearly.
Firing up a Landing Page in React using some Tailwind + ShadCN UI - oh well...
Software Design, Solution Architecture - the hard things are getting harder, not cheaper.
IKEA is great - for certain use cases. It made carpenter's work only more valuable. They thrived because of IKEA, they didn't suffer. In fact, there is more work for them to do. Is their business still hard, of course, but difficult in a different way (talent).
And all doomer's talking about the dev apocalypse - if AI takes over software development, who is in trouble then? Computer Science, software development? Or any and every job market out there?
Think twice. Develop and deploy ten considerably complex SaaS apps using AI and tell me how it went.
Access to information got cheaper. A fool with a tool is still a fool.
OutOfHere|18 days ago
themafia|19 days ago
Less maintenance and flexibility. You're not really "designing software" until you have a 20+ year old product.
Vibe coders really embody the "temporarily embarrassed billionaire" mindset so perfectly.
anonzzzies|19 days ago
bigwheels|19 days ago
TFA's take makes sense in a certain context. Getting a high-quality design which is flexible in desirable ways is now easier than ever. As the human asking an LLM for the design, maybe you shouldn't be claiming to have "designed" it, though.
MattGaiser|19 days ago
agentultra|19 days ago
There’s a reason why most vibe coded apps I’ve seen leak keys and have basic security flaws all over the place.
If you don’t know what you’re doing and you’re generating code at scale that you can’t manage you’re going to have a bad time.
The models are trained on all the slop we had to ship under time pressure and swore we’d fix later, etc. They’re not going to autocomplete the good code. They’re going to autocomplete the most common denominator code.
I don’t agree that design is cheap. Maybe for line-of-business software that doesn’t matter much.
chenmx|19 days ago
[deleted]