(no title)
NothingAboutAny | 1 month ago
If anything in my small circle the promise is waning a bit, in that even the best models on the planet are still kinda shitty for big project work. I work as a game dev and have found agents to only be mildly useful to do more of what I've already laid out, I only pay for the $100 annual plan with jetbrains and that's plenty. I haven't worked at a big business in a while, but my ex-coworkers are basically the same. a friend only uses chat now because the agents were "entirely useless" for what he was doing.
I'm sure someone is getting use out of them making the 10 billionth node.js express API, but not anyone I know.
bunderbunder|1 month ago
That would be fine if our value delivery rate were also higher. But it isn’t. It seems to actually be getting worse, because projects are more likely to get caught in development hell. I believe the main problem there is poorer collective understanding of generated code, combined with apparent ease of vibecoding a replacement, leads to teams being more likely to choose major rewrites over surgical fixes.
For my part, this “Duke Nukem Forever as a Service” factor feels the most intractable. Because it’s not a technology problem, it’s a human psychology problem.
fulladder|1 month ago
Don't get me wrong, overall I really like having AI in my workflow and have gotten many benefits. But even when I ask it to check its own work by writing test cases to prove that properties A, B and C hold, I just end up with thousands more lines of unit and integration tests that then take even more time to analyze -- like, what exactly is being tested here?, are the properties these tests purport to prove even the properties that I care about and asked the agent for in the first place, etc.
I have tried (with at least modest success) to use a second or third agent to review the work of the original coding agent(s), but my general finding has been that there is no substitute for actual human understanding from a legitimate domain expert.
Part of my work involves silicon design, which requires a lot of precision and complex timing issues, and I'll add that the best AI success I've had in those cases is a test-first approach (TDD), where I hand write a boatload of testbenches (that's what we call functional tests in chip design land), then coach my various agents to write the Verilog until my `make test` runs with no errors.
agumonkey|1 month ago