[TACIXAT]

I have the complete opposite impression w.r.t. architecture decisions. The LLMs can cargo cult an existing design, but they do not think through design consequences well at all. I use them as a rubber duck non-stop, but I think I respect less than one out of every six of their suggestions.

[daxfohl]

They've gotten pretty good IME so long as you guide it to think out of the box, give it the right level of background info, have it provide alternatives instead of recommendations, and do your best not to bias it in any particular direction.

That said, the thing it really struggles with is when the best approach is "do nothing". Which, given that a huge chunk of principal level work is in deciding what NOT to do, it may be a while before LLMs can viably take that role. A principal LLM based on current tech would approve every idea that comes across it, and moreover sell each of them as "the exact best thing needed by the organization right now!"

[XenophileJKO]

Knowing when to nudge it out of a rut (or say skip it) is probably the biggest current skill. This is why experienced people get generally much better results.

[code_martial]

I’m not sure. I keep asking the LLMs whether I should rewrite project X in language Y and it just asks back, “what’s your problem?” And most of the times it shoots my problems down showing exactly why rewriting won’t fix that particular problem. Heck, it even quoted Joel Spolsky once!

Of course, I could just _tell_ it to rewrite, but that’s different.