[tenthirtyam]

My expectation is that there'll never be a single bust-up moment, no line-in-the-sand beyond which we'll be able to say "it doesn't work anymore."

Instead agent written code will get more and more complex, requiring more and more tokens (& NPU/GPU/RAM) to create/review/debug/modify, and will rapidly pass beyond any hope of a human understanding even for relatively simple projects (e.g. such as a banking app on your phone).

I wonder, however, whether the complexity will grow slower or faster than Moore's law and our collective ability to feed the AIs.

[layer8]

Maybe software systems will become more like biological organisms. Huge complexity with parts bordering on chaos, but still working reasonably well most of the time, until entropy takes its course.

[leptons]

It's already like that, for a long time. Humans are quite capable of creating complex systems that become unwieldy the bigger they get. No one person can understand all of it. I will offer the AT&T billing system as an example that I'm all too familiar with as a customer, due to the pain it causes me. So many ridiculous problems with that system, it's been around a long time, and it is just so screwball.