[logiduck]

We may have the tools to manage it, but we are losing the ability to understand it.

AI writing software will be a exponential explosion in software complexity.

AI would very well create its own programming language to be more efficient for the AI to code in that we have no hopes of understanding. Imagine that AI started to output a large SaaS app written today in Python in Assembly because for the AI the extra cognitive overhead of using Assembly doesn't exist. At first we might resist and tell it to use a language we understand, but then as time goes on we grow more comfortable that it does the "right" thing and down the road people are just generated raw Assembly using AI without really understanding what the code is doing, only looking to see if the code behaves the way they expect.

Imagine entire codebases spun up in seconds with so many lines of code, a single person would never have hopes of understanding everything, needing to rely on AI to summarize and explain the code for them. Now imagine that massive code base being iterated and worked on for a decade over the life of a company.

AI could bring Terabyte sized code bases in a decade or so.

[throwuwu]

This already happens when you compile code for a specific architecture. The thing you fear will happen has already happened and we’re fine.

I’d worry more about generated code from an AI that doesn’t fully understand the codebase. That would be as bad as letting the junior devs run loose.

[logiduck]

Not really at all.

What you are describing is an traceable transformation of code in which there are several intermediate layers that people can inspect and understand. They can inspect the exact rules for that transformation. The process is repeatable and verifiable.

What I am describing is a black-box stochastic generation of low level code in which there is no higher level representation anymore. AI generating Assembly not by a set of rules, but using statistics. There will be no individual layers to unwind or inspect, because for AI it doesn't need them. Our separation of concerns was built for our human brains and limiting complexity of projects to our understanding.