[mjdiloreto]

It's not that it won't be better at understanding, it's that there's too many possibilities to understand. This is true for humans too, but I can use the output to make money in a particular scenario.

Take even 1 simple example - software applications on a smart watch. How many dimensions of reality are relevant? Maybe I'm a busy person, so I need a personal assistant for my calendar. Maybe my wife needs access too. Maybe I'm a bird watcher and I'd like to track the birds I see. Maybe I'm a bird researcher and those observations need to integrate with my research.... ad nauseum forever.

AI will write all the code, and make all the meaningful decisions, but the backstop of the whole thing has to be some non-virtual reality with a paying user, otherwise there is no value to extract.

I personally only care about the outcome, I don't even really care if I understand how anything else works, or any of the decisions made. My dollars go in, working code comes out to suit me.

[Archer6621]

I agree with your overall perspective here. You need the human in the loop to ground the request/direction in a reality with human needs, but that's about it.

What I was getting at is that nothing stops you from asking AI what would be the next best smartwatch app to build, and based on all its aggregated knowledge and other inputs (e.g. search) it has, it can potentially make a better estimation than you or any human of a product that would sell.

Of course whether that is actually true depends on how well its training data is able to model/mimic reality, and how grounded its inputs (e.g. internet) are. You can always help it a bit by steering it into the right direction, providing additional grounding. Was mainly wondering for how long this "additional" guidance would be a necessity, fearing that it won't be for as long as we think.

[mjdiloreto]

I agree with you. I think the "additional guidance" era we are in will be measured in single-digit years.