[wujerry2000]

Computer use agents are starting to perform well on websites/apps that are in their training distribution, but still struggle a lot when dealing with tasks outside their distribution. A big reason why is because many more niche/enterprise applications are really hard to test on in the real world, hence the need for sims!

re: labs doing this internally. They definitely are! However, the scale of sims buildout is going to be massive, probably many orders of magnitude above what we have today. We think it makes sense for one central player to do this because a really good simulator can be used by multiple people at once. It doesn’t make sense for every AI lab/company to build out their own environments if an industry standard catalog exists.

[zebomon]

Intriguing analysis. I'll be following along with interest!

[reactordev]

Conway’s law strikes again…