[duskwuff]

> Automatically updating screenshots

By running the software and getting it into an equivalent state to the original screenshot? Impressive if true.

On the other hand, if you're just "updating" the screenshot by making the AI model imagine what it would look like with some code changes - that's worse than useless; it's likely to end up generating screenshots that won't match any released version of your software at all.

[prithvi2206]

Yep. You give Promptless access to a test account and it navigates and interacts with your product to capture new/updated screenshots.

Here's an example from our own docs, where you can see the captured screenshots if you scroll down to the PNGs that Promptless added https://github.com/Promptless/docs/pull/233/changes

Getting the browser navigation to work was predictably hard, but the other thing that's very hard to get right is timing when Promptless takes the screenshots. Some teams want Promptless capturing screenshots on a staging environment, and sometimes it's on prod, and different teams have different release cycles and feature flagging processes.

For that reason, we still have to do a bit of manual set-up for each user that wants screenshot updating.

Agreed though that just asking nano banana to predict a new screenshot is negative value.

[bberenberg]

I've built something similar to this internally and absolutely agree that timing is the real challenge.

Getting an AI to navigate a process is now easy. However, the difficult problem is having it understand: 1. How to screenshot or, even better, capture a GIF of exactly the right points in time 2. How to move the mouse and pause at the right points in time such that a human understands it

[prithvi2206]

Actually, you might find this interesting: we wrote a blog post a while back about how we gave our agent a way to voice its suffering. The first example is from the screenshot updating subagent, where it got stuck after 84 browser interactions trying to log into a customer's app :'(

https://promptless.ai/blog/technical/i-must-scream