[harph]

It seems it's because OP is generating the whole screen every frame / every move. Of course that will give inconsistent results.

I wonder if this approach would work better:

1. generate the whole screen once

2. on update, create a mask for all changed elements of the underlying data

3. do an inpainting pass with this mask, with regional prompting to specify which parts have changed how

4. when moving the camera, do outpainting

This might not be possible with cloud based solutions, but I can see it being possible locally.