[gunshai]

This seems highly suseptible to what a human might consider irrelevant randomness. Given random images just shuffling indefinitely a curious individual will just give up and say, even though I can't predict the next thing it doesn't pertain to the domain of curiosity.

[dimatura]

Right, that's why the "noisy TV problem" is a "problem" - it makes the simple model of curiosity used in the research less effective, because it causes agents to get stuck watching TV instead of exploring and advancing in the game, which is what we'd want them to do. (Though this does seem somewhat reminiscent of certain human behaviors...). One possible solution is to equip agents with a more expressive predictive model, capable of discerning "interesting" randomness from "uninteresting" randomness (for some TBD definition of "interesting", of course I'm handwaving here - I'm sure in the 5 past years there's been progress on this front).

[jhardy54]

First thing that popped into my head: “interesting randomness” is when you can’t predict future frames, but [within some interval] you gain that ability.

Static on the TV is random but uninteresting, whereas morse code is “random” at first, but after enough exposure can be understood and predicted.