If you're implying that generalization isn't at play because game knowledge shows up in its training data, you can disabuse yourself of that by watching the stream and how it reasons itself out of situations. You can see its chain of thought.
It spends most of its time stuck and reasoning about what it can do. It might throw back to knowledge like "I know Pokemon games can have a ledge system that you can walk off, so I will try to see if this is a ledge" (and it fails and has to think of something else), but it's not like it knows the moment to moment intricacies of the game. It's clearly generalized problem solving.
For eg., Bug type attack is super effective against Poison type in Gen 1 but not very effective in Gen 2 and onnwards. But Claude keeps bringing Nidoran into Weedle/Caterpie.
hombre_fatal|1 year ago
It spends most of its time stuck and reasoning about what it can do. It might throw back to knowledge like "I know Pokemon games can have a ledge system that you can walk off, so I will try to see if this is a ledge" (and it fails and has to think of something else), but it's not like it knows the moment to moment intricacies of the game. It's clearly generalized problem solving.
minimaxir|1 year ago
If you watch the Twitch stream it is obvious Claude has general knowledge of what to do to win in Pokémon but cannot recall specifics.
northern-lights|1 year ago