[patal]

I like the riddle. But the framing is unfortunate. When divising riddles, you want ambiguity where it serves the riddle, but be precise elsewhere so that the solver doesn't get needlessly distracted.

Their AIW riddle is: "Alice has 4 brothers and she also has 1 sister. How many sisters does Alice’s brother have?"

Now it should've been: "How many sisters do Alice's brothers have?" or "..does each of Alice's brothers have". Why single out a specific brother, when you haven't introduced this topic, and it is irrelevant to the riddle? Naturally, a human would ask "Which brother?", fully knowing that it is not important to the riddle.

Since this grammatical distraction puts an additional burden on the LLM, the authors muddled their original goal, which was to provide an easy riddle. I think it may have also muddled their data.

[patal]

Their AIW+ riddle is just ridiculous. It contains so many ambiguities, that there are several correct answers, even though the authors claim there be only one.

Which is really unfortunate. Because now it only shows that LLMs have problems answering ill-framed riddles.