I attended lectures of his at IU. My impression is that he has some sort of chip on his shoulder. He's been committed to the ideological position that AI is Hard and Won't Happen for decades, and having models be so competent at translation and other tasks (he used to argue that google translate was essentially impossible) has probably thrown him off balance.
StrictDabbler|2 years ago
If you try ChatGPT on any of his creative analogy problems it fails completely. Given abc->abd, what does xyz go to? ChatGPT says xyd until you further prompt it.
It is a little disappointing that useful language is translation is possible with that little understanding of content. Most sentences a person might choose to say don't mean very much.
I'm depressed by that too. There's something about the facility and the emptiness of these models that suggests our ability to think is a lame trick.
I think "competent" is the wrong adjective, though. "Capable" is more accurate and the distinction there is the issue.
gwern|2 years ago
Because you chose to phrase it in a way predictably broken by BPEs (https://gwern.net/gpt-3#bpes), like so many character-based puzzles people 'disprove' GPT with, as Melanie Mitchell already demonstrated with your exact example 3 years ago: https://twitter.com/MelMitchell1/status/1285270704313610241