[MichaelBosworth]

Would be curious to hear an elaboration on this perspective. In your opinion, on which measures of intelligence would GPT-4 fail to out-perform a human with an IQ of 80? Conversely, on which measures do you imagine it would succeed at doing so? Are the latter less significant or valid than the former?

[chimprich]

Conscious thought. In biological terms it has a superhuman cerebellum but no cerebral cortex at all. It can't assess what it's doing.

GPT4 will produce stuff, but only if prodded to do so by a human.

I recently asked it to help me write some code for a Garmin smartwatch. The language used for this is MonkeyC, of which there isn't a huge amount of examples on the internet.

It confidently provided me with code, but it was terrible. There were gaps with comments suggesting what it should do, bugs, function calls that didn't exist, and many other problems.

I pointed out the issues and GPT4 kept apologising and trying new stuff, but without any improvement. There wasn't any intelligence there; the model had just intuited what a program might look like from sparse data, and then kept doing the same thing. It didn't know what it was doing; it just took directions from me. It couldn't suggest ideas when it couldn't map to a concept in memory.

A human with an IQ of 80 would know if they didn't know how to code in MonkeyC. If they thought they did, they'd soon adjust their behaviour when they realised they couldn't. They'd know where the limit of their knowledge was. They wouldn't keep trying to guess what functions were available. If they didn't have any examples in memory of what the functions might be like, they might come up with novel workarounds, or they'd appreciate what program I was trying to write and suggest a different approach.

Presumably we'll make progress on this at some point, but I think it'll take new breakthroughs, not just throwing more parameters at existing models.

[hammyhavoc]

Exactly my experiences. With a fucking NGINX configuration, for which I provided it the documentation, and the URL rewrite lines it would require. I spent days on trying to find the value that other people are claiming it has.

[reso]

Humans handily outperform GPT4 handily on the task of "write a random string of length [x]" for any x > ~25.

[olddustytrail]

If you asked most people to perform that task, they literally wouldn't have a clue what you'd just asked them to do.

[pclmulqdq]

They have a specific device to do that now. I have tried to say "write a random sentence with 6 words and 2 numbers" and it completely fails, but it can do the straightforward "write a random [x] of length [y]."

[verbify]

I got

"Here is a random string of 32 characters:

a8Jk5pYr0Dm9Nc1Vz8Qf2Bt6Hg3Lw4Uo"

[ux-app]

a 4 year old would fail at this task.

does a 4 year old have intelligence?