[willahmad]

I think this benchmark could be slightly misleading to assess coding model. But still very good result.

Yes, SVG is code, but not in a sense of executable with verifiable inputs and outputs.

[jstummbillig]

I love that we are earnestly contemplating the merits of the pelican benchmark. What a timeline.

[andrepd]

It's not even halfway up the list of inane things of the AI hype cycle.

[hdjrudni]

But it does have a verifiable output, no more or less than HTML+CSS. Not sure what you mean by "input" -- it's not a function that takes in parameters if that's what you're getting at, but not every app does.