top | item 44939288

(no title)

relevant_stats | 6 months ago

I'm very surprised according to results people struggled with identifying [3] and [4] as AI.

IMO both are simply bad and both contain usual telltales in spades (continuity problems, failed or trite metaphors/analogies, semantic failures, overall feeling of 'wtf is even being attempted here').

I'm not so surprised that people struggled with identifying [1] as human - the confounding factor is that this flash story is unpleasantly written, and it's not easy to realize that its failure modes (eg. trying to cram too much in too short a text) are rather human like. And I'm sure the fact that arguably the hardest to digest and rather bad human story opens the poll might somewhat influence the further analyses.

As others in the poll I failed to identify [5] as AI even though in hindsight the telltales are also there. That's because I rather liked it, and as a result it was harder to be vigilant. I also was very undecided on [8]. Finally I scored 6/8, but I wouldn't say it was easy.

Shame that comparing to the previous contest https://mark---lawrence.blogspot.com/2023/09/so-is-ai-writin... is not straightforward. In that one I scored 9/10 while having very easy time (I didn't even finish reading some of them before making up my mind). I also felt completely excused with my only failure, incorrectly identifying as AI the story written in the style of exhaustingly banal fan fiction. But frankly I found almost all the human stories in the previous edition better then the current ones.

In retrospect ChatGPT4 was a terrible writer. ChatGPT5 seems to be an improvement to the admittedly worrying point. Still not impossible to discover though.

However these are my impressions only and it looks maybe I was lucky and I should not generalize it? According to the website people had serious trouble discerning gpt4 writing also 2 years ago. And I'm rather shocked they did. And that they scored some of those banal AI stories positively.

If it's not luck on my part, then maybe discerning AI writing is a skill very different from 'writing' or 'being deeply interested in literature', skills of people who usually frequent this blog?

discuss

order

No comments yet.