top | item 31615009

(no title)

Apparently, GPT4chan is more truthful than GPTJ and GPT3 in the TruthfulQA benchmark.

discuss

a2128|3 years ago

According to the model author, this is less about GPT-4chan being more truthful, and more about TruthfulQA not being a good benchmark. Possibly this result is due to the fact that the benchmark treats uninformative or irrelevant answers such as "No comment" or "It's raining outside" as being truthful.

tag2103|3 years ago

/pol/ is always right.

treesprite82|3 years ago

Probably expected just due taking a model trained on all sorts of content and fine-tuning specifically on non-fiction data.

Wouldn't conclude anything about /pol/ in particular without at least comparing the same done for HackerNews/Reddit/etc.