[eeeeeeehio]

I agree with you. But what part of the blog post, or the peer review process in general, do you think ensures that only true ideas get in front of eyeballs?

I can write anything I want in the paper, but at the end of the day my experiments could do something slightly (or completely) different. Where are reviewers going to catch this?

[rocqua]

As the Article said, the first page gets a paper accepted. The remaining pages serve to not get the paper rejected. That includes actually backing up the claims on the first page.

Wrong things definitely still make it through, both mistakes and fraud. But it is a pretty strong filter.

[eeeeeeehio]

I agree that peer review can be a strong filter, but it's a filter for claims and evidence that sound true. CS papers can and do hide important details in the code (details which, I argue, would get a paper rejected if they were stated in the paper).

Regardless of the strength of the filter, if the filter's inputs are just "the paper", but the claims depend on the details in another artifact (i.e. the code), how can we argue that peer review filters for the truth?