[czr80]

So many of these discussions take it for granted that publishers don't do anything of value. This blog post, by an open access advocate, was for me very illuminating on what value they actually add (http://cameronneylon.net/blog/polecon-of-oa-publishing-i-wha...)

Quoting from that link: "One of the big challenges is discussing the costs and value added in managing peer review is that researchers who engage in this conversation tend to be amongst the best editors and referees. Professional publishers on the other hand tend to focus on the (relatively small number of) contributors, who are, not to put too fine a point on it, awful. Good academic editors tend to select good referees who do good work, and when they encounter a bad referee they discount it and move on. Professional staff spend the majority of their time dealing with editors who have gone AWOL, referees who are late in responding, or who turn out to be inappropriate either in what they have written or their conflicts of interest, or increasingly who don’t even exist! .... Much of the irritation you see from publishers when talking about why managing peer review is more than “sending a few emails” relates to this gap in perception. The irony is that the problems are largely invisible to the broader community because publishers keep them under wraps, hidden away so that they don’t bother the community."

I'm sure there must be some way to achieve this and also make the content freely available. But whatever system replaces the current one has to deal with these issues too.

[return0]

> I'm sure there must be some way to achieve this and also make the content freely available. But whatever system replaces the current one has to deal with these issues too.

Post-publication peer review could solve all of these (and the access issue too!). I'm still not convinced of the superiority of pre-publication review by only 3 (usually quite busy) people max.

[dalke]

I haven't had good experience with modern post-publication peer review so far.

My biggest complaint is that few know those comments exist. For example, I commented on one paper to highlight methodological problems that make it impossible to trust their results. (They developed a new algorithm for X. They compared it to the naive implementation. The naive implementation was poorly coded. Most of the speedup disappears by in comparison to a well-written implementation.)

I've talked to a few people about this exact paper. None noticed the link to the comment page.

I say "modern" because post-publication peer review isn't new. One of the older mechanisms was the letter to the editor. Those letters (at least in the ACS journal I'm thinking of) have a DOI and are searchable. A few of these letters have proved useful to my research.

But the modern post-publication peer reviews don't have a DOI and aren't indexed by Google Scholar or other systems, so they are less useful than a old Letter to the Editor.

(I once asked about sending a Letter to the Editor to an Open Access journal, concerning problems in one of their publications. They said they don't support those sorts of short communications, and my only option was the $1,500 to publish a full paper, which would have to go through the normal peer review process.)

[Fomite]

I have been extremely unimpressed with my experiences post-publication peer-review. Limited to no feedback for many papers, and some clearly "How dare you stray onto my turf" related comments.

[visarga]

You mean Elsevier doesn't need to earn $2 billion for hustling free editors to do their work proper? You know, it's more than just pushing emails around. You have to deal with people.

[KKKKkkkk1]

I don't understand. What is the difference between post-publication peer review and no peer review? What is a journal for if the papers are not peer reviewed?

[notalaser]

Just saying -- perhaps "editors who have gone AWOL, referees who are late in responding, or who turn out to be inappropriate" would likely be less of a problem if they were properly paid for what they're doing. I haven't been in academia for a long time (and my stint was short), but I don't remember ever hearing of a reviewer who got anything more substantial than maybe a discount or a one month of free subscription to a journal from a publisher. Managing peer review is admittedly more than "sending a few e-mails", but not in a good way.

[dalke]

What is "properly paid"?

The last review I did (for an open access journal) took ~6 hours. At minimum wage that's about $50. There were two reviewers = $100. Reviewer must get paid even if the recommendation is "don't publish", otherwise reviewers will have an incentive to say "yes". The OA fee for that journal is about $1300. If 50% of the peer reviewed papers are published, then the average price for peer review would be $200, which would raise the price to publish by about 10%.

It is an open access journal, so everyone effectively has a free subscription to it.

One alternative is a credit model, where N reviews gives a discount on the price to publish. However, as the above points out, it may be more cost effective to mow lawns on the weekend than it is to plan for that discount.

Then there's the question of perceived fairness. If one reviewer says "great job. Publish" and another gives 10 pages of useful critique, is it proper to pay each the same?

[maaku]

People would be more sympathetic if peer reviewers were actually paid for what they do.