Peer Review, Anyone?

Replies to This Discussion

Permalink Reply by The Secular One on October 9, 2009 at 4:11pm

Basically:

You submit a paper for publication to Journal X.

Journal X sends that paper out to 3 or 5 or some number of people they consider experts in the field due to their previous publications on the topic or a related topic.

These experts look over your paper and determine if it meets scientific rigor for publication. A paper can be publishable even if the expert doesn't agree with the findings.

The experts give suggestions, if necessary, about things that could be changed to make it better.

Paper is edited and sent back.

Repeat previous stuff. If paper is considered good by panel, then the paper gets published.

That's a simplified process. Maybe some of our published members can further enlighten us.

Basically though, Journal X only becomes known as a good journal if they pick appropriate peer reviewers that ensure they get good papers. If the paper is scientifically invalid and they publish it, they look like idiots. If they reject a good paper, a more fair journal can pick it up and publish it and make themselves look better.

And even if a paper gets published, if it is crap it will never be cited and fall into oblivion.

Permalink Reply by Gwen on October 9, 2009 at 7:32pm

Journal X sends that paper out to 3 or 5 or some number of people they consider experts in the field due to their previous publications on the topic or a related topic. Ideally, these people would have also published research that has gone through the peer review process as well.

Permalink Reply by Gwen on October 9, 2009 at 7:34pm

And even if a paper gets published, if it is crap it will never be cited and fall into oblivion. Some are so wrong, they are cited for the wrong reason, such as Andrew Wakefield's papers. The middling ones are ignored and fall into oblivion.

Permalink Reply by HotMess on October 9, 2009 at 9:22pm

... and this is where the peer review process falls short. The whole anti-vax movement started with him and has just grown and grown. Many people still feel that he is justified and correct and are protesting his censure. And every time you try to explain that a particular anti-vax researcher in not creditable because s/he failed the peer review process, they fault the process and not the researcher.
People need to know more about just who is qualified to issue a judgement during the review process and what factors are considered when evaluating a paper.

Permalink Reply by The Secular One on October 9, 2009 at 10:31pm

Odd how the process works for everything else given time to work it out.

Permalink Reply by Max Polun on October 9, 2009 at 4:18pm

I'm not a working scientist, and I've never tried to get a peer reviewed paper published, so I'm not an expert, but my understanding is that the reviewers are checking the methodology and experimental design/etc. There probably is some bias against unconventional ideas in peer review, but if these ideas have merit, they eventually find an audience in the scientific world. The fact that ideas that were originally laughed at (like say, continental drift) are now the consensus proves that it's possible.

The reason anti-vax research doesn't get into peer reviewed journals is because it's crap, you don't need bias to explain that.

Permalink Reply by Mike Hein on October 12, 2009 at 11:31am

Great info, Adriana. Your experience mirrors my (much less extensive) history when working on a JAMA article, long ago.

What your notes point out is that these editors have enormous power. Just the ability to choose the reviewers will influence what papers see the light of day. The amount of editing demanding by the reviewers can materially change the "conclusions" or "findings" section of research paper. It is like democracy, "the worst possible system except for all the others."

Regards,

Permalink Reply by Mike Hein on October 13, 2009 at 7:59am

fair enough and good point. I was being a bit limited in my thinking on journals.

Pssst....you're pretty f'n smart.

Cheers,

Permalink Reply by HotMess on October 12, 2009 at 9:17pm

I guess that there should be some publicly posted and universal definitions for "quality of data" and the general public should be able to access the requirements that make a research project viable or not. I understand that, within the research community, these concepts are clearly defined and understood, but when someone is unhappy with their evaluation after the peer review process and they take their case to they laypeople, they can say whatever they want. If a paper is rejected, the reasons ahould be clearly stated and easy for the public to access.
This is where the fringe element always jumps in with their qualifiers and complaints.

Permalink Reply by The Secular One on October 15, 2009 at 4:34am

There are ID people here too.