Domhnall MacAuley is a CMAJ Associate Editor and a professor of primary care in Northern Ireland, UK
Monday 28 September to Friday 2 October is “Peer Review Week“.* This gives us an opportunity to thank those who do such a superb job in assessing, advising on, and enhancing the quality of academic work. We greatly appreciate the time and effort you invest on our behalf, on behalf of our authors and, on behalf of the patients who ultimately benefit from medical research and education. We could not do our jobs without you.
Why peer review?
Academic medicine could not function without the careful and generous work of those who help maintain and improve standards of academic publication and, indeed, grant assessment. Sadly the work of peer reviewers is not rewarded, either financially or in academic esteem, and yet a good peer review of an academic paper can do so much to improve the quality of research, clarify the reporting, and help authors position their work within the context of what is already known and published. A good review probably takes between one and three hours. No journal could adequately reward you for that time. Some reviewers contribute so much to the development and content of a paper that they could almost be classed as authors. Peer review is much criticized, however, with commentators saying that it is ineffective at picking up flaws, open to bias, and, more recently. even some deeply troubling reports of fake peer reviewers. It is certainly not perfect but it seems to be the best there is at present and no one has come up with a better system, just yet. In 2011 the UK House of Commons Science & Technology Select Committee conducted an inquiry into peer review. As reported by Richard Poynder in his blog, when the Chair of the committee suggested that it might be helpful to conduct some research into the efficacy of the current system — on the grounds that “evaluation of peer review is poor”, the then director of the Wellcome Trust, Sir Mark Walport , replied: “Peer review is no more and no less than review by experts. I am not sure that we would want to do a comparison of a review by experts with a review by ignoramuses.”
What is considered a good peer review?
You are the consulting expert and, while the editor may very much value your opinion, you should be able to give an honest assessment without feeling you carry the responsibility of the final decision. The editor is ultimately responsible. The ideal peer review report starts with a short summary as a general assessment of the paper and a longer body of the report giving a more detailed critique of the individual components of the paper. The editor asked you to look at this piece of work because we feel you have greater expertise in this field than we do. We take your opinion on board but sometimes, even if you identify shortcomings, we may decide to publish. No paper is perfect and sometimes an editor may feel a paper, despite some (minor) flaws, is still worth publishing. We hope you will identify any major methodological problems and we do want you to say if a paper contributes to the literature but, sometimes reviewers can be excessively harsh. Mis-spelled words, grammar and language problems are not of prime interest to us. What we really need is to know if the work is well done, robust and adds to current knowledge. The key to being a good peer reviewer is to give a fair and honest assessment.
Who is a good peer reviewer?
YOU. As an interested expert in your field you are the person we would like to hear from. Those who are currently undertaking or have just completed their doctorate are often the best reviewers as they are immersed in their topic and, as a result of their PhD, may be one of the world’s experts in that particular specific research question at that time. Senior academics or topic research leaders may not, paradoxically, be the best reviewers. They may be overwhelmed with requests to review, write and lecture and not have time to write a comprehensive and detailed report. The electronic equivalent of a note scribbled on the back of an envelope in an airport departure lounge is not of great value.
Where is peer review going?
We don’t know. We have seen various initiatives that attempt to improve the process, with arguments for and against open peer review, blinded peer review and advocates of many hybrids and variations. While we would all encourage transparency, the main criticism of open peer review is that an identified peer reviewer is vulnerable, especially a junior researcher. On the other hand, reviewers can be excessively critical if allowed to hide behind a veil of anonymity. Signing a review is about taking responsibility. Yet reviewers who are identifiable may be less likely to criticize a colleague’s work, particularly if everyone knows each other in a small research field. There have been various experiments with the pre-publication peer review process, complete transparency of the full review process, including at CMAJ Open, transfer of peer reviews between journals, and even companies that will now provide a peer review service.
Where are we now?
Despite all the innovations in communication, multimedia dissemination, and advances in research, we all still depend on experts to help us identify and improve the quality of your publications. And, that means you, our peer reviewers. All of us at CMAJ and CMAJ Open extend to you a very sincere ‘thank you’
*Peer Review Week 2015 will run from Monday 28 September to Friday 2 October. Peer review week grew out of informal conversations between ORCID, ScienceOpen, and Wiley. This year Sense About Science has also joined the week, which will include “a series of blog posts and interviews, a social media campaign (#peerrevwk15), a webinar on trust and transparency in peer review, and more”.