"Setting and applying editorial quality standards [at Journal Of Independent Medical Research] in lieu of the conventional peer review process"

White Paper requested by Ed Sequeira (PMC) on 5/22/2003

Incorporating, by reference, "There already is a better way - 'Collaborative Publishing'" [1]

Author: Trevor G Marshall, PhD, Managing Editor, Journal of Independent Medical Research.

Introduction

Every four years the ‘Journal of the American Medical Association' and the 'BMJ Publishing Group' organize a congress on peer review. JAMA published the proceedings of the most recent conference in their June 5, 2002 theme issue.

Summarizing the congress, Rennie said [2]

"we find ourselves in the peculiar position of believing still more in the virtues of peer review, a system we know to be "time-consuming, complex, expensive and . . . prone to abuse"[3], while we acknowledge that the scientific evidence for its value is meager. Indeed, if the entire peer-review system did not exist but were now to be proposed as a new invention, it would be hard to convince editors looking at the evidence to go through the trouble and expense."

Further,

"we publish studies that fail to show any dramatic effect, let alone improvement, brought about by editorial peer review. Yet, despite this, it continues to be the experience of editors that peer review is extraordinarily effective, sometimes in saving the reputations of the authors. Why? It makes good sense that editors would want to enlist the services of those more expert in a particular subject than themselves. And there are powerful reasons why editors might wish to spread the responsibility for unfavorable decisions about manuscripts. But there is another important factor. Peer review represents a crucial democratization of the editorial process, incorporating and educating large numbers of the scientific community, and lessening the impression that editorial decisions are arbitrary." [2, 4]

 

The Journal of Independent Medical Research (JOIMR)

JOIMR was conceived and structured to enhance this "democratization of the editorial process".

‘Collaborative Publishing’ involves no fees or emoluments. All participation is voluntary. There is no advertising, no stipends. Everything is "pro bono".

JOIMR is an Open Journal. It has evolved a model of ‘Open Peer Review’ in order to establish and maintain the very highest of standards.

We have additionally put systems into place to ensure that fraud does not occur.

An author’s manuscript, together with the comments and formal reviews from his/her peers, are online for all to see. There is no secrecy to distort the "democratization". The author is able to see detailed criticisms, and to argue against them in a manner conducive to dispute-resolution. The Peer-Reviewers are able to question the author (in public) in order to better draw out any issues which may not have been properly explored in the paper.

 

JOIMR’s Collaborative-Publishing-model is cost-efficient

Another driving force behind JOIMR is to address the "expense" inherent in current models of peer review [2]. The Journal (at www.JOIMR.org) is laid out as an online message system, and requires maintenance from its editors only when inappropriate (abusive or profane) messages have been posted, and when formal peer-review comments have been left (and therefore a paper is ready to be ‘published’). JOIMR’s software takes care of the manuscript input, peer-review, and conversion to the NLM DTD compliant XML required for archiving and dissemination by PubMed Central.

Such a model has only been made viable by the existence and charter of PubMed Central. JOIMR focuses on doing those things that can be done well in a collaborative process, while the resources of the NIH and NLM effect archiving and distribution (through PubMed Central).

Fenter recently highlighted another efficiency "the concept that an author must keep submitting a paper to journal after journal until it finally passes through a peer review is ridiculously wasteful of resources." [5] JOIMR’s collaborative structure encourages mentors to help inexperienced authors improve their manuscripts, so that they can be resubmitted (to JOIMR) in a more acceptable format.

 

Internet Identity Fraud

There is an practice emerging within ‘conventional’ peer review which warrants close attention. Some journals [eg 6], have begun requesting that email addresses of potential peer reviewers be submitted by authors. All communication between editors and those peer reviewers is then carried out (expeditiously) by email.

The opportunity for identity fraud becomes significant when individuals are contacted and identified only by their email address. Does the email address ‘Prof.John.Doe@yahoo.com’ really belong to Professor John Doe? Anybody could have registered that email address with Yahoo…

JOIMR has several mechanisms in place to guard against such identity fraud. Firstly, the editor of an online Journal is less likely to be taken-in by such 'email spoofing', and, even if he/she does miss the problem, there is a very good chance that somebody else (online) will notice. Secondly, JOIMR logs a parameter called the "IP Address". This is a unique number that points to the unique computer on the internet from which the peer review was posted. The IP address and email address and supposed identity of any peer-reviewer must correlate to satisfy the editor. If there is any confusion, a telephone call will be placed to confirm authenticity of the review.

The final mechanism that JOIMR uses to guard against identity fraud is Open Peer-Review. The peer reviews are posted publicly on JOIMR, in perpetuity. They are also forwarded as appendices to PubMed Central. There is very little chance that author fraud will go unnoticed.

Qualifications Required of Potential Peer-Reviewers

JOIMR has set a very high standard for the qualifications of peer-reviewers. Before a manuscript is ‘published’ and transmitted to PubMed Central, it must be recommended for publication by two, independent, qualified peer-reviewers. The detailed review criteria are published online [7], but, briefly,

A professional is trusted to post Peer-Reviews if he/she

  1. is an active or retired academic staff member of an accredited tertiary institution who has exercised direct research and curriculum responsibilities, or
  2. is an active or retired professional who is credited as an author of at least two papers which have been indexed in MEDLINE (PubMed)

Formal JOIMR Peer-Reviews include a ‘header’ section which contains the qualification of the reviewer and a declaration of any "Competing Interests" using the aide-memoire currently promulgated by BMJ.

Summary

JOIMR has implemented an initial model of ‘Collaborative Publishing for the 21st Century’ which maintains, and seeks to improve, conventional standards of peer-review. It does this while containing costs at levels commensurate with the ‘Collaborative Publishing’ paradigm.

Everything implemented at JOIMR.org is not only ‘Open Publishing’, it is in the ‘Public Domain’. From the software that drives the Internet servers and the JOIMR website right through to the Copyright Agreement covering the manuscripts themselves.

JOIMR anticipates the likelihood that organizations such as Central Libraries and major Universities will eventually take over (and enhance) the publishing model that we have pioneered. JOIMR would regard this outcome as ‘success’, not as ‘competition’.

 

References

1. Marshall TG: There Already is a better Way - Collaborative Publishing. BMJ Rapid Response, 7 May 2003. Available from URL http://bmj.com/cgi/eletters/326/7396/945#32007

2. Rennie D: Fourth International Congress on Peer Review in Biomedical Publication. JAMA 2002; 287:2759-2760
http://jama.ama-assn.org/cgi/content/full/287/21/2759?ijkey=666795a27cd57bdc2deed98b1d2f98a28141bf78&keytype2=tf_ipsecsha

3. Horton R: The hidden research paper. JAMA 2002;287:2775-2778
http://jama.ama-assn.org/cgi/content/abstract/287/21/2775?ijkey=155a5d4bfec354c75e5e405199e2c417c3a7a4f3&keytype2=tf_ipsecsha

4. Rennie D. Problems in peer review and fraud. In: Balancing Act: Essays to Honour Stephen Lock. London, England: Keynes Press; 1991.

5. Fenter B: Building a Road from A to B. BMJ Rapid Response 19 May 2003
http://bmj.com/cgi/eletters/326/7396/945#32466

6. BMC Central Ltd: Instructions for BMC Pulmonary Medicine authors. available from URL http://www.biomedcentral.com/bmcpulmmed/ifora/

7. Journal Of Internet Medical Research: Peer Review for Collaborative Publishing. Available from URL http://www.joimr.org/phorum/peer-review.htm