From Joe Pater (firstname.lastname@example.org)
Bruce Hayes has agreed to let me post the following draft review guidelines he came up with a little while ago (though see his caveat below). I thought that putting these in the public domain might inspire reviewers to better practices, and perhaps also inspire journals to provide explicit guidance of this sort to their reviewers. I’d be interested to know about other sorts of advice that could be usefully given, and if there are existing explicit guidelines that people know of.
Sure, feel free to circulate what I wrote. Looking it over now, I realize it is far too blunt and bullying to serve as an actual “Instructions for Referees” document and I hope people will read it instead as a spontaneous cri du coeur from an experienced/beleaguered Associate Editor.
1. Please try to have your review completed within one month. If you anticipate your review taking longer, please contact the editor and obtain her approval first.
2. Your review should include both evaluative comments and a firm editorial recommendation. The latter may range from “accept without change” to “reject without invitation to resubmit”. The widely-employed intermediate recommendation, “revise and resubmit”, is too vague if it is not augmented with appropriate comments. Specifically, you can say what specific changes would need to be successfully implemented in order for the paper to become publishable. You can also say whether it is acceptable to you not to review the revised version; i.e. whether you want simply to trust the author to make the appropriate changes.
Do not use “revise and resubmit” merely as a euphemism, intended to be kindly; while a recommendation for outright rejection may seem harsher, it is likely to spare many people unnecessary work. It may also be more helpful for the journal’s goals of enforcing high scholarly standards.
3. A review can achieve greater credibility if it begins with a summary of the paper’s main point; this helps establish that the reviewer understands the paper he is reviewing.
4. Review the paper in its own terms. If it relies on a theoretical framework that has attracted publication and attention, then you should assume that it is acceptable for this framework to be used; do not use the review process as an arena for framework-warring. There is usually plenty to say about the quality of the evidence and reasoning keeping within the general assumptions of the paper.
5. Bear in mind that very long reviews can be problematic. They certainly generate extra work for authors and editors, and in the long run they change expectations about what is required of reviewers, so that reviewing becomes a heavy burden and it is difficult for editors to find volunteer reviewers. Long reviews are also likely to result in very long revised papers, crammed with passages that lack the authors’ own conviction but are intended to mollify skeptical reviewers. You may wish to read over your completed review before submitting it, trimming back items that on reconsideration seem unlikely to result in improvement.
6. It goes without saying that a review should maintain a high degree of professionalism and courtesy to the author. Rereading and editing your review before submission with this in mind can be helpful.
7. If you like to note typos and grammatical infelicities, feel free to put them at the end of your report. However, it is not required that you do this, and if it delays the completion of your review you should consider omitting this task.