In this blog post I give you a very brief idea about what I, as an editor, want to see when I receive a referee report. This is my own personal view, but I know it is shared by many other editors out there. So here is a brief guideline what to do and what not to do.
Whenever I ask a referee to judge the quality of a paper, I ask the following: I would be very happy if you can judge the article with emphasis on the following criteria: is the article novel? Does it add something interesting to the literature? Is it of high quality (the aim is top level in environmental economics)? Basically, I search for a novelty in papers, so it is not enough to have a nice paper with a good approach but there must be something catchy that readers will find interesting. I would be happy if you can judge the paper along these lines, too.
This is the request I tend to send to all reviewers, and obviously this is for a top field journal in environmental economics (Environmental & Resource Economics). Editors from other journals may not aim at such a high standard. If the paper is just another contribution, not really novel, or not that interesting, then referees should point that out. So the objective of a good referee report is to very briefly state whether and why the contribution is worthwhile (or not) to be published.
Do NOT spend time writing a review or a summary of the article. Quite a lot of referees simply paraphrase the abstract of the papers they are reviewing. Neither the editor nor the author need that information, it just wastes everyone’s time. It is an editor’s job to read the paper himself in order to make up his (or her) opinion, too. All good editors will do that. This takes time, but that is the job of an editor.
Never ever insult the author in your referee report. This just shows that you are absolutely partial and me as an editor will put much less weight on your opinion. Plus I will have to excuse myself for having chosen such a disgraceful reviewer who is hiding behind anonymity in order to contribute nothing but to insult. For example, in a recent referee report I have been called an ecofascist, I have been called a dinosaur (because I wrote that climate change impacts mankind, which for the referee seems to exclude women), and other nasty words that I don’t want to repeat. The report itself has shown that the reviewer had very little understanding of the topic but simply used the opportunity to hide insult behind anonymity. Stay polite, don’t do to others what you don’t want to have done to yourself. A simple rule.
Do not view it is as your objective to rewrite the paper for the author by adding your personal touch. If there is clearly something wrong with the setup then say so and that is sufficient reason for rejection. I very much dislike referee reports that span over several pages where reviewers accumulate a lot of minor points that an author can easily handle and then claim that is sufficient reason for rejection. It is not. If you find nothing seriously wrong with the paper then the ton of minor points can be handled by the authors in a revision. So write your report as follows: If you find a serious problem that you believe kills the article and that the authors will not be able to handle, then there is no need to write another five pages of small points that need to be addressed. Don’t waste your time on these as they will not be important for the decision.
If you find nothing wrong with the paper then don’t go fishing. If you have nothing to add then perfect, don’t. You don’t need to feel that you failed as a reviewer if you did not find crucial (or small) problems with the paper. It might just simply be good enough. Some papers are.
Be honest about whether you can review the paper. Does it really fall within your field of knowledge? Some reviewers have changed topics and the editor doesn’t know this. Editors have databases of reviewers to choose from and we choose you according to your classifications, sometimes if you have been cited in the paper, and quite definitely if you have been criticised in the paper. I don’t understand why some journals ask authors to suggest potential reviewers themselves. No author would choose someone that they don’t know (how can they?), and from the pool of those that the authors know they are certainly not going to choose someone they don’t like. All editors are aware of that bias of course.
If it is a theoretical paper then don’t argue that the authors don’t do an empirical study (and vice versa). Either there is something wrong with the paper or there isn’t. The editor will know that it is only a theoretical paper and, before having having sent out the paper to reviewers, should have read the paper in order to see whether it complies with the minimum standards of the journal and whether or not it will fit into the journal’s scope. So there is no need for you to do this job as it is the editor’s.
You can write, very briefly, if the paper is not well-written or the language is not good. But that is not a reason for rejection and it should only be a minor point that can be handled in a revision. It is the idea or the contribution that counts. Having said this, it is also my opinion that journals should do much more at the end of the review process to help polish the papers. But nobody should expect reviewers or editors to spend a considerable amount of time on this. Unfortunately this is still the case for most journals.
In a nutshell, what do editors want:
- No review but a very, very brief statement about whether or not and why the contribution of the paper is significant enough for publication.
- If there is a serious problem with the paper, as in the maths is wrong, the econometrics, or the setup is wrong, then be (kindly) destructive.
- In all other cases be constructive – don’t make the author rewrite the paper because you don’t like the style, but simply help the author by asking to clarify the one or the other unclear point.
- Do not insult the author(s).
- Be nice, constructive if a revision is in order, maybe even slightly apologetic if you feel a rejection is better. Remember, most authors have spent a significant amount of their lifetime on this paper and are certainly biased towards it. No author is happy about a rejecting, so don’t make it harder on them by trashing the paper.
So, if you are a (future) reviewer, I hope these brief points help you a bit to shape your next referee reports.