How to Review a Scientific Manuscript
But what makes a good review / reviewer for scientific journals? Pretty much all of us have gotten negative reviews, which ultimately preclude publication of a paper in a particular journal. And it’s hard not to take these comments to heart given how much effort went in to the paper. So what are the key elements to generating a balanced and fair review? I will often give manuscripts sent to me for peer-review to one of my trainees to assist. This was an opportunity afforded to me by my previous mentors and is a valuable training experience in critical evaluation of scientific data. However, I find that trainees can often become bogged down in the minutia of scientific details, particularly if the subject matter is very close to their own work. For example, comments like “why did the authors use pH 7.6 buffer instead of pH 7.4?”. While it is important to evaluate the rigor of the experimentation, this can often overlook the big picture and important first questions such as “are these findings sufficiently novel and advance the field to a degree required for publication in a particular journal?”. Dr. Emmanuelle Passegue (Journal of Experimental Medicine editor) echoed similar sentiments;
“Teaching students and postdocs to review submitted manuscripts is always difficult since they are usually largely negative without really articulating what would make them change their mind, and they focus on details rather than grasping the big picture question and importance for the field (I guess that last part came with time and experience).”
On the other hand, if the reviewer is a senior investigator who chooses to oversee the evaluation personally, there might be times when the reviewer is too busy with other tasks to give the manuscript the time it deserves for critical evaluation, resulting in a cursory review that does not carefully assess the finer details of the manuscript. There is a fine line between being overly critical in a way that doesn’t help the authors improve the manuscript, and not scrutinizing the details. In my experience, the best reviewers are tough but fair, and their comments should instigate a reappraisal of the data with some additional lines of investigation that aim to clarify or improve the manuscript. The best reviews question / state the novelty or significance of the findings, assess the rigor of the approach, and determining if the conclusions are supported by the data.
For additional insight behind peer-review, I asked the opinions of ISEH leaders who are editors of hematology-related journals. From Dr. Margaret Goodell (Blood editor);
“I try to use diverse reviewers and I have come to realize that not everyone has had good training in reviewing. A reviewer is doing two things: (1) helping the editor determine whether a manuscript is a good fit for the journal (e.g. does this uphold the standards of other papers you read in that journal), and (2) helping authors improve the manuscript for both readability and quality of the science.
The first issue of “fit” can be addressed by two separate questions. First, is the paper of sufficient impact for this journal (impact includes novelty and ability to change the way people think about a problem. Will anyone care to read the paper?). Every journal has different “impact” standards (not to be confused with impact factor) so this question should be addressed as suited for each journal.
Second, do the data presented support the claims? Is the reviewer convinced? Have the authors reached an adequate standard of proof? If the authors are way off on either of the questions, then the manuscript probably is not a good fit for the journal.
If the authors can do a bit to better “prove” their points, then it is probably worth a revision. Often, reviewers get bogged down in suggesting more experiments that won’t really change the outcome or better prove the points. Sometimes, additional experiments may help improve the novelty of the work. This tends to increase citations, which is good for the authors as well as the journal. But I prefer reviewers to really determine whether the there is sufficient proof for the claims and whether the claims are of sufficient importance (novelty/impact) to suit that journal. Ideally, reviewers will focus on the big picture and give concise and meaningful feedback that is the most helpful to everyone.
When deciding whether to use reviewers repeatedly, I also look for reviewers who seem to be fair and even-handed, and who can explain positive points as well as identify ways to improve a paper. I seek reviewers who are respectful and helpful to their colleagues, and not just nasty. We can all accept a tough but fair review- but no one likes to feel a review is unfair or overly-critical for the context.
Finally, it is certainly helpful to everyone for reviewers to improve readability by suggesting modifications to figures, legends and text. But reviewers can get carried away and want to point out every spelling or grammatical error, which usually gets caught during the publication process.”
One feature often overlooked is the difference between comments for the authors and comments (private) comments to the editor. Many reviewers will simply copy and past their reviews from one box to the other, but a good reviewer uses the editor comments to perhaps convey some areas of concern to the editor while protecting the authors from further unwarranted scrutiny. Issues of novelty, impact, and overlap with prior studies can be addressed in more depth here. The overall structure of a good manuscript review was highlighted by Dr. Connie Eaves (Experimental Hematology editor);
“A good review is one that informs the editor as well as the author as to the strengths and weaknesses of a submitted manuscript so that both can decide how to proceed – first in the editor finalizing a decision and then for the authors to revise or proceed elsewhere with the benefit of a constructive review. The review should begin with a couple of sentences that attempt to capture the reviewer’s perception of the essence of the novel thrust of the study and its strengths. This gives a general sense to the editor of the significance of the study or what it is attempting to show. It also reveals to the authors whether the reviewer understood what the authors felt the significance of the work was (or whether it was misunderstood). Then, the review should list numerically any concerns in simple clear and informed, objective language; addressing any/or all of the following: lack of novelty (why), lack of validity of conclusions (experimental design issues including inadequate controls, inappropriate stats, over- or mis-interpretation or technique), and poor presentation issues (poor English, Figs not adequately described, mislabeled, etc.).
Generally if novelty is a significant concern, or addressing inadequacies requires additional experiments requiring more than a few months, the paper will be rejected so this needs to be clear but not nasty. The goal should always be to identify and communicate constructively improvements that advance communication of new knowledge that will be useful and stand the test of time, or why this cannot be achieved in a timely fashion even though the topic addressed may be of interest.”
Given the increasing costs of research studies in both time and money, a fair and robust peer-review is critical in the scientific process. Also of note is that reviewers are becoming more savvy towards papers that contain too much overlap with existing data, studies of cell lines that do not have clear relevance or insights into actual biology, English so poor the paper is not comprehensible, clinical case studies, and preliminary findings, often leading to rejection even without detailed review. We hope this article can help highlight some of the important aspects of a good manuscript review so the community as a whole can benefit.
We would like to thank Dr. Emmanuelle Passegue (Columbia Stem Cell Initiative), Dr. Margaret Goodell (Baylor College of Medicine), and Dr. Connie Eaves (Terry Fox Laboratory) for their contributions in preparing this blog.
Associate Professor
Section of Stem Cell Biology
Division of Oncology
Department of Internal Medicine
Washington University in St. Louis
Comments
Post a Comment