 |
|
Reviewing Manuscripts for Archives of Pediatrics & Adolescent Medicine
Peter Cummings, MD, MPH;
Frederick P. Rivara, MD, MPH
Arch Pediatr Adolesc Med. 2002;156:11-13.
Peer review is a critical element in the editorial process at Archives of Pediatrics & Adolescent Medicine. The goals
are to provide expert advice to the authors regarding their work, a check
on the scientific validity of the data and methods, and information to the
editors for use in their decision about the suitability of the paper for publication
in the ARCHIVES.
The quality of journals rests, to a large degree, on the quality of
the peer review process. Like nearly all scientific journals, the ARCHIVES
relies on a volunteer pool of talented individuals who take time out of their
busy week to review manuscripts. The journal could not be published without
their help, and we are extremely grateful for their efforts. However, reviewers
are rarely given any advice, by their senior colleagues or anyone else, on
the best way to conduct reviews. Our purpose in this brief article is to provide
some suggestions for reviewers so that their time can be spent producing a
review that will be helpful to authors and editors.
GENERAL APPROACH
In approaching a review, keep in mind the Golden Rule: "Do to others
as you would have them do to you." We prefer to approach a paper as if it
were written by a colleague, and we try to give the authors suggestions that
may improve their research design, their analysis, or their writing. Even
if you think that the manuscript should be rejected, your suggestions may
improve the next effort by these authors. Keep in mind how painful it is for
authors when their work is rejected; avoid comments that are mean spirited.
If you think that the manuscript is awful, by all means tell the editor, but
try to be polite to the authors.
On the other hand, do not mislead authors. We prefer that you do not
make statements about rejection or acceptance in your comments to authors
because the final decision is not the prerogative of reviewers. However, you
should not hesitate to point out problems with their study or their writing.
If you think that something is wrong or poorly done, say so and try to tell
them how they can fix the problem.
A little humility may help. Most of us have received rejections with
harsh remarks, and more than once we have had a manuscript rejected and thought
the reviewer was an ignoramus. Consider the possibility that what the authors
have done or said is correct, even if it seems doubtful to you at first reading.
If the manuscript is unclear, tell the authors that you could not grasp their
meaning; only the most defensive author would not appreciate that if a thoughtful
reviewer could not understand a sentence, it will not be clear to the average
reader.
The more specific your advice, the more it may help the authors. For
example, if you think that they have not adequately discussed the relevant
literature, try to give them specific citations; just referring to "the paper
by Smith" is not helpful. If you think that the authors should use a different
statistical test, can you refer them to a textbook that might give them the
details they need? We keep reference lists on bibliographic software for assistance
in preparing our own articles. With this kind of software, it is easy to offer
authors appropriate citations.
How long should a review be? Rarely is a single sentence or paragraph
sufficient. Although authors may get a brief thrill from a 1-sentence review
that says the manuscript should be accepted without any changes, we believe
that nearly every manuscript can be improved. A very short review suggests
to us that little review was actually done; similarly, a 1-sentence condemnation
offers little help to the authors. A page or two may suffice for many reviews,
whereas other manuscripts may require 10 pages.
It is not the job of the reviewers to rewrite the manuscript, but it
is certainly permissible to comment on grammar and writing style. We have
wonderful copy editors who work with the ARCHIVES and can substantially improve
the quality of the writing. Many reviewers, however, have excellent editing
skills. If you find that a sentence, phrase, or paragraph seems unclear or
awkward to you, bear in mind that it probably will not seem any clearer or
less awkward to other readers. The editors will appreciate any comments that
may improve the writing. It is not necessary to point out to the authors of
international submissions that English is not their first language; they (and
the editors) know this. Offer them whatever help you can. If their article
has scientific merit and is accepted, our copy editors will work to make it
readable.
Treat the manuscript with the confidentiality that the authors have
entrusted in the journal and the editors have entrusted in you. It should
not be copied and shared with colleagues who might find it of interest. Asking
a colleague to help with the review because of special expertise is certainly
acceptable. This person should follow the same confidentiality requirements,
and his or her work should be acknowledged in the information sent back to
the editors.
PARTS OF A REVIEW
General Comments
When writing a review, start with general comments about the manuscript
before moving on to specific parts of the paper. Every study has flaws. The
question is how much the flaws bias the results and whether any of them can
be fixed. Are there fatal and irreparable flaws in the paper that affect the
validity of the work to such a degree that the results are useless? If so,
let the editor know, and point out your concerns to the authors.
Is the paper concise, or could it be shortened to make it more readable?
Do some parts need to be expanded to make it clearer?
Title and Abstract
Does the title accurately describe the study, or is it misleading or
incomplete? Is the abstract informative enough that it can stand alone as
a summary of the study? In particular, are the main numerical results of the
study given in the results section of the abstract? Is all of the material
in the abstract presented in the paper? Are the numbers in the abstract the
same as those in the manuscript? Are the abstract's conclusions appropriate?
Introduction
Do the authors succinctly present a rationale for the study? Do they
clearly state the study question?
Methods
The reviewer should pay close attention to the methods section and understand
what the authors did. Does the study follow a known study design? Did the
authors follow the principles of this design? If you have specific references
on methods or design that you think the authors should include, it is tremendously
helpful to provide that information. The ARCHIVES asks authors of randomized
controlled trials to follow the CONSORT guidelines1
in reporting the results and to complete a checklist.2
These guidelines have been designed to improve the completeness and quality
of the reporting of such trials and to make future meta-analyses more feasible.
At the ARCHIVES, we are very conscious of the ethics of research as
well as potential conflicts of interest. We wish to see evidence in all papers
that an appropriate institutional review board evaluated the study. Do the
authors clearly state the procedures for obtaining informed consent when the
study involves patients? For industry-sponsored studies, is it clear what
role was played by industry personnel, either in the methods section or in
the acknowledgments?
We have a statistical consultant as part of our ARCHIVES editorial staff.
That consultant or one of our peer reviewers with biostatistical training
is asked to review manuscripts with complex methods or analyses. Peer reviewers
should comment on the methods and analysis used and let us know if they feel
that someone with statistical expertise should review the paper. Keep in mind
that you, as a reviewer, are likely to have more expertise than the average
reader. If you are left unclear regarding the study design or analysis, it
will likely seem this way to others. A reasonable approach may be to tell
the authors that you are not sure what they did and then ask them to try to
explain their methods in a different way.
Results
Are the results presented clearly? Do they too often repeat information
that is included in the tables and figures? Did the authors account for all
participants in the study? Are the tables and figures understandable and well
labeled? Do they display the data in the clearest way possible? Are the numbers
consistent in the abstract, results, tables, and figures? It is often useful
to add up the columns in a few tables to make sure that they are correct.
Comment
Did the authors relate their study to prior work in the literature?
Is it clear how this study supports or disagrees with previous ones and why?
Did the authors acknowledge the limitations of their study and discuss how
these might affect the results and their interpretation? Are the conclusions
justified by their data?
References
Are the references appropriate for the statements they are meant to
support? Are they up-to-date, or is more recent information available?
Comments to the Editors
The editors want your advice about the fate of the manuscript. Should
it be (1) accepted in its present form; (2) accepted after modest revisions;
(3) reconsidered for acceptance after major changes; or (4) rejected at this
stage? Several considerations should go into this decision. Do you think the
data and methods are such that the results might be credible? Keep in mind
that even if the results seem surprising or counterintuitive, we generally
judge a study by the quality of its data and methods, not by the final result.
Although results may confirm or dispute our prior beliefs, they cannot be
used to measure the quality of research. Ask yourself the following question:
Would your view regarding rejection (or acceptance) change if the results
were the opposite of what was reported? If so, perhaps you should reconsider
your thoughts about the paper.
How new and important is the information in the manuscript? How much
does the paper add to the field? Even if a few articles have already reported
the same finding, confirmation of that finding in a new location or population
or by a new method may be a useful addition. You were chosen to review the
paper because of your particular expertise. Editors rely on peer reviewers
to tell them whether a study is an important contribution to the literature.
If you think that the article is important, controversial, or for some other
reason deserves an editorial, let the editor know.
TIMELINESS OF REVIEWS
All journals try to process manuscripts as rapidly as possible. Many
authors know which journals have a relatively rapid turnaround time and which
are black holes that will swallow a manuscript for many months. Most of the
processing time for manuscripts is related to the peer review process. An
editor's nightmare is a reviewer who keeps the manuscript for a month and,
when called about the review, declines to do it. This may result in substantial
delays in providing a decision to the authors. If you find that you cannot
do a review because of time commitments or other reasons, please let the editorial
office know right away (preferably by telephone or e-mail). This helps everyone
involved. When you accept a manuscript for review, we suggest a 3-step process.
First, read it soon after you receive it. If you are going to do the work,
you might as well do it sooner. You would like your own work reviewed rapidly,
so why not do so for others? You might then return to the paper in a day or
so, read it carefully, and write your review. Finally, we suggest that you
print out your review, read it, and edit it, perhaps after letting another
day go by. Most of us take a lot of care with our own manuscripts; after all,
when they are published, our name is there for all to see. Reviews, however,
are anonymous and carry little reward, which may make us give in to sloppiness.
Edit your work to be clear and to remove unnecessarily harsh language. Give
a final consideration to what you say; remember, authors much like yourself
are anxiously awaiting your thoughts. And use your spell checker to set a
good example to the authors.
CONFLICTS OF INTEREST FOR REVIEWERS
Reviewers may have a conflict of interest in reviewing a manuscript.
In general, you should not review a manuscript from a close colleague at your
institution. If you have a financial conflict of interest such as owning substantial
equity interest in a company that makes a drug tested in the study, please
declare that on the form for the editors. This does not preclude an individual
from reviewing a paper, but it allows the editor to weigh this information
when considering the review.
BENEFITS OF REVIEWING
Being a good reviewer is, like writing and editing, a skill that improves
with time. We believe that reviewing has many benefits:
- Improving a manuscript is a service that many authors
appreciate. Both of us can recall receiving thoughtful and helpful reviews
regarding our own work. We are grateful to those who took the time to enhance
our papers.
- Improving published articles performs a service
to readers.
- Good reviews may attract investigators doing quality
research back to the ARCHIVES, improving the quality of the journal.
- Reviewing enhances our own skills as investigators
and authors. It helps us improve the design of our own studies and the quality
of our writing.
The next time you are struggling with a review, try to step back for
a minute. The ultimate goal of biomedical publishing is to prevent illness,
improve care, and promote health. Thoughtful peer review is a critical part
of this process.
AUTHOR INFORMATION
Corresponding author: Frederick P. Rivara, MD, MPH, Child Health
Institute, Suite 300, 146 N Canal St, Seattle, WA 98103-8652 (e-mail: fpr{at}u.washington.edu).
From the Departments of Epidemiology (Drs Cummings and Rivara) and
Pediatrics (Dr Rivara), University of Washington, Seattle.
REFERENCES
1. Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality
of reports of parallel-group randomized trials. JAMA. 2001;285:1987-1991.
FREE FULL TEXT
2. Checklist for authors submitting reports of randomized controlled trials
[Arch Pediatr Adolesc Med Web site]. Available at: http://archpedi.ama-assn.org/info/auinst_pchk.html. Accessed October 12, 2001.
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Publication Bias: The Problem and Some Suggestions
Rivara and Cummings
Arch Pediatr Adolesc Med 2002;156:424-425.
FULL TEXT
|
