B*M
发帖数: 1418 | 1 希望年轻的PI们都能以此为榜样~~
http://www.psychologicalscience.org/observer/getArticle.cfm?id=
-------------------------------------
Twelve Tips for Reviewers
An occasional column by Henry L. Roediger, III
Many critical skills needed for becoming a successful academic are typically
not taught in graduate school, at least not in any formal way. One of these
is how to review journal articles. Few students coming out of graduate
school have much experience reviewing papers, and yet, at least for those
students continuing on in research, reviewing is a skill that will be
increasingly critical as their careers develop. In fact, being a good
reviewer can greatly help a career. If a young psychologist becomes known as
an excellent reviewer, he or she may be selected as consulting editor, then
associate editor, and then perhaps the primary editor of a journal.
How do people learn to review? I suspect most newly minted PhDs learn to
review papers in the same way that the children in Albert Bandura’s famous
1960s studies learned aggressive responses, which is to say, by imitation.
Just as children who watched one boy smack a BoBo doll with a bat (and
received praise for doing so) would tend to do the same when it was their
turn, so do young academics learn to review. Unfortunately, the analogy with
the BoBo doll experiment is apt in another way: Because reviews are often
highly negative, the new researcher implicitly learns from the negative
reviews received on his or her own submitted papers that reviews are
supposed to be negative. It is as if the implicit message is: “A reviewer’
s job is to criticize the manuscript. Find any faults in the logic, method,
results, and conclusions that the paper might have and then communicate
these to the editor.” Hence, the cycle of negative reviews is perpetuated
across academic generations.
In 2002, Robert Sternberg wrote a column in the Observer titled “On
civility in reviewing,” decrying the often hostile nature of reviews in
psychology. This led him to edit a collection of essays on different types
of reviewing and how to go about them. The book Reviewing Scientific Works
in Psychology (American Psychological Association, 2006) is excellent, and I
urge readers to purchase it (although for truth-in-advertising purposes, I
should note that I wrote one of the chapters, on reviewing chapters in
edited books). The purpose of this column is not to replace that book but
simply to provide some guidelines for reviewing papers that may be helpful
to some, especially younger, readers. Abraham Tesser and Leonard Martin have
a fine chapter in Sternberg’s book that provides further thoughts on
reviewing articles for journals.
1. Know your mission. A reviewer’s job is to evaluate a submitted article,
not (necessarily) to criticize it and certainly not to trash it. The editor
is seeking your advice on whether or not to publish the paper, so you should
point out both strengths and weaknesses of the paper and come to some
balanced conclusion. Keep the big picture in mind: Is the problem addressed
an important one in the context of the field? Does the current paper push
knowledge forward in a substantive way? The question at issue is not “does
this paper answer all the questions about this particular problem?” because
the answer to that question is always “no.” The critical issue is whether
the paper under review advances knowledge on the issue under consideration
enough to be published. Yes, this requires human judgment, a notoriously
fallible quality, but that is why there are two or three reviewers plus an
editor. (Sometimes there are even more reviewers — too many, in fact — but
I’ll save that problem for another column.) Always keep in mind that, as a
reviewer, you are just one piece in the puzzle. You offer advice to the
editor, and the editor decides on the publishability of the paper.
2. Be speedy. Your reviews should be timely. Don’t wait until the last
minute to begin the review (or worse, after you get a reminder that it is
overdue). If you want to establish a reputation as a good reviewer, be
prompt and get your review in ahead of schedule. When I was editing the
Journal of Experimental Psychology: Learning, Memory, and Cognition, I tried
a young reviewer just out of graduate school. To my surprise, he returned
the review in five days and it was excellent — fair, balanced, pithy, and
wise. Just then, I had another paper that a reviewer had returned after
holding it for a few weeks, pleading lack of time. I sent it to the same
young reviewer and got another immediate turnaround (and this was in the
days before e-mail made delivery of papers and the return of reviews
instantaneous). I quickly added the person to the editorial board, having
suddenly discovered that I couldn’t live without him. When you are young
and new to the field, you might not be asked to review much. However, if you
are a good reviewer, you may be discovered and suddenly you will have an
overflowing reviewing pile. Don’t agree to review a paper unless you think
you can do so in a timely fashion.
3. Read carefully. Yes, you need to be speedy, but not at the expense of
accuracy. Read the paper carefully, and go back over parts that are not
clear. This advice should go without saying, but I am amazed at how often
reviewers will raise a point that is based on a misreading of the paper or,
worse yet, raise a point as if it were new when, in fact, the point was
addressed and rebutted in the paper. I confess to having made this mistake
myself on occasion, and having the mistakes pointed out by the editor or
other reviewers has made me more careful.
4. Say positive things in your review. Many reviews I read (both those of my
papers and those of others’ papers when I am one of the companion
reviewers) have nary a kind word. Rather, the review consists of an
unrelenting series of criticisms that might leave the author ready to hang
him- or herself by the end of the review. (Sometimes I will even see editors
apologize in the action letter for the tone of a review.) Usually, even the
worst paper has some redeeming features — the issue addressed is
interesting, the literature review is excellent, or whatever. (Don’t push
the compliments, though. No need to comment on the excellent choice of
typography — “one hardly ever sees Franklin Gothic used in a manuscript”
— if that is the only good thing you can think to say.) In short, even if
you are recommending that the paper not be published for one reason or
another, try to include some kind words.
5. Don’t exhibit hostility or mean-spiritedness in your review. The field
suffers from too many hostile reviews, as Sternberg has noted in his essays
on this topic. If there is a confounding that undermines the research in
question, point it out directly but without being rude about it. You can say
, “Unfortunately, I believe the authors’ conclusions may be limited by the
fact that Factor X covaried with the critical Factor Y,” rather than “The
authors obviously were poorly trained, as evidenced by their blunder of
confounding Factor X with Factor Y, so this entire line of research was a
waste of time.” (I have seen much worse statements than this one.)
6. Keep it brief. The editor is a busy person. He or she wants your opinion,
but does not want a stream-of-consciousness record of every pithy thought
you had while reading the manuscript. Give your conclusion about the
publishability of the paper and provide reasons for it, but don’t go on and
on. Reviews in most fields of psychology seem to have grown longer over the
years (that is my subjective impression, at any rate), although great
individual differences exist among reviewers. During my times as editor, I
would sometimes get reviews whose length would rival that of the paper. Some
of my own have been too long as well, especially in the days when I
dictated them. I now try to keep my reviews to one or two pages, which
probably makes both the editor and the author happy. Remember, the editor
has to read the paper and the several reviews received and then write an
action letter. No one wants your logorrheic review to gum up the process.
7. Don’t nitpick. Don’t go through the paper searching for misspellings
and grammatical infelicities. Keep to the main point, the big picture. God
created copyeditors for a reason, so you needn’t do their job. If a paper
is unusually well or poorly written, that might be worth pointing out; and
if a passage or point is difficult to follow, it is fine to note that too.
However, there is no reason to go through the paper line by line, as some
reviewers do, saying that trail on page 16, line 4, should actually be trial
. (Spellcheck cannot catch this type of error; though it does take away some
of my favorites from the old days, when people would write about their “
important resluts in Experiment 2.”)
8. Develop a good reviewing style. There is no universally agreed-upon
reviewing style, so you must develop one that is comfortable for you. Some
people begin with a summary of the paper. Personally, I think this is ok if
the paper is confusing and if the editor might need a little help, but there
is no need to provide a long summary for a paper that has a straightforward
message. One typical style for reviews is to have an introductory paragraph
stating the paper’s main point and the reviewer’s initial reaction. The
reviewer might then make the case for publishing the paper, listing its
strengths and discussing its importance. Next might come a section on the
manuscript’s shortcomings and criticisms, as well as advice on how they
might be overcome (if they can be). Finally, the review may conclude with a
recommendation about the paper’s publishability, with the realization that
this is advice for the editor (as discussed in Point 11). This schema is
only a suggestion. Many other reviewing styles exist.
9. Be careful in recommending further experimentation. One of the easiest
bad habits for reviewers to develop is to routinely recommend that further
research be conducted before the paper is published. For most papers, you
will be able to think of further experiments that might be done on the topic
. However, if the author has provided several experiments already and made a
coherent contribution, the critical question is not “can further
experiments be done?” (the answer is “yes”), but “does the author need
to do further experiments to support the conclusions being made?” (Of
course, this question presupposes that one thinks that the conclusions are
of interest to the field.) When I get back a review suggesting several
further experiments I might do, I usually think “Those are fine experiments
. Let the reviewer do them. I submitted the four experiments that (in my
opinion) tell an interesting story on their own.” If you write about
further research that might be done, use a high threshold and be sure to
specify to the editor whether you think it is imperative for publication of
the paper or rather just an idea for some future paper. Yes, sometimes a
further study is absolutely required to clinch a point and make a paper
publishable, but these cases are not as routine as some reviewers and
editors make them out to be.
10. Watch for egocentrism. If you received a paper to review, chances are
you yourself have published on that topic. Nearly every member of any
scientific field is subject to the feeling of citation neglect — “The
author should be citing my work more often.” Somehow, we feel compelled to
remind the author that we made a similar point to hers, even if we embedded
it on page 646 in footnote 2 of that article we published in an obscure
journal in 1991. If you find yourself writing statements like that — which
are read as “please cite me more” — you know you’ve stepped over the
line. Yes, you know your own work better than anyone, and yes, you would
like authors to cite it appropriately, but try to resist constant advice to
cite your own work. Occasionally you might want to point out one of your
papers that is overlooked by the current authors, but don’t make a habit of
it.
11. Make a recommendation about the paper, unless the instructions from the
editor tell you not to. Most journal editors want to know what your bottom
line is. Rather than just discussing features of the paper and letting the
editor guess what your overall opinion is, state it at the end of the review
. You may have listed positive and negative factors, but how do you weight
them? Conclude with a statement such as “On balance, I do not think this
paper should be published in its current form because of [reiterate the two
main reasons],” or “I believe the current paper should be published if the
author can address [two or three major criticisms].” Don’t be dogmatic
about it, though, to give the editor leeway. Sometimes, editors permit you
to write a review and then make a statement or recommendation for only the
editor’s eyes. Although I have encouraged kinder reviews, you should never
write a flattering review of the paper and then write the editor a note that
says “I tried to be nice to the authors, but this paper is really terrible
and should be rejected.” You put the editor in an untenable position with
this practice, so make sure your recommendation is consistent with your
review, especially if only the editor sees your recommendation.
12. Sign your review. Or, if you can’t bring yourself to do that, at least
write your review as if the author will learn your identity and you wouldn’
t be embarrassed. I sign all of my reviews and have done so for many years.
I think if everyone did, most of the problems of nastiness in reviewing
would disappear. As psychologists have repeatedly shown (e.g., Zimbardo’s
prison experiment), human beings do not display their best behavior when
they are cloaked behind the mask of anonymity. Signed reviews will usually
be more polite and diplomatic, with much less tendency for brutal,
unvarnished criticism. Of course, you still want to give your honest opinion
, but (as discussed above) there are helpful and unhelpful ways of relating
that opinion. Nonetheless, many discussions over the years have convinced me
that people object to signing their reviews for all sorts of reasons. If
you fall into this category, my advice is to still write the review as if
you were going to sign it. This makes it more likely that you will follow
the golden rule of “review unto others as you would have them review unto
you.” You may still frequently need to criticize papers, but you can learn
to do so in ways that are not blatantly offensive. Signed reviews may not
win friends because often you are saying “don’t publish this paper,” but
it’s the right course of action, at least for me. Be willing to stand
behind your words, not snipe from behind the hills. Also, if you blow a
point in your review, you can be sure that the author will let you know and
you can be more careful in the future.
Peer review is critical to the scientific process. Although we do not teach
courses on reviewing, we should all be mindful of ways to improve the
process. For further reading, let me once more recommend Sternberg’s book
on Reviewing Scientific Works in Psychology.
P.S. If you have a review of mine in which I broke one or more of the rules
above, no need to send it to me. I know I haven’t always followed my own
advice in the past, but I’m trying to reform.
Henry L. Roediger, III is James S. McDonnell Professor at Washington
University in St. Louis. This column benefited from the advice of David A.
Balota, Kathleen B. McDermott, Robert J. Sternberg, Endel Tulving, and John
Wixted. | D*a
发帖数: 6830 | |
|
