Peer Review (RCR Role Plays)

Role-Play Summary
Resources on Responsibilities of Peer Reviewers
Professor Role
Graduate Student Role
Starting the Role Play
Observer Role

Role-Play Summary

This scenario is based on “What is Responsible Peer Review?”, which is based on the “Confidentiality vs. Mentor Responsibilities: A Conflict of Obligations” scenario, which is based on a real incident.

This scenario highlights several issues in the peer review of manuscripts submitted for publication:

• Conflict of interest in the role of the reviewer
• Confidentiality of information in the manuscript
• Role of the adviser in the professional development of a graduate student

Although this scenario is not based on a single actual incident, each of the scenario’s elements occurs frequently in research in science and engineering.

When a research subfield is small, a journal editor may be unable to avoid choosing a reviewer who has a conflict of interest with an author of a manuscript. The conflict of interest may bias the judgment of the reviewer. A conflict may be particularly difficult when the reviewer and the author compete with each other for priority in making discoveries or advances in the subfield. In this case, the reviewer might be tempted to delay publication of the rival author’s manuscript by recommending extensive revisions. A reviewer who has a serious conflict of interest should decline to review the manuscript and should return it promptly to the editor. The reviewer may nominate alternate reviewers.

Busy professors who are asked to review manuscripts often refer the reviewing task to one of their graduate students. The task enables the student to learn about the publication process, and to learn how to evaluate a manuscript, under the supervision of the professor. Before referring the reviewing task, however, the professor should obtain the consent of the journal editor to a change in the reviewer. The editor may have had a specific reason for selecting the professor as the reviewer.

A reviewer may wish to apply the ideas in the unpublished manuscript in his or her own research projects. Although the Society for Neuroscience guidelines, for example, allow a researcher to stop an unproductive line of research based on the manuscript’s findings, in general, a reviewer should not take advantage of the manuscript’s ideas before they are published.

Resources on Responsibilities of Peer Reviewers

American Chemical Society guidelines

Cain, J. (1999). Why be my colleague’s keeper? Moral justifications for peer review. Science and Engineering Ethics, 5, pp. 531–540.

Smith, A. J., (1990). The task of the referee. Computer, 23, 65–71.

Society for Neuroscience: Responsible Conduct Regarding Scientific Communication

Responsible Conduct of Research: Responsible Authorship and Peer Review

Professor Role

What follows is an outline of your role. You will need to improvise to some extent – be creative but try to stay within the bounds of what seems realistic.

The executive editor of the Journal of Wondrous Research has asked you to review a manuscript submitted for publication in the journal. For this journal the review process is single-blind, so you know that the manuscript comes from the laboratory of your rival S. A. Wong at Desert State University. In glancing through the manuscript, you discover that although the theoretical ideas are novel and promising, the manuscript has numerous flaws: the description of the experimental method looks internally inconsistent, the illustrations lack labels, and the statistical analysis appears to be incorrect.

You want to refer the manuscript to your third-year doctoral student, to give the student experience in reviewing a manuscript (under your supervision) because reviewing is an important professional duty. This manuscript seems like a good opportunity because it demonstrates many potential mistakes that can be made in writing up research results. In addition, you think that two theoretical ideas in the Wong manuscript might help your student overcome some obstacles that have blocked the student’s progress for the last three months. The first idea indicates that your student’s current approach is not likely to succeed, and the second idea suggests a better direction for your student’s research.

This morning you sent a brief e-mail message to your student about the Wong manuscript, and you asked the student to come to your office in the afternoon. The message stated:

[We should meet this afternoon to discuss a new manuscript from the Wong laboratory. I would like you to review the manuscript for publication, and I also think that some of the ideas in the paper may be useful to help you advance your research.]

You plan to ask the student to serve as the reviewer of the manuscript as part of the student’s professional development. You also plan to suggest an experiment to check whether the theoretical ideas might overcome the student’s obstacle. You reason that although you are obligated to keep the identity of a peer reviewer confidential, the ideas themselves are not confidential; in fact, you had previously speculated that the theoretical ideas might be true.

You are confident in the abilities of your doctoral student. In many ways, the student is a good role model: the student always takes your advice seriously and implements your ideas diligently.

Professor Role-Play Notes:

• You want your student to review the manuscript to experience reviewing
• You want the student to use the manuscript’s theoretical ideas to further the student’s research
• You are mainly focused on the student’s progress
• You respect this student and have a good relationship
• The Wong manuscript has some serious flaws, so it will probably need significant revision and therefore it won’t be published soon

Plan for your meeting:

• Write questions that you will ask the student
• Follow-up questions that you might ask
• Questions that the student might ask you, and your answers

Student Role

What follows is an outline of your role. You will need to improvise to some extent – be creative but try to stay within the bounds of what seems realistic.

You are a graduate student in your third year of graduate study. You have completed most of the course requirements and passed the doctoral qualifying examination. Because you plan to pursue an academic career, you are looking for ways to gain experience with professional service duties such as reviewing manuscripts for publication.

You have worked steadily on your doctoral research project, making good progress. For the last three months, however, you have been unable to overcome an obstacle in your project. Although you feel frustrated, your research adviser has been supportive and has expressed confidence in your abilities.

This morning you received a brief e-mail message from your research adviser, who asked to see you in the afternoon. The e-mail stated:

[We should meet this afternoon to discuss a new manuscript from the Wong laboratory. I would like to you to review the manuscript for publication, and I also think that some of the ideas in the paper may be useful to help you advance your research.]

You know that Wong and your adviser are competitors in this research area, and you have systematically read the published papers from Wong’s laboratory. You are familiar with their work and some of it has been useful to your dissertation research. You check the Wong laboratory’s public Web site weekly to make sure that you know of any developments that might affect your work, but you did not see a preprint of the new manuscript posted there.

You recently attended a session on responsible conduct of research that highlighted ethical obligations in peer review. You wonder whether it is appropriate to take advantage of ideas in an unpublished manuscript. You also wonder if you can objectively review a manuscript that has ideas useful to your current dissertation work. You have never previously contradicted your adviser, whose ideas have advanced your research. In fact, you are somewhat intimidated by your adviser, whose research insights and judgment have nearly always been correct—and much better than your own intuitions.

Student Role-Playing Notes:

• You want to review the manuscript to gain experience in professional service
• You do want to learn about the ideas in the manuscript
• You are frustrated by the lack of progress of your research project
• You are concerned about objectivity and citation of the Wong work
• You trust your adviser and want to remain on good terms

Plan for your meeting:

• Write questions that you will ask your adviser
• Follow-up questions that you might ask
• Questions that your adviser might ask you, and your answers

Starting the Role-Play

Professor: How are you today?

Grad Student: I’m good. You asked to see me about a manuscript from the Wong Lab?

Professor: Yes, I was asked to review this manuscript, and after reading through it this morning, I think that it would be a good paper for you to review.

Grad Student: That sounds good to me. I have been looking for opportunities to gain experience as a reviewer.

Professor: This manuscript has a number of problems with it, so I’d like to see if you find the same problems that I found. Then we can talk about how to write a professional review … The paper has some ideas that might help your research progress, so we can talk about them as well.

Grad Student: I should be able to work on it this week … and I have been getting somewhat frustrated by the lack of progress on my dissertation research …

Professor: That’s exactly why I thought that this would be such a good opportunity for you.

Grad Student: It does sound good … But, well, I was wondering if there might be an issue with having me review this paper? …

Observer Role

• Read both roles.
• Watch the interview and take notes.
• If the conversation appears to be stopping early, encourage discussion on topics that still haven’t been addressed.

What issues and possible courses of action did the student and professor discuss?

In what ways did the student and professor communicate their ideas and concerns? Directly or indirectly?  How did they indicate that they understood each other?

What aspects of the interaction would also be effective in a real situation? For what reasons?

What questions do you think could/should have been asked that were not? What do you think could have been said that was not?