To Review or Not: Reviewing the Competition

Case Overview

Confidentiality and Conflict of Interest

Phase 1

Phase 2

Phase 3

Case Overview

In this commentary, I will restrict my comments to the topic of the peer review of a manuscript submitted to a scientific journal for publication. As noted in the preceding commentary, there are now clearly articulated procedures designed to minimize problems with confidentiality and conflict of interest in the review of grant proposals submitted to the NIH or NSF. There is far less explicit guidance or uniformity on these issues in the context of manuscript review.

For example, in their book on Ethics and Policy in Scientific Publication, the Editorial Policy Committee of the Council of Biology Editors report journal editors' responses to a number of scenarios dealing with scientific publishing. One of these scenarios concerned a reviewer who, without the editor's knowledge, routinely asked members of a research seminar to carry out group reviews of the manuscripts he received because of the educational value of this exercise. While "most of the respondents felt that the reviewer's practice was wrong" and one wrote, "Using someone else's work as a teaching exercise before it has been published is a violation of confidentiality," some respondents "did not object to the reviewer's practice." One wrote: "I agree with the reviewer. Such a procedure can have educational value. The reviewer should take responsibility for the comments, if he or she has compiled and edited them. The author should be grateful for the additional feedback." (Editorial Policy Committee, Council of Biology Editors, 1990, pp. 88-89)

Because of this variability, it is important for scientists to know that confidentiality and conflict of interest can be problems when reviewing a manuscript, that they consider these issues in advance, that they develop their own standards and guidelines, and that they make sure that they check each journal's policies.

Back to Top

Confidentiality and Conflict of Interest

In the context of pre-publication review of research results, confidentiality does not require complete secrecy. After all, the authors have discussed the results among themselves and in their research group(s), they have submitted them to the editorial board of a journal, and they may very well have presented some or all of the results in talks at their institution(s) and at scientific meetings. Rather, what is important is the authors' control of the information. Authors have the reasonable expectation that they are the ones who control who knows what and when concerning their work, and that editors will not widely distribute their manuscript during the review process. This expectation then extends to the individual reviewer.

In this case study, conflict of interest refers to interests that the reviewer may have that could bias the reviewer's judgment. Conflict of interest in a scientific context has been said to refer to "any conflict between research or other professional scientific judgments and financial or personal interests where acting with disregard to that conflict by placing one's personal or financial interests ahead of professional interests compromises or detrimentally influences professional judgment in conducting or reporting research." (Werhane and Doering 1997, 169-170)

And, of course, the review process is part of reporting research. In this context, it is also important to remember that the perception of a conflict of interest can be as potentially harmful as an actual conflict whether or not one's professional judgment is influenced. Disclosure is critical, and management and/or avoidance are very important.

Back to Top

Phase 1

Optimally, the editor in this scenario would have contacted Slater before sending the manuscript, to determine whether Slater had time to do the requested review and whether any potential problems existed. However, despite the advent of E-mail, many editors just put the manuscript in the mail; the potential reviewer learns of the assignment only when he/she opens the envelope. So one could ask, when, in the process of looking over the manuscript, should Slater start to hear little alarm bells going off in his head? Regardless of how little he reads, his actions will be changed by what he finds in that envelope. Seeing the title and author(s), he knows that the competition is ahead of him and that Parker is at risk of being scooped. One can argue that he probably is already aware of the situation from meetings and the gossip network of science. Even if that is the case, seeing it in print is indisputable evidence. By the time Slater reads through the abstract, he can be sure that others have already done what he and Parker are working on. Reading any further will only further complicate the situation.

Because of potential problems with a conflict of interest, which, although it is not financial, could bias his judgement, he should contact the editor who sent him the manuscript as soon as possible, and preferably before carefully reading the entire manuscript. An alternative would be to insert a note declining the editor's request to review the manuscript because of a conflict of interest, taping the envelope shut to avoid temptation, and returning it to the editor immediately. However, this alternative seems a bit extreme, although not ethically objectionable. What Slater needs is to obtain an outside opinion concerning his ability to be objective in his review. The editor is the best person to consult because confidentiality will be maximized, and Slater also needs to disclose the potential conflict of interest. Slater should not show the manuscript to a colleague and ask whether he/she thinks he should go ahead with the review, and he can't rely on his own potentially biased judgment to determine whether he can be objective.

Of course, the editor knows that Slater works in the same area as the authors of the manuscript. If that were not the case, the editor never would have asked Slater to review it. However, the editor may not be aware of how close this work is to Slater's own, nor may the author(s), who might otherwise have asked that Slater not review the manuscript. The editor needs to be contacted and made aware of the details of the situation. It is entirely possible that the editor may still think that Slater is the best choice as one of the reviewers, but now the conflict has been disclosed and the editor can factor this information into his/her weighing of Slater's review. Editors are frequently scientists who do research in the same area as the authors and reviewers of the manuscripts they handle; they may have some personal knowledge of the reviewer's reputation and past behavior to draw on.

Back to Top

Phase 2

Slater has decided not to consult with the editor and just to go ahead with his review. He also decides to ask Parker's opinion of the manuscript since she is the one who has the greatest expertise. It might be interesting to modify the scenario a bit and explore whether changes in what Slater tells or shows Parker change one's conclusion regarding the appropriateness of his action. For instance, Slater could have asked Parker for her opinion on a small part of the paper without actually showing her the manuscript or telling her why he was asking. Or he could have shown her only a portion of the manuscript, perhaps the Results section but not Methods. As the case is written, he simply gave her the complete manuscript saying something like, "I've been asked to review this for the Journal of Cool Results. Since you probably know more about this topic than I do, would you look it over and give me your opinion of it?"

Question 1. It is not easy to determine whether Slater should have shown the manuscript to Parker in any circumstances, with or without the editor's permission. Conflict of interest is involved as well as confidentiality. As soon as Slater tells Parker about the manuscript or shows it to her, she has new information that will affect her behavior, just as was the case for Slater opening that envelope in Phase 1. It is likely that she will push harder to finish her experiments and prepare her own manuscript. However, while Slater was able to choose how far he read, and thus how much he knew about the competition, before he stopped and possibly contacted the editor, Parker has less freedom to choose. Unless she interrupts Slater while he is telling her about the manuscript (and few grad students would do that), she will know whatever he decides to tell her. Once Slater asks her to read the manuscript, it may be difficult for Parker to tell her adviser that she doesn't think she should read it, even assuming she is aware that there could be a problem.

Thus, Parker is pulled into an awkward situation through no fault of her own, but rather through Slater's decision. By showing the manuscript to Parker, Slater has placed her in a very difficult conflict of interest and has breached the confidentiality of the review process. How much is her judgment to be trusted in evaluating a manuscript that would scoop publication of her own dissertation work? How can she ignore the information she now has, including new ideas on techniques or materials to try in purifying her protein? The former is a conflict of interest concern, the later is a change in the situation due to the breach of confidentiality.

As mentioned in the overview of the case, the scientific and editorial community exhibits some disagreement regarding the propriety of consulting others while preparing a review. Given this uncertainty, it would be best to consult the editor first to determine the policy of the journal, and to seek his/her approval for the proposed course of action. At the very least, the editor needs to be informed of Parker's participation so that the editor can appropriately evaluate the review that he/she receives.

Back to Top

Question 2. If a graduate student is to be involved in a review, he/she needs to be made aware of the obligation of confidentiality incumbent upon prepublication reviewers. This information is not intended for dissemination during lunch with other students, nor over the Internet. Faculty members should be familiar with the conventions of confidentiality, and that would be the major difference between consulting a colleague rather than a grad student. Another difference would be that faculty members have higher standing in the scientific community and so might be better able to weather any storm associated with the review of this manuscript by Slater, a close competitor of the author(s).

Back to Top

Phase 3

Now we come to the question of what Slater and Parker should do with the insight into their own research insight they have gained by reading their competitors' manuscript. They have learned that another group managed to purify the recombinant form of the protein growth factor, a problem whose solution has eluded Parker so far. But note that the case does not say that Parker has tried everything she can think of. Rather, she is in the process of troubleshooting the protocol. It seems that she has a number of ideas that she is checking, and she may even be planning to try a technique similar to that reported in the manuscript. So it is not clear how critical the acquired information is to her progress. The case also indicates that the purification of the protein is not an experimental test of the model Slater and Parker propose, but is more analogous to the preparation of a necessary reagent for a critical experiment.

However, Slater and Parker did not suddenly and unexpectedly find themselves in possession of information concerning the purification protocol. They should have expected to find that information in the paper. In fact, were I in Parker's situation reading anything other than a prepublication manuscript, it is the first thing I would check. Besides, completing a thorough review would require them to read and evaluate the protocol. So the knowledge of the competitors' protocol should have been expected, and it was gained during the review process. Now what should they do with the information?

The case indicates that Parker and Slater successfully used the competitors' protocol as presented in the manuscript, and that they were then able to finish the work and publish in the Journal of Cool Results -- quite possibly ahead of the competitors whose manuscript they reviewed. Even assuming that their recommendation against publication was justified and that the other reviewer(s) agreed with their evaluation, there is certainly the appearance of impropriety here, and quite possibly more (see Question 2 below).

Question 1. Parker should not have used the competitors' protocol step-by-step for a number of reasons. First, on a practical level, doing so obligates her to cite the source for the technique, which is very difficult in this situation. Failing to cite the source would be improperly claiming someone else's work as her own, a type of plagiarism. Writing the citation as "Competitor, et al., unpublished results" will not work, because most journals will not allow citation of unpublished work without written permission. Second, using the protocol breaches the trust placed in a peer reviewer: Slater and Parker are deriving personal benefit from the use of privileged information. Finally, the competitor(s) and/or others familiar with their work will probably be asked to review Slater and Parker's manuscript, and they will recognize their protocol and figure out what happened.

Some might argue that because the purification protocol is not a test of the proposed model and because it has no substantive scientific relevance to Slater and Parker's conclusions, its use is not an issue. Others might argue that Parker could have planned to try a procedure very similar to the competitors' before seeing their manuscript, and thus she did not steal their ideas or gain an unfair advantage. What is most important for this question is to determine the criteria for evaluating possible courses of action. What are the probable consequences; the possible benefits and possible harms to Parker, the competitors and others who might be affected by this situation? What are Parker and Slater's obligations? What values and principles are important in such a situation, and which courses of action are consistent with them?

Back to Top

Question 2. This question is very practical: How can Slater and Parker possibly ignore what they now know? Is it even ethical to try to do so? Should Parker continue with her troubleshooting, trying everything but the particular column, for instance, that her competitors used in the hope that she will find something else that will work? Wouldn't that be a waste of time and money, probably federal money? Frequently, people will justify taking some questionable shortcut as the right thing to do because it will maximize the benefit for science or society. One must examine these situations closely to determine whether the "good of science" is not really in fact "the advancement of my scientific career."

As noted the situation for Slater and the competing group changed as soon as Slater opened that envelope and read the title of the manuscript. The more he and then Parker read, the greater the danger that trust in the peer review process would be damaged. That is why I conclude that the editor must be contacted as soon as possible and informed that Slater will be talking with the competing group either as the manuscript is on its way back to the editor, or as soon as the review is completed. This approach has potential problems, but I believe that the potential harms can be minimized by open, frank communication among the affected parties (see Question 4), and the editor needs to be informed of Slater's plan. One hopes that the editor will endorse it.

Question 4. Contact with the competitors seems to be required to avoid a potentially nasty situation when Slater and Parker eventually publish and the competitors wonder if some of their ideas were appropriated during the review process.

Contact could be initiated in a number of ways, and the two groups could agree to a number of arrangements. For instance, instead of being open about why she was contacting them, Parker could indicate that she heard through the grapevine that the competitors were working on a similar purification and ask them to give her information about their results -- for which she would cite them, of course. It seems that a more honest approach might work better, but that may depend on the history of interactions between these groups. Having made contact, the two groups might simply agree to cite each other, or perhaps they will agree to publish back-to-back in the Journal of Cool Results. They might even agree to a formal collaboration. A discussion and evaluation of various ways one might contact and cooperate with a competitor would be a useful way to conclude a discussion of this case.

References

• Editorial Policy Committee, Council of Biology Editors. Ethics and Policy in Scientific Publication. Bethesda, Md.: Council of Biology Editors, Inc., 1990.
• Werhane, P., and Doering, J. "Conflicts of Interest and Conflicts of Commitment" in D. Elliott and J. E. Stern, eds. Research Ethics: A Reader. Hanover, N. H.: University Press of New England, 1997, pp. 169-170.

Phase 1

Phase 2

Phase 3

The process of peer review is based on the premise that accomplished scientists, or "experts," in a particular field of study, are the most qualified to evaluate the scientific work or proposed research in that particular field. While most scientists would agree that this premise is valid in establishing an effective system of peer review, it has become clear that the system has some inherent problems. For the peer review system to be effective, reviewers must be able to evaluate the proposed or completed work honestly and objectively, and they must respect the confidentiality of the work being reviewed. Indeed, most ethical problems encountered during peer review are due to the need to avoid conflicts of interest and maintain confidentiality, which may be very difficult in some situations.

Many of the ethical dilemmas faced by reviewers arise from the fact that guidelines for avoiding conflicts of interest and maintaining confidentiality are often lacking or inadequate. There is a clear need for granting agencies and scientific journals to develop more explicit guidelines for reviewers in dealing with these issues, and many are beginning to adopt such policies. One starting point might be for organizations to model guidelines on those developed by the National Institutes of Health (NIH) designed to avoid conflicts of interest and maintain confidentiality during the scientific review of grant proposals. The NIH provides explicit instructions to reviewers to avoid conflicts of interest during initial review group meetings. These instructions state the following:

A member must leave the room when an application submitted by his/her own organization is being discussed or when the member, his/her immediate family, or close professional associate(s) has a financial or vested interest even if no significant involvement is apparent in the proposal being considered. If the member is available at the principal investigator's institution for discussions; is a provider of services, cell lines, reagents, or other materials, or writer of a letter of reference, the member must be absent from the room during the review. Members are also urged to avoid any actions that might give the appearance that a conflict of interest exists, even though he or she believes there may not be an actual conflict of interest. Thus, for example, a member should not participate in the deliberations and actions on any application from a recent student, a recent teacher, or a close personal friend. Judgment must be applied on the basis of recency, frequency and strength of the working relationship between the member and the principal investigator as reflected, for example, in publications. Another example might be an application from a scientist with whom the member has had long-standing differences which could reasonably be viewed as affecting the member's objectivity. Another example which might be considered is the review of a project which closely duplicates work ongoing in the member's laboratory. (National Institutes of Health 1995)

With respect to maintaining confidentiality, the NIH guidelines state:

All materials pertinent to the applications being reviewed are privileged communications prepared for use only by consultants and NIH staff, and should not be shown to or discussed with other individuals. Review group members must not independently solicit opinions or reviews on particular applications or parts thereof from experts outside the pertinent initial review group," and "privileged information in grant applications shall not be used to the benefit of the reviewer or shared with anyone. (National Insitutes of Health 1995)

These statements offer reviewers clear guidelines for ensuring that they do not have a conflict of interest, and for maintaining confidentiality during the grant review process. However, many journals do not provide such specific guidelines for the review of scientific manuscripts. For example, in response to the claim and subsequent lawsuit by Cistron Biotechnology that scientists at the Immunex Corp. "improperly used information from a paper they reviewed for Nature in their own research," Nature editor John Maddox commented that the journal does not explicitly define confidentiality. The only policy statement regarding confidentiality states that "colleagues may be consulted (and should be identified for us), but please bear in mind that this is a confidential process." (Marshall 1995, p. 1913) Furthermore, Nature does not require reviewers to identify potential conflicts of interest. Maddox continues to assert that there are unwritten rules, generally understood by reviewers, which assert, which assert hat the contents of manuscripts are not to be disclosed to the public and are not to be used to further the reviewer's own research.

Like Nature, many journals provide their reviewers with vague statements regarding confidentiality. Guidelines for avoiding conflicts of interest and maintaining confidentiality vary considerably from journal to journal. This lack of consistency is problematic: When explicit guidelines are not provided, it is difficult for reviewers to know what actions are appropriate. Furthermore, even when explicit guidelines are provided, there are many situations where the appropriate action is not obvious. One way for research groups to handle these issues would be to establish their own review procedures to help guarantee a fair and unbiased review.

Phase 1

In Phase 1 of this scenario, John Slater receives a manuscript from a competitor's laboratory to review; the title of the manuscript suggests that the work is closely related to ongoing research in Slater's laboratory. Slater should immediately recognize that there is a potential conflict of interest in his reviewing the manuscript. Slater's appropriate course of action would be to inform the editor of the journal of the potential conflict of interest prior to reviewing the manuscript.

However, when such situations arise, the editor will often ask the reviewer to review the manuscript despite the potential conflict of interest, with the understanding that the reviewer will remain honest and objective. This outcome is especially likely to occur in situations where relatively few "experts" in the particular field of study are available to review manuscripts.

Phase 2

In Phase 2, Slater decides that he can be objective in his review of the manuscript. He asks Alice Parker, a graduate student in his lab, for her evaluation of the manuscript. In this scenario, Slater's motives are only to solicit Parker's comments, as she is intimately familiar with this field of research. However, it should be noted that Slater could have shown Parker the manuscript for the sole purpose of providing her with confidential information that could benefit her research. Some journals explicitly state that reviewers may consult with colleagues regarding a manuscript as long as the reviewer discloses to the editor the names of those who were consulted. However, many journals do not explicitly state such guidelines. Furthermore, guidelines for general disclosure of the contents of a manuscript, where colleagues are not consulted for their expert opinion on the research, are often absent or extremely vague.

Many scientists would argue that disclosure within the reviewer's research group, or even within the reviewer's own institution, does not constitute a public disclosure of information. On the other hand, some reviewers adhere to a strict definition of confidentiality and do not discuss the contents of a manuscript even within their research groups, except in the situation where a colleague is consulted for his or her expertise. However, when a manuscript contains information that is relevant to the research interests in reviewers' laboratories, it may be very difficult, if not impossible, to keep the information from their research groups. Furthermore, it could be argued that keeping such information confidential would conflict with the collaborative basis of scientific research.

Phase 3

Phase 3 presents the greatest ethical dilemma for Slater. In the course of reviewing the manuscript, Slater and Parker discover that the manuscript describes a novel technique that could potentially benefit their own research efforts. In this scenario, Parker uses the technique in her research, which proves to be beneficial and results in the publication of a manuscript. Scientists generally agree that the contents of manuscripts submitted for publication are privileged information and should not be used by reviewers to further their own research efforts. However, is it reasonable to ask reviewers to disregard information that could potentially benefit his/her own research? Which is more important -- individual researchers' right to confidentiality and credit for their own work, or researchers' commitment to the collaborative basis and overall mission of the scientific enterprise?

It is not clear whether Slater attempts to credit the competitor's group for the use of the technique. In this situation, how should the reviewer cite the source of the information? Consider a situation where a reviewer does not recommend the manuscript for publication, but recognizes that both groups may benefit from a collaboration. Would it be unethical for the reviewer to contact the competitor to discuss this possibility?

This case study illustrates some of the common ethical dilemmas encountered during the peer review of manuscripts submitted for publication in scientific journals. The most common ethical dilemmas appear to revolve around attempts to avoid conflicts of interest and to maintain confidentiality during the peer review process. There is clearly a need for scientific journals to develop more explicit guidelines for handling potential conflicts of interest and safeguarding confidentiality, but as this case study illustrates, explicit guidelines may not address every ethical dilemma that may arise. For this reason, it is necessary for all scientists to have a good understanding of the ethical issues inherent in the peer review process, so that they can make sound ethical decisions when these types of situations are encountered.

References

• Marshall, Eliot. "Peer Review: Written and Unwritten Rules." Science 270 (1995): 1913.
• National Institutes of Health (NIH). "Review Procedures for Initial Review Group Meetings." Issued January 1995; revised April 1997. http://www.drg.nih.gov/guidelines/proc.htm.