Preliminary Data

This brief case includes more issues for discussion than might appear at first sight. First, it usefully draws attention to issues about the status of data that are seldom discussed. From past work on sharing data, we have learned how conventions affect what counts as data.Weil, V., and Hollander, R. "Sharing Data. Scientific Data II: Normative Issues," IRB: A Review of Human Subjects Research 12 (No. 2, March/April 1990): 7-8, reprinted as "Normative Issues in Data Sharing" in Joan E. Sieber, ed., Sharing Social Science Data. Newberry Park: Sage Publications, 1991, pp. 151-56. By bringing preliminary data to the foreground, this case raises questions about conventions for ranking data as preliminary and for presenting data in a range of contexts. Second, the case shows some of the complexity associated with identifying and dealing with misconduct in science. With its ambiguities relating to the appropriateness of the conduct of the lab director, the case presents some of the subtleties associated with identifying misconduct. Accordingly, it underlines the importance of providing explicit education in research ethics to graduate students. Third, the case highlights an important problem in the relationship between a lab director and his student when the student is progressing well and shows initiative.

What criteria should determine whether data are counted as "preliminary"? How should preliminary data be presented, and in what contexts are the data appropriate? These are questions raised by this case. It is not obvious that they can be answered globally for application to all research environments, but they must be answered locally and, specifically, for the research environment in this case. Centering on the lab director's handling of the graduate student's data in a proposal, the case is designed to address issues about misconduct in science. Because of ambiguities in the situation, it is not clear whether the lab director's treatment of the student's data is unethical. However, there are obviously serious enough problems in communication of ethical standards, laboratory policies and accepted investigative practices to have led the graduate student to suspect misconduct.

As a second year graduate student, Penelope Brighton is a relative newcomer to the world of scientific research. As far as we know, she has not had experience in other labs, unless perhaps in rotating through labs in her first year of graduate study. It does not appear that Brighton has had the advantage of an orientation to issues of scientific research ethics. So she is unprepared for considering her lab director's use of her data in the light of well and justifiably accepted practices. Dr. David Gilligan, the lab director and apparently director of Brighton's thesis research, is a "highly productive, well-published, respected investigator whose students receive prestigious post-docs." With these qualifications, he seems ideal for heading a lab and directing thesis research. Indeed, Brighton seems to be making swift progress in her graduate studies, for in her first, quick experiments as part of her thesis work she has come up with potentially interesting results that excite Gilligan.

He thinks enough of her work to use her results as preliminary data in a grant proposal he is writing. Lacking information about any conversations between Gilligan and Brighton regarding Brighton's following up the initial experiments, we do not know whether she has proceeded to follow up with in-depth, well-controlled experiments on her own initiative. Nor do we know whether Gilligan is even aware that Brighton is moving ahead and making changes in several experimental conditions. If she has proceeded on her own unbeknownst to Gilligan, he has justification for including only her earlier, cruder data and for referring to it as preliminary. Timing is a key consideration, for Gilligan must submit the proposal within a short time and may justifiably judge that there is not enough time to carry out and validate further experiments. Funding agencies depend on well proven investigators to judge in the light of accepted standards when data should be considered preliminary and yet worth presenting.

Brighton manages to complete additional, more refined experiments before Gilligan actually submits the proposal. With changes in experimental conditions to eliminate certain staining, Brighton gets results that do not look like the results from her earlier, cruder experiments. The new results tend not to fit the hypothesis that she and Gilligan had formulated from the earlier experiments. Confident that her new data are reliable, Brighton concludes that the characterization of the protein, which is the aim of her experiments, "may not be as straightforward as originally expected." Apparently Gilligan does not have time to examine Brighton's new results carefully. He may judge that he can nevertheless trust his earlier excitement over Brighton's initial data, believing that she is on her way to interesting results.

To Brighton's dismay, she cannot interest Gilligan in putting her new results in the proposal. He says he must get the grant application out the door and "will deal with the staining details later." For the proposal, he asks Brighton to supply a figure using one of the cells that fits their hypothesis. The figure appears in the proposal, apparently with no mention of the additional data that would make it appear to be an atypical result. According to the author of the case, Gilligan suggests that all of the data completely support the hypothesis. Whether he makes this suggestion merely by omitting the additional data or by explicit statement, we do not know. If he states in the proposal that all the data support their hypothesis, his action is more open to objection than if he simply omits the later data. Given the time constraint, Gilligan could justifiably decide on a cut-off point after which he excludes data that he has not examined in detail.

When Brighton reads the grant she is shocked by the "spin" Gilligan has given the data. In discussing the draft with Gilligan, she stresses that most of the data do not agree with their hypothesis. It is noteworthy that Gilligan has given the draft to Brighton to read and that he discusses it with her. Perhaps he does not allow enough discussion, but he does not use his power to shut her out and deprive her of an opportunity to speak her mind about the use of her data in the proposal. Gilligan's defense of including only Brighton's earlier data is worth examining.

His remark that standards for presenting data as preliminary results in a grant application are less stringent than those for publishing data in a journal article may be read in two different ways. If his point is that preliminary data in a grant application are not required to meet the same standards of reliability as data not identified as preliminary in a journal article, his comment is defensible. It points to complexities in the legitimate use of data in different contexts. If the remark refers to data identified as preliminary in both contexts, then Gilligan owes Brighton an explanation to account for the difference. It is more likely that he intends the first reading.

Gilligan argues that it is better, presumably prudentially better, to present the data his way. This response is not necessarily ethically objectionable. The crude data, labeled "preliminary," had excited him, a highly competent investigator. He might reasonably expect reviewers to be similarly excited. There is no evidence in the case that Gilligan has asked Brighton for the follow-up experiments intending to include them in the "preliminary" data. So we do not have a basis for judging that he excludes her data because the results do not support their hypothesis. As noted earlier, follow-up and checking of new results would take time when time is short. Rushing the process might leave them with results no longer reasonably described as preliminary, but without enough data and time to suitably revise the hypothesis. Better not to complicate the proposal with new data Brighton has managed to produce, when, as it appears, the new data have not been scrutinized for reliability and time is too short for that effort. Gilligan judges that it is a better strategy for winning funding not to mention the later findings; they might create doubt among the grant reviewers.

If Gilligan has convincing evidence that their hypothesis lacks support, his action is ethically objectionable. That Brighton is confident of her results does not show that Gilligan has convincing evidence of that kind. Brighton's view of Gilligan's conduct appears to be colored by her confidence in her new results. Yet even she concludes only that the characterization of the protein may not be as straightforward as originally expected. Unless Gilligan has clearly implied or explicitly indicated in the proposal that characterization of the protein will be straightforward, he is not misleading the reviewers. The path from preliminary, exciting data to ultimate findings is often not straightforward. Experienced investigators expect surprises that may complicate matters. Unexpected findings can add to the interest and illumination of experimental work.

This interpretation of the situation is reasonable on the basis of information in the case. Access to the proposal might put a different light on the situation, and it may be that Gilligan uses the time constraint as an excuse to simplify the proposal submission. Perhaps he avoids examining data that he suspects to be strong, but complicating. To the extent that he has reason to be skeptical of the hypothesis in the proposal and presents it without qualification, his conduct is ethically objectionable. He also takes a chance of being wrong and having to deal with the consequences.

In any event, Brighton should try to find others more experienced in this or other labs with whom to discuss her concerns. She needs to test her reactions against the responses of others. An especially good resource would be an individual or office in the university involved with research ethics, because Brighton needs a sound basis for assessing other scientists' reactions. It might be a good move for Brighton to initiate a conversation with Gilligan about dealing with the staining details, for Gilligan has committed himself to addressing those details after sending off the proposal. She will have an opportunity to ask more questions about the proposal submission and his use of her data. From that conversation Brighton should get a better sense of Gilligan's reasons for his handling of her data in the proposal. She should learn how he will deal with her unexpected, complicating findings. As an outcome, she should have a better sense of whether Gilligan is to be trusted.

If in the end, she is convinced that he has handled her data unethically in the proposal, she must take some kind of action because she cannot continue to work under someone she believes to be unworthy of her trust. To extricate herself, she must begin by seeking advice from a person or an office she can trust to take a balanced view of her situation. On the information in this case, Brighton does not have enough evidence to raise an issue of serious wrongdoing. She has reason to try to stimulate discussion of the use of preliminary data in grant proposals and what should count as preliminary data. Also worth discussing is the question of how well supported a result must be before it is presented in various settings, a seminar in the lab, in another university, in a meeting abstract, in a progress report for the department or in a published paper. Appropriate settings for such discussions would be lab meetings, graduate student gatherings, research ethics seminars or the like.

For responsible reporting of results, it is essential to explain fully and honestly the experimental basis and to make no claim to have reported all the data when withholding data. It is not clear that Gilligan has violated these strictures. If he has, he has acted unethically. And he has also failed to model ethical standards in a kind of situation that is critical for educating graduate students in research ethics.

This case is meant to raise issues involved in determining whether scientific misconduct has been committed. Recently, a uniform definition for scientific misconduct has been proposed for all federal funding agencies.The Federal Register, October 14, 1999 (Volume 64, Number 198), pp. 55722-25. http://onlineethics.org/reseth/misc.html. In the current proposal, research misconduct is defined as fabrication, falsification or plagiarism in proposing, performing or reviewing research, or in reporting research results. Fabrication is making up results and recording and reporting them; falsification is manipulating research materials, equipment or processes, or changing or omitting data or results such that the research is not accurately represented in the research record; and plagiarism is the appropriation of another person's ideas, processes, results or words without giving appropriate credit, including those obtained through confidential review of others' research proposals and manuscripts. It is important to note that research misconduct does not include honest error or honest differences of opinion.

In this case study, Brighton is concerned that her data are being falsified. In contrast, Gilligan feels he is presenting the data in a favorable manner. As scientists we are influenced by our working hypotheses, but we need to be aware of the possibility of self-deception or delusion when interpreting our results. Gilligan feels that standards for presenting data vary among formats, i.e., grants, publications, etc. Undoubtedly, many scientists would agree. When writing a grant proposal, spin doctoring or salesmanship may be needed to convince the reviewers of the importance and feasibility of a project. But when does a "positive spin" become falsification of data?

According to the proposed definition of misconduct, changing and omitting data in the research record are not permitted. The research record is defined as the record of data or results that embody the facts resulting from scientific inquiry, and includes, for example, laboratory records, both physical and electronic, research proposals, progress reports, abstracts, theses, oral presentations, internal reports and journal articles. This list suggests that high standards must be applied in all formats for data recording/presenting.

The second set of questions is designed to generate discussion of institutional and government policies for reporting misconduct. Reporting scientific misconduct is obviously a sticky situation for all scientists; however, students and post-docs are in a difficult, tenuous position when they feel their adviser/mentor is falsifying or misinterpreting data. Consult institutional policies for reporting misconduct before presenting this case study. Additionally, consult the following websites for government misconduct regulations: http://www.ori.dhhs.gov; http://www.nsf.gov/oig/resmisreg.pdf; http://ori.hhs.gov/misconduct/index.shtml.