For Engineering and Science
Crashing into Law
Parent Collection
Contributors
Editor(s)
Year
1998
Discipline(s)
Topics
Description
This case dicusses engineering design, risk assessment, mistakes in design and specifically focuse son questions such as: what rules should govern the use of models? and how to protect the interests of science and the public through judicious use of models?
Body
In designing aircraft, it is important to understand how an aircraft might be damaged in a crash. This information can be used to create safer planes. Sven Svensen, a graduate student studying under Dr. Ole Oleson, has developed a model that predicts the damage to a 747 after a crash onto a hard surface. Oleson provided the early conceptual ideas, but Sven has done most of the work in developing and testing the model. Sven validated his model by comparing its predictions of damage to the actual damage seen on three 747s that had crashed in mountainous regions.
After a Budgetair crash in the Dismal Swamp, a lawyer approached Oleson. He wanted to know whether Sven's model could be used to show what damage to the aircraft was due to the crash and what might have been due to another cause, such as explosion or fire. Oleson assured the lawyer that the model should be able to determine the source of the damage. They agreed to a contract, and the lawyer faxed data about the crash to Oleson. Oleson was pleased: The money from this work would help to pay some of his graduate students until an expected grant came through. He was particularly excited at the prospect of more work from the lawyers if this went well.
Oleson called Sven into his office and asked him to run his model using the data from the Budgetair crash. Sven ran his model for the Budgetair plane's speed and altitude and found that the model predicted a damage pattern that differed from that seen on the plane. Oleson then asked him to modify the shape of the plane in the model to conform to the DC-9 that had crashed. This change still produced a different damage pattern. Oleson then suggested they add a soft surface to the model to represent the swampy qualities of conditions in the Dismal Swamp. Sven ran the model with the new parameters and found that the model predicted some, but not all of the damage to the aircraft.
As Sven was leaving Oleson's office, Oleson picked up the phone and called the lawyer. Sven overheard him tell the lawyer that the model showed that some of the damage was not caused by the crash.
Part 1
That evening, Sven thought about Oleson's assurances to the lawyer that some of the damage was not caused by the crash. He was uncomfortable with Oleson's statement. Sven went into Oleson's office the next day and stated that he did not think the model could accurately establish that the damage was caused by factors other than the crash. He argued that 1) the model was not validated with the new input conditions and 2) the model was not created to rule out damage but rather to describe possible damage.
After Sven stated his case, Oleson replied, "I know the model isn't perfect. However, that is the problem for the defense. I promised the lawyer I could show there was evidence of an explosion. Just print out what damage the model didn't predict and let the lawyers worry about it."
Discussion Questions
1. Was Sven's model being used inappropriately? Why or why not?
2. Did Oleson have a responsibility to inform the attorney and the court about the limitations of the model?
3. Should Sven try to do something? If so, what?
Part 2
In an alternate scenario, Oleson stated, "You have an interesting argument, but I am afraid I can't agree. The principles behind the model are slid enough to encompass the new input conditions and still maintain validity. Also, the model predicted all of the crash damage in the validation cases. That is sufficient to justify believing that any damage that is not predicted by the model must be caused by something other than the crash."
Sven remained unconvinced. However, he couldn't seem to change Oleson's mind.
Oleson testified that the damage to the plane could not have come solely from the crash and that a fire or explosion must have occurred.
Discussion Questions
4. Now Oleson believes in his testimony. Does his belief in the model matter?
Notes
Used with permission of Association for Practical and Professional Ethics. Case drawn from Research Ethics: Cases and Commentaries, Volume Two, Brian Schrag, Ed., February 1998.
Citation
. . Crashing into Law. Online Ethics Center. DOI:. https://onlineethics.org/cases/graduate-research-ethics-cases-and-commentaries-volume-2-1998/crashing-law.
When I was a kid, I used to make balsa wood model airplanes. They looked real, and some of them even flew, powered by rubber bands.
When I became an engineer, I made other kinds of models -- mathematical and conceptual models -- of environmental engineering problems. And some of these models "flew": (They were accepted by the profession as useful for solving real problems).
The two types of models are similar in that they are toys that I like to make. In both cases, the models try to represent reality, and I have fun when they give results that match real world data.
These models take on ethical overtones if they are to be used by others and if their use can result in some benefit or harm. If I had suggested that my balsa wood airplanes could fly more than a few feet, I would have raised expectations in someone who might have been willing to pay me for these airplanes and thus would have invested wealth based on my assurance that the planes would perform as I claimed. Obviously, I would have been lying.
Similarly, for Oleson to say that his model will simulate the crash is raising undue expectations in the person who is about to use the model for a serious purpose -- a lawsuit. The Svenson/Oleson model is a toy. Their experience has shown that it did not correctly simulate the crash. That should be the end of it. Oleson's behavior is unethical and unprofessional, and it is a shame that he, a mentor, is presenting such a dishonorable example to Svenson.
I might suggest a simple test to determine whether Oleson is acting honorably. Take away the money. Assume that he is not getting paid. Would he still be willing to compromise his professional reputation in such a way? If not, then he has sold his professional integrity.
Author: P. Aarne Vesilind, Duke University.
Two issues arise in this case. The first is how models should be used. Sven argues that in this case, his model is not being used properly and that it is poor science for Oleson to use the model's output to prove the lawyer's points. The second issue is how an expert witness should present scientific information.
There has been a lot of discussion on what makes a good experiment, including a good experimental design, appropriate use of statistical tools, and honesty in the manner in which data are included or excluded. However, there has not been much discussion of how models should be used. Models are used more and more frequently in engineering and science to examine problems that cannot be examined by direct experimentation. For instance, in this case, it would be impractical to examine crash damage in aircraft by crashing planes of every type in each kind of terrain. This strategy would be prohibitively expensive and dangerous. However, it is important to know how an airplane might come apart in a crash in order to design safer aircraft.
What rules should govern the use of models? How might one go about protecting the interests of science and the public through judicious use of models? Here are few guidelines that I believe most modelers would agree on.
The first guideline is that a model makes appropriate use of underlying scientific principles and works within the limitations of the tools it uses. In mathematical modeling, models that are made up of lots of variables instead of a few underlying principles are sometimes described as able to "fit an elephant." That means that such models can fit every data point but might also be able to fit any other data, such as the shape of an elephant. In modeling, it isn't enough for a model to fit all available data; it should also make sense scientifically. Without scientific validity, one cannot extend a model beyond where it has been tested.
The second guideline is that a model should be validated by experiment. It is not enough that a model has a basis in some scientific principle. It should also be validated by comparing the model's predictions with experimentally measured data. These data should not be the data used as input in designing the model. This guideline can be problematic, since models are generally created to deal with problems that cannot easily be investigated experimentally. However, the more one can prove a model works, the more confidence can be placed in it.
The final guideline is the admission of the limitations of the model. Everyone who creates or uses a model should be aware of its limitations. They should have an understanding of the scientific principles and tools on which the model is based. Without such an awareness, even a good, well-validated model can be used inappropriately.
This case study calls the final guideline into question. Sven believes the limitations of his model make it inappropriate for Oleson's purposes. Sven believes that the model is being used outside its range of applicability. However, in Part 2, Oleson believes in his use of the model. This conflict makes the ethics of the case less obvious. We want to believe that modeling is science, but an element of faith is also involved. Each of the proposed guidelines enhances confidence in a model, but models inevitably contain a degree of uncertainty. In Part 2, the issue is who has a better understanding of the model. It becomes a question of whom we believe -- Sven or Oleson? Did Oleson try to convince himself the model was valid because he wanted it to be valid? Because he had already promised the lawyers that he could deliver, Sven might not have been able to persuade him to reconsider. On the other hand, Sven has less experience in the field and may not understand the limitations of the model as well as his professor.
The second issue in this case is that of the expert witness. In Engineering Ethics, Harris, Pritchard and Rabin present five guidelines for expert testimony:
Guidelines 2 and 5 are called into question in this case. In Part 1, Oleson evidently plans to withhold information on the limitations of the model. Along with the ethical issue of dishonesty, he may be acting foolishly. If the lawyers learn of the inadequacy of the model and question him in court, Oleson may be embarrassed and his professional reputation may be damaged.
In Part 2, Oleson may encounter a problem with Guideline 5. He does not seem to be open to Sven's criticism of the model. Oleson has a possible bias because he wants to keep his commitment to the lawyer that the model could find the source of the damage.
The case also raises the question of Sven's options for expressing his concerns. One option would be to include his assessment of the model's limitations, along with an analysis of its uncertainty, in any writeup. Although a writeup would still be filtered by Oleson, such a report generally goes to lawyers on both sides of a suit and would allow communication of the model's limitations to the lawyers.
This case study raises two separate issues. First is the issue of how a model should be used. Second is the issue of an expert witness's obligation to report the whole truth.
This case can provide an opener into discussing the broader question of scientists' or engineers' obligations in reporting their work, particularly modeling work, to the general public. The discussion can be expanded to include not only the role of expert witnesses, but also the roles of public policy consultants to government agencies or sources in a newspaper article or a TV news program.
Back to Top
References