Theodore Rockwell

ENSURING RESEARCH INTEGRITY
for Standards-Setting Organizations
Theodore Rockwell
Radiation, Science & Health, Inc.

Standards-setting organizations, such as the Environmental Protection Agency (EPA), the Food and Drug Administration (FDA), and the Nuclear Regulatory Commission (NRC), occupy a special niche in the scientific community. Through their regulations and rule-making, they directly affect large numbers of organizations and individuals, often requiring them to make expensive and difficult changes. Those affected sometimes respond by aggressive legal or political actions, accusing the agency of using “junk science.”1
The agency may respond by stating that they are relying on the advice of a scientific advisory committee, made up of prominent scientists from prestigious institutions, such as EPA’s Science Advisory Board or NRC’s National Council on Radiation Protection and Measurements (NCRP). Thus, when a proposed regulation is submitted for rulemaking—a legal proceeding requiring public participation and reaching findings of fact and conclusions of law—if the scientific basis is challenged, agencies have replied that they cannot open up that matter, since it is based directly on “the best science” as supplied by its advisory group. If parties challenge the advisory group, the advisors reply that they merely report on the state of scientific knowledge, and it is up to regulators to make sensible regulations. In this situation, how can research integrity be monitored, and what roles should the various organizations play in accomplishing this, particularly in view of ORI’s stated intention to reduce its monitoring function?
Because of the high stakes involved, such questions sometimes end up in court. In one case,2 responding to a legal challenge to the EPA’s finding that second-hand tobacco smoke was biologically hazardous, a federal court vacated the EPA’s finding and technical report, concluding that EPA had not met scientific standards for research integrity.
The findings of this 90+ page, five-year case stated that the EPA:
“disregarded information and made findings on selective information;
did not disseminate significant epidemiological information;
deviated from its Risk Assessment Guidelines;
failed to disclose important findings and reasoning;
and left significant questions without answers.”
These deviations from good scientific practice are strikingly similar to alleged scientific misconduct in other areas,3 where an investigator was found to have “deliberately reported selected data, omitted contradictory data, and falsified results,” and another found “not honest error, not preliminary results that later proved overly optimistic, not even carelessness, but rather intentional and conscious fraud.” But to date, no organizations seem willing to formally raise such questions with members of these advisory groups, or their sponsoring agencies.
Classically, when a scientific maverick presents work that challenges the prevailing view, that work will be thoroughly scrutinized by peer reviewers for any irregularities. In such cases, the process works well. And when a scientist has personal or professional incentives to reach certain conclusions (e.g. to support the interests of a funding party), that very fact invites careful scrutiny, to rout out any questionable procedures.
But the process becomes more clouded when the funding agency and the research group both have the same interests (e.g. in showing that a problem exists that requires more and more research and/or regulation); there, the incentive to scrutinize work may be weakened. When such work leads to public policy decisions (e.g. determining the health hazard posed by particulate air pollution or by low doses of radiation), then bad science could create a serious public burden. In such cases, scientific misconduct, when exposed, could gravely damage the credibility of the research organizations and the agency involved. (In the Liburdy case,3 when the laboratory found scientific misconduct, the funding agency moved to get some of its money back. This may make laboratories more careful about tolerating scientific misconduct; but it also may make them less eager to uncover it.
In each case of scientific misconduct, an individual scientist takes the action in question. And that scientist generally has a supervisor, who may co-author the research paper(s) in question. And the paper(s) may be peer-reviewed, accepted by an editor, and presented at a scientific meeting. What responsibility should fall on those individuals? How should we monitor the work of members of prestigious advisory committees? Does the agency that funds the committee and accepts its reports as a basis for regulation have any responsibility to check the research integrity of the committee members producing the report it pays for?
Under increasing pressure to reduce costs, we may expect that excessive restrictions based on questionable science will be challenged in the years to come. We can let this task fall to corporate lawyers and congressional staffs, or we can start now to enable our scientific institutions to have the knowledge, the will and the mechanisms to ensure that scientific truth, as well as we can know it, will be objectively applied to laws, rules and regulations. Since the legal definition of good science is “mainstream” or “generally accepted,” it is essential that scientists ensure that mainstream science is good science.
References
1. Richard Stone, “EPA Analysis of Radon in Water is Hard to Swallow,” Science, 261, 1514 (17 Sept 1993). This report charged that EPA “rejected advice from the agency’s own science advisor and drew fire from an outside panel of experts” and “virtually nobody believes that…radon in drinking water poses a major threat to public health” and quoted EPA’s science advisors saying “It troubles me the extent to which science is largely treated as an afterthought in developing regulations” and “there is no direct epidemiological or laboratory animal evidence of cancer being caused by ingestion of radon in drinking water.” In an adjoining sidebar, Stone states, “For the most part, EPA officials brush off these criticisms.”
2. U.S. District Court, Winston-Salem, NC, 6:93CV00370, filed Jul 17, 1998.
3. See, e.g. J. Kaiser, “Baylor Saga Come To an End,” Science, 283:5405-1091, !999; “Report of the Formal Investigating Committee Inquiring into Alleged Scientific Misconduct of Dr. Robert P. Liburdy” July 7, 1995, Edited Final Copy. Outcome reported in T. Beardsley, “Fat in the Fire,” Scientific American, 281:4-32, 1999; E. Marshall, “The Misconduct Case That Won’t Go Away,” Science, 286:5442-1076, 1999.