The following is mirrored from its source at: http://www.thebulletin.org/issues/1993/may93/may93Gofman.html ------------------------------------------------------------------------------ Beware the Data Diddlers by John W. Gofman The Bulletin of the Atomic Scientists May 1993 * The proposal * Un-knowledge * The rules of research * Follow the rules * Everyone wins * Violating the rules of research Permit me to put the question starkly. Although there is ample evidence that nuclear pollution presents health risks, how can we properly assess the degree of risk when the governments that have unleashed the poisons also sponsor virtually all the health research concerning nuclear radiation? That conflict-of-interest problem has a common-sense solution. We -- and "we" refers mainly to individuals and nongovernmental organizations -- must insist that independent "watchdog authorities" be established to monitor the work of those who may have a vested interest in underestimating the health risks that may be attributed to nuclear radiation. These watchdog authorities will not be responsible for conducting the actual health-risk studies. Rather, they will be charged with insuring that the raw data used in health-risk studies are obtained and maintained as objectively as possible. It's a common-sense idea. But like many simple ideas, it will be a tough sell. Valid conclusions about the health consequences of various pollutants rest upon databases that can be trusted. If a database is false -- either from careless work or from intentional bias -- it will cause innocent analysts of the data to fill medical journals and textbooks with misinformation. As surely as a skewed foundation compromises the building erected upon it, a false database turns all users into purveyors of possibly deadly information, no matter how honest they may be. The accuracy of a database is the key to every conclusion that emerges from it. The health consequences of false databases can vary from trivial to tragic. Although the principles discussed on these pages focus on radiation research, they apply with equal vigor to databases on dioxins, pesticides, mercury, and other major non-nuclear pollutants. Because it seems just a matter of common sense to have independent watchdog authorities, many people simply assume that they already exist. But they do not. Not for radiation research, and not for any other pollutant. The current situation regarding databases is unacceptable. Indeed, future generations might characterize it as criminally negligent. The proposal The Chernobyl database is already under construction by the International Program on the Health Effects of the Chernobyl Accident (IPHECA), which will operate through the World Health Organization. The main sponsors of the IPHECA study -- with $200 million suggested for its initial stages -- are the governments of the United States, Britain, Russia, France, Germany, and Japan. These governments also sponsor nuclear programs -- either nuclear power, nuclear weapons, or both. The conflict-of-interest is self-evident, and it is not limited to IPHECA's Chernobyl study. Nearly all radiation research is sponsored by governments that fiercely defend and promote nuclear energy. I believe that they recognize their goals are not aided if the public comes to believe that radiation is harmful -- even at low doses, and even if slowly delivered. The current situation in radiation research is a bit like relying on the tobacco industry to conduct all the research on the health effects of smoking. Fortunately, we don't do that. Since the 1950s, thousands of independent studies regarding the impact of smoking on health have been organized. In effect, the scientists who have conducted these tobacco studies have acted as watchdogs vis-a-vis the tobacco industry, which takes a generally benign stance toward the products it produces. Similarly, scientists with the mandate and the financial wherewithal to act independently must "watchdog" the building of radiation databases. I propose that we start with Chernobyl: * IPHECA should fund a team of independent scientists who will work inside the Chernobyl study. They would have the authority to make sure that the essential rules of research -- as detailed on page 42 -- are observed. They also would have the right to publish their own views as an integral part of every IPHECA document. Their assignment would not be to dictate a uniform analysis of the data. Rather, it would be to insure that the database itself can be trusted and that dissenting views about its handling and its meaning are not punished or silenced. * The watchdog scientists would in no way be answerable to governmental authorities. Rather, they would represent the public's right to a "second opinion" on matters of radiation and human health. An independent watchdog supervisory group could be appointed by environmental and other citizen groups to select and oversee the actual watchdog efforts. * Watchdog authorities must be permanent. Although the work that goes into the preparation of databases is most intense in the early phases, input is necessarily added for many decades, as the health of participants is followed up. Watchdog authorities must operate for the entire duration of a given study, because massive violations of the rules can occur even late in a study. (See "Violating the Rules of Research," page 43.) Un-knowledge During a lifetime in biomedical research, I've concluded that it is hard to prove anything about anything. There are sampling errors, confounding variables, and necessary equipment that has not yet been invented. The path to understanding is only darkly lit, and the stones strewn along the way are numerous. Acquiring truth about health and biology is difficult at best. In contrast, acquiring falsehoods about health and biology seems to be inordinately easy. Consider the legions of peer-reviewed professional journals that have carried "evidence" favoring various pharmaceutical, dietary, surgical, or physical therapies for almost every problem -- and the number of disappointments when the initial glowing reports are undermined by later reports suggesting no health benefits at all. I call such information "biomedical un-knowledge" -- shorthand for all the findings that are the opposite of what is true about health and disease. When I was younger, I assumed that no one wanted "un-knowledge." I no longer make that assumption. Consider the "wish-list" that nuclear energy-promoting governments seem to have for the radiation research they sponsor. It goes something like this: * Best of all would be a finding that a little extra radiation improves human health, a sort of invisible Vitamin E. This speculation has a name: hormesis. Indeed, some of its most avid proponents are already writing about the need to treat society in general for "radiation-deficiency disease." The second international conference on radiation hormesis -- with some 250 speakers and participants -- was held in Kyoto, Japan, last July. * The next best finding would be to determine that there was a threshold dose below which no harm occurs. The "safe dose­no risk" claim has become exceedingly common after the Chernobyl accident. For example, the U.S. Energy Department, in its 1987 report on the probable health consequences of Chernobyl, assigned a "zero risk" to some 500 million people exposed to low doses by fallout -- if all doses below a half rad are harmless. In a condensed version of the Energy Department's report (Science, December 16, 1988), the assertion that Chernobyl might induce zero extra cancers for persons exposed to low doses was repeated ten times in six pages. That's a fair definition of overkill. There was no mention of the powerful evidence and logic that argue against the threshold speculation. * If hormesis and thresholds are not successfully sold to the general public, the next best finding would be to claim -- as is often done -- that a dose of radiation is far less harmful if it is received slowly over time than if the same dose is received all at once. As a scientist, I have always taken these wishes and speculations seriously. I have spent years testing them with the existing evidence and with logic. I, too, would prefer for radiation to be harmless. Who would not? Unfortunately, evidence and logic do not support the wish list. On the contrary, evidence and logic suggest that low-dose ionizing radiation may well be the most important single cause of cancer, birth defects, and genetic disorders. The rules of research There are nine fundamental rules of research that any organization studying the effects of radiation on human health should be required to follow. They may seem unexceptionable, yet they have often been ignored: 1. Groups must be comparable. An essential condition for discovering the effects of radiation is reasonable certainty that exposed and non-exposed groups are so similar that the sole reason the groups might be expected to experience different rates of disease and disorder is their exposure to radiation. 2. The groups must have experienced an actual difference in dose. If the rate of disease in two groups is being compared, it is essential to establish with reasonable certainty that the groups actually received appreciably different accumulated doses. If, in fact, the groups received nearly the same total amounts of radiation, a study is predestined to find no provable difference in disease rates between the two groups. 3. The difference in dose must be sufficiently large. The dose-differences between compared groups must be large enough to allow for statistically conclusive findings despite the random variations in numbers and in population samples. Analysts can cope with the random fluctuations of small numbers both by assuring sufficiently large dose-differences between compared groups, and by assuring large numbers of people in each group. 4. The dose must be carefully reconstructed. Obviously, analysts will reach false conclusions if supposedly non-exposed individuals in a database actually received appreciable doses, and if people exposed to supposedly high doses received lower doses than the database indicates. When it comes to Chernobyl, weather patterns caused a very considerable variation in the spread of contamination, which makes this scientific pitfall a real possibility unless careful and objective dose-reconstruction is substituted for assumption. Fortunately, there are several dosimetry techniques that can reduce uncertainty about dose, even decades after the event. 5. Dose analysts must be rendered demonstrably objective. Analysts who estimate the dose subjects have received should have no idea of the medical status of the individual or the group to which the individual belongs. Health-status data and dose-related data must never appear in the same file. Analysts must do their work "blind" to protect the database from accidental or intentional bias concerning the relationship between radiation dose and health. 6. Diagnostic analysts' objectivity must be confirmed. To be credible, studies must be designed to guard against bias at every point. If any study of the effects of Chernobyl is to be valid, this principle must extend to all analysts, physicians, and technicians who diagnose the health status of study participants. They must not know whether an individual's radiation dose was high or low, and they must be denied information (such as place of residence) that would allow them to form an opinion about a likely dose. It is crucial that teams of "special experts" not be allowed to alter diagnoses at a later date-"unblinded." 7. No retroactive changes in input data must be permitted after any results are known. In a continuing study, one must not be allowed to alter, delete, or add retroactively to the original input when response results become available. Deciding to revise original data creates an opportunity to falsify the cause-effect relationships (if any) between dose and response. Any study is suspect if retroactive changes are made in diagnosis or dose, if cases are shuffled from group to group, or if any data or cases are suddenly dropped or new cases are added "as needed" from some reserve. 8. Data should not be excessively subdivided. Even the largest databases can be rendered inconclusive and misleading if analysts subdivide the data into too many categories or subsets. If analysts hope that a study will find no provable effects even if such effects exist, the outcome can be arranged by creating a "small-numbers problem," which will prevent all or nearly all of the study's results from meeting any test of statistical significance. Moreover, excessive subdivision increases the frequency of finding random effects, which may pass the test of statistical significance, but which are nevertheless false. Selective preservation of such false findings can be used to support an intentional bias. Any excessive subdivision of data should be viewed with suspicion. 9. No pre-judgments are permitted. Both pre-judgments and hypotheses are ideas held at the outset of an inquiry. Pre-judgments are assumptions, often unstated, that can cause investigators to ignore or discard particular evidence. In contrast, hypotheses are tentative explanations, openly stated, that invite challenges from other investigators. --J. G. Follow the rules My work on the risks of low-dose radiation has been controversial. Some scientists say they agree with me. Many say they do not. But whether I'm right or wrong about the low-dose question is irrelevant in evaluating the watchdog proposal. The watchdog idea serves the interests of objective, scientific inquiry. It does not promote the interest of any particular point of view regarding the possible outcomes of specific studies. It is fortunate that we do not often have disasters of the magnitude of Chernobyl. But the tragedy of Chernobyl will be compounded many-fold if we squander the opportunity to learn everything possible about the health risks associated with it, as well as its ecological consequences. Creating a trustworthy database from an event such as the Chernobyl accident is a profound obligation of the world's scientific community. If research on the radiation consequences of Chernobyl is poorly designed or biased, or both, the false conclusions will nevertheless enter the professional literature and the textbooks. Research that exaggerates the health hazard of radiation will be a disservice to humanity. But if researchers underestimate the true health hazards, the misinformation will be literally deadly, because it will result in great increases in "permissible doses" and unnecessary and preventable human exposures to radiation in the environment, in occupations, and in medical treatment. To help prevent the production of false databases and false "findings," either through bias or scientific error, medical science has developed the already noted rules of research. These rules are acknowledged implicitly or explicitly throughout the epidemiological literature. The key to creditable and credible research in the health effects of radiation is full obedience to these rules. Strict adherence to the rules simply eliminates issues regarding conflict-of-interest and scientific misconduct. No one needs to raise such issues -- if impeccable adherence to the basic rules can be demonstrated. But whenever such adherence cannot be demonstrated, society would be ill-advised to accept the purported scientific "findings." These assertions are not intended to impugn the motives or the work of most scientists who prepare or analyze radiation databases, although some misconduct exists in every field. But as already noted, if the rules of research are violated in building databases, no analyses of the data can escape the poison. The first obligation of every objective scientist is to question the believability of raw data before he or she uses them. Everyone wins Perhaps IPHECA itself should have thought about introducing the watchdog concept, as a means of obtaining credibility for its studies. However, IPHECA has not done so. Although I don't expect IPHECA's scientists to endorse the watchdog concept publicly, I suspect that most of them will privately welcome it. Most scientists, after all, would prefer to be honest and to do impeccable work. Under a watchdog authority, everyone wins. The scientists win by being liberated from humiliating pressures to please their employers. The victims of nuclear accidents win by having objective, reality-based evaluations of the harm or lack of harm they have been exposed to. And of greatest importance, humanity gains by being freed of the specter of 100 years of biomedical un-knowledge about the health effects of radiation. The watchdog proposal establishes a system that rewards and honors truth-telling, instead of punishing it by loss of employment. Further, the watchdog remedy requires no technological breakthroughs to make it feasible. The only requirement: A firm insistence that it is necessary, an idea whose time has truly come. We cannot expect governments to be the source of such insistence. The demand must come from all segments of society, not just scientists. The sooner that people create a ground swell of support for such a concept, the earlier it will become a reality. I urgently invite all individuals, groups, and organizations to let me know if they will permit themselves to be included on a list of endorsers of the watchdog concept. Last December, the concept received emphatic support from the international jury of the Right Livelihood Award Foundation in Stockholm. Subsequently, I have received very favorable reactions to the proposal from a variety of scientific, environmental, and political figures in Ukraine, Belarus, and Russia. The dollar costs of watchdog authorities over the decades will not be trivial, per database. But the work of the watchdogs is at least as important to ordinary taxpayers as is the work of the governmentally sponsored teams underwritten by their taxes. I suggest that funding for independent watchdog authorities should come from the same budget that supports governmentally funded investigators. For the sake of discussion, I suggest five percent of the budget. For the Chernobyl database, if the initial budget for IPHECA is $200 million, five percent would be $10 million. If five percent of IPHECA's budget (whatever its ultimate size) is transferred to an independent watchdog authority, I suspect that taxpayers -- or at least those who were aware of it -- would rejoice. Why are the nuclear-committed governments so ready to conduct health studies concerning Chernobyl? And Chelyabinsk? And why do they continue to govern the studies of Hiroshima and Nagasaki? They claim, of course, that they conduct such studies for the sake of truth, and for the benefit of all humanity. But at the grassroots, ordinary people may judge the sincerity of such claims by how positively governments respond to the watchdog proposal. Making the proposal is relatively easy, of course. The hard part will be building a critical mass of international support to make it a reality. The sponsors of current research on radiation and other types of pollution may fight vigorously behind the scenes to kill the watchdog idea. And after the watchdog proposal is accepted -- soon, I hope -- people must still remain vigilant. They must insure that independent experts are not -- or do not become -- sheep who wear a watchdog costume. In the end, we are all watchdogs. We owe future generations at least that much. Violating the rules of research The atomic bomb survivors database is updated and managed by the Radiation Effects Research Foundation (RERF) in Hiroshima. RERF is sponsored by the U.S. Energy Department and the Japanese Ministry of Health. In 1986, RERF virtually replaced the existing database, which had been used for about 21 years. New doses were assigned to survivors, many of the study's participants were suspended, and the remaining participants were shuffled into new groupings (cohorts). The reason given for these retroactive alterations was a set of revised dose estimates for both neutrons and gamma rays, from the 1987 publication, U.S.-Japan Joint Reassessment of Atomic Bomb Radiation Dosimetry in Hiroshima and Nagasaki: Final Report. This revision is a violation of a basic research rule -- that no retroactive changes in input data are permitted after any results are known. Parallel analysis with both old and new dose estimates, without an assault on the cohort structure of an ongoing study, is an acceptable practice to take account of new dose estimates, but RERF did not use it. Instead, its maneuvers with the A-bomb study database create a potential to revise the results to fit a preferred outcome. In contrast, credible epidemiological research makes elaborate efforts to avoid this possibility. I complained about the rule violation to Dr. Itsuzo Shigematsu, the RERF chairman. In a letter to me, he replied in part: "Concern about bias does not appear to be justified. We shall, however, be sure to consider the necessity, and if so, the feasibility of dual analyses of our data [altered and non-altered input]." RERF is already revising its revised database, which is called "the basis for any future amendments to the A-bomb survivor dosimetry that may be desirable." Perpetual, retroactive alteration seems to be RERF policy. Meanwhile, RERF provided me with the old "unshuffled" data (the original database) as well as its new "shuffled" data through 1985 for an independent examination. (This examination is described in the fourth chapter of Radiation and Chernobyl, This Generation and Beyond, forthcoming.) For radiation-induced cancer, rates were calculated using both old and new dose-estimates with the unshuffled cohorts. Both analyses show that low-dose radiation is more harmful per dose-unit than high doses of radiation. By contrast, other analysts using only the new dose estimates with a shuffled cohort, have published a curve suggesting that low dose radiation is less carcinogenic per dose-unit than high-dose radiation. Similar results were found when the database was shuffled and instances of radiation-induced mental retardation were examined. Serious mental impairment occurs when an eight to 26-week-old fetus receives an appreciable dose of radiation. The unaltered database suggests that there is no "safe" dose or threshold. But RERF analysts have frequently suggested that the atomic bomb survivor study indicates a possible threshold. Almost all the difference in interpretation can be explained by RERF's choice of the retroactively altered database; key cases of mental retardation have been moved from one dose category to another. The retroactively altered database produces data on cancer and mental retardation rates that are more favorable to nuclear polluters than the unaltered database. Database "shufflers" should be required to demonstrate that no bias was introduced by their research rule violations, particularly because their sponsors are not neutral observers. In a matter of such importance to human health, these rule violations should not be tolerated at all. Different rules of research have already been broken in the study of Chernobyl health effects. By 1989, three years after the Chernobyl accident, a number of complaints about health problems in Belarus and Ukraine had reached the press. At the request of the Soviet government, the International Atomic Energy Agency (IAEA) organized a study, led by Dr. Shigematsu. In May 1991, Shigematsu announced that the IAEA's international experts had found no relationship between illnesses in Belarus and Ukraine and the release of radiation from Chernobyl. The IAEA study received a great deal of attention in the public press, but it was sharply criticized in scientific journals -- as it should have been. It was flawed in a number of ways. The study broke more than one of the rules. IAEA teams sampled seven "contaminated" and six "uncontaminated" or "control group" villages. The researchers then selected a radiation-exposed population that had received doses that may have been less than one rem higher than the doses of those in the control group -- if they were any higher at all. Under such conditions, the search for meaningful results was doomed from the beginning, and the IAEA's conclusion was predictable from the start. The IAEA did not compare effects in "control" villages with villages where far higher doses were received (in many of the villages that were not included in the study, doses exceeded 20 rem). If only small differences in dose are examined, careful dose reconstruction is critical. But the IAEA researchers did not do this adequately. For several days, the Chernobyl 4 reactor burned graphite, and short-lived and intensely radioactive nuclides were released, which made later estimates of exposure based only on levels of cesium 137 unreliable. Using cesium 137 levels rather than careful measurements of actual biological doses contributed to inherently questionable conclusions. --J. G. See Also: Section 2: The Atomic Bomb Survivors -- A Study and Its Alteration, from Radiation-Induced Cancer From Low-Dose Exposure, An Independent Analysis (1990) by Dr. Gofman; especially, "Chapter 5, A Growing Problem: Retroactive Alteration of the Study." Copyright © 1993 Bulletin of the Atomic Scientists Reprinted for Fair Use Only. http://www.ratical.org/radiation/CNR/bewareDD.html (hypertext) http://www.ratical.org/radiation/CNR/bewareDD.txt (text only) http://www.ratical.org/radiation/CNR/bewareDD.pdf (print ready)