“Pulling the Pins on Voodoo Science”
November 20, 2003
John D. Lamb
Just this last August, I was thumbing through my latest issue of Chemical and Engineering News (that’s like the Time magazine for chemists), when I was brought up short by a full page article that read “The 10th International Conference on Cold Fusion is taking place this week in Cambridge, Mass…” The title of the article had an intriguing ring about it: “Science, Religion, and the Art of Cold Fusion”. Mentioning the term “cold fusion” brings back some chilling memories (pardon the pun) to me, because I was personally caught up in this story about 15 years ago. It’s a fascinating story, in fact one of the most fascinating in all the annals of scientific history, and I have a feeling it will be summoned up by teachers of scientific philosophy, history and ethics for centuries to come.
Many of you are probably too young to remember the cold fusion episode. The setting is the state of Utah, where it so happened that scientists at two schools were working on related, but distinct, research projects. Those two schools, long rivals like the Montagues and Capulets, were the University of Utah and BYU. The two groups were working on ways to bring about nuclear fusion at room temperature. The time was the late 1980’s.
To appreciate the full import of cold fusion, we need to take a short side trip into the interesting realm of nuclear physics. But wait—before your eyes glaze over and you find your mind wandering, trust me—it will be short. The principle is simply this: if you can get two atomic nuclei to collide and stick together, you can get a whole lot of energy out. In fact, that’s the process that powers the sun. Physicists have been trying to get nuclei to come together this way to produce useful energy for 50 years; but alas, there’s a really big problem. Atomic nuclei are positively charged, and since, as you learned in high school, “like charges repel”, getting two nuclei close enough together to stick is really hard. You have to crash them together at very high speed, speeds corresponding to temperatures in the millions of degrees. So, scientists and engineers have been frustrated in their attempts to create fusion reactors on earth. Yet they haven’t given up, because the benefits to mankind would be phenomenal. Imagine a world where energy like electricity was boundless and cheap, and the byproducts of energy production were environmentally friendly. That’s what we’re talking about. But because of the problems with trying to contain fusion reactions operating at millions of degrees, the saying was coined: “Fusion is the energy source of the future… and always will be.”
Well, back in the 1980’s there was a small community of researchers looking at some possible tricky ways—using some of the subtleties of quantum mechanics—to get fusion reactions to occur at room temperature. Of course, room temperature is “cold” compared to the sun, ergo the term “cold fusion.” It was a long shot, but a goal well worth pursuing. Anybody who could figure out how to do it would surely get the Nobel Prize, and probably become the world’s first trillionaire.
One of the early researchers in this field was Steven Jones. He worked on some cold fusion ideas just down the road from here at INEEL in Idaho Falls. Then he joined the BYU faculty in 1985. He had a proven track record in cold fusion research by 1989—that’s an important year in this story—having published a number of papers in peer-reviewed journals and having had federal funding for several years on this topic. He was claiming to see tiny bits of cold fusion reaction, far too little to be practical for producing commercial energy.
Then, in 1989, enter upon the scene two chemists from the University of Utah, Professors Pons and Fleischmann (actually Fleischmann was visiting from England). They had no record of publication or funding in the area of cold fusion, or any other fusion for that matter. They claimed to have discovered high levels of cold fusion reactions—high enough to produce usable power. This claim shocked and excited the scientific community, and indeed the whole world. But the competing claims of very low levels of fusion from the BYU group just over the hill were a major problem. Anyway, after the University of Utah made their cold fusion announcement, the story of cold fusion got very complicated, and I wish I had the time here to share with you all the lurid details. Suffice it to say that no soap opera writer in Hollywood could possibly concoct a more bizarre tale. It includes claims of scientific thievery, betrayal of trust between universities, a media feeding frenzy, congressional hearings, an international scientific “gold-rush” involving hundreds of scientists, accusations of falsified data, threats of lawsuits against fellow colleagues at the University of Utah who openly doubted cold fusion, political maneuvering to acquire huge barrels of state and federal funding, obfuscation of experimental details, laboratory explosions, accusations of tampering with experimental samples by mysterious unknown saboteurs at Texas A&M, a libel suit against a respected Italian newspaper which ended in the judge accusing the Utah scientists of being “separated from reality,” accusations of money laundering, calls for resignation of top university officials, and on and on.
For our purposes here today, it is sufficient to note only the beginning and the ending of the cold fusion story. It began in earnest this way: On March 23, 1989, the University of Utah held a fateful news conference in which they announced that Pons and Fleischmann had achieved cold fusion in simple, inexpensive electrochemical cells at a level high enough to produce usable energy. They had not published their claim in a scientific journal; instead they took their announcement straight to the press. The level of fusion activity claimed was a trillion times larger than what Steve Jones was claiming at BYU. Of course, the whole world, and especially the scientific community, went wild over the news from the University of Utah. But as weeks went by and other labs were unable to reproduce the Utah results, the elation soured, and very quickly praise turned to disdain and ultimately to contempt. The claims made by the University of Utah of high levels of cold fusion were rejected by the vast majority of scientists. And this is the way the story ends: careers ruined, huge amounts of money wasted, and the good name of a respected university, the University of Utah, sullied.
At the time, I was serving as Executive Director of Research Administration at BYU, so it was interesting for me to watch all this from the sidelines. Of course, 99% of the attention about cold fusion was directed at the University of Utah, since only they were claiming fusion at useful levels. BYU’s reputation came out of the situation more or less unscathed. But the University of Utah was left with an enormous black eye. What had gone wrong? How is it possible that well educated scientists, administrators, state legislators, congressmen, senators, media science specialists, a whole nation full of educated people could have blown it so badly? Could all this have been avoided?
I’m here today to consider together with you, not the story of cold fusion per se, but the very much-related question: How can one intelligently evaluate scientific claims? It turns out that cold fusion provides an excellent case study in considering this question. Physics professor and author Robert Park places cold fusion within the category he calls “Voodoo Science”—hence the name of my talk. My purpose here is to provide you with some of the tools necessary to decide whether a claim like cold fusion is voodoo science or not. Now, in case there are any in the audience who have come hoping that they will learn how to make voodoo dolls to get back at pesky roommates, I’m afraid you’re going to be disappointed. Voodoo science in this context has to do with scientific claims that seem plausible, but just aren’t so—like perpetual motion machines or astrology.
At this point, you might be tempted to say to yourself: Why should I care whether something is voodoo science or not? Let the scientists battle it out. But I’m afraid you can’t get off the hook that easily. Voodoo science can jump out and hit anyone at any time from any direction. You may have a sister-in-law who wants to sell you some new “alternative medicine” which cures your tendonitis; or you may hear on the news that using cell phones will give you cancer. You’re going to have to decide: are these claims reliable? Should I buy from my sister-in-law? Should I quit using my cell phone? Make the wrong choice, and you’ll end up feeling like someone has stuck a voodoo doll of you full of pins! Hopefully today we can provide you with an inoculation against the effects of voodoo science—a few tools you can use to pull those pins or prevent them from sticking you in the first place.
At the outset, I want you to imagine that you are a member of the Utah State Legislature in 1989. A delegation of University of Utah administrators and scientists has just asked your assembly to authorize spending $5M on cold fusion research. (This is indeed what happened.) You’re not a scientist, but you have a very short time to decide how you’re going to vote. What are the key questions you should be asking?
This is a topic I spend a good deal of time talking about with an honors class I teach on this subject. We list two fundamental characteristics of a dependable scientific claim:
(1) it is supported by reliable, reproducible observations
(2) it can be explained in terms of a rational model.
The key questions you should be asking the Utah scientists center on these two principles. So let’s put these principles to work for you, the Utah state legislator.
To begin, let’s ask the Utah scientists: “Do you have reliable, reproducible observations?” Of course, if they say “Yes,” then where do you go from there? Do you simply have to take their word for it? The answer is: “No, you don’t.” There are well-established procedures for checking first the reliability, then the reproducibility of scientific observations.
First, reliability: the normal procedure for disseminating new research is by publication in peer-reviewed journals. Now what’s this peer review business? When a scientist sends in her manuscript to a journal to be published, the editor of the journal sends it out to other scientists to check. If the paper passes muster, it is published. If not, it must be revised or rejected. Once the paper is published in a peer-reviewed journal, it has the equivalent of a scientific Good Housekeeping seal of approval. You may not be able to read and understand the paper yourself as a layman, but you at least have the assurance of knowing that other scientists have read it and approved it. One caveat, however: Not all scientific journals are peer-reviewed. How can you know which is which? Ask a friendly university professor, or consult that font of all modern wisdom, the Internet.
Peer review—this is the first point on which things started to go seriously wrong in the cold fusion story. In a panic to secure priority for patent rights, the University of Utah bypassed the peer-review process, and instead went straight to the public and announced their discovery in a press conference. Does this mean their scientific claims were false? Absolutely not! But where no peer review occurred, you, as a lay legislator, do not have the assurance that peer-review would normally provide. A red flag should go up in your mind—these claims may be riding on thin ice.
So much for reliability. How about reproducibility? When we talk about reproducibility, we’re talking about two different types: within the lab, from experiment to experiment, and between different labs. In science, we know we really understand a phenomenon if we make the same observation every time we set the experiment up the same way. And our confidence is significantly bolstered if someone in another lab can do the same. Short of that, we really don’t understand, and indeed we may be way off base. Of course, numerical measurements may vary from one measurement to the next, but they should be close.
So, the next question we want to ask the Utah scientists is: Are your observations reproducible from experiment to experiment. And here’s another red flag: the results were all over the map. Sometimes the Utah cells produced a little energy, sometimes a lot, sometimes none. Sometimes they seemed to produce particles or gamma rays, which are the natural byproducts of nuclear reactions, and sometimes not. Does this mean their scientific claims were false? Absolutely not! But it does mean that they did not have a clear understanding of what was going on. Hmm—the ice seems to be getting thinner!
As a follow-up, we’re going to ask about the second part: Are your observations reproduced by groups elsewhere? Uh-oh! At the time of the press conference, no one else had been able to try to reproduce their results because almost no one else knew about them. By the time the legislators were asked for the $5M, a few groups had tried; there were a few tantalizing claims of success, but most groups had failed, and none of the work had been published in the peer-reviewed literature. We’re really on thin ice now.
Okay, let’s stop and take stock. So far, we’ve covered just the first of our two fundamental principles, namely “Is the claim supported by reliable, reproducible observations?” Things aren’t looking so good on that score, I’m afraid. What about the other one, “Can the claim be explained in terms of a rational model?” Now just for clarification, you would be perfectly justified in asking at this point, “What is a model, and what determines if it is rational?” Those are good questions.
Let’s talk about models. Einstein once said: “The most incomprehensible thing about the universe is that it is comprehensible.” That’s a profound thought. Nature follows certain regular patterns, and scientists describe these patterns in the form of models. A model is a mental construct or picture which attempts to describe the features of a system. If this sounds as clear as verbiage from an insurance contract, let me give you a simple example. At one time, the model of the atom was that the electrons circle the nucleus like the planets circle the sun. From this model, you can get a mental picture of how this system (the atom) behaves. Now we don’t believe in this model any more—models come and go. But notice that the model often describes the system in terms of another system that is more familiar—the atom is like the solar system. Today, we accept a more complicated model for the atom—but that’s a subject for another day. The point is that this model, like the one that succeeded it, is rational in that it is based on accepted principles and built up using reason and logic. Scientists have painstakingly developed numerous models over the years that have become widely accepted. It is expected that most new scientific observations will fit within, that is correspond with, these accepted models.
Every once in a long while some new observations come along which don’t fit the accepted model. When that happens, scientists are caught short. Could the observations be flawed? That is always the first possibility to check. If not, then maybe the model needs to be modified, or an entirely new model needs to be developed. The latter involves what we call a revolution in science. Examples include the development of quantum mechanics (which, by the way, replaced the “solar system” model of the atom), the germ theory of disease, special and general relativity, and plate tectonics. But these kinds of revolutions are successful only when the amount of evidence is so overwhelming that changing the model is the only alternative. As the astrophysicist Carl Sagan put it: “Extraordinary claims require extraordinary proofs.”
One of the questions about a system that a model often tries to answer has to do with cause and effect. We see an effect—for example, the sun coming up in the morning. What causes it? Well, you may have noticed that the rooster crows just before sunup—maybe that’s the cause. But of course, you say, let’s not be naïve: not all events that precede the effect cause the effect. But not all cases are quite so simple, and sometimes we can misinterpret causes just this way. Consider this: The ancient pharaohs used to throw a written command into the Nile River every year to cause the river to flood—and it did! Of course, it was helpful that the local priesthood kept records going back hundreds of years showing when the regular flooding of the Nile occurred. The ancient Egyptians saw the effect; they saw a plausible cause and believed it. Silly them. Of course, we wouldn’t be so gullible today, would we? Or would we?
In the late 1990’s there was an international scare that high-tension power lines cause cancer. The media spread the word based on scanty preliminary evidence. It was easy to believe: some people living near power lines were getting cancer. The effect was clear—cancer. The presence of the power lines preceded the cancer. Ergo, the lines must be causing cancer. No one seemed to notice that there was no plausible scientific model to explain how the power lines could cause cancer. Exhaustive research was conducted, with careful controls, on thousands of individuals. Meanwhile, property values of homes near power lines plummeted. And what did the exhaustive research studies yield? No cause-effect relationship. The federal government estimates that the loss in property values over this problem, coupled with the cost of massive research into the question, was over $25B—all over what turned out to be a false alarm. Yet the impression that such a cause-effect relationship exists still lingers. In fact, I bet if you ask your aunts and uncles around the Thanksgiving dinner table next week if power lines cause cancer, some will still be under the impression that there is a connection. In some circumstances, we are just as quick to believe non-existent cause-effect relationships, as were the ancient Egyptians. It’s just human nature.
One of the standard ways to test cause and effect is to perform what we call baseline or control experiments. These experiments are extremely important, because they can identify false cause-effect leads. For example, if someone claims a pill will cure your ill, you do an experiment to see what would happen if you administered a sugar pill in its place. Such control studies are often done “blind”; that is, no one in the study, not the doctor, not the patient, knows whether the patient is taking the sugar pill or the real thing. If the sugar pill does just as well as the medicine, then clearly the medicine is not the cause of the cure. Control experiments like this are critical to a claim like cold fusion, as we shall see.
In fact, we need to get back to our cold fusion story: As good legislators, let’s ask the Utah scientists about the second major criterion of credible claims: Can your claim be explained in terms of a rational model? Here’s the answer they gave: We see enormous amounts of heat in our cells. We can’t account for the heat from a chemical reaction. What other kind of reaction could be causing it? The only other alternative is a nuclear reaction. The effect is clear—heat. The cause must be nuclear.
Okay, let’s consider this proposed cause-effect relationship. First, what about control experiments? It turns out that the Utah fusion was being done with so-called heavy water—that is water made not from normal hydrogen atoms, but hydrogen atoms with an extra neutron in the nucleus. It has been known for a long time that heavy hydrogen like this can undergo fusion by known fusion processes, although only at very high temperature. But there is no known model for normal hydrogen to undergo fusion. So, we should ask our university delegation: What happened when you performed the cold fusion experiment with normal water instead of heavy water? The logic is that if you also got heat in this case, it would point to a non-nuclear cause to the heat effect. Woops. It turns out that the Utah scientists didn’t think to perform such basic control experiments before the press conference. I think the ice is starting to crack under these cold fusion claims.
Well, control experiments aside, let’s get back to our basic question: Can the high-level cold fusion claims be explained in terms of a rational model? Beyond the lack of control experiments, we have an even more serious problem. All known fusion reactions of the sort proposed by Pons and Fleischmann produce particles or rays which can be measured—neutrons or protons or gamma rays, tritium or helium. The current “model” of fusion requires them. And at the high level of reaction they claimed, there should have been enormous numbers of particles—easy to measure—so many in fact, that the researchers should have been seriously injured if present in the lab. But in fact, these byproducts were either entirely missing or could not be found in nearly enough quantity to correspond to their heat measurements. What’s the answer? Well, there must be a kind of fusion occurring that has never been seen before. Is this possible? Of course it is! Is it probable? Well, we need to go back to what we learned about replacing one model with a new one. That’s what we’re being asked to do isn’t it? We’re being asked to consider replacing the currently accepted model of fusion with a new one that can account for this cold fusion result. But “Extraordinary claims require extraordinary proofs.” Do we have extraordinary proofs? That is, do we have huge quantities of overwhelmingly reliable evidence? Not really. And that’s really very disturbing. I think the ice just broke out from under the feet of these cold fusion claims.
Keep in mind that all we’ve said so far was not designed to answer the question “Is cold fusion possible?” Even after all these inquiries on our part, it’s still possible that cold fusion actually occurs. Wouldn’t it be nice if we could just leave it there and be philosophical about the whole thing? Yes… but you can’t. You’re in the Utah state legislature, remember? You have to make an intelligent, informed decision about the $5M you’ve been asked for. You’d be wise to make that decision not on the basis of whether economically viable cold fusion is possible, but on the basis of whether it is probable. And on this basis, these claims don’t have much going for them. Now… you might say, the stakes are so high—and it is at least possible—so let’s go for it. But is this approach any better than shoving money into a high stakes slot machine in Las Vegas?
In fact, the question is often raised when a new miracle cure or infinite energy device is promoted: Isn’t it possible that all the scientists are wrong and that there is a new, as yet undiscovered principle behind this miraculous device? The answer is: “That’s the wrong question.” Of course anything is possible. I ask you: Isn’t it possible that Elvis is really alive and sitting in the audience here today? Sure. But I don’t see any of you looking around to find him. The correct question is, “Is it probable?” And that’s a horse of a different color.
Well, as it turns out, the Utah legislature did give the University of Utah its $5M. Maybe they just felt in a Las Vegas mood. Or maybe they did what many good people are prone to do—trust the experts and not ask too many questions—Why not? Isn’t that what experts are for? Let me assure you that this whole story is not meant to criticize the good people of the Utah State legislature. Remember, even many scientists were ready to accept the University of Utah claims, which just goes to show that having a PhD after one’s name does not render him infallible. If there is a moral to this story it is that you, I, everyone should be prepared to ask questions, even of experts, and that where science-based claims are concerned, there are some key questions that must always be asked. Among these are the questions we’ve talked about today. Asking these questions won’t make you 100% immune from voodoo science, but they will go a long way to help.
So, in future, when you are faced with a science-based claim, think back to the two fundamental principles we’ve talked about today. When someone claims to have magnets that will soften your water or clean your clothes because the magnets affect the structure of water, what should you ask? May I suggest, first: Can you show me reliable, reproducible observations that support the claim? I can tell the data are reliable if they are published in a peer-reviewed journal. Don’t show me testimonials of satisfied customers—that’s not an acceptable means to support scientific claims. Show me hard data. Second, can you show me that your product works according to the currently accepted models of magnetism and water chemistry, or else show me the overwhelming peer-reviewed evidence needed to overturn those currently accepted models?
Likewise, when cousin Egbert tries to sell you a new dietary supplement that will cure your asthma, ask the right questions: Can you show me the evidence, supported by reliable control experiments using double-blind methods, which unequivocally prove that the stuff actually works? Can you show me the logical mechanism by which this product affects my body? These questions are especially important when someone is encouraging you to swallow something or apply it to your body in some way. A lot of health products you can buy off the shelf have never undergone serious testing. Of course, careful testing of new drugs under the direction of qualified specialists is something we can and should support. But don’t allow yourself to be inadvertently turned into a human lab rat to prove or disprove the efficacy of an untested treatment introduced directly into the marketplace. Better to ask first and partake after—inform yourself by asking the right questions.
Now while I’m dealing out gratuitous advice, I don’t want to leave out my good friends and colleagues in science education. You know, we science educators have to accept some of the blame when people fall victim to voodoo science. In our classrooms, we tend to dwell too much on the facts of science, and we too often teach students to solve problems of a type they will never see again in their lives. We expend far too little effort teaching students how to solve the kinds of problems they will encounter later in life—problems like how to distinguish reliable scientific claims from unreliable ones. Maybe we ought to take a hard look at our priorities, because I believe we can do a much better job of preparing students for the serious science-related challenges of the 21st century.
Well, there’s a lot more we could discuss on this topic, but my time is up. And as Mark Twain would say, “It’s a terrible death to be talked to death.” Wait, though—is cold fusion voodoo science or not? I guess in part that question depends on whether you’re talking about cold fusion ala University of Utah, or cold fusion ala BYU. Hopefully after these few minutes we’ve spent together, you’re in a better position to make up your own mind on both counts, and to avoid many of the pins that can prick the unwary when voodoo science of whatever sort comes to call. Meanwhile, there is still a small band of scientists who believe cold fusion is possible, at least at the very low levels claimed all along by Jones at BYU. Indeed, Steve Jones is still working away quietly on cold fusion to this very day. It will be interesting to see if he or someone else can answer all the right questions and come up with something truly wonderful.