This article originally appeared in The Skeptic, Volume 22, Issue 2, from 2012.
I would like to respond to many of the issues raised by Wiseman, Watt, and Roe. Unfortunately, space limitations allow me to focus on one or two. So I will confine my reply to just a couple that seem most central to my essay.
I was puzzled by Chris Roe’s statement, “Hyman makes a number of vague allusions to a lack of quality in parapsychological research…” I was unaware that I had made such allusions, vague or otherwise. I assume that Roe has been misled by a possible ambiguity when I wrote:
“These neoparapsychologists… acknowledge that the evidence for psi is inconsistent, elusive, and fails to meet accepted scientific standards.”
In this context, both the neoparapsychologists and I are not addressing the issue of whether the quality of individual experiments meets “accepted scientific standards” such as appropriate controls for possible confounds and proper application of statistical analyses. Rather, we are focusing on the consistency of outcomes across a series of experiments.
The problem at issue is not whether the individual experiments employ adequate methodology, but rather do the outcomes from separate experiments show consistent, lawful and reproducible patterns. The neoparapsychologists agree with me that they do not. The “accepted scientific standards” are those that require that a finding has to be independently reproducible before it can be accepted as a legitimate subject for scientific scrutiny.
Neoparapsychologists do find some consistent tendencies in the results of psi experiments such as experimenter effects, decline effects, reversal of effects and the like. But even these tendencies are far from predictable; along with other inconsistencies in the outcomes, they are what put the findings outside the scope of orthodox science.
Since the time of Galileo, Kepler, Harvey, and Newton, modern science has flourished just because it focused only on phenomena that were available for public scrutiny, were lawful, and could be independently replicated. Claims that challenged existing theories were taken seriously when they were supported by evidence that met these criteria of consistency. Claims such as N-rays, Martian Canals, polywater, mitogenetic radiation, and cold fusion which were based on evidence that could not be reliably replicated were rejected. It is worth noting that the proponents of these rejected claims also attributed the inability to consistently replicate to experimenter effects, reversal effects, decline effects and the sort of inconsistencies that plague psi research. As I pointed out in my essay, science can only succeed when it studies phenomena that are lawful and reliably reproducible.
Roe makes another point to which I think I should respond. He wrote:
“Hyman bemoans parapsychology’s poor replication record and… rather selectively uses one study by Broughton and Alexander (1977) to illustrate his point…”
He is correct that I deliberately selected this study to make my point. I could have just as well have chosen some other studies such as the failure to replicate the PEAR experiments (Jahn et al., 2000). Like the attempted replication of the PEAR experiments, the Broughton and Alexander experiment met all the criteria of a replication attempt that should have succeeded if, indeed, there had been a real effect. The experiment was prospective in that it was designed to test a prediction that the results of the autoganzfeld experiments were real and replicable. The design and procedures were deliberately made to be as similar as possible to original experiments. Even some of the very same equipment was used. The authors made sure to include a sufficient number of subjects and trials so that lack of power would not be a factor. They not only tested for the entire primary, but also the secondary effects that were part of the original findings. In addition, the experimenters were experienced parapsychologists.
The failure of this direct replication cannot be easily dismissed as due to low power, experimenter effect, non-standardness of design, or any of the other excuses that often are used to explain away negative findings.
Roe apparently believes that the results of psi research, while indeed showing some inconsistencies, are basically replicable. To me, the most sobering consideration for a parapsychologist in regards to replicability is that, unlike every other area of inquiry that claims to be a science, parapsychology is the only one that lacks a ‘paradigm’ experiment. As Kuhn (1977) acknowledges, his original conception of paradigms referred to standard experiments that science textbooks provide students. Each area of science has its own set of standard experiments that serve as exemplars of the field. A standard experiment could be given as a laboratory assignment with the expectation that the students could replicate the original results.
Psychology, for example, has hundreds, even thousands, of such standard experiments. Every introductory psychology lab can assign the students paradigms in sensation, perception, memory, problem solving and other areas with the confidence that the students will replicate the original results. As just one illustration, consider Ebbinghaus’s classic experiment on the laws of forgetting. The student can conduct the experiment on herself or on another person and will obtain essentially the same lawful curve that Ebbinghaus reported in 1885.
As I pointed out, parapsychology does not have even one paradigm experiment. The problem is deeper than just the inability to consistently get significant departures from chance. The loose connection between theory and experimental results in parapsychology means that the constraints on how psi should manifest itself in the data are weak. Ultimately, the decision to claim that psi is present relies on a significant departure from chance. Parapsychologists have no disciplined way to decide when psi is present and when it is not; nor do they have any principled way to decide whether an effect in experiment A is due to the same cause as the same sized effect in experiment B. This is equivalent to declaring that any outcome showing a drop in the proportion of recalled syllables over time is a successful replication of Ebbinghaus’s original experiment even if the resulting function clearly differs from the original curve.
Yet, parapsychologists often seem willing to claim a successful replication in cases where only a departure from chance occurs in the new experiment. (See the example I described wherein parapsychologists claimed that the autoganzfeld experiments successful replicated the original ganzfeld experiments.) Replication, in orthodox science, refers to reproducing a pattern of results not just an abstract ‘effect.’ Even given the loose concept of replication that parapsychologists often use, their results clearly do not replicate.
