[ RadSafe ] Article: What's Wrong With Single Hypotheses

Wed Nov 9 07:11:42 CST 2005

>From The Scientist: 
Volume 19 | Issue 21 | Page 10 | Nov. 7, 2005

The on-line version is at
http://www.the-scientist.com/2005/11/7/10/1

What's Wrong With Single Hypotheses
It's time to eschew enthrallment in science

By Don L. Jewett 
The classic description of the scientific method
begins with devising a hypothesis. The problem with
starting with a hypothesis, however, is that bias and
self-delusion can arise owing to an emotional
attachment to the hypothesis, as Chamberlin honestly
described in 1897:

The moment one has offered an original explanation for
a phenomenon which seems satisfactory, that moment
affection for [one's] intellectual child springs into
existence, and as the explanation grows into a
definite theory [one's] parental affections cluster
about [the] offspring and it grows more and more dear
... There springs up also unwittingly a pressing of
the theory to make it fit the facts and a pressing of
the facts to make them fit the theory ...1 

The temptation to misinterpret results that contradict
the desired hypothesis is probably irresistible. This
mistake occurs repeatedly in the history of science.
Some examples were collected by Langmuir,2 who
correctly called them "pathological science."

Peer-review of research can help avoid these mistakes,
so long as the reviewers are not in the thrall of the
same hypothesis. But if there is a shared enthrallment
among the reviewers in a commonly believed hypothesis,
then innovation becomes difficult because alternative
hypotheses are not seriously considered, and sometimes
not even permitted.

To speed scientific progress, Platt suggested the use
of "strong-inference,"3 a thumbnail sketch of which
includes:

1. Identify a scientifically interesting observation.

2. Enumerate all alternative hypotheses that can
account for the observation, based on present
knowledge.

3. Reject hypotheses by experimental observations
until a single hypothesis remains that has survived an
experimental test by which it could have been
rejected. The remaining hypothesis is the currently
held view of the cause of the observation.

To the laity, the remaining hypothesis is truth, but
the scientist knows that this currently held view can
change if new hypotheses arise from new knowledge. The
presence of strong-inference in publications and grant
proposals can identify those areas of science that are
rapidly evolving, and if funding were to be shifted
towards these areas, progress in science would be more
rapid and efficient.

Yet, despite its logical structure, strong-inference
does not deal with the following question: What method
can efficiently generate the observations needed to
start new cycles? Strong-inference is neither useful
nor needed for creating the critical seed observation
upon which one can base alternative hypotheses.
Instead, we need strong-inference-plus, which consists
of the exploratory, pilot, and hypotheses-testing
phases.

The exploratory phase has different goals (and uses
different methods) than the other two phases. During
the exploratory phase the factors that influence
success are unclear, so the research must proceed by
way of hunches, serendipitous observations, and
selections based on inadequate information. Pet
theories are certainly acceptable here, if they work.
Such nonscientific procedures are necessary whenever
the parameter space is too large for an exhaustive
search. It is here that scientific creativity can play
a crucial role, as years of experience may enhance or
hinder discovery. And here is where new scientific
instruments or methods might demonstrate their worth.

When the exploratory phase provides reasonably
reliable observations, the research enters the pilot
phase, during which a small number of experiments,
usually six to eight, are replicated under identical
experimental conditions. The pilot phase is the
experiential test of the exploratory phase. A
statistical power calculation of the pilot data will
predict the size of study necessary in order for the
hypotheses-testing phase to have a reasonable
probability of success.

If the analysis leads to further funding, the project
moves into the hypotheses-testing phase. None of the
experimental data from the exploratory and pilot
phases can be reused in the hypotheses-testing phase,
in order to avoid statistical bias. The
hypotheses-testing phase will be most efficient when
the multiple hypotheses of strong-inference are
utilized to guide the experiments. Quality in
strong-inference is judged on the efficiency of ruling
out alternative hypotheses. Quality in the exploratory
phase of strong-inference-plus should be judged on how
interesting the observations are, and especially
whether the observations cannot be explained by the
"standard view." The observations that are not
explainable by current scientific theories are the
most valuable, for they may propel the field forward
in the next cycle of innovation, possibly to a
paradigm shift.

Strong-inference-plus is a paradigm shift for some
scientists. For example, it is common knowledge that
R21 exploratory research grants at NIH are evaluated
not in terms of the exploratory phase, but instead,
are critiqued as if the proposal were for R01
hypothesis-testing research. Explicitly labeling the
phases of strong-inference-plus makes it easier for
students and researchers to identify changes in the
approach to experiment and to data as new knowledge is
first encountered, then elaborated on, and finally
codified.

Don L. Jewett (djewett at itsa.ucsf.edu) is an emeritus
professor at the University of California, San
Francisco, and research director at Abratech, a small
biotech R&D company. 

References 

1. TC Chamberlin "Studies for students: the method of
multiple working hypotheses," J Geol 1897, 5: 837-48. 

2. I Langmuir "Pathological science," Phys Today 1989,
42: 36-48.  
3. JR Platt "Strong inference," Science 1964, 146: 347-53.

+++++++++++++++++++
On Oct. 5, 1947, in the first televised White House address, President Truman asked Americans to refrain from eating meat on Tuesdays and poultry on Thursdays to help stockpile grain for starving people in Europe. 

-- John
John Jacobus, MS
Certified Health Physicist
e-mail:  crispy_bird at yahoo.com

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 