2000 HSS/PSA Program 1 - History of Science Society

PSA Abstracts
reliable evidential conclusions on the basis of the outcome of their experiment.
Sometimes such pre-trial planning fails, however. Experimenters then may try
to use after-trial strategies for getting around the problems introduced by failure
to adhere to pre-trial strictures. I discuss a general type of situation of this sort,
in which failing to adequately predesignate the choice of test statistic leaves
experimenters uncertain as to how they ought to characterize the experiment
that they have done, and propose a method for potentially drawing reliable results
from data even in the face of such uncertainty. In such a situation, the calculation
of statistical significance becomes impossible to calculate reliably. Significance
calculation requires that the experiment performed be specified, which in turn
determines the reference class to be used in calculating the probability of the
test result that obtained under the assumption of the null hypothesis. In the errorstatistical
account of experimental reasoning, such significance calculations are
central because of the close relationship between significance and the errorstatistical
concept of severity. I propose an extension of error-statistical methods
for assessing evidence for application to situations of uncertainty regarding the
choice of reference class for calculating significance. The method uses
calculations of apparent significance based on counterfactual choices of reference
class. That is, experimenters using this method employ a single body of data,
and ask what would have been the significance of the result that they would
have obtained with that data, under the assumption of a variety of different test
specifications. In this way, experimenters can determine how sensitive the
apparent significance of the result obtained is to particular choices of test
specification. I argue that this strategy can be used to determine whether the
apparent severity with which the hypothesis in question was tested is an artifact
of the failure of pre-trial planning. In particular, the method outlined may be
appropriate to situations in which pre-trial precautions against biasing the test
toward a particular result fail, but experimenters nevertheless wish to regain the
ability to draw reliable conclusions from the data at hand. I illustrate this approach
with an example from the search for the top quark conducted by the Collider
Detector at Fermilab collaboration in the early 90s.
242
P.␣ Kyle Stanford University of California, Irvine
Refusing the Devil’s Bargain: What Kind of Underdetermination Should We
Take Seriously?
Advocates have sought to prove that underdetermination obtains because all
theories have empirical equivalents. However, algorithms for generating
empirical equivalents simply exchange underdetermination for existing
philosophical chestnuts, while the few convincing examples of empirical
equivalents will not support the desired sweeping conclusions.
Underdetermination does not, however, depend on empirical equivalents: our
warrant for current theories is equally undermined by presently unconceived