Optimality

96 D. G. Mayo and D. R. Cox
to make the correction. In short, it would be very unwise to dismiss the possibility
of learning from data something new in a totally unanticipated direction, but one
must discriminate the contexts in order to gain guidance for what further analysis,
if any, might be required.
5. Concluding remarks
We have argued that error probabilities in frequentist tests may be used to evaluate
the reliability or capacity with which the test discriminates whether or not the
actual process giving rise to data is in accordance with that described in H0. Knowledge
of this probative capacity allows determination of whether there is strong evidence
against H0 based on the frequentist principle we set out FEV. What makes
the kind of hypothetical reasoning relevant to the case at hand is not the long-run
low error rates associated with using the tool (or test) in this manner; it is rather
what those error rates reveal about the data generating source or phenomenon. We
have not attempted to address the relation between the frequentist and Bayesian
analyses of what may appear to be very similar issues. A fundamental tenet of the
conception of inductive learning most at home with the frequentist philosophy is
that inductive inference requires building up incisive arguments and inferences by
putting together several different piece-meal results; we have set out considerations
to guide these pieces. Although the complexity of the issues makes it more difficult
to set out neatly, as, for example, one could by imagining that a single algorithm
encompasses the whole of inductive inference, the payoff is an account that approaches
the kind of arguments that scientists build up in order to obtain reliable
knowledge and understanding of a field.
References
[1] Birnbaum, A. (1977). The Neyman–Pearson theory as decision theory, and as
inference theory; with a criticism of the Lindley–Savage argument for Bayesian
theory. Synthese 36, 19–49.
[2] Carnap, R. (1962). Logical Foundations of Probability. University of Chicago
Press.
[3] Cochran, W. G. (1965). The planning of observational studies in human
populations (with discussion). J.R.Statist. Soc. A 128, 234–265.
[4] Cox, D. R. (1958). Some problems connected with statistical inference. Ann.
Math. Statist. 29, 357–372.
[5] Cox, D. R. (1977). The role of significance tests (with discussion). Scand. J.
Statist. 4, 49–70.
[6] Cox, D. R. and Hinkley, D. V. (1974). Theoretical Statistics. Chapman
and Hall, London.
[7] Cox, D. R. and Snell, E. J. (1974). The choice of variables in observational
studies. J. R. Statist. Soc. C 23, 51–59.
[8] De Finetti, B. (1974). Theory of Probability, 2 vols. English translation from
Italian. Wiley, New York.
[9] Fisher, R. A. (1935a). Design of Experiments. Oliver and Boyd, Edinburgh.
[10] Fisher, R. A. (1935b). The logic of inductive inference. J. R. Statist. Soc.
98, 39–54.
[11] Gibbons, J. D. and Pratt, J. W. (1975). P-values: Interpretation and
methodology. American Statistician 29, 20–25.