Using R for Introductory Statistics : John Verzani

Chapter 8Significance testsFinding a confidence interval for a parameter is one form of statistical inference. Asignificance test, or test of hypothesis, is another. Rather than specify a range of valuesfor a population parameter, a significance test assumes a value for the populationparameter and then computes a probability based on a sample given that assumption.■ Example 8.1: A criminal trial The ideas behind a significance test can beillustrated by analogy to a criminal trial in the United States—as seen on TV. Imagine thefollowing simplified scenario: a defendant is charged with a crime and must stand trial.During the trial, a prosecutor and defense lawyer respectively try to convince the jury thatthe defendant is either guilty or innocent. The jury is supposed to be unbiased. Whendeciding the defendant’s fate, the jurors are instructed to assume that the defendant isinnocent unless proven guilty beyond a shadow of a doubt. At the end of the trial thejurors decide the guilt or innocence of the defendant based on the strength of their beliefin the assumption of his innocence given the evidence. If the jurors believe it veryunlikely that an innocent person could have evidence to the contrary, they will find thedefendant “guilty.” If it is not so unlikely, they will rule “not guilty.”The system is not foolproof. A guilty man can go free if he is found not guilty, and aninnocent man can be erroneously convicted. The frequency with which these errors occurdepends on the threshold used to find guilt. In a criminal trial, to decrease the chance of aerroneous guilty verdict, the stringent shadow of a doubt criterion is used. In a civil trial,this phrasing is relaxed to a preponderance of the evidence. The latter makes it easier toerr with a truly innocent person but harder to err with a truly guilty one. nullLet’s rephrase the above example in terms of significance tests. The assumption ofinnocence is replaced with the null hypothesis, H 0 . This stands in contrastTable 8.1 Level of significance for range of p-valuesp-value range significance stars common description[0, .001] *** extremely significant(.001, .01] ** highly significant(.01, .05] * statistically significant(.05, .!] · could be significant(.1, 1] not significantto the alternative hypothesis, H A . This would be an assumption of guilt in the trialanalogy. In a trial, this alternative is not used as an assumption; it only gives a directionto the interpretation of the evidence. The determination of guilt by a jury is not proof ofthe alternative, only a failure of the assumption of innocence to explain the evidence well