Using R for Introductory Statistics : John Verzani

Using R for introductory statistics 208enough. A guilty verdict is more accurately called a verdict of “not innocent.” Theperformer of a significance test seeks to determine whether the null hypothesis isreasonable given the available data. The evidence is replaced by an experiment thatproduces a test statistic. The probability that the test statistic is the observed value or ismore extreme as implied by the alternative hypothesis is calculated using the assumptionsof the null hypothesis. This is called the p-value. This is like the weighing of theevidence—the jury calculating the likelihood that the evidence agrees with theassumption of innocence.The calculation of the p-value is called a significance test. The p-value is based onboth the sampling distribution of the test statistic under H 0 and the single observed valueof it during the trial. In words, we havep-value=P(test statistic is the observed value or is more extreme|H 0 ).The p-value helps us decide whether differences in the test statistic from the nullhypothesis are attributable to chance or sampling variation, or to a failure of the nullhypothesis. If a p-value is small, the test is called statistically significant, as it indicatesthat the null hypothesis is unlikely to produce more extreme values than the observedone. Small p-values cast doubt on the null hypothesis; large ones do not.What is “large” or “small” depends on the area of application, but there are somestandard levels that are used. Some R functions will mark p-values with significancestars, as described in Table 8.1. Although these are useful for quickly identifyingsignificance, the cutoffs are arbitrary, settled on more for ease of calculation than actualrelevance.In some instances, as with a criminal trial, a decision is made based on the pvalue. Ajuror is instructed that a defendant, to be found guilty, must be thought guilty beyond ashadow of a doubt. A significance test is less vague, as a significance level is specifiedthat the p-value is measured against. A typical signifi-cance level is 0.05. If the p-value isless than the significance level, then the null hypothesis is said to be rejected, or viewedas false. If the p-value is larger than the significance level, then the null hypothesis isaccepted.The words “reject” and “accept” are perhaps more absolute than they should be. Whenrejecting the null, we don’t actually prove the null to be false or the alternative to be true.All that is shown is that the null hypothesis is unlikely to produce values more extremethan the observed value. When accepting the null we don’t prove it is true, we just findthat the evidence is not too unlikely if the null hypothesis is true.By specifying a significance level, we indirectly find values of the test statistic thatwill lead to rejection. This allows us to specify a rejection region consisting of all valuesfor the observed test statistic that produce p-values smaller than the significance level.The boundaries between the acceptance and rejection regions are called critical values.The use of a rejection region avoids the computation of a p-value: reject if the observedvalue is in the rejection region and accept otherwise. We prefer, though, to find andreport the p-value rather than issue a simple verdict of “accept” or “reject.”This decision framework has been used historically in scientific endeavors.Researchers may be investigating whether a specific treatment has an effect. They mightconstruct a significance test with a null hypothesis of the treatment having no effect,