Evaluation of the Australian Wage Subsidy Special Youth ...

219
6.3 Discussion
The comparison of the employment effects of Heckman versus PSM is shown in
Table 6.9. The former unweighted results are in the first columns, to facilitate comparison.
The weighted results are shown in the last two columns, shaded.
Comparison between the weighted and unweighted results provides an additional
indicator for possible attrition bias. D’Agostino and Rubin (2000) consider the issue of
missing data in item non-response, and note that most propensity score methods are based
on complete data. They suggest that missingness needs to be controlled in PSM. The fall
in the employment gain estimated under PSM once attrition is accounted for, is in line
with the need to account for the missing data that attrition introduces. The significant
differences between the results of the weighted and unweighted models are interpreted as
indicative of the presence of attrition bias. In a general comparison with the earlier
unweighted results, it is noticeable that the weighted results are much smaller in size,
although still positive, and are also of much lower statistical significance. Whereas before
the weighting was applied, the Heckman and PSM gave very different sizes for the
employment effect, once weighted to account for attrition the effects gained are very
similar in size. This would indicate that under the different modelling assumptions of
PSM and the Heckman bivariate probit, similar results for the employment effect are
gained once selection due to attrition is accounted for.
The PSM result is significant at the 22 per cent level of significance, which is outside
normal test bounds. However, the variance estimates in the weighted estimates are also
far more conservative than for the unweighted. 152 If the unweighted Heckman probit
results had used the more conservative estimate for variance, the t statistic for the
employment effect falls to 1.49. This is then similar in size to the weighted PSM, and
also gives about a 22 per cent level of significance. Under the more conservative variance
estimates, neither Heckman probit gives statistically significant results at normal test
sizes.
152 See chapter 7 sensitivity analysis for a fuller discussion of the variance estimates used.