Evaluation of the Australian Wage Subsidy Special Youth ...

199
6: Study 4 Weighting to counteract attrition and nonresponse
in ALS
In this chapter the combined weights are applied, which were constructed in the previous
chapter to repair the data for loss due to attrition, non-response and to account for the
survey design. First the weights are applied to the Heckman bivariate probit modelling,
and the effects discussed. Then the weights are applied to the propensity score matching
(PSM). The application of weights to PSM is more complex, and the full protocol is first
presented and then the weighted results are discussed. Finally, the employment effects
found from the Heckman and PSM models in the repaired data are contrasted with those
formerly found. The discussion focuses on the limitations to each application.
6.1 Results of weighting Heckman bivariate probit
The weights accounting for attrition, non-response and survey design were constructed in
the previous section. Table 6.1 shows the effect of applying this sample reduction weight
to the bivariate probit of employment and SYETP participation. The first column shows
the unweighted replication results to better enable comparison while the final column
gives the results for the regression weighted for sample reduction and survey design.
Attention is focused on the treatment effect, as indicated by the SYETP variable.
An important change brought in by accounting for the sample reduction is the loss of
statistical significance for CEP referrals in 1984 in the SYETP participation equation. It
is now less significant, although it would pass the t test at the significance level of 10 per
cent. In the modelling of the treatment effect of SYETP, this is a key element of the
identifying restriction in the bivariate probit. Along with this change, the SYETP
treatment dummy loses size and statistical significance in the employment equation. The
change introduced by weighting for sample reduction strongly affects the interpretation
of SYETP treatment effect. Given the coincident change in the CEP referrals in the
participation equation, and it’s role in the identifying restrictions for the bivariate probit,
it is highly likely that adjusting for the selection effects due to attrition of the sample, has