Evaluation of the Australian Wage Subsidy Special Youth ...

112
4.1 Differences between the treatment and comparison group
An important problem for evaluation of programme effects is whether the treated are
compared to an adequate reference group. This is the issue of selection distortion, where
those treated have a very different profile to those in the comparison group. Selective
recruitment onto subsidy would make the treated have a different profile than the larger
group of those eligible. Controls for selection distortion can be made on observable
characteristics, such as those variables shown in Table 4.1. The Table 4.1 gives the mean
and standard deviation from the mean for a set of characteristics, with column 1 and 2
showing these for the SYETP treated group, and columns 3 and 4 showing the
comparison group characteristics. The absolute difference between the means is in
column 5, with the results of the t-test at one percent level of significance indicated by an
asterisk to show statistically significant differences.
Richardson (1998) p6 commented that the contrast between those who took part in
SYETP and those in the comparison group was “striking”. Indeed, column 1 and column
3 of Table 4.1, of which the means were also presented by Richardson (1998), shows that
the profile of the treatment and comparison groups differs strongly. In most cases, means
were quite different for the SYETP relative to the comparisons, and in some cases the
variation from the mean, as represented by the standard deviation was also very different
for each group. As Richardson (1998) highlights, SYETP participants had a different
educational attainment profile where post-school qualifications were less common, they
were about a year younger, and had poorer labour market experiences than those in the
comparison group. However, how different the SYETP group was from the comparisons
is related more clearly here by the addition of the test of the statistical significance of the
mean difference. Almost all variables have a significant divergence between the means
for the SYETP group and the comparisons. Consideration of the many statistically
significant differences in Table 4.1 makes the selection distortion for SYETP in the ALS
data apparent. The literature review also highlights other evidence that it was generally
the case for SYETP entrants to be younger than the eligible group as a whole – mostly
teenagers. Thus it is possible that the ALS sample reflects differences that existed
between the SYETP and comparison populations.