Evaluation of the Australian Wage Subsidy Special Youth ...

248
analysis was however strongly affected by the small sample size and reductions to
efficiency from various assumptions. As such the sensitivity analysis indicates that it
would be useful to provide a form of ‘confidence interval’ for model selection in
evaluation results, showing the variation employment effects are subject to for changes to
key modelling assumptions. Provision of such a confidence interval can allow for the
statistical modelling uncertainty, without which evidence for the employment effects can
be masked. This can be particularly helpful when there is no clear evidence favouring one
model and it’s assumptions over the other. In final consideration of the evidence for
SYETP found in this analysis, the conclusions of Heckman et al. (1999) are deferred to:
“…every estimator relies on identifying assumptions about the
outcome and participation processes. When a particular estimator
is applied to data, where those assumptions fail to hold, bias
results. This bias can be substantial. When different estimators are
applied to the same data, the estimates they produce will vary
because at most one set of underlying assumptions is consistent
with the data. Only if there is no problem of selection bias would
all estimators identify the same parameter.” (Heckman et al.
(1999): 2007)
As the limitations of the non-experimental data used here bound the extent of conclusions,
it is perhaps the evaluation policy which needs to be addressed. However, debating the
role of experimentation in social policy is beyond the scope of this study, and inevitably
becomes a political and ethical decision.