Evaluation of the Australian Wage Subsidy Special Youth ...
Evaluation of the Australian Wage Subsidy Special Youth ... Evaluation of the Australian Wage Subsidy Special Youth ...
258 evaluations, as shown in the review, although most of the results can only be considered with strong caveats, due to the inadequate methods employed. At least partially, due to the varying timeframes for the analyses, this gives evidence that SYETP gave employment gains in a dynamic context. In all variations, the research here found positive impacts of SYETP, even if some were not statistically significant, a difficulty influenced by the small sample and efficiency losses from some of the methods. Regardless of whether selection was based on the assumption of observables or unobservables, the impact remained positive. This contributes evidence of the robustness of the positive effect on employment for SYETP participants. The sensitivity of the specifications of both the bivariate probit and the propensity score used for the PSM were explored. In the bivariate probit this centered around the exclusion restriction, and in the propensity score matching the implications of the exclusion restriction for the comparative analysis were investigated. In the case of the bivariate probit, comparisons of the weighted and unweighted results found that the specification has great bearing on the results. An interpretation advanced here is that because sample attrition was confounded with the SYETP program treatment, the bivariate probit accounted for this with the modelling of the selection into the program. Hence, the strong differences in the outcomes of the weighted and unweighted analyses. There were serious difficulties in estimating alternative specifications in the unweighted data, indicating that the search for a plausible specification could be problematic. In the PSM, in line with recent contributions to the matching literature, the variants examined the restriction of the propensity to those variables expected on empirical or theoretical grounds to have significant relations with both SYETP participation and employment. The substantial changes to the results provide a useful illustration of the importance of the modelling assumptions, and indicate that the assumptions involved in matching are not simple. The results show that the statistical modelling uncertainty can be difficult to resolve without clear evidence of a strong basis for favouring one model over another, unless the goodness of fit assessment undertaken provides obvious resolution. It is
259 posited that that such sensitivity analysis, comparing bivariate probit and PSM where the data can lend support to both models, should be provided to enhance evaluation results and provide a form of confidence interval for model selection, and so account for the statistical modelling uncertainty. This exploration has confirmed some relationships suggested in the literature and revealed some new insights into SYETP and wage subsidy employment effects. The research challenges orthodox evaluation methods by applying not one potentially appropriate selection approach, and underlying assumption about observables and unobservables, but both. The benefits are informed overview of the role of the assumption to the evaluation outcome. To the extent that the data allows both methods to be attempted, such as sufficiently extensive data on characteristics of employment and participation, as well as variables that can act as instruments, this allows the conclusion that this approach should be adopted wherever possible. Deliberative attempts were made to constructively account for data and modelling issues. The constraints of the nonexperimental data limit the conclusions for SYETP, as without a benchmark from experimental data, it is not possible to further choose between the model of the Heckman and PSM methods. Although unsatisfying to a small extent, the remaining limitations of this research can inform further work on the evaluation of labour market programmes such as SYETP.
- Page 223 and 224: 207 6.2 Results of weighting the PS
- Page 225 and 226: 209 The distribution of the propens
- Page 227 and 228: 211 Table 6.3 Weighted probit used
- Page 229 and 230: 213 (0.76) Tradesperson mtrad 0.20
- Page 231 and 232: 215 Table 6.5 Summary statistics fo
- Page 233 and 234: 217 Table 6.7 Matching results, sin
- Page 235 and 236: 219 6.3 Discussion The comparison o
- Page 237 and 238: 221 the selection into SYETP and th
- Page 239 and 240: 223 Heteroskedasticity is a violati
- Page 241 and 242: 225 Table 7.1, Part A Employment eq
- Page 243 and 244: 227 (1.26) (1.28) (1.16) Mothers oc
- Page 245 and 246: 229 Other Post-School qualification
- Page 247 and 248: 231 7.1.2 Exclusion restriction in
- Page 249 and 250: 233 Finally, the third panel of new
- Page 251 and 252: 235 Table 7.2 summary of changes to
- Page 253 and 254: 237 schooling that was statisticall
- Page 255 and 256: 239 (0.25) (0.21) 3 years + -0.50 -
- Page 257 and 258: 241 Table 7.4 Summary of distributi
- Page 259 and 260: 243 7.2.2 Propensity score matching
- Page 261 and 262: 245 original model, but with the pe
- Page 263 and 264: 247 maintained, then the Heckman bi
- Page 265 and 266: 249 Table 7.6 Weighted Probit used
- Page 267 and 268: 251 Table 7.7 Summary of distributi
- Page 269 and 270: 253 8: Summary and Conclusions The
- Page 271 and 272: 255 over which it ran, the review m
- Page 273: 257 the Heckman and PSM methods wer
- Page 277 and 278: 261 Description Derivation and deta
- Page 279 and 280: Appendix 2 Tables 263
- Page 281 and 282: 2 years 0.28 0.07 (1.70) (1.70) 3 y
- Page 283 and 284: 267 Table A2.0b Univariate Probit o
- Page 285 and 286: 269 Religion brought up in (1.52) (
- Page 287 and 288: 271 hq9_84 Year 9 of school or less
- Page 289 and 290: 273 Table A2.1 Continued Means and
- Page 291 and 292: 275 mtrad Tradesperson 0.04 0.04 0.
- Page 293 and 294: 277 Year 9 of school or less -0.19
- Page 295 and 296: 279 Table A2.3 Probit of SYETP part
- Page 297 and 298: 281 (0.22) (0.44) (0.67) (0.89) Tra
- Page 299 and 300: 283 Table A2.5a Univariate probit f
- Page 301 and 302: 285 Manager/professional/para-profe
- Page 303 and 304: 3 years + -0.04 (2.13)* CEP referra
- Page 305 and 306: 289 Table A2.6 Part A Employment eq
- Page 307 and 308: 291 Table A2.6 Part B Selection/par
- Page 309 and 310: 293 Table A2.7 Part A Employment eq
- Page 311 and 312: 295 Table A2.7 Part B Selection/par
- Page 313 and 314: 297 Table A2.8 Part A Employment eq
- Page 315 and 316: 299 Table A2.8 Part B Selection/par
- Page 317 and 318: 301
- Page 319 and 320: BLMR Bureau of Labour Market Resear
- Page 321 and 322: Fay, R.G. (1996) "Enhancing the eff
- Page 323 and 324: Heckman J.J. and Smith, J. A. (1998
259<br />
posited that that such sensitivity analysis, comparing bivariate probit and PSM where <strong>the</strong><br />
data can lend support to both models, should be provided to enhance evaluation results<br />
and provide a form <strong>of</strong> confidence interval for model selection, and so account for <strong>the</strong><br />
statistical modelling uncertainty.<br />
This exploration has confirmed some relationships suggested in <strong>the</strong> literature and<br />
revealed some new insights into SYETP and wage subsidy employment effects. The<br />
research challenges orthodox evaluation methods by applying not one potentially<br />
appropriate selection approach, and underlying assumption about observables and<br />
unobservables, but both. The benefits are informed overview <strong>of</strong> <strong>the</strong> role <strong>of</strong> <strong>the</strong><br />
assumption to <strong>the</strong> evaluation outcome. To <strong>the</strong> extent that <strong>the</strong> data allows both methods to<br />
be attempted, such as sufficiently extensive data on characteristics <strong>of</strong> employment and<br />
participation, as well as variables that can act as instruments, this allows <strong>the</strong> conclusion<br />
that this approach should be adopted wherever possible. Deliberative attempts were made<br />
to constructively account for data and modelling issues. The constraints <strong>of</strong> <strong>the</strong> nonexperimental<br />
data limit <strong>the</strong> conclusions for SYETP, as without a benchmark from<br />
experimental data, it is not possible to fur<strong>the</strong>r choose between <strong>the</strong> model <strong>of</strong> <strong>the</strong> Heckman<br />
and PSM methods. Although unsatisfying to a small extent, <strong>the</strong> remaining limitations <strong>of</strong><br />
this research can inform fur<strong>the</strong>r work on <strong>the</strong> evaluation <strong>of</strong> labour market programmes<br />
such as SYETP.