Evaluation of the Australian Wage Subsidy Special Youth ...
Evaluation of the Australian Wage Subsidy Special Youth ...
Evaluation of the Australian Wage Subsidy Special Youth ...
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
i<br />
<strong>Evaluation</strong> <strong>of</strong> <strong>the</strong> <strong>Australian</strong> <strong>Wage</strong> <strong>Subsidy</strong><br />
<strong>Special</strong> <strong>Youth</strong> Employment and Training Program,<br />
SYETP<br />
Genevieve Margaret Knight<br />
A <strong>the</strong>sis submitted for <strong>the</strong> Degree <strong>of</strong> Doctor <strong>of</strong> Philosophy at <strong>the</strong> School <strong>of</strong> Economics<br />
and Political Science, Faculty <strong>of</strong> Economics and Business, University <strong>of</strong> Sydney<br />
August 2002
iii<br />
Acknowledgements<br />
Dedicated to my family<br />
Harry, Margaret, Yolande, Melisande, Juan, José, Isobel, Joshua, Inge, Ole, Mette, Andrew.<br />
<strong>Special</strong> thanks to Russell Ross, my supervisor.<br />
Fur<strong>the</strong>r thanks are due to<br />
James Richardson, Michael Lechner, Jeff Smith, Stefan Speckesser, Barbara Sianesi,<br />
Lorraine Dearden, Alex Heath, Henry Overman,<br />
Paul Gregg, Jonathon Wadsworth,<br />
Dave Metcalf, Richard Jackman, Richard Layard, <strong>the</strong> Centre for Economic Performance,<br />
Michael White, Alex Bryson, Dave Wilkinson, Richard Dorsett,<br />
Doro<strong>the</strong> Bonjour, Steve Lissenburgh,<br />
Joan Payne, Jim Skea, Policy Studies Institute,<br />
Denzil Fiebig, Tony Phipps, Jeff Sheen, Gordon Mills, Judy Yates, Roslyn Joyeaux,<br />
Ken Tallis and The <strong>Australian</strong> Bureau <strong>of</strong> Statistics.
v<br />
Declaration<br />
I hereby declare that this submission is my own work and<br />
that, to <strong>the</strong> best <strong>of</strong> my knowledge and belief, it contains no<br />
material previously published or written by ano<strong>the</strong>r person<br />
nor material which to a substantial extent has been accepted<br />
for <strong>the</strong> award <strong>of</strong> any o<strong>the</strong>r degree or diploma <strong>of</strong> <strong>the</strong><br />
university or o<strong>the</strong>r institute <strong>of</strong> higher learning, except where<br />
due acknowledgment has been made in <strong>the</strong> text. All errors<br />
are my own.
vi<br />
The job subsidy <strong>Special</strong> <strong>Youth</strong> Employment and Training Program (SYETP) was<br />
introduced in Australia with <strong>the</strong> aim <strong>of</strong> improving <strong>the</strong> movement into work. In 1984, <strong>the</strong><br />
SYETP was a flat rate subsidy <strong>of</strong> A$75 a week paid to employers for 17 weeks,<br />
equivalent in value to half <strong>the</strong> average teenage wage, and was available to youths aged<br />
15-24 who had been claiming unemployment benefits and not studying full-time for at<br />
least 4 <strong>of</strong> preceding 12 months.<br />
A review <strong>of</strong> <strong>the</strong>oretical literature indicates <strong>the</strong>y can give no pro<strong>of</strong> <strong>of</strong> employment gains<br />
for wage subsidies. The empirical ambiguity <strong>of</strong> employment gains is concluded<br />
unresolved, in both recent overseas and <strong>Australian</strong> literature. A contributing factor is <strong>the</strong><br />
insufficiency <strong>of</strong> <strong>the</strong> evaluation methods. Appraisal <strong>of</strong> <strong>the</strong> micro-evaluation evidence for<br />
SYETP and o<strong>the</strong>r <strong>Australian</strong> wage subsidies is also found to suffer <strong>the</strong>se deficiencies.<br />
The inadequacies <strong>of</strong> past analyses <strong>of</strong> SYETP contribute three <strong>the</strong>mes to address: suitable<br />
modelling <strong>of</strong> selection to account for <strong>the</strong> influence <strong>of</strong> observables or unobservables,<br />
dealing with non-response in <strong>the</strong> observational data, and appropriate control for <strong>the</strong><br />
differences between <strong>the</strong> SYETP and comparison groups.<br />
Past evaluation by Richardson (1998), modelling <strong>the</strong> Heckman selection bivariate probit,<br />
using <strong>the</strong> <strong>Australian</strong> Longitudinal Survey <strong>of</strong> <strong>Youth</strong>s 1984-1987 found a very large<br />
positive employment effect for SYETP participants 26 months after taking part. A key<br />
issue with <strong>the</strong> results is that no account <strong>of</strong> sample attrition was made. Theory indicates<br />
bias to be a potentially serious problem with results. Two evaluation methods are<br />
explored – <strong>the</strong> Heckman selection bivariate probit model, and matching methods, in<br />
particular propensity score matching. Both identify a parameter corresponding to <strong>the</strong><br />
mean effect <strong>of</strong> treatment on <strong>the</strong> treated, which can be used to decide whe<strong>the</strong>r <strong>the</strong><br />
programme leads to employment gains. However each method uses different assumptions<br />
to achieve this. Selection on unobservables is assumed by <strong>the</strong> Heckman bivariate probit,<br />
while selection on observables is assumed by matching methods.<br />
A series <strong>of</strong> empirical studies assesses a number <strong>of</strong> questions – what happens to <strong>the</strong><br />
evaluation outcome if selection is assumed to be based on observables instead <strong>of</strong><br />
unobservables; what is <strong>the</strong> importance <strong>of</strong> sample reduction to <strong>the</strong> evaluation outcome;<br />
and how sensitive is <strong>the</strong> employment impact to variation in modelling. To provide a<br />
foundation for useful comparison Richardson (1998) is first replicated successfully. The<br />
more recently popular propensity score matching method (PSM) is <strong>the</strong>n applied. The<br />
PSM results reduce <strong>the</strong> size and significance <strong>of</strong> <strong>the</strong> employment effect found. The effects<br />
<strong>of</strong> attrition are examined and <strong>the</strong>n accounted for, and <strong>the</strong> impact on evaluation discussed.<br />
The results are found to be smaller and have lower statistical significance. Correctly<br />
accounting for weights is found to be important in applying PSM.<br />
The Heckman and <strong>the</strong> PSM method both make strong but very different assumptions<br />
about <strong>the</strong> selection into SYETP. A comparison <strong>of</strong> <strong>the</strong> employment impacts found under<br />
each method is undertaken, toge<strong>the</strong>r with a discussion <strong>of</strong> <strong>the</strong> most suitable assumptions<br />
for this evaluation. The value <strong>of</strong> replication is validated and advocated. The research<br />
confirms that careful accounting for data and modelling problems is important. External<br />
validity for <strong>the</strong> robustness <strong>of</strong> employment gains to SYETP are provided by <strong>the</strong> variations
in method and assumptions. The orthodox approach <strong>of</strong> adopting only one potentially<br />
appropriate selection approach and underlying assumption <strong>of</strong> observables or<br />
unobservables is challenged. A sensitivity analysis showing variation to employment<br />
effects for changes in key modelling assumptions can give a confidence interval<br />
accounting for statistical modelling uncertainty. It is concluded that <strong>the</strong> benefits are an<br />
informed overview <strong>of</strong> <strong>the</strong> role <strong>of</strong> <strong>the</strong> assumption to <strong>the</strong> evaluation outcome.<br />
vii
Table <strong>of</strong> contents<br />
Table <strong>of</strong> contents.............................................................................................................. i<br />
List <strong>of</strong> Tables and Figures.............................................................................................. xi<br />
Glossary ....................................................................................................................... xiv<br />
Abstract......................................................................................................................... xv<br />
1: <strong>Wage</strong> subsidy <strong>the</strong>ory and evaluation .............................................................................. 1<br />
1.1 Structure <strong>of</strong> this research .......................................................................................... 1<br />
1.1 General introduction ................................................................................................. 2<br />
1.2 Brief summary <strong>of</strong> <strong>the</strong> <strong>the</strong>ory <strong>of</strong> wage subsidies........................................................ 7<br />
1.3 Introduction to evaluation <strong>of</strong> programs................................................................... 16<br />
1.3.1 <strong>Evaluation</strong> <strong>of</strong> programs........................................................................ 17<br />
1.3.2 <strong>Evaluation</strong> problem.............................................................................. 18<br />
1.3.3 Selection............................................................................................... 20<br />
1.3.4 Observables and unobservables ........................................................... 22<br />
1.4 Brief comment on recent overseas evidence <strong>of</strong> wage subsidy evaluations ............ 23<br />
2: <strong>Australian</strong> literature review .......................................................................................... 28<br />
2.1 Review <strong>of</strong> <strong>Australian</strong> wage subsidy evaluation evidence....................................... 30<br />
2.1.1 Adult <strong>Wage</strong> <strong>Subsidy</strong> Scheme............................................................... 31<br />
2.1.2 ‘General Training Assistance On-<strong>the</strong>-Job-Training’ ........................... 33<br />
2.1.3 Jobstart ................................................................................................. 34<br />
2.1.4 Discussion............................................................................................ 41<br />
2.2 SYETP implementation .......................................................................................... 43<br />
2.2.1 Historical development <strong>of</strong> SYETP........................................................... 43<br />
2.2.2 An overview <strong>of</strong> SYETP changes ............................................................. 48<br />
2.2.3 SYETP operation ..................................................................................... 51<br />
2.2.4 Political environment <strong>of</strong> SYETP ............................................................. 55<br />
2.2.5 Economic Context <strong>of</strong> SYETP at <strong>the</strong> time <strong>of</strong> our analysis........................ 58<br />
2.2.5.1 Role and impact <strong>of</strong> <strong>the</strong> CES.............................................................. 58<br />
2.2.5.2 Unemployment in <strong>the</strong> <strong>Australian</strong> economy over <strong>the</strong> 1980’s ............ 62<br />
2.2.5.3 Discussion......................................................................................... 69<br />
2.2.6 Context <strong>of</strong> SYETP environment and operation ....................................... 70<br />
2.2.6.1 General results for SYETP using o<strong>the</strong>r approaches.......................... 70<br />
2.2.6.2 Job characteristics ............................................................................. 71<br />
2.2.6.3 Characteristics <strong>of</strong> SYETP commencements ..................................... 72<br />
2.2.6.4 Factors affecting completion <strong>of</strong> subsidy placement.......................... 77<br />
2.3 Critical Review <strong>of</strong> <strong>Evaluation</strong> <strong>of</strong> SYETP ............................................................... 78<br />
2.3.1 Stretton (1982, 1984) .......................................................................... 79<br />
2.3.2 Baker (1984) ........................................................................................ 81<br />
2.3.3 Rao and Jones (1986)........................................................................... 84<br />
2.3.4 Richardson (1998)................................................................................ 87<br />
2.3.5 General discussion ............................................................................... 91<br />
3: Study 1 Replication....................................................................................................... 94<br />
3.1 Motivation for replication....................................................................................... 94<br />
3.2 Methods: The Heckman selection model................................................................ 95<br />
3.2.1 Self-selection and <strong>the</strong> evaluation <strong>of</strong> programmes................................ 95<br />
3.2.2 The estimated model............................................................................ 96<br />
viii
3.3 Data and variables used for estimation ................................................................... 99<br />
3.4 Replication results................................................................................................. 101<br />
3.5 Discussion............................................................................................................. 103<br />
4: Study 2 Propensity score matching for SYETP.......................................................... 111<br />
4.1 Differences between <strong>the</strong> treatment and comparison group................................... 112<br />
4.2 Propensity score matching methods...................................................................... 115<br />
4.3 Theory underlying propensity score matching methods....................................... 116<br />
4.4 The propensity score matching protocol implemented......................................... 119<br />
4.5 Estimating <strong>the</strong> probability <strong>of</strong> participation for SYETP ........................................ 122<br />
4.6 Distribution <strong>of</strong> <strong>the</strong> propensity score...................................................................... 127<br />
4.7 Common support for <strong>the</strong> treated ........................................................................... 128<br />
4.8 Results <strong>of</strong> Matching: one-to-one nearest-neighbour............................................. 132<br />
4.8.1 Discussion <strong>of</strong> <strong>the</strong> results .................................................................... 132<br />
4.8.2 Mean standardised bias statistic......................................................... 136<br />
4.9 Sensitivity analysis: all-in-Radius matching......................................................... 139<br />
4.10 Discussion........................................................................................................... 141<br />
5: Study 3 Attrition and non-response in <strong>the</strong> ALS.......................................................... 145<br />
5.1 Examining sample reduction in <strong>the</strong> ALS.............................................................. 145<br />
5.2 Theory <strong>of</strong> why attrition introduces bias................................................................ 146<br />
5.3 Empirical aspects <strong>of</strong> <strong>the</strong> effects <strong>of</strong> attrition on estimates ..................................... 149<br />
5.4 Empirical attrition test and treatment.................................................................... 150<br />
5.5 Examining <strong>the</strong> role <strong>of</strong> attrition in <strong>the</strong> ALS and in modelling <strong>of</strong> <strong>the</strong> SYETP<br />
employment effect ...................................................................................................... 152<br />
5.5.1 Extent <strong>of</strong> sample reduction .................................................................... 152<br />
5.5.2 Univariate examination <strong>of</strong> sample reduction ......................................... 155<br />
5.5.3 Sample reduction by SYETP treatment group....................................... 160<br />
5.5.3.1 Comparing those lost in attrition to those who remain................... 160<br />
5.5.3.2 The effect <strong>of</strong> sample reduction on <strong>the</strong> difference between SYETP and<br />
comparison groups...................................................................................... 164<br />
5.5.4 Attrition: natural attrition or analytical sample reduction?.................... 167<br />
5.6 Accounting for non-response and sample design ................................................. 172<br />
5.6.1 Survey design and non-response effects on modelling.......................... 176<br />
5.6.1.1 Analytical selection......................................................................... 176<br />
5.6.1.2 Effects <strong>of</strong> <strong>the</strong> non-response to 1984 Survey on <strong>the</strong> participation<br />
model........................................................................................................... 183<br />
5.7 Multivariate analysis <strong>of</strong> effects <strong>of</strong> sample reduction ............................................ 187<br />
5.7.1 Accounting for item non-response......................................................... 187<br />
5.7.2 Sample reduction effects on model <strong>of</strong> SYETP participation................. 189<br />
5.8 Model <strong>of</strong> <strong>the</strong> probability <strong>of</strong> attrition ..................................................................... 192<br />
5.9 Attrition Weights from <strong>the</strong> model......................................................................... 196<br />
5.10 Conclusions......................................................................................................... 196<br />
6: Study 4 Weighting to counteract attrition and non-response in ALS......................... 199<br />
6.1 Results <strong>of</strong> weighting Heckman bivariate probit.................................................... 199<br />
6.2 Results <strong>of</strong> weighting <strong>the</strong> PSM............................................................................... 207<br />
6.2.1 Weighting protocol ................................................................................ 207<br />
6.2.2 Effects <strong>of</strong> weighting PSM...................................................................... 208<br />
ix
6.3 Discussion............................................................................................................. 219<br />
7: Study 5 Sensitivity analysis ........................................................................................ 222<br />
7.1 Sensitivity <strong>of</strong> Heckman specification ................................................................... 222<br />
7.1.1 Potential heteroskedasticity ................................................................... 222<br />
7.1.2 Exclusion restriction in <strong>the</strong> Heckman model......................................... 231<br />
7.2 Varying <strong>the</strong> Propensity Score specification, effects on <strong>the</strong> match........................ 236<br />
7.2.1 Propensity score matching and <strong>the</strong> effect <strong>of</strong> excluding CEP referrals... 236<br />
7.2.2 Propensity score matching and <strong>the</strong> effect <strong>of</strong> reduced specification....... 243<br />
7.2.3 Fur<strong>the</strong>r discussion and conclusions ................................................................... 246<br />
8: Summary and Conclusions ......................................................................................... 253<br />
Appendix 1 Data appendix.............................................................................................. 260<br />
Table 1 Description <strong>of</strong> <strong>the</strong> data....................................................................... 260<br />
Appendix 2 Tables .......................................................................................................... 263<br />
Bibliography ................................................................................................................... 302<br />
x
List <strong>of</strong> Tables and Figures<br />
Figure 1.1: Labour market equilibrium with wage subsidy..................................... 11<br />
Figure 1.2 Employment effect <strong>of</strong> <strong>the</strong> subsidy,......................................................... 16<br />
Table 1.3 Brief overview <strong>of</strong> recent European wage subsidy evidence considered.. 27<br />
Table 2.1 Private Sector Jobstart subsidy weekly rates 1985-1987......................... 37<br />
Table 2.2 Stromback and Dockery (2000) Raw employment outcomes after Jobstart<br />
.................................................................................................................................. 40<br />
Table 2.3 Average junior award rates 1977 to 1981................................................ 46<br />
Table 2.4 SYETP rates, period and target group/eligibility criteria 1976-December<br />
1985.......................................................................................................................... 50<br />
Table 2.5 Key SYETP provisions in 1983/84.......................................................... 55<br />
Table 2.6 SYETP annual expenditure and placements1976/77-1985/86 ................ 58<br />
Table 2.7 Active steps to find work by youths looking for work in July 1980 ....... 59<br />
Table 2.8 Age distribution for CES registrants in NSW 1985-1986 ....................... 62<br />
Table 2.9 Unemployment rate Australia 1981-1990, seasonally adjusted............... 63<br />
Table 2.10 Average duration <strong>of</strong> unemployment (number <strong>of</strong> weeks) by age, August<br />
1981-1990 ................................................................................................................ 64<br />
Table 2.11 Unemployment, labour force participation and employment rates,<br />
teenagers and total working age population March 1983 and March 1988............. 66<br />
Table 2.12 Value <strong>of</strong> unemployment benefits to youths 1983-1987......................... 67<br />
Table 2.13 Smith (1984b) summary <strong>of</strong> estimates <strong>of</strong> SYETP................................... 71<br />
Table 2.14 Distribution <strong>of</strong> SYETP commencements, by age 1980-81.................... 73<br />
Table 2.15 Distribution <strong>of</strong> SYETP commencements, by age and sex 1980-81....... 74<br />
Table 2.16 Completion <strong>of</strong> SYETP placement.......................................................... 76<br />
Table 2.17 State usage <strong>of</strong> programmes in 1980/81.................................................. 77<br />
Table 2.18 Baker (1984) Post-programme full-time employment outcome............ 83<br />
Table 2.19 Baker (1984) Estimated probabilities <strong>of</strong> labour market outcomes for<br />
participants from <strong>the</strong> model <strong>of</strong> employment............................................................ 84<br />
Table 2.20 Rao and Jones (1986) Estimated percent post-programme full-time<br />
continuous employment chances 1981-1983........................................................... 86<br />
Table 2.21 Richardson (1998) Estimated marginal effect <strong>of</strong> SYETP on employment<br />
from bivariate probit modelling............................................................................... 89<br />
Table 3.1, Part A Employment equation from bivariate probit ............................. 105<br />
Table 3.1 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit.............. 108<br />
Table 4.1 Difference between treatment group and comparison group................. 114<br />
Table 4.2 Probit used to estimate propensity score for propensity score matching124<br />
Figure 4.3 Histograms <strong>of</strong> estimated propensity score prior to matching............... 129<br />
Figure 4.4 Kernel Density <strong>of</strong> propensity scores distribution for Treated SYETP and<br />
untreated................................................................................................................. 129<br />
Table 4.5 Summary statistics for distribution <strong>of</strong> propensity scores....................... 131<br />
Table 4.6 Matching results, Single nearest neighbour with replacement, within<br />
caliper..................................................................................................................... 135<br />
Table 4.7 Matching results, All-Within- caliper/Radius with replacement........... 141<br />
Table 4.8 Employment effects <strong>of</strong> Heckman versus PSM ...................................... 142<br />
Table 5.1 Sample Reduction for ALS List sample ................................................ 153<br />
xi
Table 5.2 Summary statistics for attrition effects in <strong>the</strong> final sample ................... 158<br />
Table 5.3 Summary statistics <strong>of</strong> attrition effects, for comparison and treated SYETP<br />
................................................................................................................................ 162<br />
Table 5.4 Difference between treatment group and comparison group before and<br />
after sample reduction............................................................................................ 166<br />
Table 5.5 Summary statistics for attrition effects by source <strong>of</strong> sample loss.......... 170<br />
Table 5.6: Effect <strong>of</strong> selection/response weight on 1984 survey respondents ........ 175<br />
Table 5.7 How <strong>the</strong> observations reduce to 1283 from those who respond to <strong>the</strong> 1986<br />
survey..................................................................................................................... 177<br />
Table 5.5a Summary statistics by source <strong>of</strong> sample loss due to analytical selection<br />
................................................................................................................................ 181<br />
Table 5.8 Probit <strong>of</strong> SYETP participation, showing sample reduction effects ....... 184<br />
Table 5.9: Probit results used for construction <strong>of</strong> attrition weights....................... 194<br />
Table 5.10: Performance <strong>of</strong> attrition weight as measured by unweighted and<br />
weighted pr<strong>of</strong>ile ..................................................................................................... 198<br />
Table 6.1, part A Employment equation from bivariate probit.............................. 201<br />
Table 6.1, Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit............. 204<br />
Table 6.2 Weighting protocol steps ....................................................................... 207<br />
Table 6.3 Weighted probit used to estimate propensity score for propensity score<br />
matching................................................................................................................. 211<br />
Table 6.5 Summary statistics for distribution <strong>of</strong> weighted propensity scores,<br />
comparison group and SYETP............................................................................... 215<br />
Figure 6.6 Kernel density plot <strong>of</strong> attrition weighted propensity scores, before<br />
matching................................................................................................................. 216<br />
Table 6.7 Matching results, single nearest neighbour with replacement, within<br />
caliper, weighting <strong>the</strong> propensity with combined weights for attrition, non-response<br />
and design .............................................................................................................. 217<br />
Figure 6.8 Propensity Distribution after matching, 0.001 Caliper......................... 218<br />
Table 6.9 Employment effects <strong>of</strong> Heckman versus PSM ...................................... 220<br />
Table 7.1, Part A Employment equation from bivariate probit, showing effect <strong>of</strong><br />
standard error estimate........................................................................................... 225<br />
Table 7.1, Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit, showing<br />
effect <strong>of</strong> standard error estimate ............................................................................ 228<br />
Table 7.2 summary <strong>of</strong> changes to exclusion restriction <strong>of</strong> Heckman specification235<br />
Table 7.3 Weighted Probit used to estimate propensity score for propensity score<br />
matching, exclude CEP referrals............................................................................ 238<br />
Table 7.4 Summary <strong>of</strong> distribution <strong>of</strong> propensity, exclude CEP referrals............. 241<br />
Table 7.5 Matching results, Single nearest neighbour with replacement, within<br />
caliper, weighting <strong>the</strong> propensity with combined weights: vary specification...... 242<br />
Table 7.6 Weighted Probit used to estimate propensity score for propensity score<br />
matching, new specification................................................................................... 249<br />
Table 7.7 Summary <strong>of</strong> distribution <strong>of</strong> propensity, new specification.................... 251<br />
Figure 7.8 Histograms <strong>of</strong> <strong>the</strong> propensity scores for <strong>the</strong> new specification............ 251<br />
Figure 7.9 Kernel densities <strong>of</strong> <strong>the</strong> propensities estimated for new specification... 252<br />
Table A2.0a Univariate probit for employment in 1986, as estimated in <strong>the</strong> bivariate<br />
probit replication <strong>of</strong> Richardson (1998)................................................................. 264<br />
xii
Table A2.0b Univariate Probit <strong>of</strong> participation in SYETP, as estimated in <strong>the</strong><br />
bivariate probit replication <strong>of</strong> Richardson (1998).................................................. 267<br />
Table A2.1 Means and bias after matching, one to one nearest neighbour 0.001 and<br />
0.005 propensity score radius ................................................................................ 270<br />
Table A2.1 Continued Means and bias after matching, nearest neighbour one to one<br />
0.01 and 0.05 propensity score radius.................................................................... 273<br />
Table A2.2 Applying Different methods for <strong>the</strong> missing cases in <strong>the</strong> SYETP probit<br />
before sample reduction......................................................................................... 276<br />
Table A2.3 Probit <strong>of</strong> SYETP participation, no missing dummies......................... 279<br />
Table A2.4 Summary statistics for distribution <strong>of</strong> <strong>the</strong> weights constructed for<br />
attrition, non-response and survey design.............................................................. 282<br />
Table A2.5a Univariate probit for employment in 1986, as estimated in <strong>the</strong> bivariate<br />
probit, weighted for attrition.................................................................................. 283<br />
Table A2.5b Univariate probit <strong>of</strong> participation in SYETP, as estimated in <strong>the</strong><br />
bivariate probit, weighted for attrition................................................................... 286<br />
Table A2.6 Part A Employment equation from bivariate probit where age included<br />
in employment model, attrition weights ................................................................ 289<br />
Table A2.6 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit where age<br />
included in employment model, attrition weights.................................................. 291<br />
Table A2.7 Part A Employment equation from bivariate probit where CEP referrals<br />
included in employment model, attrition weights.................................................. 293<br />
Table A2.7 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit where<br />
CEP referrals included in employment model, attrition weights........................... 295<br />
Table A2.8 Part A Employment equation from bivariate probit where CEP referrals<br />
and age included in employment model, attrition weights .................................... 297<br />
Table A2.8 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit where<br />
CEP referrals and age included in employment model, attrition weights.............. 299<br />
xiii
xiv<br />
Glossary<br />
ABS <strong>Australian</strong> Bureau <strong>of</strong> Statistics<br />
ALMP Active Labour Market Program; see also Manpower Program.<br />
ALS <strong>Australian</strong> Longitudinal Survey 1984-1988<br />
Award wage The Award wage in Australia is agreed in Federal negotiations with<br />
unions, and varies by occupation and age.<br />
CEP Community Employment Program, an <strong>Australian</strong> job creation program<br />
where public projects were sponsored based on <strong>the</strong>ir employment<br />
creating capacity.<br />
CES Commonwealth Employment Services, <strong>the</strong> <strong>Australian</strong> public<br />
employment service.<br />
DEET Department <strong>of</strong> Employment, Education and Training Commonwealth<br />
Government <strong>of</strong> Australia, Canberra see also DEIR, DEETYA, DEYA<br />
DEETYA Department <strong>of</strong> Employment, Education, Training and <strong>Youth</strong> Affairs<br />
Commonwealth Government <strong>of</strong> Australia, Canberra see also DEIR,<br />
DEET, DEYA<br />
DEIR Department <strong>of</strong> Employment and Industrial Relations, Commonwealth<br />
Government <strong>of</strong> Australia, Canberra see also DEET, DEETYA, DEYA<br />
DEYA Department <strong>of</strong> Employment and <strong>Youth</strong> Affairs, Commonwealth<br />
Government <strong>of</strong> Australia, Canberra; see also DEIR, DEET, DEETYA<br />
EPUY Education Programme for Unemployed <strong>Youth</strong>; an <strong>Australian</strong> training<br />
program part <strong>of</strong> NEAT; courses aimed to improve basic literacy,<br />
numeracy and social skills.<br />
Manpower A labour market program consisting <strong>of</strong> training or subsidies to assist<br />
program movement into work.<br />
NEAT National Employment and Training System, Australia. An active labour<br />
market program <strong>of</strong> training and wage subsidies – see for example<br />
SYETP, EPUY.<br />
NWO New Work Opportunities provided direct job creation in projects where<br />
placements had work with some training typically in environmental, age<br />
care and community sectors. Existed at <strong>the</strong> same time as Jobstart.<br />
OECD Organisation for Economic Cooperation and Development, Paris.<br />
PSM propensity score matching<br />
SYETP <strong>Special</strong> <strong>Youth</strong> Employment and Training Program, an <strong>Australian</strong><br />
employment or wage subsidy program; part <strong>of</strong> NEAT;
xv<br />
Abstract<br />
The job subsidy <strong>Special</strong> <strong>Youth</strong> Employment and Training Program (SYETP) was<br />
introduced in Australia with <strong>the</strong> aim <strong>of</strong> improving <strong>the</strong> movement into work. In 1984, <strong>the</strong><br />
SYETP was a flat rate subsidy <strong>of</strong> A$75 a week paid to employers for 17 weeks,<br />
equivalent in value to half <strong>the</strong> average teenage wage, and was available to youths aged<br />
15-24 who had been claiming unemployment benefits and not studying full-time for at<br />
least 4 <strong>of</strong> preceding 12 months.<br />
A review <strong>of</strong> <strong>the</strong>oretical literature indicates <strong>the</strong>y can give no pro<strong>of</strong> <strong>of</strong> employment gains<br />
for wage subsidies. The empirical ambiguity <strong>of</strong> employment gains is concluded<br />
unresolved, in both recent overseas and <strong>Australian</strong> literature. A contributing factor is <strong>the</strong><br />
insufficiency <strong>of</strong> <strong>the</strong> evaluation methods. Appraisal <strong>of</strong> <strong>the</strong> micro-evaluation evidence for<br />
SYETP and o<strong>the</strong>r <strong>Australian</strong> wage subsidies is also found to suffer <strong>the</strong>se deficiencies.<br />
The inadequacies <strong>of</strong> past analyses <strong>of</strong> SYETP contribute three <strong>the</strong>mes to address: suitable<br />
modelling <strong>of</strong> selection to account for <strong>the</strong> influence <strong>of</strong> observables or unobservables,<br />
dealing with non-response in <strong>the</strong> observational data, and appropriate control for <strong>the</strong><br />
differences between <strong>the</strong> SYETP and comparison groups.<br />
Past evaluation by Richardson (1998), modelling <strong>the</strong> Heckman selection bivariate probit,<br />
using <strong>the</strong> <strong>Australian</strong> Longitudinal Survey <strong>of</strong> <strong>Youth</strong>s 1984-1987 found a very large<br />
positive employment effect for SYETP participants 26 months after taking part. A key<br />
issue with <strong>the</strong> results is that no account <strong>of</strong> sample attrition was made. Theory indicates<br />
bias to be a potentially serious problem with results. Two evaluation methods are<br />
explored – <strong>the</strong> Heckman selection bivariate probit model, and matching methods, in<br />
particular propensity score matching. Both identify a parameter corresponding to <strong>the</strong><br />
mean effect <strong>of</strong> treatment on <strong>the</strong> treated, which can be used to decide whe<strong>the</strong>r <strong>the</strong><br />
programme leads to employment gains. However each method uses different assumptions<br />
to achieve this. Selection on unobservables is assumed by <strong>the</strong> Heckman bivariate probit,<br />
while selection on observables is assumed by matching methods.<br />
A series <strong>of</strong> empirical studies assesses a number <strong>of</strong> questions – what happens to <strong>the</strong><br />
evaluation outcome if selection is assumed to be based on observables instead <strong>of</strong><br />
unobservables; what is <strong>the</strong> importance <strong>of</strong> sample reduction to <strong>the</strong> evaluation outcome;<br />
and how sensitive is <strong>the</strong> employment impact to variation in modelling. To provide a<br />
foundation for useful comparison Richardson (1998) is first replicated successfully. The<br />
more recently popular propensity score matching method (PSM) is <strong>the</strong>n applied. The<br />
PSM results reduce <strong>the</strong> size and significance <strong>of</strong> <strong>the</strong> employment effect found. The effects<br />
<strong>of</strong> attrition are examined and <strong>the</strong>n accounted for, and <strong>the</strong> impact on evaluation discussed.<br />
The results are found to be smaller and have lower statistical significance. Correctly<br />
accounting for weights is found to be important in applying PSM.<br />
The Heckman and <strong>the</strong> PSM method both make strong but very different assumptions<br />
about <strong>the</strong> selection into SYETP. A comparison <strong>of</strong> <strong>the</strong> employment impacts found under<br />
each method is undertaken, toge<strong>the</strong>r with a discussion <strong>of</strong> <strong>the</strong> most suitable assumptions<br />
for this evaluation. The value <strong>of</strong> replication is validated and advocated. The research<br />
confirms that careful accounting for data and modelling problems is important. External
xvi<br />
validity for <strong>the</strong> robustness <strong>of</strong> employment gains to SYETP are provided by <strong>the</strong> variations<br />
in method and assumptions. The orthodox approach <strong>of</strong> adopting only one potentially<br />
appropriate selection approach and underlying assumption <strong>of</strong> observables or<br />
unobservables is challenged. A sensitivity analysis showing variation to employment<br />
effects for changes in key modelling assumptions can give a confidence interval<br />
accounting for statistical modelling uncertainty. It is concluded that <strong>the</strong> benefits are an<br />
informed overview <strong>of</strong> <strong>the</strong> role <strong>of</strong> <strong>the</strong> assumption to <strong>the</strong> evaluation outcome.<br />
Keywords: labour market program evaluation; non-experimental methods;<br />
panel/longitudinal survey data; attrition; missing data; Heckman bivariate probit;<br />
propensity score matching; replication.<br />
JEL classification: C14, C35, C40, J64, J68, H43.
1<br />
1: <strong>Wage</strong> subsidy <strong>the</strong>ory and evaluation<br />
1.1 Structure <strong>of</strong> this research<br />
The first chapter introduces <strong>the</strong> motivation for <strong>the</strong> substantive research that forms <strong>the</strong><br />
later chapters. Following a general introduction to <strong>the</strong> <strong>the</strong>mes which are developed in <strong>the</strong><br />
research, <strong>the</strong> <strong>the</strong>ory <strong>of</strong> wage subsidies is reviewed. A brief preface <strong>the</strong>n presents <strong>the</strong> key<br />
issues in evaluation <strong>of</strong> social programs such as wage subsidies. Empirical evidence for<br />
wage subsidies is <strong>the</strong>n treated in several parts. Firstly, a selection <strong>of</strong> conclusions from<br />
overseas evaluations for wage subsidies is canvassed in an overview that gives a<br />
background for <strong>the</strong> <strong>Australian</strong> studies that follow in subsequent chapters and form <strong>the</strong><br />
substance <strong>of</strong> this research. Only very recent work is included in this overseas appraisal, as<br />
several very comprehensive reviews have lately been published in this area. The chief<br />
aim is to draw out useful questions that are addressed later within this study. <strong>Australian</strong><br />
wage subsidies are not covered in this chapter, but are <strong>the</strong> focus <strong>of</strong> <strong>the</strong> next chapter.<br />
<strong>Australian</strong> wage subsidies are reviewed in Chapter 2. A historical essay presents an<br />
overview <strong>of</strong> <strong>the</strong> operation and economic environment <strong>of</strong> SYETP. The context is <strong>the</strong>n<br />
useful in assessing <strong>the</strong> processes for participation and employment for evaluation<br />
modelling. The past evaluation evidence relating to SYETP is critically assessed. In an<br />
effort to maintain clarity <strong>of</strong> exposition, <strong>the</strong> <strong>the</strong>ory associated with <strong>the</strong> econometric<br />
techniques applied is not in <strong>the</strong> literature review. Instead, each method is expounded<br />
within each later study where <strong>the</strong>y are applied. As a result, propensity score matching<br />
methods for example, are discussed in study 2 where <strong>the</strong>y are first applied.<br />
As a result <strong>of</strong> <strong>the</strong> critical review <strong>of</strong> past evaluation attempts <strong>of</strong> SYETP, a series <strong>of</strong> studies<br />
present new evidence for SYETP which attempt to account for <strong>the</strong> imperfections<br />
identified in past evidence. This is <strong>the</strong>n new evidence for <strong>the</strong> <strong>Australian</strong> wage subsidy<br />
program, SYETP.
2<br />
The first study presented in Chapter 3 is replication <strong>of</strong> <strong>the</strong> Richardson (1998) evaluation<br />
using Heckman bivariate modelling. Following this, propensity score matching (PSM) is<br />
applied in Chapter 4, with <strong>the</strong> observed differences between <strong>the</strong> SYETP and comparison<br />
group examined initially to establish <strong>the</strong> prospective value for propensity score matching.<br />
A number <strong>of</strong> variations to <strong>the</strong> specification are presented. In Chapter 5, data reduction is<br />
examined in detail and weights developed. The weights are <strong>the</strong>n applied in Chapter 6 to<br />
both <strong>the</strong> Heckman bivariate probit model, and PSM. In <strong>the</strong> final empirical study, in<br />
Chapter 7, sensitivity <strong>of</strong> <strong>the</strong> two modelling approaches are examined. Chapter 8<br />
concludes, drawing toge<strong>the</strong>r <strong>the</strong> <strong>the</strong>mes <strong>of</strong> <strong>the</strong> research.<br />
1.1 General introduction<br />
The central focus <strong>of</strong> this research is <strong>the</strong> <strong>Australian</strong> wage subsidy titled <strong>the</strong> <strong>Special</strong> <strong>Youth</strong><br />
Employment and Training Program (SYETP).<br />
<strong>Wage</strong> subsidies are one <strong>of</strong> <strong>the</strong> two major forms <strong>of</strong> labour market programs, <strong>the</strong> o<strong>the</strong>r<br />
being training. Labour market programs are a form <strong>of</strong> government intervention into <strong>the</strong><br />
labour market with <strong>the</strong> express aim <strong>of</strong> benefiting a target group. Theoretical evidence<br />
suggests that wage subsidies may play a useful role in raising <strong>the</strong> employment prospects<br />
<strong>of</strong> <strong>the</strong> unemployed, especially when targeted at those disadvantaged in <strong>the</strong> labour market.<br />
However as this chapter will show, <strong>the</strong> <strong>the</strong>oretical models do not conclusively prove that<br />
wage subsidies can improve employment. The resolution <strong>of</strong> this <strong>the</strong>oretical ambiguity is<br />
at least one motivating force for empirical evidence that seeks to ga<strong>the</strong>r pro<strong>of</strong> <strong>of</strong> <strong>the</strong><br />
consequences <strong>of</strong> wage subsidies. Empirical evidence is also not conclusive but remains<br />
suggestive that wage subsidies raise employment <strong>of</strong> <strong>the</strong> target group.<br />
<strong>Wage</strong> subsidies 1 are generally held in high regard in real-world policy-making as a useful<br />
tool in labour market and social policy for <strong>the</strong> problem <strong>of</strong> <strong>the</strong> unemployed. Labour<br />
market policy, consisting <strong>of</strong> programs <strong>of</strong> job creation, are promoted by <strong>the</strong> OECD as<br />
‘active’ policies for <strong>the</strong> alleviation <strong>of</strong> unemployment, as opposed to <strong>the</strong> ‘passive’<br />
1 <strong>Wage</strong> subsidies are also sometimes termed employment subsidies, recruitment subsidies, or benefit<br />
transfers (Snower, 1994), and <strong>the</strong>se terms are applied to subsidies only in private sector jobs, not <strong>the</strong> public<br />
sector.
3<br />
programs <strong>of</strong> providing benefits and income support. In weighing <strong>the</strong> positive aspects <strong>of</strong><br />
wage subsidies as a policy for treating <strong>the</strong> problem <strong>of</strong> long-term unemployment amongst<br />
o<strong>the</strong>r program choices, <strong>the</strong> OECD (1988) p58 stated:<br />
“…wage subsidies involve a number <strong>of</strong> advantages…<strong>the</strong> jobs which<br />
materialize...can be deemed to be real jobs in a mainstream labour market<br />
environment. Fur<strong>the</strong>rmore, even though one cannot impose too stringent<br />
conditions on employers to ensure retention beyond <strong>the</strong> period over which <strong>the</strong><br />
subsidy is held, many recruits taken on under such schemes are in fact<br />
retained.”<br />
<strong>Wage</strong> subsidies, or some o<strong>the</strong>r form <strong>of</strong> job creation program, are in recent times <strong>of</strong>ten<br />
considered de rigueur amongst advanced economies, where <strong>the</strong>re is little living memory<br />
<strong>of</strong> <strong>the</strong> times when <strong>the</strong>y were not applied to <strong>the</strong> population.<br />
Theoretically, several models support <strong>the</strong> notion that wage subsidies can alleviate<br />
unemployment. Various authors explore <strong>the</strong> <strong>the</strong>oretical foundations <strong>of</strong> wage subsidies,<br />
for example Kaldor (1936), Hamermesh (1978), Johnson (1980), Jackman and Layard<br />
(1980), Johnson and Layard (1986), Layard et al. (1991), Layard (1997), Calmfors (1994),<br />
Phelps (1994, 1997), Snower (1994), Millard and Mortenson (1997). A lot <strong>of</strong> <strong>the</strong><br />
literature maintains <strong>the</strong> excitement for labour market policy that existed, and was<br />
prevalent in <strong>the</strong> 1970’s and 1980’s<br />
Empirically, early evidence for <strong>the</strong> effectiveness <strong>of</strong> wage subsidies seemed very positive,<br />
but lack <strong>of</strong> sophistication in <strong>the</strong> methods means that many <strong>of</strong> <strong>the</strong>se results are doubtful or<br />
flawed. Taking this into account, <strong>the</strong> empirical information about <strong>the</strong> effectiveness <strong>of</strong><br />
wage subsidies is <strong>the</strong>n quite sketchy. As well, although many wage subsidy programs<br />
have existed and do currently exist in many countries, evaluation <strong>of</strong> <strong>the</strong> employment<br />
consequences <strong>of</strong> <strong>the</strong>se programs is more limited. Many studies that take place <strong>of</strong> <strong>the</strong>se<br />
programs are conducted by internal government agencies and general publication <strong>of</strong><br />
results is not guaranteed. While on casual overview <strong>the</strong> literature may appear to abound<br />
with research about labour market program evaluation, much <strong>of</strong> this stems from <strong>the</strong> US,<br />
is quite old, and as a result uses out-<strong>of</strong>-date methods. Additionally, most programs that<br />
were evaluated in <strong>the</strong> US were not wage subsidies but training programs or a mix where
4<br />
subsidies were not separately evaluated. Generally, <strong>the</strong> outcome <strong>of</strong> an empirical review<br />
for wage subsidies is a paucity <strong>of</strong> results for which <strong>the</strong>re are not mitigating caveats.<br />
In conducting a review <strong>of</strong> <strong>the</strong> empirical wage subsidy literature, o<strong>the</strong>r questions arise.<br />
Each empirical fact requires <strong>the</strong> bigger economic context to understand <strong>the</strong> operation. For<br />
some programs, <strong>the</strong> evaluation outcome exists and <strong>the</strong> description <strong>of</strong> <strong>the</strong> program is<br />
available yet <strong>the</strong> greater picture <strong>of</strong> <strong>the</strong> economy under which <strong>the</strong> outcome was achieved is<br />
not readily available. The general economic context can help resolve which <strong>the</strong>oretical<br />
modelling assumptions are appropriate to <strong>the</strong> outcome. To this end, <strong>Australian</strong> economic<br />
detail for <strong>the</strong> period evaluated has been briefly summarized to augment <strong>the</strong> evaluation<br />
evidence for <strong>the</strong> <strong>Australian</strong> wage subsidy SYETP.<br />
The SYETP program operated between 1976 and 1985. While SYETP operation is<br />
historical, information about <strong>the</strong> employment consequences <strong>of</strong> this program are <strong>of</strong> current<br />
relevance for a number <strong>of</strong> reasons related to economic and econometric questions.<br />
Re-evaluation <strong>of</strong> program data some time later has proved productive in <strong>the</strong> US<br />
econometric literature. The first valuable result <strong>of</strong> reanalysis is to raise debate about <strong>the</strong><br />
methods applied. Potential flaws in past program evaluation can be rectified, and <strong>the</strong><br />
results re-assessed. The outcome can be useful, and <strong>of</strong>ten shed new light on <strong>the</strong> question<br />
<strong>of</strong> how <strong>the</strong> program worked. In an influential study, Smith and Todd (2000), Smith and<br />
Todd (2003) re-examined <strong>the</strong> program evaluation <strong>of</strong> Lalonde (1986), and found that a<br />
substantial amount <strong>of</strong> bias and sensitivity in <strong>the</strong> results was due to <strong>the</strong> analytical choices<br />
made, methods, poor data and covariate restrictions. In re-analysing, methods can <strong>the</strong>n be<br />
updated for techniques ei<strong>the</strong>r not available or not chosen when <strong>the</strong> data were first<br />
analysed. Dehija and Wahba (1998, 1999) reanalysed <strong>the</strong> Lalonde (1986) data using<br />
propensity score matching, but selected only a subset <strong>of</strong> <strong>the</strong> data. Smith (2000) notes that<br />
analysis by Smith and Todd (2000) show <strong>the</strong> very different results obtained by Dehija<br />
and Wahba (1998, 1999) were strongly influenced by choice <strong>of</strong> covariates and analytical<br />
selection <strong>of</strong> <strong>the</strong> data subset.
5<br />
The empirical methods used for <strong>the</strong> evaluation <strong>of</strong> labour market programs do <strong>the</strong>mselves<br />
require assessment. It is clear that <strong>the</strong> choice <strong>of</strong> an estimator to evaluate a program<br />
requires judgments about <strong>the</strong> outcome equation, participation rules and <strong>the</strong>ir relationship<br />
[Heckman, Lalonde and Smith (1999) p2025]. Yet <strong>the</strong> testing <strong>of</strong> evaluation models on<br />
pre-program data, a specification test suggested and usually attributed to Heckman and<br />
Hotz (1989), is shown to be <strong>of</strong> no use due to <strong>the</strong> ‘alignment fallacy’ [Heckman et al.<br />
(1999): 2031]. In addition, evaluators are usually unable to test <strong>the</strong> maintained identifying<br />
assumptions <strong>of</strong> <strong>the</strong>ir empirical models. In light <strong>of</strong> this modelling uncertainty, <strong>the</strong> validity<br />
<strong>of</strong> <strong>the</strong> choice <strong>of</strong> a single method for evaluation is quite low. Benefits can accrue to<br />
varying <strong>the</strong> modelling choice in order to test <strong>the</strong> sensitivity <strong>of</strong> <strong>the</strong> outcome to <strong>the</strong> choice<br />
<strong>of</strong> assumptions. This re-analysis <strong>of</strong> SYETP enables <strong>the</strong> evaluation methods used to be<br />
assessed.<br />
An advantage to future literature reviews exists as a result <strong>of</strong> <strong>the</strong> re-evaluation <strong>of</strong> program<br />
data. For example, a recent review <strong>of</strong> <strong>the</strong> literature by Marx (2001) has already subsumed<br />
<strong>the</strong> results <strong>of</strong> <strong>the</strong> Richardson (1998) analysis <strong>of</strong> <strong>the</strong> <strong>Australian</strong> wage subsidy <strong>Special</strong><br />
<strong>Youth</strong> Employment and Training Program (SYETP). Such reviews try to gain an<br />
overview and collate <strong>the</strong> evaluation findings toge<strong>the</strong>r. Without a re-analysis, <strong>the</strong><br />
published results stand as <strong>the</strong> only ones available, even if <strong>the</strong>y are not ideal.<br />
From a scientific perspective, it is also good to ‘go back to <strong>the</strong> drawing board’. This<br />
enables a new examination <strong>of</strong> <strong>the</strong> material, from a different perspective. A new set <strong>of</strong><br />
questions can be asked from <strong>the</strong> same data. In order to provide a sound basis for this, a<br />
replication <strong>of</strong> <strong>the</strong> previous analysis can allow <strong>the</strong> new analysis to build from where <strong>the</strong><br />
former analysis finished.<br />
The econometric analyses <strong>of</strong> SYETP conducted in this research use <strong>the</strong> <strong>Australian</strong><br />
Longitudinal Survey (ALS) to provide program data for evaluation <strong>of</strong> SYETP. Program<br />
data should be seen as a valuable resource, which can be drawn upon for analysis.<br />
Program data is rarely available. Although it seems many programs have information<br />
ga<strong>the</strong>red, administratively or through surveys, most <strong>of</strong> this does not become publicly
6<br />
available. For example, in Australia, all microeconomic evaluations or programs have<br />
been carried out under <strong>the</strong> auspices <strong>of</strong> government departments. While some results have<br />
been published in departmental papers, <strong>the</strong> data are not usually made available for fur<strong>the</strong>r<br />
analysis. Often <strong>the</strong> burden <strong>of</strong> collecting and storing <strong>the</strong> data and also collating <strong>the</strong><br />
documentation needed for understanding <strong>the</strong> data are deemed too great at <strong>the</strong> time <strong>of</strong> <strong>the</strong><br />
program, or initial interest in <strong>the</strong> program wanes, or resources simply aren’t made<br />
available. Even if good intentions exist, <strong>the</strong> government involved <strong>of</strong>ten seems insecure <strong>of</strong><br />
making <strong>the</strong> data publicly available, or in any case don’t make it available. In this light,<br />
<strong>the</strong> ALS although not perfect as program data, is quite rare and should be fully utilised.<br />
The substantive analysis for this research is performed using <strong>the</strong> ALS survey data. This<br />
survey data contains information about SYETP participants and non-participants,<br />
however <strong>the</strong>se result from ordinary variation ra<strong>the</strong>r than random assignment design. The<br />
US literature has benefited from <strong>the</strong> application <strong>of</strong> experimental design for <strong>the</strong> data<br />
collection, in <strong>the</strong> sense that <strong>the</strong> information relates to a randomly selected group <strong>of</strong><br />
program subjects and control group who don’t receive services. Experimental data are<br />
held forth as <strong>the</strong> ideal data for evaluation, giving many benefits to analysis. The ‘treated’<br />
and <strong>the</strong> ‘control’ or comparison outcomes can <strong>the</strong>n be compared. 2 Experimental data can<br />
be used to develop methods and investigate comparisons <strong>of</strong> bias from <strong>the</strong> ‘true’ outcome<br />
for <strong>the</strong>se methods.<br />
Although this analysis <strong>of</strong> SYETP investigates <strong>the</strong> changes in outcomes resulting from <strong>the</strong><br />
application <strong>of</strong> various methods, because <strong>of</strong> <strong>the</strong> non-experimental data design it cannot<br />
identify a single outcome as <strong>the</strong> true outcome. Instead it provides an exploration <strong>of</strong><br />
several interesting questions about SYETP and it’s effect on employment, firstly: how <strong>the</strong><br />
program outcome changed as a result <strong>of</strong> adjusting for various data flaws, especially<br />
survey non-response and attrition; secondly: how <strong>the</strong> program outcome changed when <strong>the</strong><br />
more recently popular propensity score matching (PSM) methods were applied; thirdly:<br />
2 It has become common in <strong>the</strong> evaluation literature to use ‘control group’ to refer to groups <strong>of</strong> program<br />
non-participants generated by random assignment, and to use ‘comparison group’ to refer to groups <strong>of</strong><br />
program non-participants generated by naturally occurring variation in participation. This usage is applied<br />
throughout this work.
7<br />
how <strong>the</strong> outcomes from <strong>the</strong>se methods changed according to <strong>the</strong> type <strong>of</strong> adjustment used<br />
to correct for data flaws; fourthly: what picture do <strong>the</strong> various outcomes present for <strong>the</strong><br />
working <strong>of</strong> <strong>the</strong> <strong>Australian</strong> SYETP. In overview, it is a sensitivity analysis <strong>of</strong> <strong>the</strong><br />
underlying methodological assumptions.<br />
Finally, it is useful in itself to examine <strong>the</strong> application <strong>of</strong> <strong>the</strong> various non-experimental<br />
methods as in this study. This is because in practice <strong>the</strong>y are frequently used for program<br />
evaluation. Although some experimental data may exist for programs, in general<br />
European and <strong>Australian</strong> evaluations need to rely on non-experimental data, from<br />
administrative or survey sources. The reasons for this situation are likely to be many and<br />
diverse. However it may be influential that non-experimental data are perceived to have<br />
fewer associated political difficulties– and this counteracts <strong>the</strong> issue that <strong>the</strong>re is no<br />
public consensus that it is humane to apply experimental methods to social policy<br />
programs. Instead, it is <strong>of</strong>ten true that <strong>the</strong> social policy is applied to all through law.<br />
Sometimes, a small ‘pilot’ <strong>of</strong> a program is run, for example <strong>the</strong> New Deal Pilots in <strong>the</strong><br />
UK at <strong>the</strong> moment. The administrative and survey data can supply information that might<br />
roughly give a treated and control group, but without <strong>the</strong> random assignment <strong>of</strong><br />
experimental design. Non-experimental data methods <strong>the</strong>n need to be applied to <strong>the</strong> data<br />
try to approximate <strong>the</strong> scientific experiment. However, even experimental data can<br />
require <strong>the</strong> application <strong>of</strong> non-experimental methods. Experiments <strong>of</strong> any type can be<br />
very difficult to implement once people are involved, and data difficulties arising from<br />
loss <strong>of</strong> subject contact and o<strong>the</strong>r problems require adjustment through non-experimental<br />
analysis methods – Heckman, Lalonde and Smith (1999) detail several examples.<br />
1.2 Brief summary <strong>of</strong> <strong>the</strong> <strong>the</strong>ory <strong>of</strong> wage subsidies<br />
This section outlines <strong>the</strong> main rationale for wage subsidy programs, and presents <strong>the</strong><br />
basic conceptual framework within which this is usually analysed.<br />
<strong>Wage</strong> subsidies can be general, i.e. covering any and all workers, or specific, i.e. targeted<br />
to a particular group. Hamermesh (1978) defined 3 types <strong>of</strong> wage subsidy which differ<br />
according to what part <strong>of</strong> <strong>the</strong> firm’s total employment is covered:
8<br />
(1) applying to all <strong>of</strong> a firm’s employees, also called a wage bill subsidy<br />
(Haveman (1982));<br />
(2) applicable only to net changes in <strong>the</strong> firm’s employment, also termed a marginal stock<br />
subsidy (Haveman (1982));<br />
(3) for new hirings by a firm, which Haveman (1982) labels a recruitment subsidy.<br />
The first <strong>of</strong> <strong>the</strong>se is not treated here. Most commonly, wage subsidies are introduced with<br />
<strong>the</strong> aim <strong>of</strong> being <strong>of</strong> a marginal stock subsidy, but usually it is only practicably possible to<br />
ensure a subsidy is a recruitment subsidy. In <strong>the</strong> later analysis <strong>of</strong> SYETP, it will be<br />
noticeable that it is a targeted recruitment subsidy.<br />
Kaldor (1936) gave <strong>the</strong> early introduction <strong>of</strong> <strong>the</strong> use <strong>of</strong> wage subsidies as a remedy for<br />
‘chronic’ unemployment by reducing <strong>the</strong> cost <strong>of</strong> labour. Micro-economic <strong>the</strong>ory provides<br />
a framework for <strong>the</strong> expected effects <strong>of</strong> wage subsidies on <strong>the</strong> level and structure <strong>of</strong><br />
unemployment, with partial equilibrium analysis. In general, <strong>the</strong> labour market is<br />
assumed to be inefficient in operation, and <strong>the</strong> subsidy acts against this inefficient<br />
outcome.<br />
A wage subsidy paid to <strong>the</strong> employer is expected to act by stimulating demand for <strong>the</strong>se<br />
workers, and so raise employment. The wage subsidy is anticipated to act by reducing <strong>the</strong><br />
cost <strong>of</strong> labour relative to o<strong>the</strong>r inputs. In a traditional model, <strong>the</strong> labour and capital are<br />
balanced to pr<strong>of</strong>it maximise, and <strong>the</strong> reduction in <strong>the</strong> cost <strong>of</strong> labour leads to an incentive<br />
for a pr<strong>of</strong>it-maximising firm to substitute labour for capital. There is <strong>the</strong>n <strong>the</strong> possibility<br />
<strong>of</strong> an additional scale effect, where lower costs increase demand via reduced prices that<br />
stimulate an expansion <strong>of</strong> <strong>the</strong> firm’s scale <strong>of</strong> production, and in turn employment levels.<br />
A net employment effect <strong>the</strong>n results from <strong>the</strong> interaction <strong>of</strong> <strong>the</strong> substitution and scale<br />
effects.<br />
Where <strong>the</strong> wage subsidy is targeted at a specific group <strong>of</strong> workers, <strong>the</strong>n labour is split<br />
between subsidised and unsubsidised workers, and <strong>the</strong> substitution effect may work only<br />
so that subsidised workers replace unsubsidised workers. In this case, any net
9<br />
employment effect could be mainly due to <strong>the</strong> scale effect. The size <strong>of</strong> <strong>the</strong> substitution<br />
effects depends on <strong>the</strong> elasticity <strong>of</strong> demand for labour, both for subsidised and<br />
unsubsidised labour. The scale effect size depends on <strong>the</strong> relative cost savings against <strong>the</strong><br />
total production costs. As a result, since <strong>the</strong>se parameters remain unknown, <strong>the</strong> relative<br />
size <strong>of</strong> <strong>the</strong> substitution and scale effects remain unknown, and so <strong>the</strong>ory is ambiguous as<br />
to whe<strong>the</strong>r <strong>the</strong> wage subsidy does act to increase employment.<br />
Johnson (1980) presented <strong>the</strong> case <strong>of</strong> a subsidy to <strong>the</strong> disadvantaged, such as long-term<br />
unemployed, youths without work experience, or <strong>the</strong> low-skilled. A key assumption is<br />
rigid wages facing <strong>the</strong> unemployed, such as minimum wages, toge<strong>the</strong>r with a fairly<br />
inelastic labour supply. The model incorporates unemployment benefit payments, and<br />
only <strong>the</strong> employed skilled workers pay taxes to cover <strong>the</strong> expenditures <strong>of</strong> <strong>the</strong> subsidy<br />
program. Whe<strong>the</strong>r <strong>the</strong> skilled and unskilled workers are complementary or substitutes<br />
influences <strong>the</strong> cost-benefits <strong>of</strong> <strong>the</strong> subsidy, as well as <strong>the</strong> replacement ratio. 3 Jackman and<br />
Layard (1980), and Johnson and Layard (1986) found that if <strong>the</strong> supply elasticity for <strong>the</strong><br />
targeted subsidy group was higher than that for <strong>the</strong> taxed group, <strong>the</strong>n <strong>the</strong>re was a welfare<br />
increase. However, <strong>the</strong> results are not unambiguous as <strong>the</strong> particular parameter values<br />
assumed influence <strong>the</strong> outcome. If skilled workers and unskilled workers are<br />
complements ra<strong>the</strong>r than substitutes, and unskilled labour is subsidised, <strong>the</strong>re is a Pareto<br />
improvement.<br />
The following exposition draws on <strong>the</strong> description <strong>of</strong> <strong>the</strong> wage subsidy model by Dreze<br />
and Sneessens (1997), but is based generally in <strong>the</strong> standard microeconomic framework.<br />
They set up <strong>the</strong> model to allow for <strong>the</strong> labour market to involve a minimum wage, and to<br />
involve market segmentation where low-skill labour operates in a different market facing<br />
different wages. The wage subsidy is introduced for low-skilled labour only, and as such<br />
is targeted. This model is appealing as it more closely approximates <strong>the</strong> situation <strong>of</strong> <strong>the</strong><br />
SYETP operation in Australia, with a minimum wage system and where <strong>the</strong> subsidy was<br />
targeted to unemployed youths and was for low-level occupations only (For a full<br />
description <strong>of</strong> <strong>the</strong> SYETP operation see Chapter 2 <strong>Australian</strong> Review).<br />
3 The ratio <strong>of</strong> unemployment benefit to <strong>the</strong> wage rate.
10<br />
The labour market for low-skill labour is shown in Figure 1.1. Labour demand is LD,<br />
while Labour supply is LS, which in a competitive market might give <strong>the</strong> wage and<br />
employment outcome at A (w*, L*). The minimum wage is set to w + , which gives a<br />
market outcome for <strong>the</strong> demand for labour at B, with employment at L + . The effective<br />
labour supply corresponds to D, with unemployment <strong>the</strong> difference (L # -L + ). By <strong>of</strong>fering a<br />
wage subsidy to employers <strong>of</strong> <strong>the</strong> amount, (w + -w*), <strong>the</strong> firms raise <strong>the</strong>ir labour demand<br />
to LD2. The outcome is <strong>the</strong>n drawn to point E with <strong>the</strong> wage w + equivalent to <strong>the</strong><br />
minimum wage and employment L*. Unemployment is reduced and employment is<br />
raised.<br />
Dreze and Sneessens (1997) elaborate <strong>the</strong> model by describing a ‘ladder effect’ linking<br />
<strong>the</strong> skilled and unskilled labour markets, whereby if higher skilled wages lead to lower<br />
employment at <strong>the</strong> next higher skill level <strong>the</strong>n <strong>the</strong> unemployed from <strong>the</strong> skilled market<br />
can become part <strong>of</strong> <strong>the</strong> labour supply in <strong>the</strong> low-skilled market. The labour demand <strong>the</strong>n<br />
has skill-substitution effects, where unskilled wages are relative to o<strong>the</strong>r wages. <strong>Wage</strong><br />
subsidy programs can <strong>the</strong>n raise <strong>the</strong> employment <strong>of</strong> <strong>the</strong> person, simply by reducing <strong>the</strong><br />
costs to an employer <strong>of</strong> hiring that particular person.<br />
Katz (1996) p7 also discusses <strong>the</strong> partial equilibrium dynamics <strong>of</strong> wage subsidies in this<br />
type <strong>of</strong> model shown in Figure 1.1. In <strong>the</strong> Katz model, <strong>the</strong> labour market is for low wage<br />
ra<strong>the</strong>r than low skill workers. This is also true for <strong>the</strong> Phelps (1994, 1997) subsidy. Katz<br />
additionally considers an infinitely elastic effective supply <strong>of</strong> labour, such as when <strong>the</strong>re<br />
is structural unemployment, which would make <strong>the</strong> LS curve in Figure 1.1 horizontal. In<br />
this case, <strong>the</strong> wage subsidy does not affect wages but expands employment in proportion<br />
to <strong>the</strong> elasticity <strong>of</strong> labour demand for <strong>the</strong> low-skill (or low wage) workers. The effective<br />
wage elasticities <strong>of</strong> both labour demand and supply are key to <strong>the</strong> effects <strong>of</strong> <strong>the</strong> subsidy.<br />
However Katz (1996) also points out that in practice, design issues, administration,<br />
employer take-up, can all affect <strong>the</strong> practical effect <strong>of</strong> <strong>the</strong> subsidy.
11<br />
Real <strong>Wage</strong><br />
Employment<br />
Figure 1.1: Labour market equilibrium with wage subsidy<br />
Dreze and Sneessens (1997) Figure 8.5 p268.<br />
Layard (1991, 1997) links <strong>the</strong> wage subsidy to <strong>the</strong> concept <strong>of</strong> employability within a<br />
model <strong>of</strong> <strong>the</strong> wage-price spiral. This is <strong>of</strong>ten considered a useful general model as it links<br />
<strong>the</strong> concepts <strong>of</strong> <strong>the</strong> NAIRU (Natural rate, or Non-accelerating inflation rate <strong>of</strong><br />
unemployment) and <strong>the</strong> Phillips Curve linking inflation and unemployment, to policy for<br />
reducing unemployment. This gives a basic macroeconomic framework for <strong>the</strong> analysis<br />
<strong>of</strong> <strong>the</strong> effects <strong>of</strong> <strong>the</strong> subsidy on economic variables and processes that influence aggregate<br />
employment and unemployment rates.<br />
In defining <strong>the</strong> notion <strong>of</strong> employability, <strong>the</strong>re is a mixture <strong>of</strong> job search and skills, and<br />
employability is linked to <strong>the</strong> capacity <strong>of</strong> a person to fill a vacancy:<br />
“Near one end is A: a skilled worker who is willing to take any job and<br />
searches every day. Near <strong>the</strong> o<strong>the</strong>r end is B: unskilled worker with an<br />
excessive reservation wage who only samples <strong>the</strong> job market once a month. If<br />
<strong>the</strong>re are vacancies, A will probably be hired soon and B after a longer spell<br />
<strong>of</strong> unemployment.” (Layard (1997): 340)
12<br />
The labour demand in a period <strong>the</strong>n results in<br />
(1) H=f(V , cU) with (f 1 , f 2 >0)<br />
Where H is <strong>the</strong> total number <strong>of</strong> unemployed hired in a given period, V is <strong>the</strong> number <strong>of</strong><br />
vacancies, U <strong>the</strong> number <strong>of</strong> unemployed, f is a function and f 1 , f 2 are <strong>the</strong> first derivatives,<br />
and c is a weight that is <strong>the</strong> average employability <strong>of</strong> all unemployed people, with c i <strong>the</strong><br />
employability <strong>of</strong> an individual. It is assumed that log prices (p) are a mark-up on<br />
expected log wages (w e ), giving a simple formulation <strong>of</strong><br />
(2) p – w e = β 0<br />
Log wages are also assumed to be mark-up on expected log prices (p e ), with <strong>the</strong><br />
mark-up affected by inflationary pressures indicated by (ø) so that<br />
(3) w - p e = y 0 + ø<br />
Substituting prices for expected prices from equation 2, assuming inflation is stable:<br />
(4) w - w e = β 0 + y 0 + ø<br />
In this model a random walk for price inflation means inflation is stable when wages are<br />
equal to expected wages (w = w e ) .<br />
The model is fur<strong>the</strong>red by suggesting that evidence indicates that inflationary pressure<br />
increases with <strong>the</strong> chances <strong>of</strong> finding work for an unemployed person <strong>of</strong> given<br />
employability (Layard (1997): 340). Employability is <strong>the</strong>n introduced via <strong>the</strong> probability<br />
<strong>of</strong> finding work <strong>of</strong> an unemployed person <strong>of</strong> a given employability (H/cU), in place <strong>of</strong> ø :<br />
(5) w - w e = β 0 + y 0 + y 1 (H/cU)<br />
The target real wage increases with <strong>the</strong> rate <strong>the</strong> unemployed find employment. Assuming<br />
unemployment is constant, in equilibrium, so that hires (H) equal separations (sN), where<br />
s is <strong>the</strong> separation rate, and N is employment, <strong>the</strong>n wages equal expected wages and <strong>the</strong><br />
real wage depends on<br />
(6) w = w e = β 0 + y 0 + y 1 (sN/cU)<br />
Thus Layard concludes that for a given inflation path, unemployment is inversely<br />
proportional to average employability <strong>of</strong> <strong>the</strong> unemployed. It is <strong>the</strong>n assumed that <strong>the</strong>re
13<br />
are two types <strong>of</strong> unemployment, short term and long-term unemployment, and long term<br />
unemployment results in people who are less employable, c L < c S and<br />
(7) c S U S + c L U L =constant .<br />
It is assumed that <strong>the</strong> wage subsidy raises <strong>the</strong> employability <strong>of</strong> <strong>the</strong> long-term unemployed,<br />
on whom it is targeted. The wage cost to <strong>the</strong> employer for employing <strong>the</strong> long term<br />
unemployed is <strong>the</strong>n lowered by <strong>the</strong> subsidy, so <strong>the</strong>y employ <strong>the</strong>m. Short-term<br />
unemployment is unaffected, but long-term unemployment U L is lower by <strong>the</strong> same<br />
proportion that <strong>the</strong> average employability c L is raised. The time factor is <strong>the</strong> important<br />
key to this model, as <strong>the</strong> hiring <strong>of</strong> unemployed increases in <strong>the</strong> transition to <strong>the</strong> next<br />
equilibrium due to a rise in <strong>the</strong> hiring rate <strong>of</strong> long-term unemployed. A wage subsidy<br />
provides <strong>the</strong> link to employability, as <strong>the</strong> long term unemployed person who was targeted<br />
moves directly into <strong>the</strong> vacancy, albeit a subsidised employment. The subsidised<br />
employment raises <strong>the</strong> employability <strong>of</strong> <strong>the</strong> long-term unemployed, after which <strong>the</strong>y<br />
ei<strong>the</strong>r retain <strong>the</strong> job after <strong>the</strong> end <strong>of</strong> <strong>the</strong> subsidy or gain ano<strong>the</strong>r job.<br />
Lewis (1963) introduced <strong>the</strong> idea <strong>of</strong> accounting for indirect effects in general equilibrium<br />
analysis <strong>of</strong> <strong>the</strong> macroeconomy. A number <strong>of</strong> indirect effects on wages, regular labour<br />
demand and <strong>the</strong> effective labour force can occur as a result <strong>of</strong> <strong>the</strong> subsidy. These need to<br />
be taken into account in establishing <strong>the</strong> macroeconomic net effect <strong>of</strong> <strong>the</strong> subsidy. These<br />
can reinforce or counteract <strong>the</strong> direct effects <strong>of</strong> <strong>the</strong> subsidy.<br />
The most commonly discussed indirect effects are deadweight losses, substitution effects<br />
and displacement effects. Calmfors (1994) p17 defines <strong>the</strong> substitution effect as “…<strong>the</strong><br />
extent to which jobs created for a certain category <strong>of</strong> workers simply replace jobs for<br />
o<strong>the</strong>r categories, because relative wage costs are changed…”. This is most likely to occur<br />
if <strong>the</strong> subsidy is targeted on a subgroup. De La Dehesa (1997) p355 in providing a<br />
critique to <strong>the</strong> Layard model pointed out that substitution could occur where low wage<br />
jobs are switched into <strong>the</strong> subsidy scheme and away from normal employment, and thus<br />
making <strong>the</strong> subsidy mostly deadweight. Deadweight loss occurs when <strong>the</strong> outcome <strong>of</strong> <strong>the</strong><br />
program is no different from what would have happened without <strong>the</strong> program.<br />
Deadweight here occurs when <strong>the</strong> subsidy is paid for those who would have been
14<br />
employed anyway. The displacement effect <strong>of</strong> a program occurs if it crowds-out regular<br />
employment. Usually, displacement is defined in reference to <strong>the</strong> fall in output for firms<br />
without subsidised workers where <strong>the</strong> subsidy leads to increased output for firms with<br />
subsidised workers.<br />
Millard and Mortenson (1997) develop a modified version <strong>of</strong> job creation and destruction.<br />
There is a two sided matching process where workers search and employers hire, but<br />
<strong>the</strong>re is friction in <strong>the</strong> matching process and wages are determined by bargaining, and<br />
<strong>the</strong>y simulate <strong>the</strong> effects <strong>of</strong> untargeted wages subsidies. In this framework, a small hiring<br />
subsidy is found to reduce equilibrium unemployment only if <strong>the</strong>re are zero redundancy<br />
payments. It is plausible that this is practically true in a low wage or low-skill labour<br />
market, although this is not discussed. However, in order to examine outcomes in <strong>the</strong><br />
model, it is calibrated with estimates <strong>of</strong> key parameters, and <strong>the</strong> effects <strong>of</strong> empirical mismeasurement<br />
are not trivial to <strong>the</strong> model outcomes.<br />
Snower (1994) describes <strong>the</strong> wage subsidy for his model as a ‘benefit-transfer’. This is<br />
due to his condition that <strong>the</strong> subsidy amount is linked to <strong>the</strong> amount <strong>of</strong> unemployment<br />
benefit <strong>the</strong> unemployed person would have received instead <strong>of</strong> <strong>the</strong> employment subsidy.<br />
The subsidy is again argued to overcome labour market inefficiencies, due to benefit<br />
systems, efficiency wages, insider-outsider processes or unions. Snower (1994) devises<br />
<strong>the</strong> subsidy value to depends positively on <strong>the</strong> length <strong>of</strong> unemployment and <strong>the</strong> amount <strong>of</strong><br />
training <strong>the</strong> firm provides in <strong>the</strong> subsidy placement. This design feature is added to<br />
mitigate displacement and deadweight, based on <strong>the</strong> low employment probabilities faced<br />
by long-term unemployed and lower lay-<strong>of</strong>f chance for trained employees in a firm.<br />
The model Snower (1994) puts forward focuses on <strong>the</strong> pr<strong>of</strong>it-maximizing firm. There are<br />
a fixed number <strong>of</strong> firms, producing output using labour input (L), with a revenue function<br />
that is a positive function <strong>of</strong> L, and faces constant firing cost (f):<br />
(8) R= aL – (½)cL 2<br />
Each firm hires entrants L e , and all workers have mortality rate σ set equal to <strong>the</strong> birth<br />
rate, and entrants become incumbents L i after one period <strong>of</strong> employment at rate (1-σ).
15<br />
Real wages are predetermined when firms consider employment. Incumbents get <strong>the</strong><br />
market wage w*. Entrants Le have no market power and no disutility <strong>of</strong> work, so <strong>the</strong><br />
wage received is equal to <strong>the</strong> exogenous unemployment benefit level (b). The subsidy<br />
amount (v) means <strong>the</strong> firms pay <strong>the</strong> entrant (b-v). The difference in marginal revenue and<br />
marginal costs for zero time discount factor is<br />
(9) (a-cL-w*) / σ<br />
The entrants are hired until <strong>the</strong> present value <strong>of</strong> <strong>the</strong> difference is zero<br />
(10) [(a-cL) – b +v] +(1-σ) (a-cL-w*) / σ = 0<br />
The incumbents’ market wage w* is set to <strong>the</strong> present value <strong>of</strong> <strong>the</strong>ir difference in<br />
marginal revenue and marginal costs plus <strong>the</strong> positive constant firing cost (f):<br />
(11) (a-cL-w*) / σ = - f<br />
The equilibrium level <strong>of</strong> employment is <strong>the</strong>n<br />
(12) L* = (1/c)(a-b+ v- (1-σ) f )<br />
The subsidy constraint is <strong>the</strong>n introduced to close <strong>the</strong> model, and it ensures that <strong>the</strong> value<br />
<strong>of</strong> <strong>the</strong> subsidy (v L* e ) is no greater than <strong>the</strong> saving on unemployment benefits, which are<br />
a function <strong>of</strong> <strong>the</strong> change in employment due to <strong>the</strong> subsidy (b∆L ) where ∆L is <strong>the</strong><br />
difference in equilibrium employment without <strong>the</strong> voucher L - and with <strong>the</strong> voucher L*<br />
( b∆L=L*- L - ):<br />
(13) subsidy constraint v L* e = vσL* ≤ b∆L<br />
Figure 1.2 shows <strong>the</strong> Snower (1994) subsidy constraint with <strong>the</strong> curve representing<br />
equilibrium employment (LE), in a model <strong>of</strong> <strong>the</strong> subsidy v and employment L. The<br />
employment gain <strong>of</strong> <strong>the</strong> subsidy (∆L) is maximized at v*, where L - < b/ c σ , that is<br />
where <strong>the</strong> subsidy constraint curve is steeper than equilibrium employment (LE).<br />
The (1994) model is <strong>the</strong>n expanded to incorporate deadweight and displacement effects,<br />
<strong>the</strong> replacement ratio <strong>of</strong> benefits to <strong>the</strong> average wage, and subsidies are restricted to <strong>the</strong><br />
long-term unemployed. There is <strong>the</strong>n a new subsidy constraint that only a fraction <strong>of</strong> <strong>the</strong><br />
unemployment benefits <strong>of</strong> <strong>the</strong> long-term unemployed should be reflected in <strong>the</strong> value <strong>of</strong><br />
<strong>the</strong> subsidy, and that <strong>the</strong> share depend inversely on <strong>the</strong> deadweight and displacement
16<br />
coefficients in <strong>the</strong> model. As before <strong>the</strong> elasticity <strong>of</strong> labour demand and labour supply,<br />
but now also <strong>the</strong> replacement ratio, will affect <strong>the</strong> effectiveness <strong>of</strong> <strong>the</strong> employment gain<br />
due to <strong>the</strong> subsidy.<br />
V<br />
v*<br />
∆L<br />
L -<br />
b/c σ<br />
L<br />
Figure 1.2 Employment effect <strong>of</strong> <strong>the</strong> subsidy,<br />
Snower (1994) Figure 1 p67<br />
All <strong>of</strong> <strong>the</strong>se <strong>the</strong>oretical models provide evidence but do not <strong>of</strong>fer unequivocal pro<strong>of</strong> that<br />
<strong>the</strong> wage subsidy can increase employment, as <strong>the</strong>y retain unknown parameters in <strong>the</strong>ir<br />
final solutions. The ambiguity that <strong>the</strong>se empirically unknown parameters introduce <strong>the</strong>n<br />
leads to <strong>the</strong> conclusion that <strong>the</strong>re is <strong>the</strong>oretical potential for employment gains <strong>of</strong><br />
subsidies. This infers <strong>the</strong> need to empirically assess <strong>the</strong> employment gains from wage<br />
subsidies in practice. It is in this context that <strong>the</strong> importance <strong>of</strong> empirical analysis <strong>of</strong><br />
wage subsidy programs arises.<br />
1.3 Introduction to evaluation <strong>of</strong> programs<br />
In examining <strong>the</strong> evidence for whe<strong>the</strong>r a wage subsidy program improves <strong>the</strong><br />
employment prospects for <strong>the</strong> individual, it is necessary to conduct an evaluation. Several<br />
recent surveys <strong>of</strong> evaluation and evaluation techniques exist in <strong>the</strong> economic literature –
17<br />
Vella (1998), Heckman, Lalonde and Smith (1999), Blundell and Costa Dias (2000),<br />
Smith (2000). The aim here is to provide a brief introduction to <strong>the</strong> issues.<br />
1.3.1 <strong>Evaluation</strong> <strong>of</strong> programs<br />
<strong>Evaluation</strong> <strong>of</strong> <strong>the</strong> program can focus on <strong>the</strong> microeconomic level, or on <strong>the</strong><br />
macroeconomic level. A microeconomic evaluation can look at <strong>the</strong> effect <strong>of</strong> <strong>the</strong> program<br />
upon <strong>the</strong> individual who has participated, such as whe<strong>the</strong>r <strong>the</strong>y are in employment. A<br />
macroeconomic evaluation can examine <strong>the</strong> effect <strong>the</strong> program has on <strong>the</strong> economy as a<br />
whole, such as employment levels in <strong>the</strong> economy. In this <strong>the</strong>sis, only microeconomic<br />
evaluation is attempted. Substitution effects, and o<strong>the</strong>r indirect effects <strong>of</strong> <strong>the</strong> program,<br />
cannot be measured in microeconomic evaluation <strong>of</strong> program participants but only in a<br />
macroeconomic evaluation. However, demonstrating in a microeconomic evaluation<br />
whe<strong>the</strong>r <strong>the</strong> program has improved <strong>the</strong> employment <strong>of</strong> <strong>the</strong> targeted individuals is a<br />
necessary condition for establishing <strong>the</strong> potential for macroeconomic effects.<br />
Bryant (1978) succinctly introduces program evaluation, and <strong>the</strong> underlying concept that<br />
programs should be evaluated on <strong>the</strong> basis <strong>of</strong> participant outcomes. The principal<br />
evaluation question is stated as:<br />
“To what extent are <strong>the</strong> participants better <strong>of</strong>f… as a result <strong>of</strong> participation<br />
than <strong>the</strong>y would have been if <strong>the</strong>re had been no such program.” (Bryant<br />
(1978): 44).<br />
In a microeconomic evaluation, participation is equated to <strong>the</strong> direct outcome <strong>of</strong> <strong>the</strong><br />
program and non-participation equated to <strong>the</strong> outcome <strong>of</strong> <strong>the</strong> ‘no program’ state<br />
(Heckman, Lalonde and Smith (1999): 1880). The most difficult problem facing <strong>the</strong><br />
evaluator is <strong>the</strong> problem <strong>of</strong> attribution. Experimental design can assist in attributing cause<br />
and effect, using randomisation <strong>of</strong> treatment and control status. Bryant (1978) however<br />
cautions that even in evaluations using experimental designs, <strong>the</strong>re can be restrictions on<br />
<strong>the</strong> generality <strong>of</strong> results and gives examples where this can arise due to <strong>the</strong> need to obtain<br />
voluntary cooperation, <strong>the</strong> artificiality <strong>of</strong> <strong>the</strong> treatment in a differently organized system
18<br />
and <strong>the</strong> possible impact <strong>of</strong> repeated interviewing on behaviour. Additionally, <strong>the</strong>re is <strong>the</strong><br />
issue <strong>of</strong> overlapping social programs, which can affect even experimental program<br />
evaluations, and make it difficult to quantify <strong>the</strong> unique contribution <strong>of</strong> each program.<br />
In <strong>the</strong> absence <strong>of</strong> controlled experimentation, a comparison group is selected comprised<br />
<strong>of</strong> persons who are not participants in <strong>the</strong> program, but who have similar characteristics.<br />
The gain in employment <strong>the</strong>y make serves as <strong>the</strong> benchmark against which <strong>the</strong> progress<br />
<strong>of</strong> <strong>the</strong> participants is compared. This is <strong>of</strong>ten termed <strong>the</strong> matched comparison group<br />
design. It is argued that because <strong>the</strong> comparisons are subjected to similar economic<br />
conditions at <strong>the</strong> time <strong>of</strong> <strong>the</strong> treatment, <strong>the</strong>n any greater gain in employment due to<br />
participants than <strong>the</strong> comparisons is due to <strong>the</strong> program. The problem subsequently, is<br />
how to select <strong>the</strong> comparison group, and adjust <strong>the</strong> treatment and comparison groups in<br />
order to make <strong>the</strong>m more comparable and so enable attribution <strong>of</strong> <strong>the</strong> effect to <strong>the</strong><br />
program.<br />
1.3.2 <strong>Evaluation</strong> problem<br />
Heckman, Lalonde and Smith (1999) p1877 use <strong>the</strong> phrase ´<strong>the</strong> evaluation problem´ and<br />
succinctly describe <strong>the</strong> underlying difficulty in evaluating programs as <strong>the</strong> issue <strong>of</strong> reconstructing<br />
counterfactuals. To find <strong>the</strong> effect <strong>of</strong> <strong>the</strong> program on <strong>the</strong> individual, a<br />
comparison must be made between <strong>the</strong> observed employment for that treated individual<br />
and <strong>the</strong> outcome that would have occurred if that person had not participated. Yet what is<br />
observed is <strong>the</strong> outcome for participants and <strong>the</strong> outcome for non-participants. Inference<br />
about <strong>the</strong> impact <strong>of</strong> <strong>the</strong> program involves speculation about outcomes had <strong>the</strong>y not<br />
received treatment. The counterfactual <strong>of</strong> what would have occurred if that person had<br />
not participated cannot be observed. Instead, all methods used for evaluation attempt to<br />
infer an estimate <strong>of</strong> <strong>the</strong> counterfactual from <strong>the</strong> observed data, and <strong>the</strong>n use this to find<br />
<strong>the</strong> program effect. Hujer and Caliendo (2000) p9 describe <strong>the</strong> <strong>the</strong>oretical framework <strong>of</strong><br />
this approach as <strong>the</strong> potential outcome approach, most <strong>of</strong>ten attributed to Rubin (1974).<br />
Early authors associated with this model are Fisher (1951), Roy (1951), and Quandt<br />
(1972) (Heckman, Ichimura and Todd (1997): 608). Identifying assumptions, which can<br />
differ for each method, are introduced to help identify <strong>the</strong> causal effect <strong>of</strong> <strong>the</strong> program.
19<br />
Each individual has two potential outcomes, Y t treatment in <strong>the</strong> program and Y c no<br />
treatment, i.e. <strong>the</strong>y are in <strong>the</strong> comparison group. Actual participation can be denoted with<br />
D, with values D=1 in treatment, or D=0 no treatment. The difference between potential<br />
outcomes defines <strong>the</strong> treatment effect, ∆:<br />
(14) ∆ = Y t - Y c<br />
The observed outcome for each individual is:<br />
(15) Y= DY t + (1-D) Y c<br />
As noted, Y t and Y c can never be observed simultaneously. In equation (14), <strong>the</strong><br />
unobserved component is <strong>the</strong>n termed <strong>the</strong> counterfactual outcome. In examining<br />
individuals <strong>the</strong>re is an underlying assumption <strong>of</strong> Stable Unit Treatment Value (SUTVA)<br />
(Rubin (1986): 961). This infers that <strong>the</strong> treatment effect ∆ on each individual is<br />
independent <strong>of</strong> <strong>the</strong> treatment <strong>of</strong> o<strong>the</strong>r individuals. SUTVA assumes that <strong>the</strong> outcome for<br />
any exposure to <strong>the</strong> treatment is <strong>the</strong> same regardless <strong>of</strong> <strong>the</strong> mechanism <strong>of</strong> assignment to<br />
<strong>the</strong> treatment. Rubin (1986) p961 notes that if <strong>the</strong>re exist unrepresented versions <strong>of</strong> <strong>the</strong><br />
treatment, so that <strong>the</strong> outcome might depend on which version <strong>of</strong> treatment received, <strong>the</strong>n<br />
SUTVA is violated. The population has an average gain from treatment, usually termed<br />
average treatment effect on <strong>the</strong> treated:<br />
(16) E ( ∆ | D = 1 ) = E (Y t | D=1 ) - E (Y c | D=1 )<br />
The second term in equation (16) is <strong>the</strong> unobservable component for <strong>the</strong> treated. To allow<br />
<strong>the</strong> mean outcome <strong>of</strong> non-participants to proxy for <strong>the</strong> counterfactual <strong>of</strong> participants <strong>the</strong><br />
following assumption must be invoked:<br />
(17) E (Y c | D=1 ) = E (Y c | D=0 )
20<br />
This identifying assumption is valid if <strong>the</strong>re is randomised assignment <strong>of</strong> individuals into<br />
treatment and controls. Randomised assignment can ensure that <strong>the</strong> potential outcomes<br />
are independent ( ╨ ) <strong>of</strong> <strong>the</strong> assignment to <strong>the</strong> treatment:<br />
(18) Y t , Y c ╨ D<br />
With conditional independence, or ignorable assignment (Rubin (1974)), <strong>the</strong>n it is true<br />
that:<br />
(19) E (Y c | D=1 ) = E (Y c | D=0 ) and E (Y t | D=1 ) = E (Y t | D=0 )<br />
And so <strong>the</strong> estimate <strong>of</strong> <strong>the</strong> causal effect <strong>of</strong> treatment can be estimated without bias while<br />
using <strong>the</strong> mean outcome <strong>of</strong> non-participants to proxy for <strong>the</strong> counterfactual <strong>of</strong><br />
participants.<br />
1.3.3 Selection<br />
The economic model <strong>of</strong> <strong>the</strong> effect <strong>of</strong> <strong>the</strong> program on employment can be described in two<br />
parts, participation in <strong>the</strong> program and <strong>the</strong> process <strong>of</strong> job entry. In summarizing <strong>the</strong><br />
process <strong>of</strong> <strong>the</strong>se two parts, both observable and unobservable influences may play a role.<br />
The main evaluation interest is whe<strong>the</strong>r <strong>the</strong> program influences job entry. Heckman and<br />
Hotz (1989) pointed out that selection might occur on observables or unobservables. If<br />
some observable or unobservable component that leads to program participation also has<br />
a role in job entry <strong>the</strong>n selection bias can arise. In this case, <strong>the</strong> assumption in equation<br />
(19) does not hold, and instead<br />
(20) E (Y c | D=1 ) ≠ E (Y c | D=0 ).<br />
As a result, <strong>the</strong> use <strong>of</strong> non-participants as a comparison group in an evaluation <strong>of</strong> <strong>the</strong><br />
program is subject to selection bias.<br />
An evaluation <strong>of</strong> <strong>the</strong> program effect must account for selection. A notable exception<br />
might be where an experiment could assign eligible people randomly to program<br />
treatment or no treatment and also measure all observable influences upon outcomes. In
21<br />
this case, if <strong>the</strong>re was no data loss such as due to loss <strong>of</strong> contact or o<strong>the</strong>r confounding<br />
problems, <strong>the</strong>n <strong>the</strong> experiment might avoid <strong>the</strong> selection problem. Random assignment<br />
creates a control group that comprises individuals, which within sampling variation<br />
should have identical distributions <strong>of</strong> observable and unobservable characteristics to<br />
those in <strong>the</strong> treatment group. Random participation can thus defeat <strong>the</strong> potential for<br />
selection bias. In <strong>the</strong> absence <strong>of</strong> this, non-experimental methods for evaluation should be<br />
used. However, data problems <strong>of</strong> non-response, or treatment differing from assigned<br />
treatment, can re-introduce selection problems into even experimental data. For example<br />
Heckman et al. (1999) pp1905 -1914 discuss <strong>the</strong> data problems <strong>of</strong> economic program<br />
experimental data, such as sample attrition. Fay (1996) Table 4a p47 also lists some<br />
problems <strong>of</strong> experiments as randomisation bias, sample contamination, treatment<br />
contamination, site self-selection, substitution bias, crossover bias, and program entry<br />
effects.<br />
Heckman et al. (1999) examine in detail <strong>the</strong> form <strong>of</strong> evaluation counterfactual estimated<br />
using various evaluation methods. Both <strong>the</strong> Heckman selection model and matching<br />
methods produce <strong>the</strong> parameter corresponding to <strong>the</strong> mean effect <strong>of</strong> treatment on <strong>the</strong><br />
treated, which Heckman et al. (1999) derive fully. Hujer and Caliendo (2000) p10 note<br />
this parameter answers <strong>the</strong> question “what is <strong>the</strong> expected outcome gain to individuals<br />
who received treatment, to <strong>the</strong> hypo<strong>the</strong>tical situation <strong>the</strong>y had not received it?”. Heckman<br />
et al. (1999) and Heckman et al. (1997, 1998) point out that this question focuses directly<br />
on actual participants, and can help decide whe<strong>the</strong>r <strong>the</strong> program is a success. However,<br />
<strong>the</strong> identifying assumptions used to estimate <strong>the</strong> unobserved counterfactual term from <strong>the</strong><br />
observed non-participant outcomes, that are required for <strong>the</strong> Heckman selection model,<br />
differ from those required for matching to produce a valid estimation <strong>of</strong> <strong>the</strong> mean effect<br />
<strong>of</strong> treatment on <strong>the</strong> treated. Each <strong>of</strong> <strong>the</strong>se modelling methods and <strong>the</strong>ir assumptions is<br />
treated in detail in <strong>the</strong> later chapters <strong>of</strong> this research.
22<br />
1.3.4 Observables and unobservables<br />
Friedlander et al. (1997) express succinctly <strong>the</strong> difference between selection on<br />
observables and selection on unobservables. The behaviour <strong>of</strong> <strong>the</strong> participants is<br />
expressed in <strong>the</strong> econometric model:<br />
(21) Y it = c t X i + b t D io + u it t>0 employment<br />
(22) D io = a 0 Z i + e io participation<br />
Again, Y it is <strong>the</strong> outcome, such as employment, for <strong>the</strong> ith person in period t , where t=0<br />
is <strong>the</strong> period where <strong>the</strong> treatment occurs. The X i and Zi are sets <strong>of</strong> exogenous<br />
characteristics for <strong>the</strong> ith individual, that can be overlapping and are measured pre –<br />
program. The importance <strong>of</strong> measuring pre-program is to avoid endogeneity, where <strong>the</strong><br />
characteristics may change after program entry due to <strong>the</strong> program, <strong>the</strong>se would confound<br />
measurement <strong>of</strong> <strong>the</strong> program effect. D io is <strong>the</strong> program participation binary variable, D io<br />
=1 if participant and D io =0 if not (<strong>the</strong> comparison group); u it, e io are error terms. The<br />
mean effect <strong>of</strong> <strong>the</strong> program in period t after participation is <strong>the</strong> coefficient b t . If <strong>the</strong>re is<br />
no correlation between <strong>the</strong> participation and <strong>the</strong> error in <strong>the</strong> employment equation<br />
E(D io , u it ) =0, <strong>the</strong>n an unbiased estimate <strong>of</strong> <strong>the</strong> coefficient b t is obtained simply from <strong>the</strong><br />
employment equation (21).<br />
Correlations between <strong>the</strong> participation and <strong>the</strong> employment error E(D io , u it ) ≠ 0, can<br />
arise from selection due to observables or unobservables (Friedlander et al. (1997),<br />
Heckman and Robb (1985), Heckman and Hotz (1989)). The correlation between <strong>the</strong><br />
participation and <strong>the</strong> employment error E(D io , u it ) ≠0 can arise through ei<strong>the</strong>r Z i or e io, in<br />
<strong>the</strong> equation (22) representing participation . If <strong>the</strong> error terms for employment and<br />
participation are uncorrelated E(u it, e io ) = 0 , but <strong>the</strong>re is correlation between <strong>the</strong> error <strong>of</strong><br />
<strong>the</strong> employment equation and <strong>the</strong> characteristics affecting participation E( Z i ,u it ) ≠ 0 <strong>the</strong>n<br />
selection is on observables. Conversely, when <strong>the</strong>re is selection on unobservables, <strong>the</strong><br />
error terms for employment and participation are correlated E(u it, e io ) ≠ 0 but <strong>the</strong>re is no
23<br />
correlation for <strong>the</strong> error <strong>of</strong> <strong>the</strong> employment equation and <strong>the</strong> characteristics affecting<br />
participation E( Z i ,u it ) = 0.<br />
The Heckman selection model assumes selection on unobservables and attempts to<br />
control for that, while propensity score matching assumes selection on observables. In <strong>the</strong><br />
later chapters, <strong>the</strong>se modelling methods and <strong>the</strong>ir assumptions are presented and applied.<br />
1.4 Brief comment on recent overseas evidence <strong>of</strong> wage subsidy evaluations<br />
This section reviews some existing information for wage subsidy programs in o<strong>the</strong>r<br />
countries. This appraisal is limited to recent evidence since <strong>the</strong> 1990’s, as older material<br />
is well reviewed elsewhere. Only micro-economic evaluation evidence is addressed. The<br />
main aim is to consolidate a general perspective <strong>of</strong> <strong>the</strong> wage subsidy evidence found<br />
overseas. The exposition is <strong>the</strong>n not comprehensive with regard to <strong>the</strong> details <strong>of</strong> <strong>the</strong><br />
programs or a critical review <strong>of</strong> <strong>the</strong> evaluation evidence. Instead, key <strong>the</strong>mes are<br />
identified.<br />
A number <strong>of</strong> published reviews provide recent overviews <strong>of</strong> wage subsidies and o<strong>the</strong>r<br />
programs. To avoid repetition, <strong>the</strong>ir conclusions are summarized here.<br />
Katz (1996) pp31-33 and Table 4 found that some targeted wage subsidies gave positive<br />
gains. It was found that <strong>the</strong> US Targeted Jobs Tax Credit and YIEPP private sector wage<br />
subsidy, might have modestly raised disadvantaged youth employment rates. However it<br />
was concluded that <strong>the</strong>re was little satisfactory formal evidence <strong>of</strong> <strong>the</strong> impacts <strong>of</strong> wage<br />
subsidies, and that many studies were not compelling in <strong>the</strong>ir evidence due to flawed<br />
evaluation methods or poor data.<br />
Friedlander, Greenberg and Robins (1997) concentrate on US programs targeted on <strong>the</strong><br />
disadvantaged, including wage subsidies. However <strong>the</strong> focus <strong>of</strong> <strong>the</strong> literature review was<br />
to get an overview <strong>of</strong> whe<strong>the</strong>r, as a whole, any social programs had gains for any groups<br />
<strong>of</strong> men, women and youths. The main recent wage subsidy program in <strong>the</strong> US has been<br />
<strong>the</strong> JTPA-II-A subsidised on-<strong>the</strong>-job training, for which a random assignment experiment
24<br />
was carried out. In this respect, <strong>the</strong>y found that women appeared to consistently have had<br />
gains, while <strong>the</strong>re were no youth gains and for men <strong>the</strong>re was great variation and<br />
uncertainty <strong>of</strong> <strong>the</strong> presence <strong>of</strong> gains.<br />
Heckman, Lalonde and Smith (1999) noted that <strong>the</strong> major evaluations in <strong>the</strong> US have<br />
<strong>of</strong>ten focused on earnings ra<strong>the</strong>r than employment gains from programs. They concluded<br />
that <strong>the</strong> evidence from both <strong>the</strong> North American and European studies indicated only a<br />
modest gain in <strong>the</strong> probability <strong>of</strong> employment. However, it was found that many nonexperimental<br />
evaluations were lacking in <strong>the</strong>ir exploration <strong>of</strong> methodological issues, such<br />
as <strong>the</strong> appropriate choice <strong>of</strong> evaluation method. For both Friedlander et al. (1997) and<br />
Heckman et al. (1999) <strong>the</strong> overview encompassed an immense variety <strong>of</strong> programs, time<br />
periods and methods. As a result <strong>the</strong> conclusions are not specific, but instead <strong>the</strong>y provide<br />
an impression <strong>of</strong> <strong>the</strong> uncertainty that still remains as to <strong>the</strong> empirical effectiveness <strong>of</strong><br />
programs in achieving gains.<br />
Fay (1996) conducted a review <strong>of</strong> OECD active labour market program evidence,<br />
including wage subsidies to <strong>the</strong> private sector. Subsidies were concluded to be useful for<br />
long-term unemployed or women re-entrants, with this drawn mostly from <strong>the</strong> US JTPA-<br />
II-A experimental evidence. It was found impossible to harmonise <strong>the</strong> results for <strong>the</strong><br />
different outcome measures and means <strong>of</strong> achieving <strong>the</strong> evaluations and it was<br />
commented that robust evaluation results were scarce. In particular <strong>the</strong>y recommended it<br />
was important to make non-experimental evaluations more rigorous, and consideration<br />
made to testing alternative model specifications, toge<strong>the</strong>r with greater data collection<br />
(Fay (1996): 33).<br />
Marx (2001) reviews targeted employment/wage subsidies, for OECD countries mostly<br />
in Europe but also Australia. The conclusion is that little evidence exists for a beneficial<br />
effect on employment prospects, with a variety <strong>of</strong> negative impacts, limited impact size,<br />
and some few positive results.
25<br />
Table 1.3 collects toge<strong>the</strong>r some recent European employment effects found in evaluation<br />
<strong>of</strong> wage subsidies. The direction and significance <strong>of</strong> <strong>the</strong> estimated effect <strong>of</strong> <strong>the</strong> program<br />
is presented only, as <strong>the</strong> scope <strong>of</strong> this limited overview precludes reference to <strong>the</strong> details<br />
<strong>of</strong> <strong>the</strong> various programs, <strong>the</strong>ir targeting, and <strong>the</strong> empirical methods employed. As for <strong>the</strong><br />
reviews already summarized, <strong>the</strong>re is variation in <strong>the</strong> effects even when <strong>the</strong> size <strong>of</strong> <strong>the</strong><br />
effect is not considered. Some are not statistically significant, some are negative and<br />
o<strong>the</strong>rs are positive. Once more, <strong>the</strong> uncertainty remains <strong>of</strong> whe<strong>the</strong>r wage subsidies<br />
provide employment gains empirically.<br />
However, it is unlikely that any overview or literature review can account for <strong>the</strong><br />
extraordinary amount <strong>of</strong> variation that could be <strong>the</strong> source <strong>of</strong> <strong>the</strong>se varying evaluation<br />
results. A meta-analysis might generate some conclusions. Meta-analysis, in particular<br />
meta-regression analysis, has developed in <strong>the</strong> past two decades into one <strong>of</strong> <strong>the</strong> most<br />
important instruments for <strong>the</strong> syn<strong>the</strong>sis <strong>of</strong> quantified evaluation findings. In metaanalysis,<br />
<strong>the</strong> dependent variable is a standardized measure <strong>of</strong> effect estimated in each <strong>of</strong> a<br />
set <strong>of</strong> studies that is representative <strong>of</strong> a field <strong>of</strong> inquiry. This dependent variable may <strong>the</strong>n<br />
be used in two very different ways. One approach, which is dominant in medicine and<br />
psychology, focuses chiefly upon <strong>the</strong> derivation <strong>of</strong> an appropriately weighted average<br />
effect from <strong>the</strong> set <strong>of</strong> studies. Meta-analysis, in this approach, is employed to extract a<br />
superior estimate <strong>of</strong> what is regarded as <strong>the</strong> true effect. The o<strong>the</strong>r approach, which is<br />
more appropriate to economic evaluations, is primarily concerned with explaining <strong>the</strong><br />
nature and sources <strong>of</strong> <strong>the</strong> between-studies variation in <strong>the</strong> estimates. This variation is<br />
regarded as representing genuine heterogeneity in impacts. This would be <strong>the</strong> more<br />
appropriate analysis for generating conclusions in this context. Examples are Stanley and<br />
Jarrell (1998), who examined estimates <strong>of</strong> <strong>the</strong> gender pay gap in <strong>the</strong> USA, and<br />
Ashenfelter et al. (1999) who examined rates <strong>of</strong> return to schooling from several<br />
industrialised countries. Although a useful proposal for future research, a meta-analysis is<br />
however not within <strong>the</strong> range <strong>of</strong> this research.<br />
However, as <strong>the</strong> economic environment within a country can set up a unique context, <strong>the</strong><br />
<strong>the</strong>ory <strong>of</strong> wage subsidies does not predict that wage subsidies will work in all <strong>the</strong>se
26<br />
environments. As <strong>the</strong> former section showed, <strong>the</strong> ambiguity in <strong>the</strong> <strong>the</strong>ory arises from<br />
whe<strong>the</strong>r within a given economic context, labour demand and supply elasticities,<br />
reservation wages and subsidy amount, a gain to employment eventuates. In this sense,<br />
every program needs to be assessed, and reassessed over time, in order to discover under<br />
what environment a gain can be found. This takes into account <strong>the</strong> dynamic aspects <strong>of</strong><br />
programs, as microeconomic evaluation can hinder <strong>the</strong> appreciation <strong>of</strong> this characteristic.<br />
This is because <strong>the</strong> static evaluation process loses <strong>the</strong> sense <strong>of</strong> <strong>the</strong> variation in conditions<br />
that is naturally part <strong>of</strong> <strong>the</strong> macroeconomic setting.<br />
In conclusion, two strong <strong>the</strong>mes are identified. Firstly, <strong>the</strong> non-experimental evidence is<br />
not sufficiently thorough in methodology. The key suggestion seems to be that <strong>the</strong>re is<br />
insufficient testing <strong>of</strong> alternative methods, assumptions and model specifications.<br />
Secondly, <strong>the</strong>re remains empirical ambiguity as to whe<strong>the</strong>r wage subsidies provide<br />
employment gains. Some <strong>of</strong> this uncertainty stems from <strong>the</strong> inadequate non-experimental<br />
evaluation evidence, so that <strong>the</strong> employment outcomes attributed to <strong>the</strong> wage subsidy<br />
cannot be deemed well established. As well, not all evaluations have found positive gains<br />
to employment following wage subsidies, although this to some extent is again due to <strong>the</strong><br />
caveats pertinent to <strong>the</strong> quality <strong>of</strong> <strong>the</strong> non-experimental evaluation methods.<br />
In <strong>the</strong> following analysis, <strong>the</strong>se <strong>the</strong>mes are addressed via re-analysis <strong>of</strong> <strong>the</strong> <strong>Australian</strong><br />
wage subsidy <strong>Special</strong> <strong>Youth</strong> Employment and Training Program (SYETP). The chief<br />
goal is to submit a fur<strong>the</strong>r contribution as to whe<strong>the</strong>r this wage subsidy gave employment<br />
gains. Alongside this, by exploring alternative specifications and methods, <strong>the</strong> subsidiary<br />
aim is to more rigorously extend <strong>the</strong> non-experimental evaluation <strong>of</strong> SYETP.
27<br />
Table 1.3 Brief overview <strong>of</strong> recent European wage subsidy evidence considered<br />
Author Country Program<br />
period/data<br />
Method<br />
<strong>Wage</strong> subsidy employment<br />
outcome<br />
Bonjour et al.<br />
(2001)<br />
Britain 1998 Propensity score<br />
matching<br />
+ wage subsidy (Employment<br />
option) relative to o<strong>the</strong>r<br />
Bonnal et al.<br />
(1994)<br />
France 1986-88 Parametric<br />
Econometric<br />
modelling<br />
Breen (1991) Ireland 1982-88 Parametric<br />
Econometric<br />
modelling<br />
Cockx et al.<br />
(1996)<br />
Eichler and<br />
Lechner (1998)<br />
Gerfin and<br />
Lechner (2000)<br />
Harkman and<br />
Johansen<br />
(2000)<br />
Lalive et al.<br />
(2000)<br />
Belgium 1991-93 Parametric<br />
Econometric<br />
modelling<br />
East Germany<br />
(Sachsen-<br />
Anhalt)<br />
1991-97 Propensity score<br />
matching<br />
Switzerland 1997-98 Propensity score<br />
matching<br />
Sweden 1996 Parametric<br />
Econometric<br />
modelling<br />
Switzerland 1997-99 Parametric<br />
Econometric<br />
modelling<br />
Larsson (2000) Sweden 1991-97 Propensity score<br />
matching<br />
O’Connell and Ireland 1992 Parametric<br />
Mcginnity<br />
Econometric<br />
(1997)<br />
modelling<br />
Rosholm (1998) Denmark 1983-90 Parametric<br />
Econometric<br />
modelling<br />
options in NDYP.<br />
Used in most programs with<br />
training, not separately<br />
evaluated<br />
Used in temporary<br />
employment programs, not<br />
separately evaluated<br />
+ if combined with training<br />
+ emp (neg unemp) at 1yr<br />
+ emp<br />
+ / n.s. (emp) at 1yr<br />
+ females emp. transition<br />
n.s. males<br />
+ YP v. LMT at 1yr (emp and<br />
earnings)<br />
+ emp<br />
+ earnings/wage<br />
n.s priv.sector males 16-24y,<br />
NEG females 16-24y<br />
Sianesi (2001, Sweden 1994-99 Propensity score + emp<br />
2002)<br />
matching<br />
Notes: n.s.=not statistically significant;+ = positive effect or coefficient, neg=negative effect or coefficient<br />
emp=employment;
28<br />
2: <strong>Australian</strong> literature review<br />
This analysis focuses on wage subsidies, and in particular <strong>the</strong> <strong>Australian</strong> SYETP program.<br />
Although <strong>the</strong>re have been a few past reviews, <strong>the</strong>se have proceeded with <strong>the</strong> general aim<br />
<strong>of</strong> summarising labour market programs in Australia from various historical economic or<br />
socio-political perspectives [Edwards (1987), Ross (1988), Stretton and Chapman (1990),<br />
Webster (1998) Harris (2001)], including governmental reviews [OECD (2001), BLMR<br />
(June 1984)]. Some <strong>of</strong> <strong>the</strong>se have also tried to assess <strong>the</strong> effectiveness <strong>of</strong> <strong>the</strong> programs by<br />
canvassing <strong>the</strong> evaluation evidence to date. The focus here is on extracting information<br />
about wage subsidies only and in particular SYETP.<br />
The chief intention is to summarise <strong>the</strong> results and critically assess <strong>the</strong> micro-evaluation<br />
evidence. Some effort is made to retain detail <strong>of</strong> <strong>the</strong> data, methods and variables used to<br />
achieve <strong>the</strong> evaluation results, as <strong>the</strong>se can have some bearing on <strong>the</strong> outcome and can<br />
help place our results amongst <strong>the</strong> evidence to date. The empirical literature covered is<br />
limited to microeconomic evaluations. These evaluate programs by looking at <strong>the</strong> effect<br />
on <strong>the</strong> participant and can establish whe<strong>the</strong>r <strong>the</strong> program changes <strong>the</strong> outcome for <strong>the</strong><br />
selected individuals after <strong>the</strong> program has ended, mainly examining employment. An<br />
important aspect <strong>of</strong> evaluations <strong>of</strong> this type is that <strong>the</strong>y try to establish whe<strong>the</strong>r <strong>the</strong><br />
participant would have a job anyway, and try to assess this using information about what<br />
happens after <strong>the</strong> program has ended. The brevity <strong>of</strong> <strong>the</strong> evaluation review reflects <strong>the</strong><br />
paucity <strong>of</strong> formal evaluation in <strong>the</strong> <strong>Australian</strong> literature.<br />
Empirical evidence for <strong>Australian</strong> wage subsidies is treated in several parts. Only recent<br />
work completed during or after 1980 is included, as this corresponds to <strong>the</strong> period <strong>of</strong><br />
SYETP analysed later, but also because this is when published evaluations first occurred<br />
in Australia. <strong>Australian</strong> wage subsidies o<strong>the</strong>r than SYETP are firstly briefly treated.<br />
Because <strong>of</strong> <strong>the</strong> broad extent <strong>of</strong> SYETP, <strong>the</strong>re were few o<strong>the</strong>r subsidies and this section is
29<br />
necessarily brief. Then, <strong>the</strong> evidence for SYETP is reviewed. Finally, <strong>the</strong> past evaluation<br />
evidence relating to SYETP is critically appraised.<br />
As a result <strong>of</strong> <strong>the</strong> critical review <strong>of</strong> past evaluation attempts <strong>of</strong> SYETP, <strong>the</strong> later chapters<br />
present new evidence for SYETP which attempt to account for <strong>the</strong> imperfections<br />
identified in past evidence. This is <strong>the</strong>n new evidence for <strong>the</strong> <strong>Australian</strong> wage subsidy<br />
program, SYETP.<br />
In an effort to maintain clarity <strong>of</strong> exposition, <strong>the</strong> <strong>the</strong>ory associated with <strong>the</strong> econometric<br />
techniques applied is not in <strong>the</strong> literature review. Instead, each method is expounded<br />
within each later study where <strong>the</strong>y are applied. As a result, propensity score matching<br />
methods for example, are discussed in study 2 where <strong>the</strong>y are first applied.
30<br />
2.1 Review <strong>of</strong> <strong>Australian</strong> wage subsidy evaluation evidence<br />
The brief overseas review in <strong>the</strong> first chapter showed relatively few evaluations <strong>of</strong> wage<br />
subsidies exist, and empirical evidence for wage subsidies is not well founded. As this<br />
section will show, wage subsidy programs in Australia also have little publicly available<br />
evaluation evidence. Microeconomic evaluation is selected, using individual data,<br />
addressing <strong>the</strong> issue <strong>of</strong> whe<strong>the</strong>r <strong>the</strong> subsidy leads to better employment experience for<br />
participants after <strong>the</strong> program ends. The evidence is limited to published items that can be<br />
accessed by <strong>the</strong> public.<br />
Only recent evidence is compiled, corresponding approximately to <strong>the</strong> period starting<br />
with <strong>the</strong> 1980’s and onward, since this is <strong>the</strong> period <strong>of</strong> <strong>the</strong> later analysis. However it<br />
becomes evident in reviewing <strong>the</strong> literature that in fact most programs only started within<br />
<strong>the</strong> 1970’s, and publicly available evaluation work occurred mostly in <strong>the</strong> mid-1980’s<br />
and l990’s. Paterson (1982) in discussing evaluation activities in Australia pointed out<br />
that “…In Australia government involvement in <strong>the</strong> area <strong>of</strong> manpower programs is a<br />
relatively new phenomenon. Experience with manpower program evaluation is <strong>the</strong>refore<br />
limited.” (Paterson (1982): 1). The timing <strong>of</strong> this statement marked <strong>the</strong> general beginning<br />
<strong>of</strong> <strong>Australian</strong> evaluation activities and publications.<br />
<strong>Wage</strong> subsidies form only a part <strong>of</strong> <strong>the</strong> array <strong>of</strong> labour market programs available, and so<br />
evaluation <strong>of</strong> <strong>the</strong>se in turn forms only a small part <strong>of</strong> <strong>the</strong> existing <strong>Australian</strong> evaluation<br />
literature available. <strong>Evaluation</strong> evidence for o<strong>the</strong>r program types such as training for<br />
Australia can be found generally reviewed in Webster (1998, 1997a).<br />
Although subsidised apprenticeships were available, for example CRAFT<br />
(Commonwealth Rebate for Apprenticeship Full-time Training), <strong>the</strong>se are not dealt with<br />
as <strong>the</strong>y are deemed to be mostly a training subsidy. 4 Also not covered are programs with<br />
4 In support <strong>of</strong> this, in references such as Kesteven (1987) and Hoy (1983) <strong>the</strong>y are categorized as<br />
education programs. The design did allow for a small amount <strong>of</strong> on-<strong>the</strong>-job training. For example Merrilees<br />
(1984) refers to wage subsidies in CRAFT for this reason, however <strong>the</strong> grand focus was apprenticeship<br />
training.
31<br />
limited applications, such as for special needs groups only, for example <strong>the</strong> disabled or<br />
Aboriginal and Torres Strait Islanders. This is because <strong>the</strong> limited application means such<br />
programs were not reasonably comparable i.e. a wage subsidy targeted to a reasonable<br />
extent <strong>of</strong> <strong>the</strong> population as <strong>the</strong> SYETP was. Because so few wage subsidy evaluations<br />
exist, some information about <strong>the</strong> existence <strong>of</strong> o<strong>the</strong>r wage subsidy programs in Australia<br />
is also provided even if no evaluation evidence could be found for <strong>the</strong>m.<br />
SYETP is covered separately in <strong>the</strong> next section. However, SYETP was <strong>the</strong> first<br />
widespread wage subsidy program applied in Australia and so also <strong>the</strong> first wage subsidy<br />
program in Australia for which evaluation attempts were made. SYETP was not <strong>the</strong> only<br />
employment based program <strong>of</strong> ‘work experience’ available that was subsidy based, in<br />
1983. O<strong>the</strong>r, usually brief programmes, have also operated, sometimes alongside SYETP.<br />
SYETP was however by far <strong>the</strong> largest subsidy, making up 69.2 cent <strong>of</strong> all new<br />
programme placements (flow only) in 1980/81 (BLMR (1983) p16, table 3.1, final<br />
column). More recently, o<strong>the</strong>r subsidy programmes have been evaluated and <strong>the</strong> details<br />
<strong>of</strong> <strong>the</strong>se are now presented.<br />
2.1.1 Adult <strong>Wage</strong> <strong>Subsidy</strong> Scheme<br />
The list <strong>of</strong> labour market programmes compiled by Routley (1984) describes <strong>the</strong> Adult<br />
<strong>Wage</strong> <strong>Subsidy</strong> Scheme (AWSS), which was introduced during 1983. This programme<br />
briefly coexisted with SYETP within <strong>the</strong> National Employment and Training System<br />
(NEAT), an active labour market programme combining training and wage subsidies.<br />
Chapman (1985) p101 points out that AWSS was introduced by <strong>the</strong> outgoing Fraser<br />
Liberal government in response to <strong>the</strong> <strong>Australian</strong> recession starting after 1981, and<br />
initially retained by <strong>the</strong> incoming Hawke Labour government. Subsidies in amounts <strong>of</strong> up<br />
to $125 per week, for an up to 52 week period, were available to long term unemployed,<br />
fur<strong>the</strong>r defined as “…adults who have been unemployed for a lengthy period” (Routley<br />
(1984): 3). BLMR (June 1984) p168 indicates that AWSS operated in a similar fashion to<br />
<strong>the</strong> extended SYETP in 1983-4, with a 17 week period at $100 per week followed by<br />
ano<strong>the</strong>r 17 week period at $75 per week. Those aged over 24 with 8 <strong>of</strong> <strong>the</strong> last 12 months<br />
registered with <strong>the</strong> CES as unemployed and away from full-time education were eligible.
32<br />
For those aged over 45 and similarly long-term unemployed, an ‘extended AWSS’ was<br />
available, at $125 per week for up to 52 weeks.<br />
Edwards (1987) p86 notes that <strong>the</strong>re were limitations on <strong>the</strong> proportion <strong>of</strong> <strong>the</strong> employer’s<br />
workforce which could be employed under all/any wage subsidy programmes from <strong>the</strong><br />
Federal government and cites <strong>the</strong> guidelines to CES 5 as stating that at any one time, a<br />
single physical location <strong>of</strong> factory, shop or <strong>of</strong>fice with 1-3 staff could have 1 placement;<br />
while those with 4-7 staff could have 2 placements; for those with 8-100 staff, not more<br />
than 25 per cent <strong>of</strong> staff could be placements; and for those with more than 100 staff not<br />
more than 10 per cent <strong>of</strong> staff could be placements. This would have limited both AWSS<br />
and SYETP placements. Routley (1984) indicates that in <strong>the</strong> first four months <strong>of</strong><br />
operation in 1983, 1,642 <strong>of</strong> <strong>the</strong> eligible target group were approved for placements.<br />
Approvals <strong>the</strong>n rose to 15,353 over 12 months in 1983/4 and <strong>the</strong>n 14,388 in 1984/5<br />
(Edwards (1987) p88 Table 7.1, sourced from DEIR (1985) Annual Report figures). Ross<br />
(1988) p48 shows that expenditure on AWSS rose with <strong>the</strong> increase in placements: for<br />
<strong>the</strong> corresponding initial 4 month period to June 30 1982/3 <strong>the</strong> expenditure was<br />
$0.4million in current prices, while in 1983/4 it rose to $23.4 million, and 1984/5 to<br />
$35.1million, before $25.4 million in <strong>the</strong> part <strong>of</strong> <strong>the</strong> final year <strong>of</strong> operation to December<br />
1985.<br />
An evaluation report was due to be made for AWSS in 1985 (Routley (1984)). However<br />
this never achieved publication. Ross (1988) p48 Charts 1 and 2 and appendix Table <strong>of</strong><br />
Acronyms , noted that having started in March 1983, in December 1985 <strong>the</strong> programme<br />
had already ended by being subsumed into Jobstart. An evaluation for AWSS is not<br />
separately listed amongst <strong>the</strong> BLMR evaluations listed in Mckay and Hope (1986)<br />
Attachment A pp19-21, and it is likely <strong>the</strong> evaluation was sidelined with <strong>the</strong> knowledge<br />
that policy interests had already determined <strong>the</strong> programme, at least in name, would end.<br />
Although no general analysis <strong>of</strong> AWSS was made, Edwards (1987) examined <strong>the</strong> impact<br />
for <strong>the</strong> Wollongong and Newcastle regions <strong>of</strong> New South Wales using a survey <strong>of</strong> firms<br />
5 Department <strong>of</strong> Employment and Industrial Relation (1984) Adult <strong>Wage</strong> <strong>Subsidy</strong> Scheme in CES<br />
Operating Manual, Volume 6 section 5; Volume 7.
33<br />
who participated. It was concluded that <strong>the</strong> AWSS was used mostly by small firms, and<br />
those firms used it for <strong>the</strong> recruitment <strong>of</strong> low wage and lower skilled workers. No<br />
evidence <strong>of</strong> churning was found, where employers lay <strong>of</strong>f placements when <strong>the</strong> subsidy<br />
expired, as most placements that completed <strong>the</strong> subsidised period were retained. However,<br />
it was also found that most placements were at <strong>the</strong> expense <strong>of</strong> unsubsidised employment<br />
<strong>of</strong> o<strong>the</strong>r similar workers. There was no modelling <strong>of</strong> employment.<br />
2.1.2 ‘General Training Assistance On-<strong>the</strong>-Job-Training’<br />
The ‘General Training Assistance On-<strong>the</strong>-Job-Training’ [GTA-OTJ] was also available to<br />
youths at <strong>the</strong> same time as SYETP in 1983, but in contrast was only 9.9 per cent <strong>of</strong> all<br />
placements. The GTA was a highly flexible programme, which also appears to have been<br />
highly discretional. This was described as an 'age tiered' wage subsidy to unemployed for<br />
on-<strong>the</strong>-job training in ‘occupations in demand’, with a subsidy in 1983 <strong>of</strong> $50.80 to<br />
juniors and $69.30 for adult award rates. It was available to all ages for those ‘at risk’,<br />
with eligible positions based on trade or non-trade skills which were subject to ‘labour<br />
market demand’ – this was tested on departmental criteria so that CES <strong>of</strong>ficers instigated<br />
a GTA position if ‘no suitably experienced qualified person is available when employers<br />
lodge vacancies’ (BLMR (1983) p 7 Table 1.3, footnote (c)). The period <strong>of</strong> assistance<br />
‘varied according to occupational skill level’. This programme likely best falls into <strong>the</strong><br />
category <strong>of</strong> ‘special assistance’, as it was not generally available. Officially, it was not<br />
considered a true wage subsidy. It was pointed out that <strong>the</strong> primary purpose was training<br />
workers with past work experience in situations where <strong>the</strong>re were no workers with <strong>the</strong><br />
requisite skills, and it was not considered to be a marginal stock subsidy programme<br />
(DEYA (1983): 166). However, despite <strong>the</strong> unusual targeting, it clearly fits within <strong>the</strong><br />
definition <strong>of</strong> wage subsidy outlined in section 1.2.<br />
GTA-OTJ was analysed alongside SYETP and o<strong>the</strong>r programmes and generally found to<br />
have a positive impact on post-programme employment. Each <strong>of</strong> <strong>the</strong>se analyses is treated<br />
in depth later in considering SYETP evaluation, and so fur<strong>the</strong>r details and critique <strong>of</strong> <strong>the</strong><br />
methods can be found <strong>the</strong>re. BLMR (1983) used administrative records for<br />
commencements flowing onto all government programmes to youths in 1980-81 and
34<br />
showed employment entry flows for GTA-OTJ, but <strong>the</strong>re was no modelling <strong>of</strong><br />
employment outcomes. Stretton (1982, 1984) found that GTA-OTJ performed better on<br />
post-programme employment than training in EPUY 6 , but was no different to SYETP.<br />
Baker (1984) also found that GTA-OTJ performed better than EPUY, and no different to<br />
SYETP. Before modelling, it was found that 58.9 per cent <strong>of</strong> GTA were in continuous<br />
full-time work, made up <strong>of</strong> 45.4 per cent retained in <strong>the</strong>ir placement job and 13.5 per cent<br />
not retained, with 29.8 per cent having non-continuous full-time work, while <strong>the</strong><br />
estimated probability for continuous full-time employment after <strong>the</strong> programme for those<br />
with 17 weeks <strong>of</strong> unemployment was 0.569 (Baker (1984) p19 Table 5.2, p47 Table A8,<br />
p48 Table A10). Rao and Jones (1986) estimated post-programme full-time continuous<br />
employment chances for GTA-OTJ 1981-1983 relative to <strong>the</strong>ir quasi-control group as<br />
being 59.8 per cent for <strong>the</strong> least disadvantaged and 14.6 per cent for <strong>the</strong> most<br />
disadvantaged. This positive employment effect was greater than that found for SYETP.<br />
2.1.3 Jobstart<br />
Jobstart began in December 1985, when SYETP and <strong>the</strong> AWSS ended by being<br />
subsumed into <strong>the</strong> new Jobstart programme as all labour market programmes were<br />
restructured (Ross (1988) p34 footnote 9). The specialist programmes, ‘<strong>Special</strong> Needs<br />
Job Sector <strong>Subsidy</strong>’ and ‘Disabled on-<strong>the</strong>-job subsidy’, were also merged into Jobstart<br />
(Kesteven (1987): 45). All aspects <strong>of</strong> SYETP, including <strong>the</strong> Commonwealth SYETP, as<br />
described later in <strong>the</strong> section about SYETP, turned up as components in Jobstart. At<br />
inception, Jobstart had two key components, private sector Jobstart and Commonwealth<br />
Jobstart. The ‘Commonwealth Work Experience programme’ was identical to<br />
Commonwealth SYETP – it lasted for 17 weeks subsidy, and was limited to 15-24 year<br />
olds, with <strong>the</strong> department fully reimbursed for <strong>the</strong> wage paid to <strong>the</strong> placement. However,<br />
in May 1987 <strong>the</strong> Commonwealth component ceased. After this, Private Sector Jobstart<br />
became synonymous with <strong>the</strong> term Jobstart.<br />
Eligible groups for Jobstart were divided into two: Jobstart basic rates applied to those<br />
unemployed for 6 <strong>of</strong> <strong>the</strong> last 9 months, while Jobstart <strong>Special</strong> rates applied to those long-<br />
6 Education Programme for Unemployed <strong>Youth</strong>; an <strong>Australian</strong> training program part <strong>of</strong> NEAT; consisting<br />
<strong>of</strong> courses aimed to improve basic literacy, numeracy and social skills.
35<br />
term unemployed for over 12 <strong>of</strong> <strong>the</strong> last 15 months, or <strong>the</strong> disadvantaged groups listed.<br />
The designated disadvantaged groups were “...disabled, sole supporting parents, migrants<br />
with English language difficulties, Aborigines” (Kesteven (1987): 45). The positions<br />
were required to be full-time, but with exceptions possible for <strong>the</strong> disadvantaged groups,<br />
and available for 26 weeks continuous employment. Job vacancies had to be notified with<br />
<strong>the</strong> CES, and at least <strong>the</strong> Award wage 7 was paid to placements. There was a special<br />
Jobstart self-service board with all wage subsidy jobs displayed on it, and <strong>the</strong> CES could<br />
also give a card called, a ‘self-canvassing card’, that could be shown to employers at <strong>the</strong><br />
job interview (Victorian State Office (1985) p7 and DEET (1994) p38).<br />
Jobstart was described as a general wage subsidy scheme, where employers received<br />
subsidy for placements where <strong>the</strong> rate varied with age. Private Sector Jobstart subsidy<br />
rates for 1985-87 are shown below in Table 2.1. The Jobstart <strong>Special</strong> rate was higher than<br />
<strong>the</strong> basic rate, and rates increased with age. In 1985/86 35,098 placements were approved<br />
in private sector Jobstart, with expenditure for <strong>the</strong> year at $18,787 million (Kesteven<br />
(1987): 46).<br />
Jobstart was expanded considerably during <strong>the</strong> <strong>Australian</strong> recession, but was more<br />
commonly referred to within <strong>the</strong> Job Compact package. After 1989, it was part <strong>of</strong> <strong>the</strong> Job<br />
Compact along with <strong>the</strong> National Training <strong>Wage</strong> [NTW]. The NTW was a training wage<br />
but was “...to be supplemented by subsidies to entice employers to provide training”<br />
(OECD (2001): 198). The Job Compact was part <strong>of</strong> <strong>the</strong> Working Nation package <strong>of</strong><br />
initiatives, and centred on an <strong>of</strong>fer <strong>of</strong> 6-12 months job placement in <strong>the</strong> private sector, but<br />
usually 9 months, to all who had been in receipt <strong>of</strong> benefits for more than 18 months<br />
(OECD (2001): 198). This eligibility required a longer period <strong>of</strong> unemployment than in<br />
1987, and so <strong>the</strong> targeting <strong>of</strong> <strong>the</strong> programme was tightened in this new form, however<br />
against this, <strong>the</strong> subsidy length was extended to 12 months. This had strong effects on <strong>the</strong><br />
programme. There were issues <strong>of</strong> low take-up. Sheen and Tre<strong>the</strong>wey (1991) p38 point<br />
7 Australia has extensive union wage agreements, which specify a complex system <strong>of</strong> minimum wages.<br />
These are termed Award <strong>Wage</strong>s.
36<br />
out that in 1989-90 <strong>the</strong>re were substantial difficulties in finding placements with private<br />
sector employers up to <strong>the</strong> full Budget allocation to <strong>the</strong> programme.<br />
During <strong>the</strong> period to 1996, <strong>the</strong>re was an expansion <strong>of</strong> job creation programmes ra<strong>the</strong>r<br />
than wage subsidies, which was mostly due to <strong>the</strong> new case management incentive<br />
system where <strong>the</strong> case payment for outcomes was <strong>the</strong> same for entry to any programme<br />
placement or unsubsidised job (OECD (2001): 200). It was found that several factors<br />
meant that <strong>the</strong> NWO 8 direct job creation programme got most Job Compact placements<br />
ra<strong>the</strong>r than Jobstart: employers were reluctant to take on <strong>the</strong> long term unemployed and<br />
considered <strong>the</strong> 9 month placement too long; dismissal laws meant employers were<br />
reluctant to take on potentially unsuitable employees (DEETYA (1996): 46). However<br />
importantly, in 1994/5 <strong>the</strong>re was a strong administrative push to give preference to NTW<br />
over Jobstart and it was perceived that <strong>the</strong>re was competition between <strong>the</strong> programmes as<br />
employers could get a higher subsidy under NWO than Jobstart without having to<br />
commit to a long period <strong>of</strong> placement employment (DEETYA (1996): 41-49). Stretton<br />
and Chapman (1990) p42 described <strong>the</strong> selection process for Jobstart as follows: “The<br />
filling <strong>of</strong> Jobstart vacancies does involve some selection bias as <strong>the</strong> CES selects those<br />
eligible clients who are judged to be ‘job ready with assistance´ for referral to Jobstart<br />
vacancies. Employers <strong>the</strong>n select <strong>the</strong>ir subsidised employee from among a number <strong>of</strong><br />
referrals made by <strong>the</strong> CES.”<br />
The subsidy varied with age, education and length <strong>of</strong> unemployment (DEETYA (1996):<br />
131). In 1995, Jobstart subsidies could run for 6 to 12 months, and subsidies varied by<br />
unemployment length and were received in tiers so that those 18 months unemployed<br />
received $200 for <strong>the</strong> first 13 weeks, followed by $100 per week for <strong>the</strong> next 26 weeks,<br />
and a lump sum to employers <strong>of</strong> $500 after 12 months <strong>of</strong> continuous employment was<br />
reached (Piggot and Chapman (1995): 4). In 1994/5 <strong>the</strong>re were 40,200 Jobstart<br />
placements for Job Compact clients, which was a fall <strong>of</strong> 25,000 from <strong>the</strong> previous year<br />
(DEETYA (1996): 46). Employer survey findings suggested that employers found <strong>the</strong><br />
8 New Work Opportunities (NWO) provided direct job creation in projects where placements had work<br />
with some training typically in environmental, age care and community sectors (OECD (2001): 199).
37<br />
importance <strong>of</strong> <strong>the</strong> subsidy secondary to <strong>the</strong> quality and suitability <strong>of</strong> <strong>the</strong> worker, and<br />
references were made to <strong>the</strong> problems <strong>of</strong> low skills, poor attitudes to work and low<br />
motivation <strong>of</strong> long-term unemployed as perceived by employers (DEETYA (1996): 47).<br />
In October 1995, Jobstart was adjusted to allow shorter placements and extension <strong>of</strong><br />
eligibility to those 6-12 months unemployed.<br />
Table 2.1 Private Sector Jobstart subsidy weekly rates 1985-1987<br />
Age <strong>of</strong> placement Jobstart Basic rate $ Jobstart <strong>Special</strong> rate $<br />
15-17 50 75<br />
18-20 75 110<br />
21-44 100 150<br />
45+ 125 180<br />
Source: Kesteven (1987) p45<br />
A flurry <strong>of</strong> evaluation evidence started to exist for Jobstart from about <strong>the</strong> time <strong>of</strong> this<br />
programme’s inception, and through <strong>the</strong> 1990’s. These evaluations were usually by <strong>the</strong><br />
government department responsible for <strong>the</strong> programme. Most <strong>of</strong> <strong>the</strong> more recent studies<br />
use <strong>the</strong> matched comparison group design, or exact matching methods. 9 The early<br />
evaluation DEET (1989) used data on 1000 participants and non-participants, and found<br />
post-programme continuous full-time employability increased on average by 33<br />
percentage points at 3,5 and 14 months. 10<br />
The next analysis <strong>of</strong> Jobstart was published in 1993. The raw post-programme<br />
employment share, sourced from <strong>the</strong> quarterly Post Program Survey, showed that for<br />
1992 <strong>the</strong>re were 95,600 new placements and 57 per cent <strong>of</strong> Jobstart participants were in<br />
unsubsidised employment over <strong>the</strong> 12 months to September 1992 (DEET (1994) p38<br />
Table 5.3). This was <strong>the</strong> highest employment outcome amongst all programmes, with <strong>the</strong><br />
average for all programmes at 32 per cent. Younger jobseekers were more likely to<br />
receive Jobstart 11 , and men were more likely to get Jobstart placements. 12 SYETP became<br />
Jobstart, and after 10 years in place it seems likely that SYETP had become a generally<br />
9 See earlier in Chapter 1 for <strong>the</strong> introduction to evaluation methods, and Chapter 4.4 for a description <strong>of</strong><br />
matching.<br />
10 DEET (1989) “Jobstart evaluation” DEET Program Review and Income Support Branch, Canberra. Cited<br />
in Webster (1998) Table 1 p194.<br />
11 DEET (1994) p45 and p33 “participation by youth in Jobstart, especially 18-20 years olds was well<br />
above average placement rates.”<br />
12 DEET (1994) p33 “32 places to women per 1000 registrants compared to 44 for men”.
38<br />
accepted part <strong>of</strong> <strong>the</strong> labour market, so employers may have continued to use <strong>the</strong><br />
programme as <strong>the</strong>y had done so in <strong>the</strong> past, under <strong>the</strong> new name <strong>of</strong> Jobstart.<br />
The net impact analysis in 1993 used a survey <strong>of</strong> a sample <strong>of</strong> programme participants<br />
approximately 6 months after programme participation, and matched <strong>the</strong>se on age, sex<br />
and unemployment duration to a similarly surveyed sample <strong>of</strong> CES registrants who had<br />
not been placed in that programme. The mean employment outcomes <strong>of</strong> <strong>the</strong> matched<br />
treated and comparison groups were <strong>the</strong>n compared to create <strong>the</strong> impact measure. Two<br />
comparison groups were constructed, <strong>of</strong> those with no programme experience and <strong>of</strong><br />
those who had received training assistance instead <strong>of</strong> Jobstart. Jobstart was found to have<br />
a net impact <strong>of</strong> 30 percentage points above those who had no programme assistance. It<br />
was found that Jobstart had a similar impact to o<strong>the</strong>r programmes <strong>of</strong> training assistance<br />
such as Jobtrain, Skillshare and Jobclub when compared to a comparison group <strong>of</strong> those<br />
who had taken part in a training programme and <strong>the</strong>n found employment (DEET (1994):<br />
42). Relative to training programmes, Jobstart was concluded to be <strong>the</strong> most successful<br />
programme for raising post-programme employment.<br />
In reviewing <strong>the</strong> 1993 Jobstart evaluation, it is unclear that non-response was accounted<br />
for in analysing <strong>the</strong> survey data, but it is presumed that it was not. The method <strong>of</strong> directmatching<br />
can only account for <strong>the</strong> factors used in <strong>the</strong> matching, in this case only 3 i.e. age,<br />
sex and unemployment length. Although <strong>the</strong>se might presumably have been considered<br />
<strong>the</strong> most important variables, it is likely <strong>the</strong>re were restrictions on <strong>the</strong> information<br />
available in <strong>the</strong> data, and also limits on <strong>the</strong> number <strong>of</strong> variables it was possible to match<br />
on. The general difficulty <strong>of</strong> direct matching is that it becomes increasingly difficult to<br />
find matches as <strong>the</strong> number <strong>of</strong> factors is increased, so curbing <strong>the</strong> number <strong>of</strong> variables<br />
chosen. Thus <strong>the</strong>re were constraints on <strong>the</strong> number <strong>of</strong> variables that could be accounted<br />
for. The aim <strong>of</strong> <strong>the</strong> matching is to control for observed differences in pre-programme<br />
characteristics between <strong>the</strong> treated and comparison groups. Matching on <strong>the</strong>se three<br />
variables would eliminate bias due to non-random selection from o<strong>the</strong>r sources only to<br />
<strong>the</strong> extent that those o<strong>the</strong>r factors are correlated with <strong>the</strong>se 3 variables. Any o<strong>the</strong>r<br />
variables are not controlled for entirely, even if <strong>the</strong>y are observed, and so strong
39<br />
differences in characteristics that affect <strong>the</strong> employment outcome can remain. As a result<br />
it is likely that <strong>the</strong> matching performed does not fully diminish potential bias arising from<br />
<strong>the</strong>se o<strong>the</strong>r sources.<br />
Some raw statistics suggesting <strong>the</strong> positive effect <strong>of</strong> Jobstart upon employment were<br />
presented in <strong>the</strong> Working Nation policy document, however no more systematic<br />
evaluation evidence was presented. The raw post-programme employment share, derived<br />
from <strong>the</strong> quarterly Post Programme Survey [PPM], showed that for 1996 <strong>the</strong>re were 41.1<br />
per cent <strong>of</strong> Jobstart participants in unsubsidised employment over <strong>the</strong> 12 months to<br />
September 1992 (DEETYA (1996) p53 Table 4.5). This was higher than <strong>the</strong> employment<br />
share for <strong>the</strong> training programmes Jobskills and NWO 13 . The employment effect was not<br />
evaluated fur<strong>the</strong>r with any modelling.<br />
A fur<strong>the</strong>r evaluation <strong>of</strong> Jobstart in 1997 14 also used <strong>the</strong> matching design. Again age, sex<br />
and unemployment duration were used for matching with o<strong>the</strong>r factors such as education<br />
and location not accounted for, as <strong>the</strong>se were not deemed as important to describing<br />
employment outcomes. 15 Using administrative data, participants were matched to a<br />
comparison group <strong>of</strong> non-participants. Jobstart participants had post-programme<br />
employment 3 months after <strong>the</strong> programme <strong>of</strong> 50 per cent while <strong>the</strong> matched<br />
comparisons had 22 per cent, a net impact <strong>of</strong> 28 percentage points. At 12 months postprogramme,<br />
<strong>the</strong> net employment impact had fallen to 15 percentage points.<br />
Stromback and Dockery (2000) used <strong>the</strong> panel data 1994-1997 Survey <strong>of</strong> Employment<br />
and Unemployment Patterns (SEUP) to examine employment outcomes for programme<br />
participants, including <strong>the</strong> Jobstart. The survey collected 3 interviews between September<br />
1994 and 1997, for a panel <strong>of</strong> persons aged 15-59 years. Labour market history for each<br />
month <strong>of</strong> <strong>the</strong> past year was collected at each interview by recall, recorded as periods <strong>of</strong><br />
13 New Work Opportunities (NWO). NWO provided direct job creation in projects where placements had<br />
work with some training typically in environmental, age care and community sectors [OECD (2001): 199].<br />
14 DEETYA (1997) “The net impact <strong>of</strong> labour market programs: improvements in <strong>the</strong> prospects <strong>of</strong> those<br />
assisted” DEETYA <strong>Evaluation</strong> and Monitoring Branch report 2/97, Economic and Policy Division,<br />
Canberra. Cited Table 1 p194 Webster (1998) and Richardson (1998) p3.<br />
15 DEETYA (1997) Appendix B cited Webster (1998) p200 footnote 5.
40<br />
working, looking for work and absence from <strong>the</strong> labour market. The survey data was<br />
linked to administrative records for programme participation, and <strong>the</strong> PPM survey<br />
outcomes. Only consenting persons could be linked, and not all linked cases had full<br />
administrative data or PPM survey outcomes, and all incomplete records were dropped<br />
for analysis. The matched data showed that raw outcomes for employment 3 months after<br />
<strong>the</strong> Jobstart programme were substantially different in <strong>the</strong> matched PPM outcome and <strong>the</strong><br />
SEUP Labour market history, as shown in Table 2.2. The difference in <strong>the</strong> employment<br />
outcomes measured could be partly due to <strong>the</strong> differing data sources, collection times,<br />
response rates, recall biases, <strong>the</strong> shares <strong>of</strong> incomplete/missing data (as all incomplete<br />
cases were dropped), and also <strong>the</strong> share <strong>of</strong> cases where <strong>the</strong>re was an inability to link data.<br />
Table 2.2 Stromback and Dockery (2000) Raw employment outcomes after Jobstart<br />
Matched PPM<br />
SEUP<br />
% in Employment 38.4% 51.0%<br />
after Jobstart<br />
Known outcomes 318 787<br />
Source: Stromback and Dockery (2000) p32 Table 8.3 Only includes episodes for which PPM outcomes are<br />
known. Reported labour market activity is known for all episodes. SEUP calculated as per cent <strong>of</strong> labour<br />
market spells, Matched PPM calculated as per cent <strong>of</strong> spells with known outcomes.<br />
Exact matching analysis on gender, four age groups and 8 categories <strong>of</strong> unemployment<br />
spell durations was used to estimate <strong>the</strong> net employment impact using <strong>the</strong> SEUP data. A<br />
comparison group was constructed <strong>of</strong> those registered unemployed in <strong>the</strong> administrative<br />
data, who were not in <strong>the</strong> programme at <strong>the</strong> point <strong>the</strong> Jobstart participants left <strong>the</strong><br />
programme (note this means <strong>the</strong>y could have entered <strong>the</strong> programme after this point). The<br />
employment outcomes 3 months after <strong>the</strong> programme participation reference period were<br />
<strong>the</strong>n compared. For Jobstart, <strong>the</strong> net impact on employment 3 months after <strong>the</strong><br />
programme in 1995 was found to be 11.8 per cent, with 19.3 per cent <strong>of</strong> Jobstart<br />
participants in employment and 7.5 per cent <strong>of</strong> <strong>the</strong> comparison group. During <strong>the</strong> later<br />
survey periods <strong>the</strong>re was a change to <strong>the</strong> programme rules, which meant that employers<br />
were required to retain employees at least 3 months after <strong>the</strong> subsidy period, and because<br />
this corresponds to <strong>the</strong> post-programme evaluation period chosen for analysis <strong>the</strong>se later<br />
periods cannot be usefully considered (OECD (2001): 217).
41<br />
Matching methods are <strong>the</strong>oretically based upon <strong>the</strong> consideration <strong>of</strong> characteristics prior<br />
to programme entry, and <strong>the</strong> definition <strong>of</strong> <strong>the</strong> comparison group used here flouts this<br />
fundamental premise and would bias and generally invalidate <strong>the</strong>se estimates. This is<br />
because Stromback and Dockery (2000) argue that <strong>the</strong>ir comparison group definition<br />
excludes <strong>the</strong> ‘lock-in’ period, where placements must be in subsidised employment until<br />
<strong>the</strong> end <strong>of</strong> <strong>the</strong> subsidy, which <strong>the</strong> comparison group does not face. They also point out<br />
that defining <strong>the</strong> comparison group at programme entry as matching methods require<br />
would give <strong>the</strong> comparison group an employment rate nearly 3 times higher, and so<br />
Jobstart would end up with a negative employment impact. However, this alludes to<br />
choices made to achieve a desired positive employment impact ra<strong>the</strong>r than providing a<br />
satisfactory reason for choosing this econometric strategy.<br />
In June 1996 <strong>the</strong> new coalition government announced reductions for labour market<br />
programme expenditures, and abolition <strong>of</strong> several programmes. However <strong>the</strong>re was some<br />
political confusion and substantial ambiguity about Jobstart expenditures and continuity,<br />
with various conflicting announcements (OECD (2001): 201 –202). It eventuated that<br />
Jobstart continued into 1997-98 but as a programme for <strong>the</strong> disabled, and was <strong>the</strong>n<br />
abolished in 1998/99. There is no wage subsidy programme that operates in Australia in<br />
<strong>the</strong> current period.<br />
2.1.4 Discussion<br />
<strong>Australian</strong> evaluation evidence for wage subsidies is scant. The AWSS was not evaluated.<br />
The GTA-OTJ was only evaluated alongside SYETP. 16 It could in any case be argued,<br />
that <strong>the</strong> relevance <strong>of</strong> <strong>the</strong>se subsidies is more limited than that <strong>of</strong> SYETP or Jobstart. Both<br />
AWSS and GTA-OTJ were very small in scale, with very little expenditure and few<br />
placements being made to <strong>the</strong>se programmes. GTA-OTJ is very difficult to define clearly,<br />
as it was so discretionary. However, as GTA-OTJ was only for ‘jobs-in-demand’ it was<br />
not a general wage subsidy available to all employers. The later Jobstart evaluations<br />
mostly took place after Jobstart had been changed from a general wage subsidy tiered by<br />
age to a subsidy programme only for long-term unemployed and disadvantaged or<br />
16 A critical review <strong>of</strong> <strong>the</strong>se evaluations is given in <strong>the</strong> later section 2.3 <strong>of</strong> this chapter
42<br />
targeted minority groups. The SYETP in contrast was a more generally targeted flat rate<br />
wage subsidy programme.<br />
The later <strong>Australian</strong> evaluation <strong>of</strong> <strong>the</strong> Jobstart wage subsidy used direct matching<br />
techniques. These methods however require a limited set <strong>of</strong> characteristics and large data<br />
sample in order to enable <strong>the</strong> analysis to proceed. The more characteristics it is desirable<br />
to match on, <strong>the</strong> fewer matches become available amongst <strong>the</strong> treatment and comparison<br />
groups. As <strong>the</strong> earlier modelling <strong>of</strong> SYETP showed, controlling for individual<br />
characteristics was important, and <strong>the</strong> subsequent Richardson (1998) evidence indicates<br />
<strong>the</strong> breadth <strong>of</strong> characteristics that might be useful (reviewed in <strong>the</strong> next section).<br />
Matching, such as that <strong>of</strong> Jobstart, which does not control for <strong>the</strong>se o<strong>the</strong>r characteristics<br />
that affect both employment and participation would <strong>the</strong>n be subject to bias. Importantly,<br />
education <strong>of</strong> participants and nonparticipants is not accounted for in <strong>the</strong> Jobstart<br />
evaluations. As well, for matching to be valid, definition <strong>of</strong> <strong>the</strong> characteristics must be<br />
prior to programme entry, and this was not <strong>the</strong> case for <strong>the</strong> comparison group constructed<br />
for <strong>the</strong> Stromback and Dockery (2000) Jobstart evaluation using matching.<br />
The review <strong>of</strong> o<strong>the</strong>r <strong>Australian</strong> wage subsidy evaluation evidence makes it clear that<br />
evaluation <strong>of</strong> <strong>the</strong> SYETP remains useful. Apart from its later namesake Jobstart, SYETP<br />
was <strong>the</strong> largest wage subsidy programme <strong>of</strong> general application. There is also less useful<br />
evaluation evidence for o<strong>the</strong>r wage subsidy programmes than for SYETP, as can be seen<br />
in <strong>the</strong> later review <strong>of</strong> SYETP evidence.
43<br />
2.2 SYETP implementation<br />
As SYETP does not now operate, this review is essentially a historical essay ga<strong>the</strong>ring<br />
toge<strong>the</strong>r information about <strong>the</strong> programme and its process. Due to <strong>the</strong> fact that much <strong>of</strong><br />
<strong>the</strong> information is contained in government publications and conference papers, some<br />
sources are difficult to locate, and as such <strong>the</strong> review provides a valuable resource.<br />
Considerable effort has been made to associate <strong>the</strong> context <strong>of</strong> o<strong>the</strong>r aspects <strong>of</strong> <strong>the</strong><br />
<strong>Australian</strong> economy that may have influenced <strong>the</strong> working <strong>of</strong> SYETP. The policy does<br />
not operate within a vacuum, and although <strong>the</strong> <strong>the</strong>oretical model may outline a set <strong>of</strong><br />
assumptions it is <strong>of</strong> interest when assessing <strong>the</strong> policy outcome whe<strong>the</strong>r in fact <strong>the</strong>se<br />
assumptions were realistic. In particular, this is necessary as many models have <strong>the</strong> same<br />
outcome, but use different sets <strong>of</strong> assumptions to set up how <strong>the</strong> outcome is achieved; or<br />
various models can achieve different outcomes by altering <strong>the</strong> assumption set. So this<br />
context might help define which model best approximated <strong>the</strong> circumstances under which<br />
<strong>the</strong> outcome was achieved. The description <strong>of</strong> <strong>the</strong> economic context <strong>of</strong> labour market<br />
programme operation is <strong>of</strong>ten not clear, or is overlooked in a review, for example – <strong>the</strong><br />
benefit system payments, <strong>the</strong> wage system, whe<strong>the</strong>r it operated in a recession or boom?<br />
Due to <strong>the</strong> wide field <strong>of</strong> influences to <strong>the</strong> labour market, it is not possible to fully convey<br />
<strong>the</strong> economic influences at work in a brief review. However, I have selected what I<br />
believe to be <strong>the</strong> most relevant aspects. The information is <strong>the</strong>n reconciled in <strong>the</strong><br />
discussion.<br />
2.2.1 Historical development <strong>of</strong> SYETP<br />
The SYETP was active policy in Australia from late 1976 to December 1985. The history<br />
and functioning <strong>of</strong> SYETP have been documented in various sources, and are<br />
summarised here. The terms <strong>of</strong> <strong>the</strong> scheme and operating environment are factors that are<br />
important to understanding <strong>the</strong> conditions under which <strong>the</strong> programme performed. They<br />
provide <strong>the</strong> background for <strong>the</strong> findings <strong>of</strong> possible impacts <strong>of</strong> <strong>the</strong> scheme. The
44<br />
characteristics <strong>of</strong> <strong>the</strong> programme and <strong>the</strong> labour market environment are <strong>the</strong> focus <strong>of</strong><br />
discussion, with emphasis on details pertinent to <strong>the</strong> period <strong>of</strong> our analysis.<br />
SYETP was <strong>the</strong> first wage subsidy scheme to be introduced in Australia (Chapman<br />
(1985): 101). The scheme started operation in September/October 1976. 17 The stated<br />
objective was to encourage <strong>the</strong> employment <strong>of</strong> young unemployed persons. Over <strong>the</strong><br />
subsequent period to December 1985 when SYETP ended, almost all labour market<br />
programmes in Australia were targeted at young people. 18 There was also a distinct<br />
emphasis on private sector involvement in SYETP. The subsidy was given at a flat rate<br />
for a limited time period.<br />
SYETP was initially quite closely targeted. However, it gradually evolved broader<br />
application and was tweaked to vary according to age and unemployment duration. The<br />
following discussion draws heavily on <strong>the</strong> historical progression <strong>of</strong> SYETP found in<br />
BLMR (1984), Smith (1983, 1984a), Chapman (1985), Hoy (1983), Kesteven (1987),<br />
Ross (1988), and Stretton and Chapman (1990).<br />
On initial introduction in October 1976, SYETP was for 15-19 year olds who had left<br />
school in 1975 (BLMR (1984): 159). Employers received $58 per week for six months.<br />
The eligibility was rapidly extended by December 1976 to all 15-19 year olds who had<br />
been unemployed and away from full-time education for at least 6 <strong>of</strong> <strong>the</strong> last 12 months.<br />
The age range was extended to 15-24 year olds who had been unemployed and away<br />
from full-time education for at least 6 <strong>of</strong> <strong>the</strong> last 12 months in July 1977, and <strong>the</strong> subsidy<br />
increased to $63. In October 1977, <strong>the</strong> unemployment criterion was relaxed to those<br />
unemployed and away from full-time education for at least 4 <strong>of</strong> <strong>the</strong> last 12 months, ra<strong>the</strong>r<br />
than 6, and <strong>the</strong> rate rose slightly to $66.<br />
17 BLMR (1984) p127 gives September, p159 gives October.<br />
18 BLMR (1984) p128. There were some exceptions, such as <strong>the</strong> Regional Employment Development<br />
Scheme (REDS) which started in September 1974, but was ended in 1975 and fully phased out by <strong>the</strong> end<br />
<strong>of</strong> 1977. This program gave grants for specific labour intensive projects, costs <strong>of</strong> wages were paid to <strong>the</strong><br />
employer for up to 12 months, and award wages paid to placements. Eligible participants were unemployed<br />
with preference given to those with dependents and with longer unemployment (Kesteven (1987): 65).
45<br />
In July1978, <strong>the</strong> subsidy was calculated to be equivalent to 62 per cent <strong>of</strong> <strong>the</strong> average<br />
junior award pay rate for <strong>the</strong> applicable ages, as can be seen in Table 2.3. The<br />
information in Table 2.3 can only be used as a broad guide as award rates and weekly<br />
earnings in <strong>the</strong> award system vary by age, gender, occupation and industry. <strong>Subsidy</strong><br />
participants were guaranteed at least <strong>the</strong> minimum award rate from employers, but could<br />
be <strong>of</strong>fered more, although this is less likely. As <strong>the</strong> number <strong>of</strong> hours worked could vary<br />
<strong>the</strong> average weekly wage, <strong>the</strong> average hourly earnings can be a more useful indicator.<br />
Windshuttle (1985) p20 also pointed out that males and females faced differing hourly<br />
rates on average and so <strong>the</strong> overall average for juniors can be misleading. Windshuttle<br />
(1985) p21 also argued that in times <strong>of</strong> growing employment, adults were more likely to<br />
receive overtime than juniors. In light <strong>of</strong> this, <strong>the</strong> junior ratio to <strong>the</strong> adult average hourly<br />
earnings for fulltime employees is also shown for <strong>the</strong> same period, with a breakdown by<br />
sex as a point <strong>of</strong> comparison. The issue <strong>of</strong> <strong>the</strong> lower rate for juniors relative to adults is<br />
returned to in later discussion <strong>of</strong> <strong>the</strong> economic environment.<br />
Expansion <strong>of</strong> <strong>the</strong> eligibility criteria and subsidy amount were perceived to be <strong>the</strong> source<br />
<strong>of</strong> <strong>the</strong> noticeable rise in SYETP placements (see later Table 2.6 SYETP annual<br />
expenditure and placements1976/77-1985/86), but most noticeable to Government was<br />
that <strong>the</strong> expenditure on SYETP rose most dramatically. Indeed Table 2.6 shows that<br />
change, in <strong>the</strong> year to 78/9 from <strong>the</strong> previous year, was remarkable for a doubling <strong>of</strong><br />
expenditure but little change in placement numbers. A considerable lag in stock would<br />
have been evident due to <strong>the</strong> 26 week placement, so although new placements (flow) did<br />
not change much <strong>the</strong> length <strong>of</strong> <strong>the</strong> subsidy meant that <strong>the</strong> stock on SYETP at a point in<br />
time would rise quite high over <strong>the</strong> period.
46<br />
Table 2.3 Average junior award rates 1977 to 1981<br />
Date<br />
Average<br />
junior award<br />
wage<br />
$ per week<br />
Junior males<br />
Average<br />
hourly<br />
earnings for<br />
fulltime<br />
employees:<br />
$ per hour<br />
Ratio to<br />
adult male<br />
Average<br />
hourly<br />
earnings for<br />
fulltime<br />
employees<br />
%<br />
Junior<br />
females<br />
Average<br />
hourly<br />
earnings for<br />
fulltime<br />
employees:<br />
$ per hour<br />
Ratio to<br />
adult female<br />
Average<br />
hourly<br />
earnings for<br />
fulltime<br />
employees<br />
%<br />
1977 1.1.77 96.08 2.71 55.19 2.66 66.66<br />
1.7.77 101.46<br />
1978 1.1.78 105.82 2.89 54.84 2.83 66.24<br />
1.7.78 108.82<br />
1979 1.1.79 113.41 3.13 54.15 3.02 66.09<br />
1.7.79 117.04<br />
1980 1.1.80 119.97 3.52 54.15 3.38 64.68<br />
1.7.80 125.96<br />
1981 1.1.81 134.07 3.72 50.81 3.61 61.50<br />
1.7.81 144.69<br />
1982 1.1.82 156.11 4.43 51.73 4.31 62.92<br />
Source: Column1: Bureau <strong>of</strong> Labour Market Research (1984) Table 4.14 p 160.<br />
Column2: Windshuttle (1985) p20 Table 2 and p21 table 3 Average hourly earnings for non-managerial<br />
fulltime employees in private enterprise, all industry groups, ABS Cat 6304.0 various years to 1983. The<br />
ratios in <strong>the</strong> 5 th and 7 th columns have been multiplied by 100 to represent percentages.<br />
Cutbacks in <strong>the</strong> subsidy period and amount <strong>of</strong> subsidy were introduced in August 1978.<br />
The subsidy period dropped from 6 months to 4 months. The subsidy amount fell to $45 19 ,<br />
a fall from about 45 per cent <strong>of</strong> <strong>the</strong> average junior award pay rate for <strong>the</strong> applicable ages<br />
paid for 6 months to 30 per cent <strong>of</strong> <strong>the</strong> average junior award rate payable for only 4<br />
months (Ross (1988): 37). As Ross (1988) points out, this was a fall from a total <strong>of</strong> $1716<br />
paid to employers for a full subsidy period to only $765, a 55 per cent fall in <strong>the</strong> total<br />
subsidy for a full subsidy period.<br />
In late 1978, ministerial announcements were used to emphasise that SYETP was<br />
supposed to be used for additional new jobs that would not o<strong>the</strong>rwise have occurred. Also,<br />
guidelines to this effect were issued to CES <strong>of</strong>fices. 20 It is unlikely this provoked much<br />
change to operation however.<br />
19 For commencements after August 1978 <strong>the</strong> subsidy was $45 (Smith (1984)), [BLMR (1984) Table 4.14 p<br />
160]. The average junior award rates as estimated in BLMR (1983) and used for calculations are repeated<br />
in Table 2.3.<br />
20 Hoy (1983), cited p19 Smith (1984a).
47<br />
In January 1979, variations were made to <strong>the</strong> types <strong>of</strong> employer to whom <strong>the</strong> subsidy<br />
could apply. The State and Commonwealth government could now apply for SYETP for<br />
employees, as well as private sector employers. <strong>Special</strong> rates applied to <strong>the</strong><br />
Commonwealth government, who received <strong>the</strong> full wage cost in SYETP subsidy. State<br />
and private sector employers received <strong>the</strong> flat subsidy rate. 21 However, in August 1981<br />
<strong>the</strong> State governments ceased to be eligible for SYETP subsidies. The States again<br />
became eligible in 1983.<br />
In February 1981 ‘extended-SYETP’ was introduced. The eligibility for this required<br />
longer unemployment, and employers received a doubly long subsidy period <strong>of</strong> 34 weeks,<br />
with <strong>the</strong> subsidy paid at a higher rate. Those unemployed and away from full-time<br />
education for at least 8 <strong>of</strong> <strong>the</strong> last 12 months were eligible for ‘extended-SYETP’, and<br />
employers got A$80 per week for <strong>the</strong> first 17 weeks, <strong>the</strong>n A$55 for <strong>the</strong> next 17 weeks.<br />
This was equivalent to 60 per cent and <strong>the</strong>n 41 per cent <strong>of</strong> <strong>the</strong> average junior award pay<br />
rate for <strong>the</strong> applicable ages (BLMR (1984) p160 Table 4.14). The standard rate <strong>of</strong><br />
SYETP was commensurate with <strong>the</strong> second phase <strong>of</strong> extended SYETP.<br />
The SYETP rates were again changed in August 1982. The standard rate <strong>of</strong> SYETP and<br />
<strong>the</strong> second phase <strong>of</strong> extended-SYETP rose to $75, while <strong>the</strong> initial phase rate <strong>of</strong><br />
extended-SYETP rose to $100 per week. The procedure for SYETP subsidies was also<br />
loosened at this time. Whereas before <strong>the</strong> Commonwealth Employment Service <strong>of</strong>fice<br />
was <strong>the</strong> prime intermediary, in August 1982 <strong>the</strong> young unemployed were given <strong>the</strong> right<br />
to make direct contact with employers. This was seen as a “liberalization <strong>of</strong><br />
administrative procedures” (Smith (1984a): 20), whereby jobseekers could get a card<br />
indicating <strong>the</strong>ir eligibility for <strong>the</strong> subsidy, which <strong>the</strong>y could present to prospective<br />
employers. Aside from this change, it was always a requirement <strong>of</strong> SYETP that <strong>the</strong><br />
participant was registered with <strong>the</strong> CES, and <strong>the</strong> length <strong>of</strong> registration determined<br />
21 Sloan (1985) p114 pointed out that any overview <strong>of</strong> <strong>the</strong> operation <strong>of</strong> SYETP, and any labour market<br />
program, was likely to underestimate <strong>the</strong> important interaction between State and Federal government roles<br />
in funding in Australia. This aspect <strong>of</strong> expenditure can be seen here in <strong>the</strong> different treatment State and<br />
Commonwealth employees received at differing times in <strong>the</strong> history <strong>of</strong> SYETP.
48<br />
eligibility for programmes. Eligible youths were referred to employers by <strong>the</strong> CES and<br />
administration <strong>of</strong> <strong>the</strong> programme was carried out by <strong>the</strong> CES.<br />
In 1983, SYETP rates varied by age, with a lower $75 for 15-17 year olds but $100 for<br />
18-24 year olds.<br />
In 1984 programme conditions for employers included <strong>the</strong> development <strong>of</strong> a training plan<br />
for <strong>the</strong> new employee, and <strong>the</strong> employer had to pay <strong>the</strong> relevant award wage (Smith<br />
(1985): 15). In 1984, <strong>the</strong> standard rate <strong>of</strong> SYETP and <strong>the</strong> second phase <strong>of</strong> extended-<br />
SYETP remained at $75, and <strong>the</strong> initial phase rate <strong>of</strong> extended-SYETP stayed at $100 per<br />
week. In 1984, this was equivalent to “almost half <strong>the</strong> median full-time weekly earnings<br />
<strong>of</strong> teenagers” (Smith (1984a): 15).<br />
The SYETP programme ended at <strong>the</strong> end <strong>of</strong> December 1985. However, from a practical<br />
viewpoint all subsidised employment in place would have continued until ei<strong>the</strong>r <strong>the</strong><br />
placement ended or expiry <strong>of</strong> <strong>the</strong> subsidy period. Thus <strong>the</strong> scheme is considered to be<br />
completed in practice at <strong>the</strong> end <strong>of</strong> September 1986, or between at most 17-34 weeks<br />
after December 1985.<br />
2.2.2 An overview <strong>of</strong> SYETP changes<br />
The following Table 2.4 allows an overview <strong>of</strong> <strong>the</strong> changes to SYETP programme<br />
definition over time. As a result <strong>of</strong> <strong>the</strong> variations introduced <strong>the</strong> programme became<br />
increasingly complex. The age and unemployment duration necessary for eligibility<br />
altered over time, as did <strong>the</strong> subsidy period and payment.<br />
While ostensibly a flat rate, SYETP did have some tiers introduced. Tiers can alter <strong>the</strong><br />
target group as it leads to favouring <strong>of</strong> a particular group through more advantageous<br />
subsidy, but in practice <strong>the</strong> effect might not be as expected due to <strong>the</strong> interaction with<br />
o<strong>the</strong>r labour market features. A key aspect <strong>of</strong> change to <strong>the</strong> programme is that at various<br />
stages, <strong>the</strong> subsidy rate paid varied by age, as did <strong>the</strong> unemployment benefit rate paid.<br />
However, while early on <strong>the</strong>re was no age variation for <strong>the</strong> subsidy but unemployment
49<br />
benefits were paid at a slightly higher rate to older ages, later both <strong>the</strong> age variation in<br />
unemployment benefit and SYETP subsidy grew to be quite unfavourable for 15-17 year<br />
olds relative to o<strong>the</strong>r ages. Yet this adds to <strong>the</strong> variation in award rate payments by age,<br />
sex and occupation/industry so that <strong>the</strong>re was a complex picture <strong>of</strong> variation in how <strong>the</strong><br />
subsidy comparison worked for 15-24 year olds. In practice <strong>the</strong> subsidy to 15-19 year<br />
olds was positive in relation to <strong>the</strong> proportionately higher share <strong>of</strong> <strong>the</strong> award rate it<br />
covered for this age group. The impact <strong>of</strong> this is discussed in <strong>the</strong> evaluation evidence to<br />
follow where it was found that for subgroups <strong>of</strong> <strong>the</strong> eligible target group, <strong>the</strong> subsidy fell<br />
particularly favourably with employers leading to a greater intensity <strong>of</strong> placements for<br />
<strong>the</strong>se groups. 22<br />
It becomes clear from Table 2.4 that <strong>the</strong> SYETP subsidy paid to employers was usually<br />
higher than <strong>the</strong> benefit payable to <strong>the</strong> unemployed participant. 23 When benefits shown in<br />
Table 2.4 are compared to <strong>the</strong> average award rate shown in Table 2.3, it can be seen that<br />
it is likely that participants were better <strong>of</strong>f remunerated under <strong>the</strong> wages subsidy than<br />
with benefits, although non-pecuniary related benefits cannot be accounted for in this<br />
comparison. As well, <strong>the</strong> age/gender/occupation variation in award rates is not obvious<br />
from Table 2.3, and <strong>the</strong>se would be important. However <strong>the</strong>se figures can be used to<br />
roughly indicate underlying reservation wages for this group.<br />
22 Girls 15-17 years old got SYETP mostly, as <strong>the</strong>se faced <strong>the</strong> lowest award rates, so <strong>the</strong> wage discount<br />
from <strong>the</strong> subsidy was greatest.<br />
23 In Australia in 1983, unemployment benefit was income tested on own income, but not parental benefit,<br />
and paid to those registered with <strong>the</strong> CES, payable to those unemployed who were able and willing to work<br />
and taking reasonable steps to obtain work. To gauge <strong>the</strong> scale <strong>of</strong> payments to youths: At June 1983 about<br />
300,000 15-24 year olds were paid unemployment benefit, benefits to 15-24 year olds were roughly<br />
estimated to cost $990 million in 1982/3 (DEYA (1983): xxix).
50<br />
Table 2.4 SYETP rates, period and target group/eligibility criteria 1976-December 1985<br />
Date <strong>of</strong> change Age <strong>Subsidy</strong> Rate<br />
$ per week<br />
Max weeks<br />
<strong>of</strong> subsidy<br />
Eligible group <strong>of</strong> unemployed<br />
CES registrants<br />
Unemployment<br />
benefit rate<br />
October 1976 15-17 58 26 15-19 school leavers<br />
36<br />
18-19 58 26<br />
41.25<br />
November 1976 15-17 59 26 All 15-19 at least 6 <strong>of</strong> last 12 36<br />
18-19 59 26<br />
months not in fulltime education 43.50<br />
August 1977 15-17 63 26 All 15-24 at least 6 <strong>of</strong> last 12 36<br />
18-24 63 26<br />
months not in fulltime education 47.10<br />
October 1977 15-17 66 26 All 15-24 at least 4 <strong>of</strong> last 12 36<br />
18-24 66 26<br />
months not in fulltime education 49.30<br />
August 1978 15-17 45 17 No change<br />
36<br />
18-24 45 17<br />
51.45<br />
November 1978 15-17 50 17 No change<br />
36<br />
18-24 50 17<br />
51.45<br />
April 1980 15-17 50 17 No change but if unemployed for 36<br />
18-24 50 17<br />
4 months continuously after 51.45<br />
SYETP <strong>the</strong>n eligible for a second<br />
SYETP placement<br />
November 1980 15-17 55 17 No change but for ex-STWTP 24 36<br />
18-24 55 17<br />
participants who were <strong>the</strong>n 53.45<br />
immediately eligible for a SYETP<br />
placement<br />
February 1981 15-17 55 standard rate 17 Extended SYETP introduced for 36<br />
18-24<br />
18-24<br />
55 standard rate<br />
80 extended<br />
17<br />
First 17 <strong>of</strong> 34<br />
18-24 unemployed at least 8 <strong>of</strong><br />
last 12 months; with 2 periods <strong>of</strong><br />
17 weeks, <strong>the</strong> first at <strong>the</strong> higher<br />
53.45<br />
53.45<br />
‘extended SYETP’ rate and <strong>the</strong><br />
second 17 weeks at <strong>the</strong> standard<br />
rate.<br />
August 1982 15-17 75 standard rate 17 No change<br />
36<br />
18-24 75 standard rate 17 58.10<br />
August 1983<br />
18-24 100 extended First 17 <strong>of</strong> 34<br />
58.10<br />
15-17 75 standard rate 17 No change to eligibility, but start 40<br />
18-24 100 standard rate 17 <strong>of</strong> age tiers to subsidy<br />
68.65<br />
18-24 75 extended First 17 <strong>of</strong> 34<br />
68.65<br />
August 1984- 15-17 50 standard rate 17 No change to eligibility<br />
45<br />
December 1985 18-19 75 standard rate 17 78.60<br />
20-24 100 standard rate 17 78.60<br />
18-19 50 extended First 17 <strong>of</strong> 34 45<br />
20-24 75 extended First 17 <strong>of</strong> 34<br />
78.60<br />
Source: Ross (1988) p39 Table 4. The benefit rate is in $ per week.<br />
Notes: The SYETP became part <strong>of</strong> Jobstart from January 1986. This table relates to private SYETP rates –<br />
Commonwealth SYETP mainly differed only in that <strong>the</strong> employer received <strong>the</strong> full wage costs for <strong>the</strong><br />
subsidy period. However see section 2.2.3 for o<strong>the</strong>r differences to private SYETP.<br />
24 School to Work Transition Program. This was a program made up <strong>of</strong> pre-employment education and<br />
training transition courses in Technical and Fur<strong>the</strong>r Education (TAFE) institutes – courses generally <strong>of</strong> 12<br />
weeks or up to 52 weeks full-time. From October 1980 STWTP was supported by a ‘transition allowance’,<br />
but until <strong>the</strong>n participants were not eligible for unemployment benefit (Kesteven (1987): 57).
51<br />
2.2.3 SYETP operation<br />
Earlier references to SYETP explained it in various ways. SYETP was initially described<br />
as a training programme that was part <strong>of</strong> <strong>the</strong> National Employment and Training System<br />
(NEAT 25 ), in a 1980 information pamphlet for CES services and Manpower Programmes<br />
(DEYA (1980)). In 1982 SYETP was listed amongst manpower programmes as a work<br />
experience programme, which was part <strong>of</strong> <strong>the</strong> <strong>Youth</strong> Training Programmes, with <strong>the</strong><br />
following remit:<br />
“The work experience programme helps employers (both private and<br />
Commonwealth) take on young people who have found it difficult to get<br />
stable employment because <strong>the</strong>y lack <strong>the</strong> required experience and<br />
qualifications by providing a subsidy for <strong>the</strong>ir employment.” (Paterson (1982)<br />
Appendix 1, p3).<br />
However, research documenting <strong>the</strong> programmes pointed out that <strong>the</strong> SYETP was<br />
essentially a wage subsidy to employers to take on unemployed young people (BLMR<br />
(1983): 2). In this same research, SYETP was also defined as a ‘work experience<br />
programme’. Yet <strong>the</strong> real emphasis was never on training. In 1984 programme conditions<br />
for employers included <strong>the</strong> development <strong>of</strong> a training plan for <strong>the</strong> new employee,<br />
however this seems <strong>the</strong> greatest extent <strong>of</strong> training under SYETP. Smith (1983) pointed<br />
out that this training plan could cover normal orientation for new employees. Thus it was<br />
a fairly straightforward employment subsidy programme with no real training provisions<br />
attached.<br />
In a submission to <strong>the</strong> OECD, <strong>the</strong> department in charge <strong>of</strong> administering manpower<br />
programmes described SYETP as “…one <strong>of</strong> <strong>the</strong> major programmes designed to improve<br />
access to and equity in <strong>the</strong> labour market”, one <strong>of</strong> two toge<strong>the</strong>r with <strong>the</strong> job creation<br />
programme CEP (Community Employment Programme). It was fur<strong>the</strong>r described as<br />
“…aimed at improving <strong>the</strong> employability <strong>of</strong> longer-term unemployed young people aged<br />
25 An active labour market ‘umbrella’ program, NEAT consisted <strong>of</strong> a mixture <strong>of</strong> separate training and wage<br />
subsidy programs.
52<br />
15-24…employers are expected to provide work experience but <strong>the</strong>re is no obligation on<br />
<strong>the</strong>m to retain subsidised young people at <strong>the</strong> end <strong>of</strong> <strong>the</strong> subsidy period…<strong>the</strong> subsidies …<br />
provide employers with a greater incentive to take on people...” (DEYA (1983) synopsis:<br />
xxvii).<br />
Employers received <strong>the</strong> weekly subsidy, and it was described in 1980 as a form <strong>of</strong> on-<strong>the</strong>job-training<br />
where “Trainees must be 15 to 24 years <strong>of</strong> age; unemployed for 4 <strong>of</strong> <strong>the</strong> last<br />
12 months; away for full-time education for 4 months in <strong>the</strong> last 12 months; registered<br />
with <strong>the</strong> CES.” It was fur<strong>the</strong>r stipulated that “Employers must provide proper trainee<br />
supervision and pay <strong>the</strong> trainee <strong>the</strong> award or going rate.” Although <strong>the</strong> chief aim was<br />
private employers, Commonwealth and State government could also participate in<br />
SYETP. There was no rule on retention <strong>of</strong> <strong>the</strong> SYETP participant after <strong>the</strong> subsidy ended,<br />
however “…<strong>the</strong>re was an assumption that employers o<strong>the</strong>r than Commonwealth<br />
Departments and Authorities would retain <strong>the</strong> participants after <strong>the</strong> subsidy period or that<br />
participants would at least be able to obtain jobs on <strong>the</strong> basis <strong>of</strong> <strong>the</strong>ir programme<br />
experience” (Baker (1984): 15).<br />
SYETP operated as a 17 week placement for work-experience and on-<strong>the</strong>-job training,<br />
with a limited wage subsidy to employers (BLMR (1983) p6, Table 1.3). As <strong>the</strong> section<br />
tracing <strong>the</strong> historical development <strong>of</strong> SYETP shows, <strong>the</strong> subsidy limit changed in<br />
response to new policy directives and to keep in line with price/wage increases. In some<br />
cases, a ‘second-serve’ <strong>of</strong> SYETP operated as a follow-on to SYETP – where a person<br />
who had been on SYETP, but <strong>the</strong>n following this was unemployed for 4 consecutive<br />
months, <strong>the</strong>y <strong>the</strong>n became eligible for a second 17 week placement. 26 There were several<br />
variations <strong>of</strong> SYETP, which existed in a small number <strong>of</strong> cases.<br />
For <strong>the</strong> Commonwealth Government employers, <strong>the</strong> subsidy covered full reimbursement<br />
<strong>of</strong> wage costs. For Commonwealth employers, placements were not subject to any ‘staff<br />
26 BLMR (1983) p33 and footnote 4 on page 35; Baker, S. (1984) p7 refers to this as a ‘second-serve’.
53<br />
ceiling constraints’ applied to Commonwealth Departments at <strong>the</strong> time. 27 Fur<strong>the</strong>rmore,<br />
<strong>the</strong>re were restrictions on dismissal with a time limit so that after <strong>the</strong> first 2 weeks <strong>of</strong><br />
placement, any placement dismissed could not be replaced. 28 Entrants to Commonwealth<br />
SYETP were restricted though and could not have already experienced a placement lost<br />
due to dismissal or voluntary withdrawal, <strong>the</strong>reby precluding people with former negative<br />
experiences. Additionally, <strong>the</strong> whole cost <strong>of</strong> <strong>the</strong> award wage paid to <strong>the</strong> participant was<br />
recovered from SYETP, while for private SYETP only <strong>the</strong> payment limit was paid to <strong>the</strong><br />
employer by SYETP (Baker (1984): 6).<br />
‘Extended SYETP’ was also available, which was a slight alteration to <strong>the</strong> target group<br />
and subsidy structure. Extended SYETP was for those slightly older, over school age, and<br />
unemployed for a longer period. The structure <strong>of</strong> <strong>the</strong> subsidy for <strong>the</strong> Extended SYETP<br />
was changed from a flat rate for a single period to a two-tiered rate with a higher rate in<br />
<strong>the</strong> earlier period. The time tiered subsidy was $80 for <strong>the</strong> first 17 week period and <strong>the</strong>n<br />
$55 for <strong>the</strong> second 17 week period, in 1983. Although <strong>the</strong> subsidy was paid to employers,<br />
<strong>the</strong> subsidy entitlement period was allocated to ‘trainees’, not to <strong>the</strong> placement, so if <strong>the</strong>y<br />
left or were dismissed before <strong>the</strong> expiry <strong>of</strong> <strong>the</strong> period, <strong>the</strong>n <strong>the</strong>y retained eligibility for<br />
ano<strong>the</strong>r placement position for <strong>the</strong> unused balance <strong>of</strong> <strong>the</strong> original period. This was termed<br />
a split placement (BLMR (1983) p6, Table 1.3, footnote (d)). Table 2.5 shows <strong>the</strong> key<br />
SYETP provisions at <strong>the</strong> time <strong>of</strong> analysis. The delivery arrangements required employers<br />
to respond to <strong>the</strong> programme and provide placements through <strong>the</strong> national network <strong>of</strong><br />
CES <strong>of</strong>fices operated by <strong>the</strong> Department <strong>of</strong> Employment and Industrial Relations. The<br />
CES was both <strong>the</strong> administrative and delivery mechanism, with CES <strong>of</strong>fices at local level<br />
dealing with both <strong>the</strong> employer and <strong>the</strong> ‘trainee’.<br />
Employers could specify a vacancy was exclusively available for an SYETP placement<br />
(AIMA (1985): 90). O<strong>the</strong>r operational procedures were also noted by AIMA (1985).<br />
Vacancies for SYETP placements could be on a special SYETP self-service notice-board,<br />
27 BLMR (1983) section 3 p17 footnote 2. Additionally, entrants to Commonwealth SYETP were restricted<br />
and could not have already experienced a placement lost due to dismissal or voluntary withdrawal<br />
(precluding people with former negative experiences). However, after <strong>the</strong> first 2 weeks <strong>of</strong> placement, any<br />
placement dismissed could not be replaced. Dismissal was <strong>the</strong>n uncommon.<br />
28 Dismissal was <strong>the</strong>n uncommon for Commonwealth SYETP as can be seen in section 2.2.6.
54<br />
or on <strong>the</strong> general CES vacancy display board. Alternatively, SYETP placements could be<br />
made by nomination <strong>of</strong> <strong>the</strong> employer, or self-canvassing by jobseekers from general<br />
vacancies. An informal estimate suggests that in 1984 60 per cent <strong>of</strong> SYETP placements<br />
were from those displayed on <strong>the</strong> SYETP self-service display board (AIMA (1985): 90).<br />
The display board self-service process was <strong>the</strong> same as for general vacancies, but in <strong>the</strong><br />
interview and referral process for an SYETP vacancy <strong>the</strong> CES <strong>of</strong>fice had to confirm <strong>the</strong><br />
applicant’s eligibility for <strong>the</strong> subsidy with an initial screening. A subsidy agreement had<br />
to be signed between <strong>the</strong> CES and <strong>the</strong> employer, <strong>the</strong> CES administered <strong>the</strong> subsidy<br />
monies, and CES <strong>of</strong>ficers could make ‘supervisory visits’ to employers. CES <strong>of</strong>fices were<br />
directed to refer any eligible jobseeker, but after June 1984 <strong>the</strong>y were to select from<br />
registrants suitable for referral ‘those who have <strong>the</strong> longest periods <strong>of</strong> unemployment’. 29<br />
There is evidence that, at least in 1984, <strong>the</strong>re were limitations as to how many staff could<br />
be SYETP placements, or o<strong>the</strong>r ‘subsidy’ placements. 30 Edwards (1987) p86 notes that<br />
<strong>the</strong>re were limitations on <strong>the</strong> proportion <strong>of</strong> <strong>the</strong> employer’s workforce which could be<br />
employed from wage subsidy programmes from <strong>the</strong> Federal government and cites <strong>the</strong><br />
guidelines to CES 31 as stating that at any one time, a single physical location <strong>of</strong> factory,<br />
shop or <strong>of</strong>fice with 1-3 staff could have 1 placement; those with 4-7 staff could have 2<br />
placements; for 8-100 staff not more than 25 per cent <strong>of</strong> staff could be placements; for<br />
more than 100 staff not more than 10 per cent <strong>of</strong> staff could be placements.<br />
There is also evidence that <strong>the</strong> CES were vigilant for abuse <strong>of</strong> wage subsidy programmes<br />
by employers. Edwards (1987) p86 cites CES guidelines about programme subsidies with<br />
regard to possible termination <strong>of</strong> a programme subsidy if it was alleged that employers<br />
dismissed existing employees to hire <strong>the</strong> placement, or <strong>the</strong>re was evidence an employer<br />
was seeking use <strong>of</strong> <strong>the</strong> programme for an on-going supply <strong>of</strong> subsidised labour, or where<br />
<strong>the</strong> employer sought to employ a former employee through <strong>the</strong> placement, or where<br />
29 AIMA (1985) p90 citing DEIR directive minute 84/55, 27 June 1984.<br />
30 Note that apprenticeship subsidy payments under CRAFT were considered by <strong>the</strong> government at <strong>the</strong> time<br />
as a wage ‘subsidy’, although <strong>the</strong>y are classified in this analysis as ‘training’. The AWSS Adult <strong>Wage</strong><br />
<strong>Subsidy</strong> Scheme operated from 1 March 1983 until 30 November 1985 as well.<br />
31 Department <strong>of</strong> Employment and Industrial Relations (1984) Adult <strong>Wage</strong> <strong>Subsidy</strong> Scheme CES Operating<br />
Manual, Volume 6 section 5; Volume 7.
55<br />
Award Conditions for employment were being breached. Edwards (1987) p87 also notes<br />
that for <strong>the</strong> Wollongong and Newcastle areas, evidence for such employer behaviour with<br />
regard to subsidies was ‘fairly minimal’. However, in practice <strong>the</strong> responsibility was<br />
<strong>the</strong>re, but CES <strong>of</strong>ficers’ monitoring would be limited and discretional.<br />
Table 2.5 Key SYETP provisions in 1983/84<br />
Target group <strong>Subsidy</strong> structure and period Eligible positions<br />
SYETP 32 , private 33<br />
Semi-skilled<br />
employer<br />
Extended SYETP<br />
15-24 years,<br />
unemployed<br />
registered with<br />
CES and away<br />
from fulltime<br />
education for 4 <strong>of</strong><br />
<strong>the</strong> past 12 months<br />
18-24 years<br />
registered with<br />
CES; unemployed<br />
and away from<br />
fulltime education<br />
for 8 <strong>of</strong> <strong>the</strong> past 12<br />
months<br />
Source: BLMR (1983) p6, Tables 1.2 and 1.3.<br />
flat rate for 17 weeks, for all<br />
placements in <strong>the</strong> age group<br />
(some age tiers from 1984)<br />
34 weeks: First 17 weeks higher<br />
rate, second 17 weeks lower rate<br />
Semi-skilled<br />
2.2.4 Political environment <strong>of</strong> SYETP<br />
SYETP operated across a long period <strong>of</strong> time, with a varying political background. As<br />
Stretton and Chapman (1990) p1 and section 3 point out, <strong>the</strong> expenditure on programmes<br />
and <strong>the</strong> mix <strong>of</strong> programmes <strong>of</strong>fered changed over time due to differing government<br />
economic philosophies, observed changes in <strong>the</strong> unemployment structure and rate, and<br />
ambiguity about <strong>the</strong> role <strong>of</strong> programmes and <strong>the</strong>ir effectiveness. Despite this, SYETP<br />
continued to operate, although with resulting changes to size and operation, as<br />
summarized in o<strong>the</strong>r sections.<br />
Ross (1988) identifies SYETP with two key periods <strong>of</strong> government, distinguished by <strong>the</strong><br />
Prime Ministers, which led <strong>the</strong>m. SYETP started during <strong>the</strong> Fraser Government (1975-<br />
1983) – a Liberal government led by Malcolm Fraser, and continued into <strong>the</strong> time <strong>of</strong> <strong>the</strong><br />
32 Also known as Standard SYETP.<br />
33 Also termed non-Commonwealth SYETP.
56<br />
Hawke Government, a Labour government led by Bob Hawke, from 1983. Harris (2001)<br />
also analyses in retrospect <strong>the</strong> political context for social policy during <strong>the</strong> SYETP period<br />
<strong>of</strong> operation. According to Harris (2001), during <strong>the</strong> Fraser period, SYETP operated<br />
within <strong>the</strong> context <strong>of</strong> deflationary budgets, except for during 1981-1982. Harris (2001)<br />
claims that little policy was implemented to combat unemployment, with a policy-making<br />
background where it was widely held within Treasury that unemployment levels could<br />
play a role in dampening inflationary expectations. Labour market programmes were<br />
adopted to deal with unemployment from a social perspective.<br />
The Fraser period was however a time during which an effort was made to develop a<br />
number <strong>of</strong> labour market programmes, <strong>of</strong> which SYETP was one within <strong>the</strong> NEAT 34<br />
umbrella. The build up to <strong>the</strong> introduction <strong>of</strong> labour market programmes started between<br />
1973 and <strong>the</strong> end <strong>of</strong> 1975, when unemployment rose and it became clear that it would not<br />
be short-lived (Ross (1988): 29). The significant rise in unemployment was particularly<br />
amongst young unemployed (Hoy and Paterson (1983): 2). A direct link is drawn by Ross<br />
(1988) p29 between <strong>the</strong> introduction <strong>of</strong> <strong>the</strong>se “...small number <strong>of</strong> large-scale<br />
programmes...” and <strong>the</strong> <strong>the</strong>n recently held government inquiries into Overseas Manpower<br />
and Industry Policies (1973), Labour Market Training (1973) and Unemployment<br />
Statistics (1973). In reference to <strong>the</strong> Fraser period, Harris (2001) cites <strong>the</strong> introduction<br />
during this period <strong>of</strong> a 6 week waiting period for school leavers and those who had left<br />
<strong>the</strong>ir last job, and points out that unemployment benefit criteria were tightened alongside<br />
<strong>the</strong> labour market programmes. Chapman (1985) p101 refers to <strong>the</strong> balancing act played<br />
by SYETP in policy:<br />
“...<strong>the</strong> conservative government needed to be seen to be concerned with<br />
burgeoning youth unemployment, but at <strong>the</strong> same time, <strong>the</strong>re were perceived<br />
political risks associated with direct job creation programmes…<strong>Youth</strong><br />
oriented wage subsidies accomplished <strong>the</strong> twin goals <strong>of</strong> targeted employment<br />
assistance and private sector support”.<br />
34 National Employment and Training system, an active labour market program <strong>of</strong> training and wage<br />
subsidies.
57<br />
Harris (2001) claims that during <strong>the</strong> later Hawke regime, expansionary budgets were<br />
introduced, and citing Watts (1999) 35 also claims that <strong>the</strong>re was a ´willingness to spend´<br />
on labour market programmes but this mostly went to Job Creation, such as <strong>the</strong> CEP. An<br />
overview <strong>of</strong> how SYETP expenditure and placements varied across <strong>the</strong>se periods, indeed<br />
<strong>the</strong> life <strong>of</strong> <strong>the</strong> programme, can be gained from Table 2.6. This information, sourced from<br />
Ross (1988), gives details <strong>of</strong> <strong>the</strong> fluctuations in spending allocated to SYETP and<br />
numbers treated by <strong>the</strong> programme. It is clear <strong>the</strong> amount spent on SYETP doubled in<br />
1983/4 compared to <strong>the</strong> prior year, yet <strong>the</strong> SYETP share <strong>of</strong> labour market programme<br />
expenditure did not rise but fell slightly. To gauge <strong>the</strong> scale <strong>of</strong> SYETP programme<br />
expenditure to <strong>the</strong> cost <strong>of</strong> unemployment benefit payments to youths: at June 1983<br />
slightly more than 300,000 15-24 year olds were paid unemployment benefit, and<br />
benefits to 15-24 year olds were roughly estimated to cost $990 million in 1982/3 (DEYA<br />
(1983): xxix). In contrast, in <strong>the</strong> same year only $120 million was spent on SYETP. The<br />
expenditure on SYETP was <strong>the</strong>n not great relative to <strong>the</strong> general cost <strong>of</strong> supporting <strong>the</strong><br />
unemployed. This is despite <strong>the</strong> subsidy rate generally being higher than <strong>the</strong> benefit<br />
amount (see Table 2.4).<br />
35 Cited p 9, Harris (2001) referring to Watts, R. (1999) “The future <strong>of</strong> work: economists and employment<br />
policy 1983-1999” conference paper presented at <strong>the</strong> National Social Policy Conference Sydney, Social<br />
Policy Research Centre, University <strong>of</strong> New South Wales.
58<br />
Table 2.6 SYETP annual expenditure and placements1976/77-1985/86<br />
Political regime Year Expenditure<br />
($millions)<br />
Fraser government<br />
era<br />
Hawke government<br />
era<br />
Expenditure % <strong>of</strong><br />
total annual<br />
expenditure on<br />
labour market<br />
programmes<br />
Number <strong>of</strong><br />
placements<br />
1976/77 6.6 6.5 9,590<br />
1977/78 47.1 29.6 66,000<br />
1978/79 82.6 40.7 66,350<br />
1979/80 24.2 17.8 44,300<br />
1980/81 41.3 21.9 65,309<br />
1981/82 53.7 24.8 51,696<br />
1982/83 63.6 17.8 66,270<br />
1983/84 120.2 15.3 87,582<br />
1984/85 97.7 10.8 68,874<br />
1985/86 61.7 7.3 30,107<br />
Source: Ross (1988) p37 Table 3<br />
Note: <strong>the</strong> SYETP became part <strong>of</strong> Jobstart from January 1986 and 1985-6 figures reflect <strong>the</strong> end <strong>of</strong> <strong>the</strong><br />
programme under this name.<br />
2.2.5 Economic Context <strong>of</strong> SYETP at <strong>the</strong> time <strong>of</strong> our analysis<br />
The main aim <strong>of</strong> this section is to provide a syn<strong>the</strong>sis <strong>of</strong> research for selected relevant<br />
features <strong>of</strong> <strong>the</strong> <strong>Australian</strong> labour market for <strong>the</strong> period, in particular focusing on youths.<br />
2.2.5.1 Role and impact <strong>of</strong> <strong>the</strong> CES<br />
Wielgosz (1984) examined <strong>the</strong> referral and placement role <strong>of</strong> <strong>the</strong> Commonwealth<br />
Employment Service (CES). They noted that since establishment in 1946, <strong>the</strong> primary<br />
function <strong>of</strong> <strong>the</strong> <strong>Australian</strong> CES had been to match registered jobseekers and vacancies,<br />
but later <strong>the</strong> administration <strong>of</strong> manpower programmes was added to <strong>the</strong> brokerage<br />
function. CES effectiveness in executing such roles would affect <strong>the</strong> extent to which<br />
employers and jobseekers utilized <strong>the</strong> CES. In turn <strong>the</strong> CES performance in matching<br />
jobseekers and vacancies would be influenced by <strong>the</strong> characteristics and volume <strong>of</strong> <strong>the</strong>se<br />
received.<br />
Hoy and Lampe (1982) p6 noted that after 1977, efforts were made to improve <strong>the</strong> CES<br />
performance <strong>of</strong> <strong>the</strong> brokerage role. They listed two initiatives: job self-service, where
59<br />
display boards listed details <strong>of</strong> vacancies and jobseekers could select which to apply for;<br />
and improved systems circulating vacancies between CES <strong>of</strong>fices. Table 2.7 indicates <strong>the</strong><br />
use <strong>of</strong> CES by youths looking for work. Most youths were registered with <strong>the</strong> CES, and<br />
most indicated <strong>the</strong>y also took on active job search methods contacting employers. It was<br />
also observed by Hoy and Lampe (1982) that most married women were not eligible for<br />
unemployment benefits, which led to lower CES registration for that group.<br />
Table 2.7 Active steps to find work by youths looking for work in July 1980<br />
Active steps to find work Persons looking for work July 1980<br />
Unemployed males<br />
15-24 years<br />
Unemployed females<br />
15-24 years<br />
Registered with CES And took no o<strong>the</strong>r steps 4.3 6.9<br />
And applied to<br />
75.8 68.2<br />
prospective employers<br />
in person or by post or<br />
telephone<br />
And took o<strong>the</strong>r active 3.5 *<br />
steps<br />
Total registered 83.6 76.9<br />
Not registered with CES And applied to<br />
14.5 21.1<br />
prospective employers<br />
in person or by post or<br />
telephone<br />
And took o<strong>the</strong>r active * *<br />
steps<br />
Total not registered 16.4 23.1<br />
Total 100 100<br />
Source: Hoy and Lampe (1982) p34 Table 4 cited source ABS Cat 6222.0 ‘Persons looking for work,<br />
Australia’.<br />
A 1977 review <strong>of</strong> CES services to employers found limited employer use <strong>of</strong> <strong>the</strong> CES. It<br />
was established that employers estimated only 48 per cent <strong>of</strong> <strong>the</strong>ir current vacancies were<br />
notified to <strong>the</strong> CES. Limited occupations were related to <strong>the</strong> CES notified vacancies: 36<br />
per cent <strong>of</strong> clerical, administrative and sales worker vacancies were notified to <strong>the</strong> CES,<br />
and 50 per cent <strong>of</strong> vacancies for tradesmen. 36<br />
Wielgosz (1984) pointed out that <strong>the</strong> CES had no direct control over <strong>the</strong> volume or nature<br />
<strong>of</strong> vacancies received, and that only a minority <strong>of</strong> available vacancies was lodged with<br />
36 Hoy and Lampe (1982) citing Norgard, J.D. (1977) “Review <strong>of</strong> <strong>the</strong> Commonwealth Employment Service<br />
1977”, Report prepared for <strong>the</strong> Minister <strong>of</strong> Employment and Industrial Relations, <strong>Australian</strong> Government<br />
Printing Service, Canberra, pp253-254.
60<br />
<strong>the</strong> CES. In support <strong>of</strong> this <strong>the</strong>y cited statistics 37 reflecting that about 30 percent <strong>of</strong> all<br />
vacancies were registered with <strong>the</strong> CES. In addition to <strong>the</strong> volume constraint, <strong>the</strong>y noted<br />
that <strong>the</strong> majority <strong>of</strong> vacancies were concentrated in low-skilled occupations. In contrast to<br />
<strong>the</strong> low vacancy attraction, <strong>the</strong> majority <strong>of</strong> jobseekers made use <strong>of</strong> <strong>the</strong> CES. It was noted<br />
that 80 to 90 per cent <strong>of</strong> jobseekers looking for full-time work used <strong>the</strong> CES. 38 However,<br />
it was also made clear that although <strong>the</strong> majority registered with <strong>the</strong> CES, as registration<br />
was a condition for receipt <strong>of</strong> unemployment benefits, more than 90 per cent additionally<br />
used o<strong>the</strong>r methods, so it was not <strong>the</strong> sole method and likely not <strong>the</strong> predominant method<br />
for job search.<br />
Referral and placement 39 data for <strong>the</strong> Brisbane area, regarded as a good example <strong>of</strong><br />
average CES activities, was analysed by Wielgosz (1984). In examining referral to<br />
notified vacancies <strong>the</strong> underlying assumption about CES functioning in referral to<br />
vacancies was clarified as follows: where “…employers have certain preferences and<br />
expectations about <strong>the</strong> type <strong>of</strong> labour <strong>the</strong>y require, that <strong>the</strong> CES is aware <strong>of</strong> <strong>the</strong>se<br />
preferences and in it’s desire to fill vacancies and fill <strong>the</strong>m as quickly as possible, will<br />
refer those unemployed applications who are most likely to be placed easily and quickly”<br />
(Wielgosz (1984): 12).<br />
Using a probit econometric analysis, Wielgosz (1984) found referrals to have several<br />
statistically significant factors amongst applicant characteristics, being negatively related<br />
to age, educational attainment (years <strong>of</strong> schooling) and duration <strong>of</strong> unemployment<br />
(current spell in weeks) <strong>of</strong> applicants, with positive effects for applicants in<br />
clerical/administrative and skill trade occupations (relative to a base <strong>of</strong> all o<strong>the</strong>r<br />
occupations except semi-skilled and unskilled). Wielgosz (1984) separately modelled<br />
successful placement in a job, with a variable for <strong>the</strong> number <strong>of</strong> referrals received and<br />
found this significantly increased <strong>the</strong> probability <strong>of</strong> placement using <strong>the</strong> Heckman<br />
method to control for sample selection (Wielgosz (1984) p19 Table 7). No details <strong>of</strong> <strong>the</strong><br />
Heckman sample selection model are presented, and so it is not clear what exclusion<br />
37 Wielgosz (1984) p4 footnote 4 citing ABS Cat.6231.0 ‘Job vacancies, Australia’ various issued to 1984.<br />
38 Wielgosz (1984) p4 citing ABS Cat.6222.0 ‘Persons looking for work, Australia’ various issued to 1984.<br />
39 Note that placement here is in subsidised or unsubsidised jobs.
61<br />
restriction was used. If <strong>the</strong>re was no exclusion restriction applied, and identification was<br />
based solely on functional form, <strong>the</strong>n Monte Carlo evidence in <strong>the</strong> literature indicates that<br />
<strong>the</strong> modelling suffers from poor performance. The estimation procedure also lacks fur<strong>the</strong>r<br />
detail, and so it is not clear whe<strong>the</strong>r <strong>the</strong> two equations needed for <strong>the</strong> Heckman sample<br />
selection model were estimated simultaneously, but it appears that a two step procedure<br />
was applied. If both equations are estimated as probits, <strong>the</strong>n this is inappropriate due to<br />
<strong>the</strong> nonlinearity <strong>of</strong> <strong>the</strong> probit. It was found that <strong>the</strong> factors affecting referrals did not have<br />
an independent affect on placement. Duration <strong>of</strong> <strong>the</strong> current unemployment spell also had<br />
an additional negative effect on placement if selection bias was not controlled for.<br />
Wielgosz found that tests for significance <strong>of</strong> <strong>the</strong> selection correction factor indicated no<br />
selection bias. But it was concluded that <strong>the</strong> very high correlation between <strong>the</strong> selection<br />
correction factor and <strong>the</strong> duration <strong>of</strong> unemployment, coupled with <strong>the</strong> significance <strong>of</strong> this<br />
variable when selection bias was not controlled for, suggested that that <strong>the</strong> sample<br />
selection bias was “…closely and solely related to duration <strong>of</strong> unemployment” (Wielgosz<br />
(1984): 19). It was fur<strong>the</strong>r noted that this was because to be referred, and so included in<br />
<strong>the</strong> sub-sample, was almost completely dominated by <strong>the</strong> length <strong>of</strong> unemployment spell.<br />
As a result, due to <strong>the</strong>ir correlation <strong>the</strong> coefficients for selection correction and<br />
unemployment duration were both insignificant in <strong>the</strong> placement equation because <strong>the</strong>y<br />
reflected <strong>the</strong> same phenomenon.<br />
Aungles and Stewart (1986) used <strong>the</strong> same referral and placement data as Wielgosz (1984)<br />
to examine referrals and <strong>the</strong> duration <strong>of</strong> unemployment, toge<strong>the</strong>r with exits from<br />
unemployment. Aungles and Stewart (1986) modelled durations <strong>of</strong> unemployment, and<br />
also found that <strong>the</strong> number <strong>of</strong> referrals jobseekers receive is related to <strong>the</strong> probability <strong>of</strong><br />
leaving unemployment. They found that jobseekers receipt <strong>of</strong> CES referrals were most<br />
likely to occur early in <strong>the</strong>ir unemployment spell, but once referred by <strong>the</strong> CES,<br />
differences did not exist in <strong>the</strong> probability <strong>of</strong> leaving unemployment. The hazard function<br />
for leaving unemployment initially rose to peak at 11 days, less than two weeks, after<br />
which it fell.
62<br />
A regional analysis <strong>of</strong> NSW registrations <strong>of</strong> unemployment and vacancies with <strong>the</strong> CES<br />
was conducted in 1986 by McGillicuddy et al. (1986). Over <strong>the</strong> year from June 1985 to<br />
June 1986, CES registrations had risen by 1.1 per cent. It was noted that regional analyses<br />
<strong>of</strong> <strong>the</strong> distribution <strong>of</strong> registered unemployed and vacancies emphasized geographical<br />
limitations to <strong>the</strong> functioning <strong>of</strong> <strong>the</strong> labour markets within NSW. This analysis could be<br />
extended to Australia as related to rural/urban features. The share <strong>of</strong> notified vacancies<br />
per unemployed was higher in Sydney (0.26 vacancies per person) than o<strong>the</strong>r NSW areas<br />
(0.13 vacancies per person) and <strong>the</strong> unfilled vacancies followed a similar pattern (Sydney<br />
0.06 unfilled vacancies per person; rest <strong>of</strong> NSW 0.03) (McGillicuddy et al. (1986): 6).<br />
Most unemployment registrations were for older age groups in <strong>the</strong> periods studied, with<br />
youths taking a lower share, see Table 2.8. However it was clear that those 15-19 years<br />
had lower registrations in Sydney than <strong>the</strong> rest <strong>of</strong> NSW, but those 20-24 years always had<br />
lower shares outside Sydney than within Sydney.<br />
Table 2.8 Age distribution for CES registrants in NSW 1985-1986<br />
Sydney<br />
Rest <strong>of</strong> NSW<br />
June 1985 June 1986 June 1985 June 1986<br />
15-19 19.5 19.1 22.5 21.8<br />
20-24 25.1 24.3 23.7 22.6<br />
25-44 41.1 42.6 39.8 41.4<br />
45+ 14.3 13.8 13.9 14.2<br />
Source: McGillicuddy et al. (1986) p6 Table 2. Columns add to 100 per cent.<br />
2.2.5.2 Unemployment in <strong>the</strong> <strong>Australian</strong> economy over <strong>the</strong> 1980’s<br />
Labour supply and demand factors, overall unemployment and <strong>the</strong> business cycle, and<br />
relative youth wages are factors that can be viewed as having a significant impact on<br />
youth unemployment. They also play a role in <strong>the</strong> operation <strong>of</strong> wage subsidies, as<br />
discussed in chapter 1. These issues are now considered for <strong>the</strong> <strong>Australian</strong> economic<br />
background to <strong>the</strong> operation <strong>of</strong> <strong>the</strong> SYETP.<br />
The <strong>Australian</strong> economy encountered a peak in <strong>the</strong> unemployment rate during <strong>the</strong> period<br />
<strong>of</strong> our data analysis, followed by improvements with falling unemployment from 1985 on
63<br />
to <strong>the</strong> end <strong>of</strong> <strong>the</strong> 1980’s. An overview <strong>of</strong> <strong>the</strong> change to unemployment across this period<br />
is shown in Table 2.9. It can be seen that <strong>the</strong> unemployment rate for those aged 15-19<br />
years was substantially higher than for <strong>the</strong> general population, but that it also fluctuated<br />
in line with <strong>the</strong> business cycle. In <strong>the</strong> context <strong>of</strong> <strong>the</strong> <strong>Australian</strong> business cycle, it was<br />
generally considered that <strong>the</strong> period from late 1981 to mid-1983 was a recession (see for<br />
example EPAC (June 1992) p14 and Windschuttle (1985) p3).<br />
Teenage unemployment rose dramatically between 1981 and 1983. Although teen<br />
participation rates may also have changed with more teenagers remaining in education<br />
(see later discussion) <strong>the</strong>se fluctuations indicate that <strong>the</strong> share <strong>of</strong> teens looking for work<br />
was considerable over this period. Paterson and Mackay (1982a) make it clear that<br />
demographic growth was not a source <strong>of</strong> increased youth labour supply. They note that in<br />
Australia youths had lower than average rates <strong>of</strong> population increase.<br />
Table 2.9 Unemployment rate Australia 1981-1990, seasonally adjusted<br />
Per cent Total unemployment rate 15-19 years<br />
unemployment rate<br />
January July August<br />
1981 5.9 5.9 13.9<br />
1982 6.0 7.0 16.7<br />
1983 9.3 10.4 22.6<br />
1984 10.3 8.4 21.0<br />
1985 9.3 7.8 18.2<br />
1986 8.5 7.9 19.1<br />
1987 8.9 7.8 18.7<br />
1988 8.3 6.6 15.6<br />
1989 7.4 5.9 13.7<br />
1990 6.7 6.9 16.5<br />
Source: Sheen and Tre<strong>the</strong>wey (1991) Total unemployment: p12 Table 2 Cited from sources ABS Cat<br />
6203.0 Labour Force Australia February 1986, February 1990, January 1991.15-19 years unemployment:<br />
Table “Labour force status <strong>of</strong> <strong>the</strong> civilian population aged 15-19: school attendance”.<br />
An additional understanding <strong>of</strong> unemployment during this period can be gained from <strong>the</strong><br />
distribution <strong>of</strong> unemployment durations amongst <strong>the</strong> various age groups can be seen from<br />
Table 2.10. <strong>Youth</strong>s in general usually had lower durations <strong>of</strong> unemployment than older<br />
groups, however school leavers would limit <strong>the</strong> average duration for teens. Over <strong>the</strong><br />
changing business cycle, a rise in <strong>the</strong> share <strong>of</strong> newly unemployed would lower <strong>the</strong><br />
unemployment duration average for each group.
64<br />
Table 2.10 Average duration <strong>of</strong> unemployment (number <strong>of</strong> weeks) by age, August 1981-<br />
1990<br />
1981 1982 1983 1984 1985 1986 1987 1988 1989 1990<br />
15-19 25 23 32 29 29 30 33 28 26 23<br />
20-24 32 32 42 46 46 43 41 46 34 33<br />
25-34 31 28 40 45 50 43 45 53 37 40<br />
35-44 38 40 44 55 50 57 52 54 56 57<br />
45-54 56 52 56 61 73 85 82 71 80 60<br />
55-59 69 46 63 79 86 89 86 115 84 99<br />
60-64 83 58 60 67 123 98 121 91 107 126<br />
Source: Sheen and Tre<strong>the</strong>wey (1991) citing unpublished data ABS Cat 6203.0 Labour Force Australia.<br />
Analysis <strong>of</strong> unemployment by educational attainment in February 1991 also found that<br />
<strong>the</strong> unemployment rate was clearly higher for those with less formal education, especially<br />
young people. 40 It was noted that unemployment rates for those 20-24 who did not finish<br />
schooling to year 12 was 20 per cent, which was about twice that <strong>of</strong> those who did finish<br />
schooling. Overall, <strong>the</strong> observed youth unemployment rate was 23 per cent for those 15-<br />
24 years without schooling to year 12.<br />
Ross (1988) usefully examined in detail <strong>the</strong> relative performance <strong>of</strong> teenagers in <strong>the</strong><br />
labour force across <strong>the</strong> period 1983-1988. The distribution <strong>of</strong> labour force activities for<br />
teenagers and <strong>the</strong> total working age population for Australia can be seen in Table 2.11.<br />
Teenagers had different activity patterns to <strong>the</strong> rest <strong>of</strong> <strong>the</strong> working age population in<br />
general. Ross (1988) points out that for teens, while employment rose across <strong>the</strong> period,<br />
and unemployment fell, <strong>the</strong>ir labour force participation had changed.<br />
Particularly for male teens <strong>the</strong>re was a rise in <strong>the</strong> school participation rate, and Ross<br />
(1988) made clear <strong>the</strong> change to <strong>the</strong> education and employment decision was an<br />
important aspect <strong>of</strong> <strong>the</strong> teen labour market over <strong>the</strong> period. A breakdown in employment<br />
showed that full-time employment for teens fell, due to a decline in female employment,<br />
and part-time employment rose around 60 per cent for teens (Ross (1988): 4). It was<br />
found that school student’s labour force participation had risen, and while employment<br />
40 EPAC (1992) p 14 Chart 1.7 and citing ABS CAT 6235.0 Labour force states and educational attainment,<br />
Australia February 1991.
65<br />
for teens overall had risen, employment rose most for school students, whose<br />
employment was mostly part time 41 averaging 4-6 hours a week – <strong>the</strong> Saturday job effect<br />
(Ross (1988): 11).<br />
Ross (1988) found <strong>the</strong> most suitable interpretation was that employers were placing<br />
greater reliance on casual and part-time labour while teens were aware <strong>of</strong> lack <strong>of</strong><br />
experience and qualifications affecting employment chances. Thus, teens were remaining<br />
in school longer, and accepting part-time employment as a first-step to full-time<br />
employment, which gave work-experience. The growth in in-school employment was<br />
interpreted as an attempt to combine work experience with higher educational attainment.<br />
Stricker and Sheehan (1981) and Merrilees (1981), in commenting on <strong>the</strong> rise in<br />
schooling participation, noted that those who wanted a job but were unable to find one<br />
could remain in school, and <strong>the</strong> state <strong>of</strong> <strong>the</strong> youth labour market had thus affected<br />
schooling rates.<br />
41 Fulltime employment was defined as 35 hours or more per week, with part-time less than 35 hours per<br />
week.
66<br />
Table 2.11 Unemployment, labour force participation and employment rates, teenagers<br />
and total working age population March 1983 and March 1988.<br />
Rate as<br />
percent<br />
15-19 All working age<br />
male female all male female all<br />
March Unemployment 23.2 25.6 24.3 9.9 11.2 10.4<br />
1983<br />
Labour force<br />
63.9 59.8 61.9 77.3 45.6 61.3<br />
participation<br />
Employment 49.1 44.5 46.9 69.6 40.5 54.9<br />
School<br />
35.2 40.0<br />
participation<br />
In school<br />
Unemployment 28.7 24.9<br />
Labour force 20.0 23.5<br />
participation<br />
Employment 14.3 17.6<br />
Out <strong>of</strong> school<br />
Unemployment 22.5 25.7<br />
Labour force 87.8 80.2<br />
participation<br />
Employment 68.0 59.6<br />
March Unemployment 18.2 21.2 19.7 7.3 9.1 7.5<br />
1988<br />
Labour force<br />
61.9 60.0 61.0 76.2 50.9 63.3<br />
participation<br />
Employment 50.6 47.3 49.0 70.6 46.3 58.2<br />
School<br />
40.9 41.3<br />
participation rate<br />
In school<br />
Unemployment 20.8 21.4<br />
Labour force 26.3 34.4<br />
participation<br />
Employment 20.8 27.0<br />
Out <strong>of</strong> school<br />
Unemployment 17.7 21.1<br />
Labour force 86.3 78.0<br />
participation<br />
Employment 71.1 61.5<br />
Source: Ross (December 1988) p2 table 1 citing ABS Cat 6203.0 Labour Force Australia March 1983 and<br />
March 1988. Also, selected statistics from Ross (December 1988) p5 Table 3 and p 12 Table 5.<br />
In concluding, Ross (1988) examined <strong>the</strong> value <strong>of</strong> unemployment benefits for teens. In<br />
<strong>the</strong> context <strong>of</strong> <strong>the</strong> value <strong>of</strong> benefits relative to prices and average weekly earnings, it was<br />
found that <strong>the</strong> real value <strong>of</strong> unemployment benefits had fallen only for those 16-17 years.<br />
The value <strong>of</strong> benefits relative to prices and average weekly earnings is shown in Table<br />
2.12. Webster (1997b) p10 modelled <strong>the</strong> Beveridge curve <strong>of</strong> Unemployment-Vacancies
67<br />
for Australia using data from <strong>the</strong> period 1978-1997. This analysis found that <strong>the</strong> ratio <strong>of</strong><br />
unemployment benefits to minimum award wages was an important shift variable <strong>of</strong> <strong>the</strong><br />
Beveridge curve, with a rise in <strong>the</strong> ratio <strong>of</strong> benefits to wages raising <strong>the</strong> unemployment<br />
rate. The relative values <strong>of</strong> employment and unemployment activities in <strong>the</strong> labour<br />
market seem to have been detrimentally influencing youth labour participation.<br />
Table 2.12 Value <strong>of</strong> unemployment benefits to youths 1983-1987<br />
16-17 years 18-20 years<br />
Benefits /CPI Benefits /AWE Benefits /CPI Benefits /AWE<br />
1983 100.0 100.0 100.0 100.0<br />
1984 106.3 105.3 108.0 106.8<br />
1985 101.8 97.8 114.0 109.6<br />
1986 103.6 104.8 113.5 115.1<br />
1987 94.6 100.9 107.2 114.8<br />
Source: Ross (December 1988) p14 CPI is <strong>the</strong> Consumer Price Index for all goods weighted average for 8<br />
capital cities ABS Cat 6401.0. AWE is average weekly earnings from annual survey <strong>of</strong> earnings.<br />
Paterson and Mackay (1982) note that <strong>the</strong> share <strong>of</strong> youths aged 15-20 years in public<br />
sector employment suffered a large fall <strong>of</strong> 33 per cent from 1971 to 1981. Over <strong>the</strong> same<br />
period, public sector employment grew strongly by 29 per cent whereas private sector<br />
employment rose by only 8 per cent across this period, and in 1981 public sector<br />
employment was 30 per cent <strong>of</strong> total employment (Paterson and Mackay (1982): 28). A<br />
large part <strong>of</strong> this was likely due to <strong>the</strong> public sector refocusing employment towards<br />
graduates and skilled workers ra<strong>the</strong>r than taking on school-leavers and allowing<br />
'working-up-<strong>the</strong>-ranks'. This would have led to a contraction in what was a substantial<br />
part <strong>of</strong> <strong>the</strong> youth employment market.<br />
The role <strong>of</strong> relative labour costs can have particular relevance to <strong>the</strong> operation <strong>of</strong> a wage<br />
subsidy. Windshuttle (1985) presented statistics for <strong>the</strong> ratio <strong>of</strong> average hourly earnings<br />
for adults relative to juniors, see Table 2.3 earlier. In 1983, for full-time private sector<br />
employees <strong>the</strong> ratio <strong>of</strong> average hourly earnings for adults relative to juniors was 52.7 for<br />
males and 63.4 for females. For females, this was a fall in <strong>the</strong> ratio from 66.6 in 1977,<br />
while for males it was also a fall but from 55.2 in 1977. Windshuttle (1985) p6 notes that<br />
public sector wages were higher than those for private sector over <strong>the</strong> same period.<br />
Windshuttle (1985) also observed that <strong>the</strong>re was a greater difference in relative hourly
68<br />
earnings between juniors and adults in <strong>the</strong> private sector. However, this provides no<br />
modelling evidence that <strong>the</strong>se wage differentials were important factors influencing youth<br />
unemployment.<br />
More formal modelling by Lewis (1983) examined <strong>the</strong> role <strong>of</strong> relative wages in <strong>the</strong><br />
substitution between young and adult workers in Australia. It was concluded that for<br />
young males, demand was not particularly elastic to any o<strong>the</strong>r group except adult females,<br />
while young females had high demand elasticity relative to any adult workers. Thus it<br />
was suggested a change in labour costs would have a significant effect on young women<br />
and unskilled young men. The SYETP wage subsidy would introduce such a change in<br />
<strong>the</strong> relative costs for <strong>the</strong> eligible groups. The main restriction on substitutability foreseen<br />
by Lewis (1983) was reasoned to be skills and work experience.<br />
Miller and Volker (1987) examined skills and work experience as <strong>the</strong> determinants <strong>of</strong><br />
youth earnings. They found that education and experience were major determinants <strong>of</strong><br />
youth earnings. They also found that recent unemployment did not have a strong<br />
influence on earnings for youths. They reasoned that this was due to institutional features<br />
and trade union effects on wages and “…while <strong>the</strong> unemployment record appears to be<br />
used as a cheap screen at <strong>the</strong> hire stage, it does not have much impact on wage<br />
determination” (Miller and Volker (1987): 35). A wage subsidy like SYETP can help<br />
address <strong>the</strong> screening problem by allowing a low cost trial period for unemployed. As<br />
SYETP operated in <strong>the</strong> period <strong>the</strong>y analysed, it could have contributed to <strong>the</strong> weak<br />
influence <strong>of</strong> unemployment on wages by influencing human capital <strong>of</strong> participants<br />
through on <strong>the</strong> job training.<br />
Daly (1991) examined <strong>the</strong> returns to experience in a comparison <strong>of</strong> formal education and<br />
on-<strong>the</strong>-job training by modelling earnings in Australia 1981. It was found that <strong>the</strong><br />
experience pr<strong>of</strong>iles <strong>of</strong> different formal education groups converged. The conclusion<br />
drawn was that formal education and on-<strong>the</strong>-job training were to some extent<br />
substitutable methods for <strong>the</strong> accumulation <strong>of</strong> human capital, and thus <strong>the</strong>y were<br />
substitute and not complementary activities. <strong>Wage</strong> subsidies, while not ostensibly
69<br />
training, can provide a form <strong>of</strong> on <strong>the</strong> job training simply through work experience (for<br />
example 2.2.1 notes that a condition <strong>of</strong> SYETP was a training plan). This is advantageous<br />
for <strong>the</strong> functioning <strong>of</strong> a wage subsidy since <strong>the</strong> unemployed who benefit from <strong>the</strong> subsidy<br />
can develop greater human capital and subsequent improvement in <strong>the</strong>ir labour market<br />
position (in turn allowing <strong>the</strong> wage subsidy to influence employment gains). As SYETP<br />
existed in this period, it is difficult to determine whe<strong>the</strong>r this aspect arose due to SYETP,<br />
or pre-existed it and provided SYETP with helpful conditions in which to act as conduit<br />
for <strong>the</strong> activation <strong>of</strong> human capital.<br />
2.2.5.3 Discussion<br />
The analysis that our study produces operated with <strong>the</strong>se background features to <strong>the</strong><br />
youth labour market. Over <strong>the</strong> period this study covers, 1984-1986, <strong>the</strong> <strong>Australian</strong> youth<br />
labour market had several interesting aspects. It faced institutionalised lower relative<br />
wages than adults. Although school participation rose, overall youth unemployment was<br />
high relative to adults, but generally youth unemployment was falling and employment<br />
was rising. Regional and rural/urban differences existed in <strong>the</strong> incidence <strong>of</strong><br />
unemployment. Educational attainment was related to patterns <strong>of</strong> unemployment.<br />
Amongst youths, teens had shorter average spells <strong>of</strong> unemployment because <strong>the</strong>y were<br />
mostly school leavers.<br />
CES efficacy had a strong influence on <strong>the</strong> path out <strong>of</strong> unemployment for registrants, and<br />
getting referrals was an integral factor. The CES faced a limited supply <strong>of</strong> registered<br />
vacancies, which seemed to favour lower skilled occupations. In reviewing <strong>the</strong> evidence<br />
from programme evaluation for OECD countries, Fay (1996) noted that <strong>the</strong> process itself,<br />
relating here to <strong>the</strong> CES administration, might be an important determinant <strong>of</strong> <strong>the</strong><br />
outcome from a programme (Fay (1996): 11). However it was also found in <strong>the</strong> review<br />
that <strong>the</strong> role <strong>of</strong> delivery was not usually included in impact studies <strong>of</strong> employment, and<br />
that while potentially important <strong>the</strong>y might be difficult or impossible to include in<br />
evaluations <strong>of</strong> net employment impact.
70<br />
Many <strong>of</strong> <strong>the</strong> labour market features described here would form part <strong>of</strong> <strong>the</strong> model<br />
assumptions in <strong>the</strong>oretically ascribing a role for a youth wage subsidy. There were<br />
minimum wages, which would give institutionalised inefficiency <strong>of</strong> <strong>the</strong> labour market.<br />
O<strong>the</strong>r factors, which have been reviewed here, would also have a role in affecting <strong>the</strong><br />
functioning <strong>of</strong> <strong>the</strong> SYETP subsidy, including lower relative youth labour costs, elastic<br />
demand for youths relative to adults, and substitutability <strong>of</strong> formal education for on <strong>the</strong><br />
job training in human capital. The <strong>the</strong>ory <strong>of</strong> wage subsidies outlined in Chapter 1<br />
indicates how <strong>the</strong>se might influence <strong>the</strong> potential for employment gains from a wage<br />
subsidy targeted at youths. Toge<strong>the</strong>r, <strong>the</strong>se features favour <strong>the</strong> suggestion that SYETP<br />
might have given employment gains to those eligible. However, it is impossible to<br />
determine whe<strong>the</strong>r <strong>the</strong>se features were caused by SYETP, or were merely <strong>the</strong> backdrop<br />
influencing SYETP operation. Fur<strong>the</strong>r consideration <strong>of</strong> <strong>the</strong> macro-economic environment<br />
is beyond <strong>the</strong> remit <strong>of</strong> this analysis.<br />
2.2.6 Context <strong>of</strong> SYETP environment and operation<br />
A brief exposition <strong>of</strong> useful research about <strong>the</strong> operation <strong>of</strong> SYETP is included as<br />
informative <strong>of</strong> <strong>the</strong> nature <strong>of</strong> SYETP close to <strong>the</strong> time <strong>of</strong> our analysis. The research<br />
included here is relevant to SYETP but was not an evaluation <strong>of</strong> employment, or used<br />
employer data, or was not based on micro-economic evaluation <strong>of</strong> individual data.<br />
2.2.6.1 General results for SYETP using o<strong>the</strong>r approaches<br />
Hoy and Paterson (1983) and <strong>the</strong> NEAT evaluation (BLMR (1984)) provide early BLMR<br />
analysis with data for <strong>the</strong> years from 1977 to 1980. Unfortunately, SYETP operation<br />
cannot be distinguished from o<strong>the</strong>r programmes and so this is not fur<strong>the</strong>r discussed.<br />
Smith (1984b) conducted a macro-econometric analysis <strong>of</strong> SYETP, where administrative<br />
data for SYETP participants was related to aggregate unemployment survey measures for<br />
youth’s unemployment and duration. The chief aim was to relate <strong>the</strong> number <strong>of</strong> subsidy<br />
placements to variations in youth unemployment over time. The result for estimates were<br />
compared to employer survey estimates, and <strong>the</strong> summary is shown in Table 2.13. The
71<br />
employer survey estimates were <strong>the</strong> shares observed for questions asking about what<br />
hiring choices employers would have made if <strong>the</strong> subsidy placements had not been<br />
available. A useful conclusion <strong>of</strong> <strong>the</strong> analysis <strong>of</strong> SYETP was that <strong>the</strong> following extreme<br />
macro-outcomes for <strong>the</strong> subsidy could be considered excluded: a total windfall gain or<br />
complete substitution <strong>of</strong> employees for SYETP placements; that all SYETP placements<br />
took place without substitution <strong>of</strong> o<strong>the</strong>r workers; a 100 percent gain in employment per<br />
subsidy to <strong>the</strong> target group or <strong>the</strong> economy.<br />
Table 2.13 Smith (1984b) summary <strong>of</strong> estimates <strong>of</strong> SYETP<br />
Immediate impact <strong>of</strong> 100 subsidised jobs on Employment<br />
Period <strong>of</strong> data Target group O<strong>the</strong>rs Aggregate<br />
Smith (1984) 1978-83 64 -16 49<br />
DEIR (1984) 42 1983 42 -30 12<br />
Hoy and Ryan 1981 43 -24 19<br />
(1984)<br />
DEYA (1980) 43 1979 94 -61 33<br />
Source: Smith (1984) p 8a Table 1<br />
2.2.6.2 Job characteristics<br />
SYETP jobs were mostly low-paid, had a high job turnover, and private SYETP<br />
placements were more commonly for 15-17 year old girls. 44 Over <strong>the</strong> period 1976-1982,<br />
approximately 90 percent <strong>of</strong> placements were for 15 to 19 year olds, although <strong>the</strong> eligible<br />
age group was 18-24 years (Hoy and Lampe (1982): 24). Over this same period, <strong>the</strong> share<br />
<strong>of</strong> females always outnumbered males in placements relative to <strong>the</strong> number <strong>of</strong> 15-19 year<br />
olds employed full-time (Hoy and Lampe (1982) p24 and Figure 1). In August 1978, <strong>the</strong><br />
number <strong>of</strong> young women in SYETP peaked at 9 per cent <strong>of</strong> those in full time<br />
employment, but after this fluctuated between 2-4 per cent with males slightly lower at 1<br />
to 3 per cent. 45 Hoy and Lampe (1982) pointed out that 49 per cent <strong>of</strong> teenage females<br />
assisted under SYETP were employed in Manufacturing and Retail occupations. It was<br />
42 DEIR “Telephone survey <strong>of</strong> wage subsidy employers conducted in November 1983” as Cited p8a Smith<br />
(1984), and as referenced p55 Smith (1984). This reference is unpublished.<br />
43 DEYA (1980) “SYETP in <strong>the</strong> private sector: follow-up survey <strong>of</strong> April 1979 placements” Melbourne,<br />
unpublished. Cited Smith (1984) p8a. This reference is unpublished.<br />
44 Cited p128 BLMR (1984) and see also Hoy and Lampe (1982) p22.<br />
45 Hoy and Lampe (1982). Calculations based on placement numbers relative to <strong>the</strong> ABS Labour Force<br />
Survey estimates <strong>of</strong> Labour Force in employment for 15-19 year olds.
72<br />
additionally noted that <strong>the</strong> average weekly earnings for non-managerial employees under<br />
21 in Retail Industry in May 1981 were lower for females at $131.40 compared to males<br />
at $147.20; in Manufacturing <strong>the</strong> corresponding figures for females were $149.70 while<br />
for males $170.50. These industries were in turn below <strong>the</strong> all-industry average earnings<br />
for <strong>the</strong> corresponding groups - females $154 but males $171. Thus, <strong>the</strong> bulk <strong>of</strong> industry<br />
and occupational placements for SYETP reflected <strong>the</strong> greater share <strong>of</strong> <strong>the</strong> wage covered<br />
by <strong>the</strong> subsidy. Smith (1983) concluded that this could be interpreted as employers<br />
responding to <strong>the</strong> wage incentives.<br />
2.2.6.3 Characteristics <strong>of</strong> SYETP commencements<br />
Departmental research in 1983 46 examined <strong>the</strong> administrative records for<br />
commencements flowing onto all government programmes for youths in 1980-81.<br />
Although this was completed as a form <strong>of</strong> ‘evaluation’, it is not included in <strong>the</strong> evaluation<br />
critique, as it contains no modelling. A brief exposition <strong>of</strong> this is included as informative<br />
<strong>of</strong> <strong>the</strong> nature <strong>of</strong> SYETP close to <strong>the</strong> time <strong>of</strong> our analysis. This is useful as it gives some<br />
background to <strong>the</strong> nature <strong>of</strong> SYETP placements according to administrative data sources.<br />
These data are less likely to suffer <strong>the</strong> problems <strong>of</strong> sample surveys, although instead<br />
suffering from o<strong>the</strong>r data problems and errors. The timing is quite close to that <strong>of</strong> our<br />
analysis.<br />
They found for all programmes, such as SYETP and EPUY, <strong>the</strong>re was a substantially<br />
higher share <strong>of</strong> placements for 15-19 year old males (BLMR (1983): 11). They also<br />
concluded that 16-17 year olds were most <strong>of</strong>ten assisted amongst those 15-25 years.<br />
Commencements to extended SYETP had mean unemployment experience <strong>of</strong> 44.9 weeks,<br />
much greater than <strong>the</strong> minimum pre-requisite 34 weeks or more. The proportion <strong>of</strong><br />
commencements that had experienced 8 months or more unemployment prior to<br />
programme entry was 42.7 per cent for SYETP Commonwealth placements, and 22.3 per<br />
cent for SYETP private placements (BLMR (1983): 14). They observed <strong>the</strong>re was no<br />
strong age variation amongst <strong>the</strong> 2 types <strong>of</strong> SYETP programme, although <strong>the</strong> eligibility<br />
46 BLMR (1983) Figures only refer to ‘flow’ into <strong>the</strong> program and not <strong>the</strong> total number or ‘stock’ being<br />
treated at any one time.
73<br />
provisions for SYETP and extended SYETP had a small amount <strong>of</strong> age variation with<br />
extended SYETP excluding 15-17 year olds. They concluded that a smaller share <strong>of</strong><br />
commencements for older ages in <strong>the</strong> target groups was likely due to <strong>the</strong> flat age payment<br />
<strong>of</strong> SYETP subsidy whereas employers paid wages within an age related award structure.<br />
Table 2.14 shows <strong>the</strong> age breakdown <strong>of</strong> SYETP commencements in 1980/81, while Table<br />
2.15 gives <strong>the</strong> age/sex breakdown. Total wage reimbursement to Commonwealth<br />
employers and <strong>the</strong> ability to avoid ‘staff ceiling constraints’ led to older target ages being<br />
more common in this sort <strong>of</strong> placement. The Extended SYETP was not in operation until<br />
<strong>the</strong> ‘March 1981’ quarter.<br />
In Table 2.15, <strong>the</strong> number <strong>of</strong> commencements gives an idea <strong>of</strong> <strong>the</strong> scale <strong>of</strong> <strong>the</strong><br />
programmes in terms <strong>of</strong> total treated ‘flow’ cases in <strong>the</strong> period, and <strong>the</strong> proportion <strong>of</strong> <strong>the</strong><br />
total each programme represents is also shown. It is clear <strong>the</strong>re was a heavy reliance on<br />
<strong>the</strong> private SYETP, amongst all SYETP, and also <strong>the</strong>re was more widespread use <strong>of</strong><br />
SYETP amongst <strong>the</strong> range all programmes.<br />
Table 2.14 Distribution <strong>of</strong> SYETP commencements, by age 1980-81<br />
Quarter Sept 1980 Dec 1980 Mar 1981 Jun1981<br />
Age 15-19<br />
years<br />
20-24<br />
years<br />
25 47<br />
years<br />
15-19<br />
years<br />
20-24<br />
years<br />
25<br />
years<br />
15-19<br />
years<br />
20-24<br />
years<br />
25<br />
years<br />
15-19<br />
years<br />
20-24<br />
years<br />
25<br />
years<br />
SYETP<br />
58.6 41.4 0.0 52.3 47.5 0.2 52.2 46.6 1.2 62.7 36.5 0.8<br />
Commonwealth<br />
2nd SYETP 52.8 47.2 0.0 42.9 57.1 0.0 46.9 53.1 0.0 46.9 50.0 3.1<br />
Commonwealth<br />
SYETP private 86.8 13.1 0.1 84.8 15.0 0.2 84.1 15.7 0.2 90.1 9.8 0.1<br />
2nd SYETP 86.2 13.8 0.0 82.5 17.1 0.3 81.4 18.5 0.1 85.8 14.2 0.0<br />
private<br />
Extended SYETP 58.0 41.7 0.3 53.5 46.0 0.5<br />
Source: BLMR (1983) p42 Appendix Table B1. Data is Department <strong>of</strong> Employment and Industrial<br />
Relations administrative records for commencements to <strong>the</strong> programme in 1980-81, which thus only refer<br />
to ‘flow’ into <strong>the</strong> programme and not <strong>the</strong> total number or ‘stock’ being treated at any one time. Table shows<br />
proportions in <strong>the</strong> three age groups that sum to 100 per cent <strong>of</strong> commencements for each quarter and<br />
programme. ‘2 nd SYETP’ placements refer to commencements where this was <strong>the</strong> 2 nd eligible period <strong>of</strong><br />
SYETP assistance – i.e. multiple placements. The Extended SYETP was not in operation until <strong>the</strong> ‘March<br />
1981’ quarter.<br />
47 The 25 year olds were not eligible but <strong>the</strong>se commencements are explained by ‘Birth dates falling just<br />
prior to or after commencement dates, coding errors’ (BLMR (1983) p42 Appendix Table B1footnote (b)).
74<br />
Table 2.15 Distribution <strong>of</strong> SYETP commencements, by age and sex 1980-81<br />
15-17 years 18-19 years 20-24 years Numbers <strong>of</strong><br />
commencements<br />
% <strong>of</strong> all<br />
programme<br />
commencements<br />
Male Female Male Female Male Female<br />
SYETP<br />
7.2 10.5 14.6 24.2 23.3 20.1 3315 3.8<br />
Commonwealth<br />
2nd SYETP 2.1 0.8 17.5 27.5 26.2 25.8 240 0.3<br />
Commonwealth<br />
SYETP private 27.2 32.0 12.7 14.8 7.4 5.9 47500 54.2<br />
2nd SYETP 19.6 21.2 20.9 22.2 8.3 7.8 23631 3.0<br />
private<br />
Extended SYETP 0.9 0.9 25.1 28.9 25.4 18.8 6825 7.8<br />
Total all<br />
87558<br />
programmes<br />
Source: BLMR (1983) p16 Table 3.1. Data is Department <strong>of</strong> Employment and Industrial Relations<br />
administrative records for commencements to <strong>the</strong> programme in 1980-81, which thus only refer to ‘flow’<br />
into <strong>the</strong> programme and not <strong>the</strong> total number or ‘stock’ being treated at any one time. Table shows<br />
proportions in that sum across to 100 per cent for <strong>the</strong> programme. ‘2 nd SYETP’ placements refer to<br />
commencements where this was <strong>the</strong> 2 nd eligible period <strong>of</strong> SYETP assistance – i.e. multiple placements. The<br />
Extended SYETP was not in operation until <strong>the</strong> ‘March 1981’ quarter.<br />
Private SYETP placements were mostly in selling and unskilled occupations, with 14.4<br />
per cent in general administrative positions, 23.1 per cent in selling, and 34.4 per cent in<br />
semi-skilled jobs. Extended SYETP had 42 per cent <strong>of</strong> placements in semi-skilled jobs.<br />
Private SYETP placements occurred to a large extent in Retail trade (30.9 per cent) and<br />
‘o<strong>the</strong>r services’ which excluded public/health or education (11.8 per cent), and<br />
manufacturing (33.4 per cent). Extended SYETP had a similar industrial distribution. In<br />
comparison, Commonwealth SYETP had relatively higher shares in clerical (both<br />
keyboard clerical 21.5 per cent and 25.1 per cent in general administrative) and skilled<br />
manual jobs, influenced by <strong>the</strong> restricted range <strong>of</strong> tasks in <strong>the</strong> agencies. 48<br />
Not all placements were completed to <strong>the</strong> full length <strong>of</strong> <strong>the</strong>ir subsidy period. Overall,<br />
more placements terminated due to voluntary withdrawal <strong>of</strong> trainees than due to dismissal.<br />
However, for private sector placements only three quarters <strong>of</strong> approved subsidy time was<br />
used due to non-completion <strong>of</strong> <strong>the</strong> full subsidy period. 49 Overall, trainee dismissals and<br />
48 Of course, this meant that <strong>the</strong> industry was made up <strong>of</strong> 96.2 per cent ‘public/health or education/o<strong>the</strong>r<br />
services’.<br />
49 Entrants to Commonwealth placements could not have already experienced a placement lost due to<br />
dismissal or voluntary withdrawal (precluding people with former negative experiences). However, after
75<br />
withdrawals occurred at similar junctures in <strong>the</strong> approved time used, with dismissals on<br />
average occurring at 46.3 per cent <strong>of</strong> <strong>the</strong> allocated time, and withdrawals at 46.7 per cent<br />
<strong>of</strong> <strong>the</strong> subsidy period (BLMR (1983): 24). There was variation by State in <strong>the</strong> completion<br />
<strong>of</strong> placements, mainly due to <strong>the</strong> mix <strong>of</strong> Commonwealth/private SYETP. It was noted<br />
that in particular Western Australia, which had a higher share <strong>of</strong> private ra<strong>the</strong>r than<br />
Commonwealth SYETP placements, <strong>the</strong> dismissal rate was highest at 25.1% <strong>of</strong> all<br />
approved subsidies, whereas in Tasmania, <strong>the</strong> dismissal rate was low with 9.4 per cent <strong>of</strong><br />
all subsidies ending early with dismissal. It was noted that a previous SYETP<br />
evaluation 50 found that voluntary withdrawal in <strong>the</strong> first 8 weeks in private SYETP<br />
placements was more likely to occur due to not liking <strong>the</strong> work or people, but leaving to<br />
go to ano<strong>the</strong>r job was more likely to occur at 8-16 weeks <strong>of</strong> placement. As <strong>the</strong> share <strong>of</strong><br />
time used is quite high for SYETP programmes, shown in Table 2.16, <strong>the</strong>n it is likely<br />
most withdrawals would have been to go to ano<strong>the</strong>r job.<br />
There was also State variation in placements to <strong>the</strong> different education and employment<br />
programmes available. Queensland, Western Australia and Victoria were found to have<br />
much higher SYETP placement rates than o<strong>the</strong>r states. Table 2.17 repeats information<br />
about <strong>the</strong> State placements to <strong>the</strong> various programmes. The greatest share <strong>of</strong> placements<br />
in each State went to SYETP, and mostly to private SYETP, although <strong>the</strong> concentration<br />
in each state varied from state to state. Western Australia had <strong>the</strong> highest SYETP<br />
participation rate.<br />
<strong>the</strong> first 2 weeks <strong>of</strong> placement, any placement dismissed could not be replaced. Dismissal was <strong>the</strong>n<br />
uncommon in public SYETP placements. (BLMR (1983) p23 footnote 13).<br />
50 BLMR (1983) p33 citing Department <strong>of</strong> Employment and <strong>Youth</strong> Affairs (1980) “<strong>Special</strong> <strong>Youth</strong><br />
Employment and Training Program SYETP in <strong>the</strong> private sector: follow-up survey <strong>of</strong> April 1979<br />
placements”, Melbourne, unpublished.
76<br />
Table 2.16 Completion <strong>of</strong> SYETP placement<br />
% completing % dismissed % voluntarily<br />
withdrew<br />
% <strong>of</strong> approved<br />
time used<br />
SYETP<br />
76.9 4.4 18.4 91.0<br />
Commonwealth<br />
2nd SYETP 81.2 2.9 14.6 92.1<br />
Commonwealth<br />
SYETP private 55.3 18.5 25.5 76.6<br />
2nd SYETP 52.6 23.7 23.1 74.9<br />
private<br />
Extended SYETP 23.1 24.9 32.7 56.7<br />
(1)<br />
For All SYETP 52.8 17.4 25.8<br />
Source: BLMR (1983) p 22 Table 3.6. Data is Department <strong>of</strong> Employment and Industrial Relations<br />
administrative records for commencements to <strong>the</strong> programme in 1980-81, which thus only refer to ‘flow’<br />
into <strong>the</strong> programme and not <strong>the</strong> total number or ‘stock’ being treated at any one time. Durations are for all<br />
completed or terminated placements up to Jan 1982. Table shows proportion completing <strong>the</strong> full subsidy<br />
period, terminating for o<strong>the</strong>r reason, and <strong>the</strong>se percentages approximately sum across to 100 for each<br />
programme, with any discrepancy <strong>the</strong> few who were still in training at <strong>the</strong> analysis time point. The<br />
percentage <strong>of</strong> approved time used is <strong>the</strong>n <strong>of</strong> <strong>the</strong> eligible subsidy period. ‘2 nd SYETP’ placements refer to<br />
commencements where this was <strong>the</strong> 2 nd eligible 17 week period <strong>of</strong> SYETP assistance – i.e. multiple<br />
placements. (1) 19.4 % <strong>of</strong> extended SYETP were still in training at <strong>the</strong> time.<br />
Importantly, an intended sequence to programme attendance was sometimes used for<br />
placements. Thus, several programmes might be attended, for example combinations <strong>of</strong><br />
education-based EPUY and employment in SYETP in sequence. It was found that in<br />
<strong>the</strong>se data, which covered <strong>the</strong> period <strong>of</strong> one year and were <strong>the</strong>n subject to censoring<br />
limitations, 8.3 per cent were multiple placements, and <strong>of</strong> <strong>the</strong>se 24 per cent were<br />
combinations <strong>of</strong> education and employment based programmes and a fur<strong>the</strong>r 30 per cent<br />
were split SYETP placements (indicating a high level <strong>of</strong> movement between SYETP<br />
programme placements within <strong>the</strong> 17 weeks <strong>of</strong> subsidy).
77<br />
Table 2.17 State usage <strong>of</strong> programmes in 1980/81<br />
New South Victoria Queensland South Western Tasmania<br />
Wales<br />
Australia Australia<br />
SYETP<br />
4.5 3.3 2.9 3.7 3.0 6.1<br />
Commonwealth<br />
2nd SYETP 0.2 0.2 0.2 0.4 0.7 0.3<br />
Commonwealth<br />
SYETP private 52.2 58.6 58.5 49.7 53.8 45.3<br />
2nd SYETP 2.0 2.4 3.3 3.6 6.5 1.7<br />
private<br />
Extended SYETP 7.3 6.6 8.5 10.6 8.3 6.9<br />
All SYETP 66.2 71.2 73.5 68.1 72.5 60.4<br />
O<strong>the</strong>r programmes 33.8 28.8 26.5 31.9 27.5 39.6<br />
for youths<br />
Source: BLMR (1983) p43 Appendix Table B2. Data is Department <strong>of</strong> Employment and Industrial<br />
Relations administrative records for commencements to <strong>the</strong> programme in 1980-81, which thus only refer<br />
to ‘flow’ into <strong>the</strong> programme and not <strong>the</strong> total number or ‘stock’ being treated at any one time. Table shows<br />
proportions in <strong>the</strong> state that sum to 100 per cent <strong>of</strong> commencements for each state.<br />
2.2.6.4 Factors affecting completion <strong>of</strong> subsidy placement<br />
Once SYETP placements were made, <strong>the</strong> placement faced normal employment<br />
conditions, allowing quits and dismissals to occur within <strong>the</strong> approved subsidy period.<br />
Ginpil and Hoy (1984) used both employer survey and programme participant survey<br />
data to examine <strong>the</strong> factors affecting completion <strong>of</strong> subsidy placement. In log-linear<br />
analysis <strong>the</strong>y modelled <strong>the</strong> completion <strong>of</strong> <strong>the</strong> full subsidy period.<br />
Using <strong>the</strong> employer data, <strong>the</strong>y found that lower completion rates were linked to<br />
employers recording both poor job skills and ‘<strong>the</strong> occurrence <strong>of</strong> problems with work or<br />
behaviour’ for a placement, and poor work habits. However <strong>the</strong> ‘good work habits’ <strong>of</strong> a<br />
placement did not lead to higher completion except when <strong>the</strong>re was no ‘occurrence <strong>of</strong><br />
problems with work or behaviour’ for placements. Higher than average gross wages 51 to<br />
placements also led to higher completion rates. More placements were fully completed in<br />
larger firms, and completion was found to be more generally related to training and<br />
support to placements if problems with work or behaviour were perceived.<br />
51 The question in <strong>the</strong> 1982 survey based this amount around <strong>the</strong> value $140 or more per week.
78<br />
For <strong>the</strong> placement data, higher completion rates for private SYETP placements were<br />
found for school leavers, who had not held a past full-time job and those with higher<br />
levels <strong>of</strong> schooling (not early school leavers). The placements where it was perceived that<br />
no employer help was received when problems were experienced had lower chance <strong>of</strong><br />
completion. Completion <strong>of</strong> placement was found to be related to <strong>the</strong> higher morale <strong>of</strong><br />
placements, where <strong>the</strong>y thought <strong>the</strong>y had a better chance <strong>of</strong> employment now than before<br />
<strong>the</strong> placement.<br />
2.3 Critical Review <strong>of</strong> <strong>Evaluation</strong> <strong>of</strong> SYETP<br />
This review is limited to micro-economic evaluation evidence <strong>of</strong> <strong>the</strong> employment effect<br />
<strong>of</strong> SYETP, where individual record data are used. It thus covers works which were<br />
carried out on <strong>the</strong> same general basis as our later analysis. Some papers are not included<br />
in <strong>the</strong> evaluation critique, where it was not deemed an evaluation if <strong>the</strong> analysis contained<br />
no modelling <strong>of</strong> employment impact. Some useful information about <strong>the</strong> SYETP<br />
programme operation and characteristics <strong>of</strong> participants from <strong>the</strong>se types <strong>of</strong> study are in<br />
<strong>the</strong> earlier sections, as are some macro-economic analyses such as Smith (1983, 1984b).<br />
Early evaluation <strong>of</strong> <strong>the</strong> SYETP during 1980 is referred to in BLMR (1983). These were<br />
studies carried out by <strong>the</strong> Departments and unpublished 52 , which makes it difficult to<br />
assess <strong>the</strong>ir evidence. The usefulness <strong>of</strong> <strong>the</strong>ir findings is however commented on in<br />
BLMR (1983). Reference is made to <strong>the</strong>ir covering various different time periods, and it<br />
is pointed out that “<strong>the</strong> methodologies used have made it difficult to make comparisons<br />
between programmes or to consider <strong>the</strong> combined impact <strong>of</strong> <strong>the</strong> programmes” (BLMR<br />
(1983): 3). Baker (1984) also refers to <strong>the</strong>se evaluations and comments that <strong>the</strong>y did not<br />
control for individual characteristics, but instead used contingency tables or crosstabulation.<br />
52 Cited p2 BLMR (1983): Department <strong>of</strong> Employment and <strong>Youth</strong> Affairs (1980) “<strong>Special</strong> <strong>Youth</strong><br />
Employment and Training Program: employer study”, Melbourne, unpublished; Department <strong>of</strong><br />
Employment and <strong>Youth</strong> Affairs (1980) “<strong>Special</strong> <strong>Youth</strong> Employment and Training Program: a study <strong>of</strong> <strong>the</strong><br />
initial 1000 trainees”, Melbourne, unpublished; Department <strong>of</strong> Employment and <strong>Youth</strong> Affairs (1980)<br />
“<strong>Special</strong> <strong>Youth</strong> Employment and Training Program SYETP in <strong>the</strong> private sector: follow-up survey <strong>of</strong> April<br />
1979 placements”, Melbourne, unpublished.
79<br />
2.3.1 Stretton (1982, 1984) 53<br />
Stretton (1984) used survey data <strong>of</strong> participants in 5 programmes collected in November<br />
1981, where <strong>the</strong> 5 programmes were EPUY, GTA on-<strong>the</strong> job and 3 types <strong>of</strong> SYETP-<br />
Commonwealth, private, and second serve. The survey took place about 6 months after<br />
programme participation, was <strong>of</strong> a sample <strong>of</strong> 1500 participants who completed,<br />
terminated or withdrew from <strong>the</strong> placement in April/May 1981 and had a response rate <strong>of</strong><br />
63 per cent with 945 cases for analysis. Stretton (1984) himself pointed out that he did<br />
not have a comparison group, and that <strong>the</strong> possibility <strong>of</strong> selection bias existed with regard<br />
to modelling employment, and non-response bias might be present. An analysis <strong>of</strong> nonresponse<br />
was made using <strong>the</strong> administrative data for <strong>the</strong> sample. It was found that<br />
younger persons, females and programme completers were more likely to respond.<br />
Participation in one programme was compared to participation in ano<strong>the</strong>r programme, by<br />
including a set <strong>of</strong> programme dummies. Employment (two-types: full or part-time job at<br />
interview, or any job since programme) was modelled using a logit <strong>of</strong> labour market<br />
status. The regressions were run with age, continuous age, education (dummy with 1 for<br />
school leaver at 16 years or younger), duration <strong>of</strong> unemployment prior to programme<br />
entry, programme (EPUY was <strong>the</strong> base), and state (Tasmania was <strong>the</strong> base). Positive<br />
coefficients, statistically significant at <strong>the</strong> one and five per cent level, were present for all<br />
programme dummies, and for SYETP this was true in both employment at interview and<br />
any employment since <strong>the</strong> programme.<br />
To investigate retention, <strong>the</strong> same model was run on just <strong>the</strong> sample <strong>of</strong> those who were<br />
not retained by <strong>the</strong>ir placement employer. No programme dummies were significant. It<br />
was concluded completers not retained in <strong>the</strong>ir placement job did no better than those<br />
who did not complete. It was noted that <strong>the</strong> observed retention rates differed for <strong>the</strong><br />
various programmes, with only 60 per cent <strong>of</strong> GTA retained, 70 per cent <strong>of</strong> SYETP<br />
private, 70 per cent or SYETP second serve, and 80 per cent <strong>of</strong> SYETP Commonwealth<br />
(Stretton (1984): 86). Also, less than 5 per cent <strong>of</strong> SYETP or GTA programme<br />
participants left early (Stretton (1984): 87). Then <strong>the</strong> employment model was adjusted to<br />
include dummies about completion, <strong>the</strong> resulting model interpreted as comparing those<br />
53 As <strong>the</strong> first <strong>of</strong> <strong>the</strong>se is early findings, only <strong>the</strong> second is discussed here.
80<br />
who completed job-based programmes and were not retained to those who left early to no<br />
job. The results <strong>of</strong> this model were held to show that <strong>of</strong> those who withdrew early<br />
without a job to go to, those who began job-based programmes had higher probability <strong>of</strong><br />
employment than those in EPUY. Overall, it was concluded SYETP and GTA had better<br />
employment outcomes than EPUY. Although this was <strong>the</strong> conclusion, <strong>the</strong> size <strong>of</strong> <strong>the</strong><br />
employment effects for <strong>the</strong>se programmes were not calculated but were instead inferred,<br />
with conclusions drawn from <strong>the</strong> positive and statistically significant coefficients from<br />
<strong>the</strong> SYETP and GTA regressions.<br />
Stretton (1984) did not have a control group. Instead, participation in one programme was<br />
compared to participation in ano<strong>the</strong>r programme by including dummies. The main<br />
disadvantage to this is that it alters <strong>the</strong> evaluation question from whe<strong>the</strong>r <strong>the</strong> programme<br />
was effective in improving post-programme employment in an absolute sense, to <strong>the</strong><br />
relative issue <strong>of</strong> whe<strong>the</strong>r <strong>the</strong> programme was effective compared to ano<strong>the</strong>r programme.<br />
The participation in different programmes, which was included as right hand side<br />
variables, might be endogenous. The various programme participations are not modelled<br />
and this introduces <strong>the</strong> problem <strong>of</strong> selection bias. No adjustment for non-response is<br />
made, although <strong>the</strong> potential presence is briefly treated. The investigation <strong>of</strong> retention<br />
holds a number <strong>of</strong> problems. EPUY remained in <strong>the</strong> first model as <strong>the</strong> base, and yet postprogramme<br />
employer retention was impossible for this education programme. The<br />
dummies to reflect completion combined completion with all <strong>the</strong> programme types, and<br />
are not ideally constructed. The construction is problematic as retention rates varied by<br />
programme, as well as completion rates, and <strong>the</strong> length <strong>of</strong> each programme was very<br />
different. In discussion Stretton points out that two thirds <strong>of</strong> EPUY participants left early<br />
to start a job, but <strong>the</strong>n <strong>of</strong> <strong>the</strong>se 40 per cent started an SYETP job. This indicates that in<br />
<strong>the</strong> second retention model <strong>the</strong> ‘leaving early to a job’ is really a type <strong>of</strong> EPUY-SYETP<br />
programme combination. That less than 5 per cent <strong>of</strong> SYETP or GTA programme<br />
participants left early indicates <strong>the</strong> base would be very small. Bases for <strong>the</strong> regressions<br />
are not shown in <strong>the</strong> results presented so this cannot be assessed. In concluding, Stretton<br />
pointed out that non-response bias could also play a role in <strong>the</strong> ‘early leaver’ analysis.
81<br />
Stretton attributed <strong>the</strong> success <strong>of</strong> SYETP to retention. This falls in with <strong>the</strong> generally<br />
stated belief held in <strong>the</strong> BLMR analysis that <strong>the</strong> chief advantage <strong>of</strong> <strong>the</strong> subsidy was firstly<br />
<strong>the</strong> immediate impact on <strong>the</strong> employment status, where <strong>the</strong>ir ILO labour market status<br />
was now employment instead <strong>of</strong> unemployment, and <strong>the</strong> strong potential to remain in <strong>the</strong><br />
job after <strong>the</strong> subsidy finished. Non-completion was however common for most<br />
programmes with only 54.5 per cent <strong>of</strong> all programmes completed, and a large part <strong>of</strong><br />
early analysis focused on withdrawal and completion rates, toge<strong>the</strong>r with <strong>the</strong>ir reasons<br />
(BLMR (1983) p22 Table 3.6). Earlier BLMR (1983) analysis reported that <strong>the</strong><br />
programme provisions <strong>the</strong>mselves were a key source <strong>of</strong> this variation. The earlier section<br />
2.2.6.3 and Table 2.16 gave SYETP completion rates in <strong>the</strong> administrative data and<br />
showed that Private SYETP had low completion rates. But for Private SYETP in<br />
particular, a uniquely large number, approximately half <strong>of</strong> early terminations, were<br />
dismissals. It was in contrast, not surprisingly, extremely rare for EPUY training<br />
participants to be terminated. 54 In BLMR (1983) it was concluded that job search<br />
accounted for <strong>the</strong> large part <strong>of</strong> voluntary withdrawal. Stretton did not account for<br />
dismissals in his modelling. Of early leavers with no job to go to, employment chances<br />
would likely be strongly affected by dismissal, in a different way to voluntary withdrawal<br />
for job search or o<strong>the</strong>r reasons.<br />
2.3.2 Baker (1984)<br />
Baker (1984) used survey data <strong>of</strong> participants to evaluate <strong>the</strong> employment impact <strong>of</strong> a<br />
number <strong>of</strong> programmes, including SYETP. The survey was <strong>of</strong> 3713 persons who were<br />
surveyed in May 1982 after <strong>the</strong>y were selected from programme administrative data. The<br />
overall response rate was 66 per cent, but varied by <strong>the</strong> programme sampled: <strong>the</strong> response<br />
rate for private SYETP was 60 per cent, but 69 per cent for Commonwealth SYETP.<br />
Employment and labour market activity was observed at about six to eight months after<br />
programme participation. The raw full-time employment share at interview date for<br />
participants <strong>of</strong> SYETP private was 60.6 per cent, slightly lower for Commonwealth<br />
54 For EPUY <strong>the</strong> statistics were: 4.5 per cent dismissals, 37.6 per cent voluntary withdrawal, and a 57.5 per<br />
cent completion rate; Mean approved program period 14 weeks, average observed approved time used 80.8<br />
per cent Source: Table 3.6, p22 BLMR (1983).
82<br />
SYETP at 53.6 per cent, 45.6 per cent for repeat SYETP and 52.4 per cent for extended<br />
SYETP. A small share between 5-6 per cent was in part time employment. Fulltimeemployment<br />
prior to SYETP was lower for SYETP Private at 52 per cent, so before<br />
controlling for individual characteristics <strong>the</strong>y were more likely to hold a job after than<br />
before <strong>the</strong> programme (Baker (1984) p16, Tables A4 and 5.1). For o<strong>the</strong>r SYETP, post<br />
programme employment was lower or <strong>the</strong> same.<br />
A probit model <strong>of</strong> response to <strong>the</strong> survey was found to indicate response bias might be<br />
present. 55 The various programme dummies were significant in modelling response,<br />
where response was a full response with no missing items. The post-programme full-time<br />
employment outcome was modelled using a variable with 3 categories: continuous<br />
employment, including those retained in <strong>the</strong> job and those who left after <strong>the</strong> subsidy but<br />
got a job within a month; non-continuous employment, including those retained in <strong>the</strong> job<br />
for only 1 month after subsidy expiry but who later lost it by <strong>the</strong> survey date, and those<br />
who left after <strong>the</strong> subsidy and took longer than one month to find a job; no employment,<br />
including unemployed, education or o<strong>the</strong>r states <strong>of</strong> not in <strong>the</strong> labour force. This variable<br />
combined holding a job since <strong>the</strong> programme and <strong>the</strong> durability <strong>of</strong> <strong>the</strong> jobs held since<br />
programme, as well as <strong>the</strong> job search success, to perhaps to reflect employability. Table<br />
2.18 shows <strong>the</strong> distribution <strong>of</strong> this variable for <strong>the</strong> various SYETP. It can be seen for<br />
example, that <strong>the</strong> continuous full-time work category for SYETP private is much lower at<br />
41.1 per cent than <strong>the</strong> share in employment at interview (60.6 per cent). However, about<br />
80 per cent <strong>of</strong> SYETP private participants had held some full-time employment.<br />
The employment variable with three categories (no employment, continuous and noncontinuous<br />
employment) was <strong>the</strong>n modelled as an ordered probit, as if it were count data.<br />
The selection process onto programme participation was not modelled. However, <strong>the</strong><br />
ordered employment probit was adjusted for non-response using <strong>the</strong> Heckman model. To<br />
adjust for non-response, <strong>the</strong> non-response model was used “…to estimate a correction<br />
factor LAMBDA…included as an additional independent variable”, and this variable was<br />
55 Baker (1984) p49 Table A11 Variables included age, sex, unemployment (weeks in 52 week year before<br />
program entry), multiple program participation, dismissal, voluntary withdrawal, 10 program dummies with<br />
SYETP private as Base, state dummies. State dummies were not statistically significant.
83<br />
included in <strong>the</strong> employment model. This is <strong>the</strong> Heckman selection correction model<br />
where <strong>the</strong> lambda is <strong>the</strong> inverse mills ratio. The employment model had <strong>the</strong> variables age,<br />
sex, year left school (but not qualifications), post-school formal training (any post-school<br />
qualifications), a post-school job (pre-programme), weeks unemployment (weeks in 52<br />
week year before programme entry), multiple programme participation, dismissal,<br />
voluntary withdrawal, 10 programme dummies with SYETP private as base, state<br />
dummies and <strong>the</strong> LAMBDA correction factor. The response model used a subset <strong>of</strong> <strong>the</strong>se<br />
variables, so that <strong>the</strong> exclusion restriction variables were year left school, post school<br />
formal training and post school job (pre-programme).<br />
No additional results were presented showing <strong>the</strong> sensitivity <strong>of</strong> <strong>the</strong> employment model to<br />
whe<strong>the</strong>r or not nonresponse was accounted for, so it is not clear whe<strong>the</strong>r <strong>the</strong> additional<br />
Heckman selection model performed better than a simpler model without nonresponse.<br />
Baker estimated predicted probabilities for employment for various subgroups using <strong>the</strong><br />
model outcomes. Predicted probabilities were estimated for <strong>the</strong> eligibility period <strong>of</strong> 17<br />
weeks unemployment in <strong>the</strong> past 52 weeks, for a participant who left school in year 10/11,<br />
did post-school training, took part in no o<strong>the</strong>r programme and completed <strong>the</strong> programme<br />
<strong>of</strong> participation. They found that SYETP private generally performed better than<br />
education based programmes, but did no better than o<strong>the</strong>r SYETP, and was significantly<br />
worse than extended SYETP. The results are shown below for SYETP in Table 2.19. It<br />
can be seen that <strong>the</strong> ‘out <strong>of</strong> work’ outcome had a very low probability for all types <strong>of</strong><br />
SYETP.<br />
Table 2.18 Baker (1984) Post-programme full-time employment outcome<br />
Continuous<br />
fulltime work<br />
Non-<br />
Continuous<br />
fulltime work<br />
Retained Not-retained Total<br />
21.1 14.3 35.5 33.7 30.8<br />
SYETP<br />
Commonwealth<br />
SYETP private 34.9 6.3 41.1 38.4 20.4<br />
2nd SYETP 24.3 7.3 31.6 41.2 27.2<br />
Extended SYETP 37.1 5.5 42.6 30.6 26.8<br />
Source: Baker (1984) p19 Table 5.2 May 1982 post-programme Survey <strong>of</strong> participants<br />
Out <strong>of</strong> work
84<br />
Table 2.19 Baker (1984) Estimated probabilities <strong>of</strong> labour market outcomes for<br />
participants from <strong>the</strong> model <strong>of</strong> employment<br />
Continuous fulltime Non-Continuous Out <strong>of</strong> work<br />
work<br />
fulltime work<br />
SYETP<br />
Commonwealth<br />
.414 .400 .186<br />
SYETP private .517 .358 .124<br />
2nd SYETP .418 .399 .183<br />
Extended SYETP .611 .307 .082<br />
Source: Baker (1984) p48 Table A10 May 1982 post-programme Survey <strong>of</strong> participants. Predicted<br />
probabilities were estimated for <strong>the</strong> eligibility period <strong>of</strong> 17 weeks unemployment in <strong>the</strong> past 52 weeks, for a<br />
participant who left school in year 10/11, did post-school training, took part in no o<strong>the</strong>r programme and<br />
completed <strong>the</strong> programme <strong>of</strong> participation.<br />
A chief difficulty with <strong>the</strong> results <strong>of</strong> modelling is <strong>the</strong> likely endogeneity <strong>of</strong> <strong>the</strong><br />
programmes. As <strong>the</strong> authors also point out, <strong>the</strong>y did no selection modelling for<br />
programme entry. They did however try to account for <strong>the</strong> survey response problem,<br />
which could introduce bias and inconsistency to <strong>the</strong> estimates if unaccounted for. The<br />
means <strong>of</strong> implementing <strong>the</strong> Heckman selection adjustment was to enter <strong>the</strong> selection<br />
correction term linearly into <strong>the</strong> ordered probit model, but this is problematic due to nonlinearity<br />
<strong>of</strong> <strong>the</strong> probit functional form. An appropriate model combining response and<br />
employment would have estimated <strong>the</strong> equations jointly and allowed for correlated errors.<br />
A fur<strong>the</strong>r difficulty with <strong>the</strong>ir nonresponse modelling is that it combines item nonresponse<br />
and survey non-response, which requires <strong>the</strong> assumption that <strong>the</strong> same model<br />
can account properly for <strong>the</strong>m both. There is no non-participant comparison group for<br />
programme participants, only participants <strong>of</strong> o<strong>the</strong>r programmes are available for<br />
comparison. This is because <strong>the</strong> data are from a survey <strong>of</strong> programme participants only.<br />
2.3.3 Rao and Jones (1986)<br />
Rao and Jones (1986) used <strong>the</strong> same survey data as used for Baker (1984), but with<br />
additional data from a second follow-up survey conducted in May 1983. The additional<br />
survey allowed employment outcomes at 18-20 months to be examined, and <strong>the</strong>y<br />
combined <strong>the</strong> survey data with <strong>the</strong> CES administrative data. They defined a quasi-control<br />
group <strong>of</strong> those who did not complete <strong>the</strong> programme, where <strong>the</strong>y used <strong>the</strong> cut<strong>of</strong>f <strong>of</strong> one<br />
third <strong>of</strong> programme period to separate <strong>the</strong> treatment from <strong>the</strong> quasi-control <strong>of</strong> non-
85<br />
completers. Their argument was that such a short experience did not constitute treatment,<br />
because <strong>the</strong>y would not have had time to benefit much from <strong>the</strong> programme (Rao and<br />
Jones (1986): 1). To control for differences in eligibility and characteristics <strong>of</strong><br />
participants, <strong>the</strong>y estimated <strong>the</strong> post-programme employment chances <strong>of</strong> “clearly defined<br />
socio-demographic groups in different programmes” (Rao and Jones (1986): 5). There<br />
were 2 groups – most disadvantaged and least disadvantaged. 56 They estimated logistic<br />
regressions <strong>of</strong> <strong>the</strong> probability <strong>of</strong> full-time continuous employment or not, where <strong>the</strong><br />
employment variable’s basic definition was defined as for Baker (1984). The regression<br />
included grouped age (15-19, 20-24), sex, year <strong>of</strong> schooling, post-school training and<br />
completion, post-school job, pre-programme unemployment 57 , past programme<br />
experience, NSW resident or not, and twelve programme dummies including four SYETP<br />
dummies for Commonwealth, private, second serve and extended. From <strong>the</strong> regression<br />
estimates, <strong>the</strong> percentage probabilities were calculated.<br />
They found that individual characteristics as well as <strong>the</strong> type <strong>of</strong> programme affected postprogramme<br />
employment outcomes. Completing <strong>the</strong> programme or near-completion gave<br />
better employment outcomes than early withdrawal which was <strong>the</strong> quasi-control group.<br />
Employment based programmes performed better than training/education-based<br />
programmes. They found that <strong>of</strong> <strong>the</strong> employment based programmes, GTA on-<strong>the</strong>-job<br />
performed better than SYETP, and that this was because GTA did not prescribe prior<br />
unemployment or age criteria and was ‘occupational demand’ based whereas SYETP was<br />
aimed at disadvantaged youths. Amongst SYETP, private SYETP and extended SYETP<br />
were more effective than Commonwealth SYETP. A ‘second serve’ SYETP, where<br />
former SYETP participants had again experienced long-term unemployment and were<br />
given a second SYETP placement performed worst <strong>of</strong> all employment programmes. The<br />
most-disadvantaged were proportionally more likely to benefit from <strong>the</strong> programmes.<br />
However <strong>the</strong>y also concluded that as <strong>the</strong> probability <strong>of</strong> post-programme employment was<br />
56 Rao and Jones (1986) p6. Most disadvantaged: 15-19 years, female, schooling to year 9, started but<br />
incomplete post-school training, no post-school full-time job, unemployed for more than 12 months prior to<br />
placement, no o<strong>the</strong>r program participation, NSW resident. Least disadvantaged: 20-24 years, male,<br />
completed schooling to year 11/12, completed post-school training, had a post-school job, no<br />
unemployment prior to placement, o<strong>the</strong>r past program experience, not NSW resident.<br />
57 The pre-program unemployment groups are: none, less than 3 months, 3-6 months, 6-12 months, >12<br />
months.
86<br />
still much lower for <strong>the</strong> most-disadvantaged, <strong>the</strong>y were still worse <strong>of</strong>f than <strong>the</strong> leastdisadvantaged<br />
after <strong>the</strong> programme. This arose because <strong>of</strong> <strong>the</strong> difference in employment<br />
chances for non-participant– i.e. <strong>the</strong> quasi-control group for <strong>the</strong> most-disadvantaged and<br />
least-disadvantaged had very different employment chances. Table 2.20 shows <strong>the</strong>ir<br />
results for SYETP. The results show a positive impact on full-time continuous<br />
employment for all groups after SYETP participation, however <strong>the</strong> size ranges from 5 to<br />
55 per cent.<br />
Table 2.20 Rao and Jones (1986) Estimated percent post-programme full-time<br />
continuous employment chances 1981-1983<br />
Least Disadvantaged<br />
Most Disadvantaged<br />
Completed<br />
programme<br />
Difference to<br />
quasi- control<br />
Completed<br />
programme<br />
Difference to<br />
quasi-control<br />
SYETP<br />
63.4 44.8 8.2 7<br />
Commonwealth<br />
SYETP private 72.0 53.4 11.7 10.5<br />
2nd SYETP 55.7 37.1 6.1 4.9<br />
Extended SYETP 74.4 55.8 13.0 11.8<br />
Quasi-control 18.6 1.2<br />
Source: Rao and Jones (1986) Table 3, p24<br />
This analysis made an effort to define comparison and treatment groups based on<br />
observed characteristics, while lacking true control data in a survey <strong>of</strong> participants. The<br />
quasi-control would be subject to a reasonably large extent <strong>of</strong> contamination bias, as <strong>the</strong><br />
data are based on survey data recalling labour market history. Contamination bias is<br />
where <strong>the</strong> treatment and comparison groups are not cleanly defined and mixing can occur.<br />
There is also <strong>the</strong> strong possibility <strong>of</strong> selection bias. The Baker (1984) results showed<br />
some evidence that completers and non-completers had different characteristics, and had<br />
different raw employment outcomes. Rao and Jones (1986) confirmed that controlling for<br />
individual characteristics was essential. It is clear that individual characteristics might<br />
also influence which programme an individual participated in however this was not<br />
controlled for. There was no modelling <strong>of</strong> programme entry. Again, <strong>the</strong> various<br />
programmes were simply dummies in <strong>the</strong> employment equation, which is subject to <strong>the</strong><br />
same difficulties as <strong>the</strong> Baker (1984) analysis. Baker (1984) showed that non-response
87<br />
was an issue for <strong>the</strong> data. Unlike <strong>the</strong> Baker (1984) analysis, survey non-response was<br />
ignored by Rao and Jones (1986).<br />
2.3.4 Richardson (1998)<br />
Richardson (1998) used panel data spanning 4 years <strong>of</strong> repeat surveys <strong>of</strong> <strong>the</strong> <strong>Australian</strong><br />
Longitudinal Survey (ALS). The analysis is based upon <strong>the</strong> ALS list sample, which was a<br />
sample selected from <strong>the</strong> CES records for young unemployed aged 15-24. The ALS list<br />
sample was a nationally representative sample <strong>of</strong> <strong>Australian</strong> youths aged 15-24 who had<br />
been registered as unemployed with <strong>the</strong> Commonwealth Employment Service for at least<br />
3 months in June 1984. They were interviewed in September-October each year from<br />
1984-1987. 58 Using <strong>the</strong> labour market history, those who entered SYETP job placements<br />
during 1984 until <strong>the</strong> 1985 interview defined <strong>the</strong> SYETP treatment group and comparison<br />
group reference period. The subsequent survey data on labour market history until <strong>the</strong><br />
1987 interview date was used to examine post-programme employment outcomes.<br />
An analysis <strong>of</strong> attrition was made by examination <strong>of</strong> <strong>the</strong> means. It was concluded that <strong>the</strong><br />
characteristics were virtually identical and so attrition bias was treated as a minor<br />
problem (Richardson (1998): 5). It was however noted that those lost to attrition were<br />
more likely to have less than year 10 education, more likely to have held a job for at least<br />
1 year and less likely to have never held a job. Accordingly, no treatment <strong>of</strong> non-response<br />
was made. It was noted that <strong>the</strong> treated SYETP had different characteristics than <strong>the</strong><br />
comparison group <strong>of</strong> non-participants. It was observed that in <strong>the</strong> raw data, employment<br />
in 1986 was 14 percentage points higher for those who had participated in SYETP than<br />
<strong>the</strong> comparison group, and in 1987 it was 5 per cent higher than <strong>the</strong> comparison group.<br />
First, separate probit models <strong>of</strong> employment and <strong>of</strong> participation were estimated. Then<br />
Richardson used a Heckman bivariate probit which controlled for programme selection<br />
by modelling both <strong>the</strong> participation and post-programme employment. To identify <strong>the</strong><br />
model, a variable showing CES referrals to ano<strong>the</strong>r programme, and age <strong>of</strong> <strong>the</strong><br />
58 Fur<strong>the</strong>r details <strong>of</strong> <strong>the</strong> data are in <strong>the</strong> later chapter dealing with <strong>the</strong> replication study.
88<br />
participant, were excluded from <strong>the</strong> employment equation and included in <strong>the</strong><br />
participation equation. Additionally, some variables such as marital status were 1984<br />
dated in <strong>the</strong> participation equation but updated to 1985/6 values for <strong>the</strong> employment<br />
equation. Employment was any non-subsidised post-programme period employment.<br />
Additional variables that were included in both <strong>the</strong> SYETP and employment models were<br />
sex, children in 1984, children interacted with sex, ethnicity, State <strong>of</strong> residence in 1984,<br />
marital status, education: school type, year left school, post-school qualifications; initial<br />
labour market experience: duration <strong>of</strong> longest job to 1984 reference period; past<br />
programme participation, pre-programme unemployment in <strong>the</strong> year to reference period,<br />
health problems affecting work, attitude to women working and attitude interacted with<br />
sex, parental education and occupation, number <strong>of</strong> siblings, English skills, spousal<br />
employment in 1984, parental education and occupation, religion, urban/rural/overseas<br />
residence before aged 14.<br />
The bivariate probit results gave a negative correlation coefficient and Wald tests <strong>of</strong> <strong>the</strong><br />
correlation coefficient rejected <strong>the</strong> hypo<strong>the</strong>sis that <strong>the</strong> errors <strong>of</strong> <strong>the</strong> employment and<br />
participation equations were uncorrelated. The comparison for <strong>the</strong> bivariate probit to <strong>the</strong><br />
univariate probits showed a strong difference in <strong>the</strong> measured employment effects. The<br />
marginal effects <strong>of</strong> SYETP on post-programme employment were calculated to give<br />
positive employment effects <strong>of</strong> SYETP participants over non-participants <strong>of</strong> 26 per cent<br />
in 1986 and 20 per cent in 1987 (Richardson (1998): 11). To examine retention <strong>of</strong><br />
subsidised jobs, <strong>the</strong> modelling was repeated for employment in 1987 and <strong>the</strong> sample was<br />
limited to exclude those in <strong>the</strong> treatment or comparison groups who were continuously in<br />
a job from <strong>the</strong> 1984/5 reference period until <strong>the</strong> 1986 interview. It was noted that similar<br />
shares <strong>of</strong> <strong>the</strong> SYETP and comparison groups were excluded. The results were similar to<br />
those found from <strong>the</strong> first bivariate probit model. The marginal effect calculated for this<br />
equation showed again a positive employment effect for SYETP participants over <strong>the</strong><br />
comparisons <strong>of</strong> 23.7 per cent (Richardson (1998): 12). Retention was concluded to be<br />
important, but not <strong>the</strong> only source <strong>of</strong> positive post-programme employment effects for<br />
SYETP participants. Table 2.21 summarises <strong>the</strong> employment effects found for<br />
Richardson (1998).
89<br />
Table 2.21 Richardson (1998) Estimated marginal effect <strong>of</strong> SYETP on employment from<br />
bivariate probit modelling<br />
effect on employment 1986 effect on employment 1987<br />
All data used 26.4 19.7<br />
Excluding jobs or placements<br />
retained to 1986<br />
23.7<br />
Source: Richardson (1998) p22 Table 6, and p12.<br />
The Richardson (1998) analysis usefully modelled both <strong>the</strong> participation in SYETP and<br />
<strong>the</strong> post-programme employment using <strong>the</strong> Heckman selection method in a bivariate<br />
probit. Earlier SYETP evaluations only modelled employment, none modelled<br />
participation in SYETP. The modelling included numerous variables that account for<br />
individual characteristics. This was by far <strong>the</strong> largest set <strong>of</strong> individual characteristics used<br />
in modelling <strong>the</strong> post-programme employment for SYETP. Generally, most <strong>of</strong> <strong>the</strong>se have<br />
been used in <strong>the</strong> economics literature, if available in <strong>the</strong> data, to explain employment,<br />
education, or programme participation. The wide variety <strong>of</strong> variables included illustrates<br />
<strong>the</strong> very rich dataset <strong>the</strong> ALS has to draw on. The inclusion <strong>of</strong> a wide set <strong>of</strong> individual<br />
characteristics can help demonstrate that all <strong>the</strong> useful observable characteristics have<br />
been controlled for in modelling both <strong>the</strong> employment and programme participation.<br />
However some <strong>of</strong> <strong>the</strong> variables, such as religion, are possibly more unusual for<br />
describing economic models <strong>of</strong> employment or programme participation.<br />
Although <strong>the</strong> non-response was briefly examined, <strong>the</strong> examination was terse. The<br />
conclusions about <strong>the</strong> possible effects <strong>of</strong> non-response were based on little serious<br />
contemplation <strong>of</strong> non-response. Some earlier SYETP evaluations (Stretton (1984), Baker<br />
(1984)) modelled non-response for <strong>the</strong>ir observational data and found non-response was a<br />
potential problem. Richardson (1998) contains no modelling <strong>of</strong> non-response, yet <strong>the</strong><br />
potential for non-response is much higher in a panel constructed from 4 surveys. As for<br />
earlier SYETP evaluations, again no treatment for non-response was made, although in<br />
<strong>the</strong> case <strong>of</strong> Richardson (1998) this was in line with <strong>the</strong> conclusions whereas for earlier<br />
evaluations this was in spite <strong>of</strong> <strong>the</strong> conclusions about <strong>the</strong> existence <strong>of</strong> a potential nonresponse<br />
problem for estimates.
90<br />
An advantage over earlier SYETP analyses is that <strong>the</strong> comparison group was nonparticipants<br />
ra<strong>the</strong>r than participants in o<strong>the</strong>r programmes. This allows <strong>the</strong> more pertinent<br />
question for evaluation to be asked: instead <strong>of</strong> whe<strong>the</strong>r <strong>the</strong> programme was effective<br />
relative to ano<strong>the</strong>r programme to whe<strong>the</strong>r <strong>the</strong> programme was effective in improving<br />
post-programme employment in an absolute sense compared to non-participants.<br />
Retention in <strong>the</strong> post-programme job is addressed through sample selection for those<br />
where continuous employment to <strong>the</strong> end <strong>of</strong> 1986 was not <strong>the</strong> case. This has potential to<br />
introduce <strong>the</strong> need to fur<strong>the</strong>r model <strong>the</strong> selection process <strong>of</strong> retention or tenure in <strong>the</strong> one<br />
job, as it may be endogenous. Essentially, it selects a sample with short tenure jobs, and<br />
durations are censored at 1986. This might introduce o<strong>the</strong>r difficulties related to duration<br />
modelling and censoring <strong>of</strong> spells too. Richardson (1998), in relation to concerns for <strong>the</strong><br />
sample reduction, examines <strong>the</strong> means <strong>of</strong> <strong>the</strong> retained amongst <strong>the</strong> treated and<br />
comparisons, and <strong>the</strong> effect on size <strong>of</strong> <strong>the</strong> treated sample against that <strong>of</strong> <strong>the</strong> comparisons.<br />
The conclusion <strong>of</strong> no bias problems relies on <strong>the</strong> observation that <strong>the</strong> retained are a<br />
similar share <strong>of</strong> treated and <strong>of</strong> comparisons, and that <strong>the</strong> proportion <strong>of</strong> SYETP before and<br />
after sample reduction is <strong>the</strong> same. This examination is more related to balance <strong>of</strong> <strong>the</strong><br />
retention characteristic in <strong>the</strong> treated and comparison groups than whe<strong>the</strong>r retention is<br />
related to participation in SYETP.
91<br />
2.3.5 General discussion<br />
Some general observations are apparent in comparing <strong>the</strong> SYETP analyses. All analyses<br />
<strong>of</strong> SYETP proceeded with survey data to enable <strong>the</strong> recording <strong>of</strong> post-programme<br />
employment. The earlier analyses <strong>of</strong> SYETP did not have a non-participant comparison<br />
group, whereas <strong>the</strong> Richardson (1998) analysis did. The results enable a gauge <strong>of</strong> whe<strong>the</strong>r<br />
participation in <strong>the</strong> SYETP programme was effective in improving post-programme<br />
employment. All analyses found positive effects.<br />
Three key <strong>the</strong>mes emerge from <strong>the</strong> past analyses <strong>of</strong> SYETP. These relate to accounting<br />
for selection into programme participation, non-response in <strong>the</strong> observational data used,<br />
and controlling for differences in individual characteristics between <strong>the</strong> comparison and<br />
treatment groups. These <strong>the</strong>mes are fur<strong>the</strong>r developed in <strong>the</strong> later chapters <strong>of</strong> this study.<br />
In <strong>the</strong> following discussion, <strong>the</strong> past SYETP literature is assessed with regard to <strong>the</strong>se<br />
points.<br />
Richardson (1998) is <strong>the</strong> evaluation <strong>of</strong> SYETP motivating this current study. The format<br />
<strong>of</strong> <strong>the</strong> Richardson (1998) evaluation forms a key basis for our analyses, which are<br />
contained in <strong>the</strong> later chapters starting with <strong>the</strong> Replication Study. In brief, Richardson<br />
used a Heckman bivariate probit which controlled for programme selection to analyse <strong>the</strong><br />
SYETP using ALS panel data. A strong positive effect on post-programme employment<br />
was found. In this review, this evaluation is considered to contain <strong>the</strong> most sophisticated<br />
analysis <strong>of</strong> SYETP to date, allowing for both programme selection and individual<br />
characteristics. This evaluation also showed evidence that accounting for selection into<br />
SYETP participation in <strong>the</strong> modelling <strong>of</strong> employment was important in two ways: by<br />
testing <strong>the</strong> correlation coefficient and finding it statistically significant; and by showing<br />
<strong>the</strong> comparison to <strong>the</strong> unadjusted univariate employment results, which were also very<br />
different despite <strong>the</strong> employment equation being o<strong>the</strong>rwise specified identically. The<br />
modelling <strong>of</strong> <strong>the</strong> effects on employment was <strong>the</strong> best developed <strong>of</strong> those evaluations<br />
reviewed, since it took into account selection into programme participation.
92<br />
However, <strong>the</strong> Richardson (1998) examination <strong>of</strong> non-response was less developed than<br />
<strong>the</strong> earlier SYETP analyses where non-response was considered. In common with all <strong>the</strong><br />
SYETP analyses, survey data was used. Stretton (1984) and Baker (1984) modelled nonresponse<br />
in <strong>the</strong>ir observational data and found non-response was a potential problem. As<br />
both <strong>of</strong> <strong>the</strong>se had <strong>the</strong> administrative sample frame to identify some variables for<br />
modelling non-response to <strong>the</strong> survey, <strong>the</strong>y were at an advantage. Richardson (1998)<br />
could not benefit from <strong>the</strong> administrative sample frame as this was not available with <strong>the</strong><br />
survey data deposited, and this precluded this type <strong>of</strong> analysis. However it was possible<br />
to choose to use <strong>the</strong> non-response weights provided with <strong>the</strong> data, which had been<br />
developed from <strong>the</strong> non-response analysis for <strong>the</strong> first survey using <strong>the</strong> administrative<br />
data. Fur<strong>the</strong>r, <strong>the</strong> panel <strong>of</strong> data used had 3 subsequent surveys, to which non-response 59<br />
could be modelled from <strong>the</strong> first survey.<br />
In common with <strong>the</strong> earlier analyses, no treatment for non-response was applied. In <strong>the</strong><br />
case <strong>of</strong> Richardson (1998) this was in line with <strong>the</strong> conclusions maintained from <strong>the</strong><br />
analysis <strong>of</strong> non-response. For earlier evaluations by Stretton (1984) and Baker (1984) this<br />
was in spite <strong>of</strong> <strong>the</strong> conclusions about <strong>the</strong> existence <strong>of</strong> a potential non-response problem<br />
for estimates as a result <strong>of</strong> useful models <strong>of</strong> survey non-response from <strong>the</strong> administrative<br />
data sample. For Rao and Jones (1986), which used data from Baker (1984) but with<br />
additional survey data, ignoring non-response was in conflict with <strong>the</strong> Baker (1984)<br />
analysis, and <strong>the</strong> aggravation <strong>of</strong> non-response problems due to fur<strong>the</strong>r attrition to <strong>the</strong> later<br />
survey was also ignored. In review it is likely that <strong>the</strong> choice <strong>of</strong> models potentially<br />
available with computing resources at <strong>the</strong> time limited modelling in <strong>the</strong>se earlier analyses<br />
<strong>of</strong> SYETP. A choice had to be made as to what was <strong>the</strong> most important modelling<br />
consideration, given <strong>the</strong> computational capabilities <strong>of</strong> <strong>the</strong> time. As a result, non-response<br />
was not fur<strong>the</strong>r accounted for. This is an adequate modelling assumption, but fur<strong>the</strong>r<br />
invokes <strong>the</strong> assumption that <strong>the</strong> non-response was ignorable for <strong>the</strong>ir modelling. In<br />
Stretton (1984) and Baker (1984), <strong>the</strong> examination <strong>of</strong> non-response indicated this was not<br />
generally tenable. This issue is dealt with in <strong>the</strong> study <strong>of</strong> non-response that follows.<br />
59 Usually termed attrition in this case.
93<br />
Controlling for differences in individual characteristics between <strong>the</strong> comparison and<br />
treatment groups was carried out in all <strong>of</strong> <strong>the</strong> SYETP evaluations using econometric<br />
modelling <strong>of</strong> <strong>the</strong> post-programme employment. This accounts for observed differences in<br />
individual characteristics between <strong>the</strong> comparison and treatment groups by including<br />
<strong>the</strong>m in <strong>the</strong> model. The earlier modelling showed that controlling for individual<br />
characteristics was important, but <strong>the</strong> set <strong>of</strong> characteristics was limited. The key issue<br />
<strong>the</strong>n is whe<strong>the</strong>r all <strong>the</strong> observable characteristics that might affect employment have been<br />
included. The Richardson (1998) analysis had a very broad set <strong>of</strong> characteristics and it<br />
showed that some <strong>of</strong> <strong>the</strong>se, such as education, are indeed important for modelling<br />
employment. Leaving out important variables for modelling employment is likely to lead<br />
to specification bias.<br />
Controlling for differences in individual characteristics between <strong>the</strong> control and treatment<br />
groups is treated fur<strong>the</strong>r in this study when different modelling methods are used. As <strong>the</strong><br />
earlier review <strong>of</strong> wage subsidy evidence shows, matching techniques can also be used to<br />
resolve differences in individual characteristics between <strong>the</strong> comparison and treatment<br />
groups. Later <strong>Australian</strong> evaluation <strong>of</strong> <strong>the</strong> Jobstart wage subsidy used direct matching<br />
techniques. These methods however require a limited set <strong>of</strong> characteristics and large data<br />
sample in order to enable <strong>the</strong> analysis to proceed. The more characteristics it is desirable<br />
to match on, <strong>the</strong> fewer matches become available amongst <strong>the</strong> treatment and comparison<br />
groups. As <strong>the</strong> earlier modelling <strong>of</strong> SYETP showed, controlling for individual<br />
characteristics was important, and <strong>the</strong> Richardson (1998) evidence indicates <strong>the</strong> breadth<br />
<strong>of</strong> characteristics that might be useful. Matching, such as that <strong>of</strong> <strong>the</strong> Jobstart evaluations,<br />
which does not control for <strong>the</strong>se o<strong>the</strong>r characteristics that affect both employment and<br />
participation would <strong>the</strong>n be subject to bias. As such, it is <strong>the</strong>n most useful to control for a<br />
fairly broad set <strong>of</strong> characteristics. O<strong>the</strong>r recent overseas evidence has used propensity<br />
score matching (PSM) techniques, which allow a greater number <strong>of</strong> characteristics to be<br />
controlled for, for example Bonjour et al (2001). This method has not been used for any<br />
published <strong>Australian</strong> evaluation so far. The PSM method is applied in <strong>the</strong> following study.
94<br />
3: Study 1 Replication<br />
In this section, <strong>the</strong> first aim is to first replicate <strong>the</strong> bivariate probit analysis <strong>of</strong> SYETP as<br />
reported by Richardson (1998). The key value <strong>of</strong> <strong>the</strong> replication is to validate <strong>the</strong> results,<br />
and to ensure that <strong>the</strong> later analyses are carried out using comparable data. The<br />
Richardson (1998) analysis was <strong>the</strong> most sophisticated <strong>of</strong> <strong>the</strong> SYETP analyses, as shown<br />
in <strong>the</strong> earlier review. This study has selected to build upon Richardson (1998) for this<br />
reason.<br />
Firstly, <strong>the</strong> motivation for replication is presented. Then, <strong>the</strong> methods needed to carry out<br />
<strong>the</strong> replication are described. The data are briefly detailed. Finally, <strong>the</strong> results <strong>of</strong> <strong>the</strong><br />
replication are shown and discussed.<br />
3.1 Motivation for replication<br />
A number <strong>of</strong> papers have discussed <strong>the</strong> importance <strong>of</strong> replication. King (1995, 2002) has<br />
been one <strong>of</strong> <strong>the</strong> more vocal in recently maintaining <strong>the</strong> importance <strong>of</strong> <strong>the</strong> scientific step<br />
<strong>of</strong> replication. However, earlier work by Dewald, Thursby and Anderson (1986) also<br />
presented strong evidence <strong>of</strong> <strong>the</strong> value <strong>of</strong> replication. King (2002) p1 describes <strong>the</strong><br />
‘Replication Standard’:<br />
“Sufficient information exists with which to understand, evaluate and build<br />
upon a prior work if a third party can replicate <strong>the</strong> results without any<br />
additional information from <strong>the</strong> author.”<br />
Empirical evidence that meets <strong>the</strong> replication standard allows <strong>the</strong> conclusions <strong>of</strong> <strong>the</strong> work<br />
to be maintained and built upon in accordance with <strong>the</strong> scientific method. King (1995)<br />
argues convincingly that “…Without complete information about where <strong>the</strong> data have<br />
come from and how we measured <strong>the</strong> real world and abstracted from it, we cannot truly<br />
understand a set <strong>of</strong> empirical results…” (King (1995): 445). King (1995) p 445 also<br />
suggests that replication is a prerequisite to fur<strong>the</strong>r development and is <strong>the</strong> most<br />
productive method <strong>of</strong> building on existing research, by “…following <strong>the</strong> precise path
95<br />
taken by a previous researcher and <strong>the</strong>n improve on <strong>the</strong> data or methodology in some way<br />
or ano<strong>the</strong>r…”.<br />
Dewald et al. (1986) provide evidence <strong>of</strong> <strong>the</strong> value <strong>of</strong> <strong>the</strong> application <strong>of</strong> <strong>the</strong> replication<br />
standard. They point out that <strong>the</strong> confirmation <strong>of</strong> research findings through replication by<br />
o<strong>the</strong>r researchers is an essential component <strong>of</strong> scientific methodology. To examine <strong>the</strong><br />
role <strong>of</strong> replication in empirical economic research <strong>the</strong>y took one year's worth <strong>of</strong> articles<br />
from The Journal <strong>of</strong> Money, Credit and Banking and attempted replication <strong>of</strong> all <strong>the</strong><br />
articles. Extensive efforts were reported but also largely failed attempts to replicate <strong>the</strong><br />
findings as <strong>the</strong>y were reported. They <strong>of</strong>ten found that replication was impossible based on<br />
<strong>the</strong> information in <strong>the</strong> articles, and also subject to ambiguities, errors and oversights in<br />
<strong>the</strong> reporting. They emphasized that replication is an essential element in <strong>the</strong> evaluation<br />
and unification <strong>of</strong> <strong>the</strong> results for a group <strong>of</strong> studies.<br />
3.2 Methods: The Heckman selection model<br />
The specification to be replicated is <strong>the</strong> bivariate probit <strong>of</strong> employment, where <strong>the</strong><br />
selection equation modelled is that <strong>of</strong> selection into SYETP. The analysis based on <strong>the</strong><br />
‘1986 data’ is attempted here, and <strong>the</strong> results as shown in Richardson (1998) Tables 4 and<br />
5 are replicated.<br />
3.2.1 Self-selection and <strong>the</strong> evaluation <strong>of</strong> programmes<br />
Evaluating <strong>the</strong> benefits <strong>of</strong> a social programme is one <strong>of</strong> <strong>the</strong> major uses <strong>of</strong> self-selection<br />
models. There are many variations <strong>of</strong> <strong>the</strong> self-selection model depending on <strong>the</strong> problem<br />
<strong>the</strong> model is applied to solve. The terminology is that <strong>of</strong> <strong>the</strong> experiment, where <strong>the</strong><br />
programme is <strong>the</strong> treatment being evaluated. The individuals who participate in <strong>the</strong><br />
programme are <strong>the</strong> treatment group and o<strong>the</strong>rs not participating are in <strong>the</strong> comparison<br />
group. In <strong>the</strong> case <strong>of</strong> <strong>the</strong> SYETP, it is assumed that <strong>the</strong> programme administrator<br />
essentially makes <strong>the</strong> assignment <strong>of</strong> individuals to <strong>the</strong> treatment groups, but <strong>the</strong> process is<br />
jointly determined with <strong>the</strong> employers and <strong>the</strong> participant. Selection onto <strong>the</strong> programme<br />
could have led to systematic differences between those who went on <strong>the</strong> SYETP and
96<br />
those who did not. Modelling only <strong>the</strong> employment infers that <strong>the</strong> selection onto <strong>the</strong><br />
programme was a random process. The selection problem arises where programme<br />
participants are not selected randomly from <strong>the</strong> population. The aim <strong>of</strong> <strong>the</strong> programme is<br />
to improve employability. Employability after <strong>the</strong> programme is not observed, but instead<br />
post-programme employment is observed. The probability <strong>of</strong> participating is not<br />
observed, but participation is observed. The evaluation <strong>the</strong>n needs to resolve <strong>the</strong> link<br />
between employment and programme participation. The selection modelling problem<br />
occurs if <strong>the</strong>re are unobserved characteristics affecting both participation and postprogramme<br />
employment, for example <strong>the</strong>ir employability before <strong>the</strong> programme.<br />
3.2.2 The estimated model<br />
Richardson (1998) states his estimated model is <strong>of</strong> <strong>the</strong> form used by van de Ven and van<br />
Praag (1981). In van de Ven and van Praag (1981) <strong>the</strong> model was used for <strong>the</strong> decision to<br />
prefer health insurance with a deductible, to one with complete coverage, and <strong>the</strong> sample<br />
selection problem arose as a result <strong>of</strong> non-response. Their extension to <strong>the</strong> Heckman<br />
(1979) method <strong>of</strong> sample selection control was to apply it in <strong>the</strong> probit analysis context,<br />
where not only <strong>the</strong> first but also <strong>the</strong> second stage has a discrete dependent variable.<br />
Maddala (1983) points out that <strong>the</strong> econometric discussion <strong>of</strong> <strong>the</strong> consequences <strong>of</strong> selfselectivity<br />
began with <strong>the</strong> studies <strong>of</strong> Gronau (1974), Lewis (1974) and Heckman (1974),<br />
but that it was not until Heckman (1976) that <strong>the</strong> two stage selection model was<br />
suggested as <strong>the</strong> solution. Essentially, by fully modelling <strong>the</strong> entry into participation, <strong>the</strong>n<br />
allowing a correction factor for <strong>the</strong> participation to be included in <strong>the</strong> employment model,<br />
<strong>the</strong> sample selection process affecting employment can be reasonably accounted for.<br />
However, <strong>the</strong> bivariate probit can also be seen as a system <strong>of</strong> simultaneous equations, a<br />
simultaneous probit model. More recently, Knapp and Seaks (1998) describe a two<br />
equation recursive probit model and note that <strong>the</strong> bivariate probit model can be analogous<br />
to <strong>the</strong> results by Lahiri and Schmidt (1978) for <strong>the</strong> triangular system <strong>of</strong> simultaneous<br />
equations with cross equation error correlation.
97<br />
If employability is assumed to be a latent variable y*, and <strong>the</strong> probability <strong>of</strong> being<br />
selected onto <strong>the</strong> programme is d*, <strong>the</strong>n <strong>the</strong> bivariate probit <strong>of</strong> <strong>the</strong> joint probabilities <strong>of</strong><br />
selection and employment <strong>the</strong>n takes <strong>the</strong> form:<br />
1) y i *=αd i + β′x i + ε i employment equation<br />
where y i =1 for y i *>0<br />
y i =0 o<strong>the</strong>rwise<br />
2) d i *=γz i + v i participation equation<br />
where d i =1 for d i *>0<br />
d i =0 o<strong>the</strong>rwise<br />
d i = dummy variable with value 1 for participation in wage subsidy, 0 o<strong>the</strong>rwise<br />
d i *=probability <strong>of</strong> being selected for participation in wage subsidy<br />
x i =vector <strong>of</strong> exogenous individual characteristics<br />
v i = error term, distributed as standard normal<br />
ε i =error term, distributed as standard normal<br />
y i =employment with value 1 for employed in a time period, 0 o<strong>the</strong>rwise<br />
y i *=employability<br />
z i = vector <strong>of</strong> exogenous individual characteristics<br />
Each subscript i indicating <strong>the</strong> individual.<br />
Employment y is observed where y=1 if <strong>the</strong> periods is employed in some period, and y=0<br />
o<strong>the</strong>rwise. If y=1 when y*>0 and y=0 o<strong>the</strong>rwise, <strong>the</strong>n equation 1 can be estimated as a<br />
probit if it is assumed that ε i is distributed as a standard normal. However, it may be <strong>the</strong>re<br />
is a selection problem as programme participants are not randomly selected from <strong>the</strong><br />
population. If <strong>the</strong> probability <strong>of</strong> being selected onto <strong>the</strong> programme d* is determined by<br />
equation 2, and v i are distributed as a standard normal, and <strong>the</strong>re are unobserved<br />
characteristics that affect programme entry and subsequent employability, <strong>the</strong>n ε i and v i<br />
are correlated. The bivariate probit assumes that ε i and v i are correlated, and that <strong>the</strong>y<br />
follow a bivariate normal distribution.<br />
E(ε i v i )=ρ<br />
Prob (y i =1, d i =1) = Φ (αd i + β′x i , γz i , ρ)
98<br />
Φ = cumulative distribution function for <strong>the</strong> standard bivariate normal<br />
This approach has been applied in several evaluation problems for solving <strong>the</strong> issue <strong>of</strong><br />
selection bias, where programme participants are not randomly selected, as discussed in<br />
<strong>the</strong> brief review <strong>of</strong> Maddala (1983). It allows for selection into participation in <strong>the</strong> wage<br />
subsidy on <strong>the</strong> basis <strong>of</strong> variables that are unobservable, or more usually variables that are<br />
unobserved in <strong>the</strong> data. The model is made operational by assuming a bivariate normal<br />
distribution for <strong>the</strong> errors in <strong>the</strong> participation and employment equations. This assumption<br />
makes <strong>the</strong> unobservable characteristics that jointly influence participation and<br />
employment follow <strong>the</strong> bivariate normal distribution. The participation equation is<br />
explicitly modelled to provide a variable that is <strong>the</strong>n used to control for <strong>the</strong> part <strong>of</strong><br />
unobserved variation <strong>of</strong> <strong>the</strong> employment equation that is correlated with <strong>the</strong> unobserved<br />
variation in <strong>the</strong> participation decision. With full information maximum likelihood<br />
methods, <strong>the</strong> selection and employment equations are estimated jointly, in which case <strong>the</strong><br />
correlation between <strong>the</strong> unobservables is estimated directly.<br />
The sets <strong>of</strong> exogenous variables x i , z i can be overlapping, in which case only <strong>the</strong><br />
functional form assumption provides identification. However, in practice this is not<br />
recommended as identification based solely on functional form is likely to be empirically<br />
fragile. The model is more appropriately identified if at least one variable in <strong>the</strong> set z i is<br />
not in <strong>the</strong> set x i , as <strong>the</strong> exclusion restriction improves <strong>the</strong> identification <strong>of</strong> <strong>the</strong> parameters<br />
<strong>of</strong> interest. Thus, practical implementation usually requires a variable included in <strong>the</strong><br />
participation equation estimated that is excluded from <strong>the</strong> employment equation. 60 This<br />
exogenous variable should be suitable in that it influences selection but not employability.<br />
An important aspect <strong>of</strong> this estimation approach is <strong>the</strong> identification <strong>of</strong> a credible<br />
instrument for <strong>the</strong> exclusion restriction. Also, <strong>the</strong> results <strong>of</strong> estimation rest upon <strong>the</strong><br />
60 Wilde (2000) points out that earlier references presented contradictory opinions on this matter, so that<br />
Maddala (1983) held that a model with overlapping sets <strong>of</strong> exogenous variables x i , z i was only identified if<br />
at least one variable in <strong>the</strong> set z i is not in <strong>the</strong> set x i , while Heckman (1978) maintained that <strong>the</strong> functional<br />
form sufficed for identification. However Wilde (2000) clarifies that Maddala’s argument is only valid for<br />
his particular example, and is not <strong>the</strong> case generally due to <strong>the</strong> nonlinear relationships between z and <strong>the</strong><br />
probability P(d=1 | z), and fur<strong>the</strong>r concluded it could be avoided by assuming each equation contains at<br />
least one varying exogenous regressor. Wilde (2000) points out that this is a ra<strong>the</strong>r weak assumption in<br />
economic applications. Hence, Maddala’s argument commonly holds in econometric practice.
99<br />
suitability <strong>of</strong> <strong>the</strong> underlying assumption <strong>of</strong> <strong>the</strong> bivariate normal distribution for <strong>the</strong> errors<br />
in <strong>the</strong> participation and employment equations.<br />
3.3 Data and variables used for estimation<br />
The first stage <strong>of</strong> <strong>the</strong> replication, whereby <strong>the</strong> variables used in <strong>the</strong> analysis are reconstructed,<br />
is made easier by access to <strong>the</strong> final data set used as <strong>the</strong> basis for <strong>the</strong><br />
Richardson (1998) analysis. 61 It was pointed out by Dewald et al. (1986) that access to<br />
<strong>the</strong> author’s data, with <strong>the</strong> variables in <strong>the</strong> final form used for <strong>the</strong> analysis, is an<br />
invaluable assistance in replication, but observed that this was an unusual occurrence.<br />
They attributed this to angst held by authors as <strong>the</strong>y “…may interpret <strong>the</strong> very act <strong>of</strong><br />
replication as a challenge to <strong>the</strong>ir pr<strong>of</strong>essional competence and integrity” (Dewald et al.<br />
(1986): 601). In light <strong>of</strong> this, <strong>the</strong> access granted by <strong>the</strong> author James Richardson is<br />
strongly appreciated.<br />
As a result <strong>of</strong> access to <strong>the</strong> transformed data, replicating <strong>the</strong> exact construction <strong>of</strong> <strong>the</strong><br />
variables is simplified. This would normally be an exacting task, as <strong>the</strong> construction <strong>of</strong><br />
complex variables such as those derived from <strong>the</strong> work history is extremely difficult to<br />
repeat without <strong>the</strong> exact details <strong>of</strong> construction, and even <strong>the</strong> exact syntax coding. Any<br />
variation in construction would make replication far more difficult, and overcoming this<br />
hurdle also allows interpretation <strong>of</strong> <strong>the</strong> replication results to be more precise. It should<br />
not be overlooked that this stage <strong>of</strong> a replication would normally be most difficult. The<br />
variables are described in <strong>the</strong> data appendix.<br />
A brief introduction to <strong>the</strong> data is informative. The <strong>Australian</strong> Longitudinal Survey List<br />
Sample (Mcrae et al. 1984-1987) is used. This was a sample drawn from an<br />
administrative sample frame. The ALS list sample was a nationally representative sample<br />
<strong>of</strong> <strong>Australian</strong> youths aged 15-24 who had been registered as unemployed with <strong>the</strong><br />
Commonwealth Employment Service for at least 3 months in June 1984. The 1984<br />
survey took place with interviews between September 1984 and November 1984. The<br />
survey was repeated in each <strong>of</strong> <strong>the</strong> later years 1985, 1986 and 1987, following up where<br />
61 The data were transformed from <strong>the</strong> ALS original SPSS files in extensive STATA processing by<br />
Lorraine Dearden, Alex Heath, Henry Overman, and James Richardson.
100<br />
possible <strong>the</strong> same people with whom interviews were conducted in 1984. Information<br />
was collected about job history, job search behaviour, job training, transitions from<br />
education to work, education, occupation, health, attitude to women working, parental<br />
education and occupation, racial origin, country <strong>of</strong> birth, age, sex, size <strong>of</strong> household,<br />
spousal education and occupation, religion, income, and urban/rural residence. The<br />
background variables available for study are <strong>the</strong>n quite extensive. Fur<strong>the</strong>r discussion <strong>of</strong><br />
<strong>the</strong> data details is covered in <strong>the</strong> later sections to which it is immediately relevant.<br />
The breakdown <strong>of</strong> SYETP types as indicated in Chapter 2 is not available in <strong>the</strong> data. 62<br />
The key issue <strong>of</strong> programme heterogeneity would appear to be problematic. It is however<br />
possible to identify whe<strong>the</strong>r <strong>the</strong> SYETP job was in <strong>the</strong> public or private sector, from <strong>the</strong><br />
job characteristics. All SYETP placements identified using <strong>the</strong> SYETP variable in <strong>the</strong><br />
data were for private sector SYETP. The possibility <strong>of</strong> Extended SYETP placements is<br />
addressed and he notes that <strong>the</strong> analysis <strong>of</strong> <strong>the</strong> spells indicates that <strong>the</strong>re were not spells<br />
<strong>of</strong> lengths indicative <strong>of</strong> extended SYETP placements (Richardson (1998): 9).<br />
Careful attention is paid to <strong>the</strong> description <strong>of</strong> <strong>the</strong> sample selected for analysis as reported<br />
by <strong>the</strong> author, in order to enable <strong>the</strong> same base to be used for <strong>the</strong> replication. The limits <strong>of</strong><br />
<strong>the</strong> sample are well described detailing <strong>the</strong> exclusion from those interviewed in 1984 <strong>of</strong><br />
those who were over 25 years at <strong>the</strong> interview date, those who are in full-time education,<br />
or for whom responses were missing in 1984, 1985 or 1986 (Richardson (1998): 5). The<br />
final observations number 1283, <strong>of</strong> which 104 are in <strong>the</strong> treatment group ‘SYETP’. 63<br />
Applying <strong>the</strong>se rules, <strong>the</strong> same number <strong>of</strong> cases is successfully achieved, and <strong>the</strong><br />
replication <strong>of</strong> <strong>the</strong> estimation is repeated on this data. Replication <strong>of</strong> <strong>the</strong> sample can be<br />
highly informative, as this is where most <strong>of</strong> <strong>the</strong> difference in results arose in <strong>the</strong> Smith<br />
and Todd (2000, 2003) replication <strong>of</strong> work by Dehejia and Wahba (1998,1999).<br />
62 Jobs which are placements to any SYETP program are simply described as SYETP, with <strong>the</strong> program<br />
variations not accounted for.<br />
63 As Richardson (1998) details, roughly half <strong>of</strong> <strong>the</strong>se 104 SYETP had program participation start weeks in<br />
1984, with <strong>the</strong> rest in 1985. The average start week is <strong>the</strong> week beginning 13 January 1985, and <strong>the</strong> range<br />
<strong>of</strong> start weeks is 72 from week 40 to week 112.
101<br />
Heckman, Lalonde and Smith (1999) p1912 clearly point out that different choices about<br />
data handling such as variable choice and decisions about whom to include in <strong>the</strong> sample<br />
can strongly influence <strong>the</strong> measurement <strong>of</strong> treatment impacts. To avoid differences<br />
arising from this source, <strong>the</strong> initial replication is made as close as possible to that <strong>of</strong> <strong>the</strong><br />
original. Using <strong>the</strong> reported bivariate probit equation in Richardson (1998) Tables 4 and 5,<br />
<strong>the</strong> variables matching those described are selected for use in <strong>the</strong> estimation. This means<br />
that <strong>the</strong> same identifying restrictions are applied. Thus, CEP referrals and age are<br />
excluded from <strong>the</strong> employment equation but included in <strong>the</strong> selection equation,<br />
additionally in <strong>the</strong> employment equation qualifications, children, marital status and health<br />
affecting work are 1985 or 1986 dated and in <strong>the</strong> selection equation <strong>the</strong>y are 1984 dated.<br />
The same set <strong>of</strong> base variables are used for <strong>the</strong> equations as that <strong>of</strong> Richardson (1998) by<br />
reference to <strong>the</strong> reported bivariate probit results. The variables are described in detail in<br />
Appendix 1: Data Appendix, Table 1. Analysis was carried out in STATA 7.0.<br />
3.4 Replication results<br />
The results <strong>of</strong> replication for <strong>the</strong> bivariate probit are shown in Table 3.1, parts a and b. To<br />
facilitate comparison <strong>of</strong> <strong>the</strong> replication result with <strong>the</strong> original results, <strong>the</strong> estimates as<br />
shown in Richardson (1998) Tables 4 and 5 are repeated in column 1 <strong>of</strong> Table 3.1 part a<br />
and part b. Table 3.1 part a shows <strong>the</strong> employment equation <strong>of</strong> <strong>the</strong> bivariate probit, while<br />
Table 3.1 part b shows <strong>the</strong> selection equation <strong>of</strong> <strong>the</strong> bivariate probit which models <strong>the</strong><br />
entry into SYETP. The second column <strong>of</strong> Table 3.1 gives <strong>the</strong> replication estimates.<br />
The replication is found to be generally successful. The replicated results match closely<br />
those <strong>of</strong> Richardson (1998). There is some minor variation, but at <strong>the</strong> 2 decimal places<br />
level, most estimates <strong>of</strong> <strong>the</strong> coefficients are identical. There is slightly more disparity in<br />
<strong>the</strong> t-statistics. The coefficient <strong>of</strong> key interest is <strong>the</strong> SYETP variable in <strong>the</strong> employment<br />
equation, which measures <strong>the</strong> treatment effect. In this case, <strong>the</strong> replication result is<br />
slightly larger, and <strong>the</strong> t-statistic is also larger. However, in no instance does <strong>the</strong><br />
replication report results that are not closely comparable to those reported in Richardson<br />
(1998). The appendix Tables A2.0a and A2.0b show <strong>the</strong> replicated univariate probits, for
102<br />
which <strong>the</strong> employment equation was also reported in Richardson (1998) Table3, p16.<br />
These are identically specified to <strong>the</strong> bivariate probit, but <strong>of</strong> course are separate and<br />
independent. When compared to <strong>the</strong> Richardson estimates, <strong>the</strong>se simple probit replicates<br />
are identical. Additionally, <strong>the</strong> means for all <strong>the</strong> variables were calculated and found to<br />
match those <strong>of</strong> Richardson 1998 Table 1 (<strong>the</strong>se can be seen later in Chapter 5, for<br />
example Table 5.2). As <strong>the</strong> simpler replication exercises produced results identical to<br />
those <strong>of</strong> Richardson (1998), <strong>the</strong> small discrepancies observed are thought to arise from<br />
minor differences in <strong>the</strong> estimation algorithm. These analyses are conducted in STATA<br />
7.0, <strong>the</strong> Richardson (1998) analyses were in Stata 5.0, in later versions algorithm changes<br />
have been made to <strong>the</strong> code for implementing <strong>the</strong> bivariate probit.<br />
It is useful to briefly discuss <strong>the</strong> factors found to impact on employment and participation<br />
in SYETP. Although this is a replication, <strong>the</strong> results <strong>of</strong> <strong>the</strong> estimation are <strong>of</strong> interest for<br />
comparison with later results. For reference to <strong>the</strong> variable definition, see <strong>the</strong> data<br />
appendix. Only statistically significant coefficients are discussed.<br />
In <strong>the</strong> employment equation, Table 3.1 part a column 2, SYETP has a positive coefficient,<br />
indicating a positive effect upon employment chances, while o<strong>the</strong>r positive effects on<br />
employment were associated with <strong>the</strong> partner being in employment, having attended a<br />
private school, higher post-school qualifications and having left school in year 11 relative<br />
to <strong>the</strong> base <strong>of</strong> year 10 highest qualification 64 , and having held a job for 3 years prior to<br />
1984. These all represent higher levels <strong>of</strong> human capital and work experience, and greater<br />
labour supply and are in <strong>the</strong> direction <strong>the</strong>y are expected to work. Negative effects on<br />
employment chances were found for women with children, those with health limiting<br />
<strong>the</strong>ir work, those with longer pre-programme unemployment, those who had o<strong>the</strong>r<br />
programme experience. Of <strong>the</strong> more unusual variables, Catholic religion had a positive<br />
64 Note that in <strong>the</strong> <strong>Australian</strong> schooling system, a certificate is achieved in year 10 and <strong>the</strong>n ano<strong>the</strong>r in year<br />
12 after successful completion <strong>of</strong> <strong>the</strong> study program. Those who left in year 10 could include those who<br />
achieved <strong>the</strong> certificate or left during <strong>the</strong> year prior to obtaining <strong>the</strong> certificate or failed <strong>the</strong> certificate<br />
studies. Those who left in year 11 had definitely obtained <strong>the</strong>ir year 10 certificate and so qualified to<br />
continue <strong>the</strong>ir studies. Each <strong>Australian</strong> state has a different system <strong>of</strong> education and each certificate has a<br />
different name and required studies.
103<br />
effect on employment relative to <strong>the</strong> base <strong>of</strong> Church <strong>of</strong> England. 65 Surprisingly, fair to<br />
poor English had a positive effect on employment relative to natural English speakers,<br />
however this was a self-assessed question and <strong>the</strong>re might have been a tendency for those<br />
with non-English-speaking-background to describe <strong>the</strong>ir English optimistically 66 , also <strong>the</strong><br />
smallest share had fair-poor English so this rests on a small number <strong>of</strong> cases. Those who<br />
gave opinions expressing a negative attitude towards women in work also had lower<br />
chances <strong>of</strong> employment – as this is attitudinal material, this may also reflect o<strong>the</strong>r<br />
attitudinal features such as a degree <strong>of</strong> anti-social behaviour. Maternal background in <strong>the</strong><br />
plant operative occupations also had a negative effect on employment outcomes.<br />
In <strong>the</strong> participation equation, age was negative, indicating that older eligible youths had<br />
lower chances <strong>of</strong> placements – this is in line with <strong>the</strong> findings in o<strong>the</strong>r SYETP literature.<br />
The age <strong>of</strong> <strong>the</strong> individual was one <strong>of</strong> <strong>the</strong> variables excluded from <strong>the</strong> employment<br />
equation. The key variable for <strong>the</strong> selection was however CEP referrals, and this had a<br />
positive impact on placement chances. O<strong>the</strong>r negative effects on placements were from<br />
health problems limiting work and childhood background in a country town. Positive<br />
effects on placement chances came from having highest qualification <strong>of</strong> year 12<br />
schooling (relative to year 10 schooling base) and a longer qualifying unemployment<br />
period also raised placement chances, in line with <strong>the</strong> eligibility criteria for SYETP.<br />
Maternal background <strong>of</strong> plant-operative occupation raised chances <strong>of</strong> a placement, which<br />
was opposite to <strong>the</strong> negative effect it had on employment.<br />
3.5 Discussion<br />
It is worth noting, in light <strong>of</strong> ‘<strong>the</strong> replication standard’, that no additional information was<br />
needed to carry out this replication. The replication was achieved using only Richardson<br />
(1998) and <strong>the</strong> supplied data set. For a replication data set to be useful King (1995) notes<br />
that it must include all information necessary to replicate <strong>the</strong> empirical results. As such,<br />
65 But this effect may arise toge<strong>the</strong>r with <strong>the</strong> type <strong>of</strong> schooling being private – <strong>the</strong>re is a greater share <strong>of</strong><br />
Catholic private schools.<br />
66 The base natural English speaking held <strong>the</strong> greatest share <strong>of</strong> <strong>the</strong> 1283 cases (1161 or 91 per cent), with<br />
good English 93 cases or 7 per cent, and fair to poor 29 cases or 2 per cent. In <strong>the</strong> 104 SYETP cases, only 1<br />
had fair-poor English, while 8 had good English and <strong>the</strong> rest were native English speakers (unweighted<br />
percentages).
104<br />
<strong>the</strong> quality <strong>of</strong> this enabled <strong>the</strong> reproduced work, and so <strong>the</strong> original work met ‘<strong>the</strong><br />
replication standard’.<br />
In summary, it is concluded that it is evident replication has a key role as a starting point<br />
for useful development <strong>of</strong> <strong>the</strong> analysis presented by Richardson (1998). By replicating, it<br />
has been shown that <strong>the</strong> results <strong>of</strong> fur<strong>the</strong>r analysis can be isolated as not due to deviation<br />
in <strong>the</strong> basic data. In turn, any variations in results achieved in <strong>the</strong> following studies can be<br />
clearly sourced to adjustments in <strong>the</strong> methods applied.
105<br />
Table 3.1, Part A Employment equation from bivariate probit<br />
Employment equation from Replication results<br />
bivariate probit <strong>of</strong><br />
Richardson (1998)<br />
Model <strong>of</strong> ever employed in 1986 survey<br />
SYETP 1.590* 1.596**<br />
(2.45) (2.85)<br />
Gender=Female -0.398** -0.397**<br />
(-3.99) (-4.04)<br />
married -0.055 -0.054<br />
(-0.28) (-0.28)<br />
children -0.238 -0.238<br />
(-0.98) (-0.99)<br />
Children*female -1.212** -1.211**<br />
(-3.79) (-3.80)<br />
Spouse employed 1984 0.542* 0.542*<br />
(2.40) (2.41)<br />
Aboriginal/Torres Strait Islander -0.272 -0.271<br />
(-1.17) (-1.18)<br />
O<strong>the</strong>r ethnic minority -0.318 -0.318<br />
(-1.66) (-1.66)<br />
Work limited by health -0.306* -0.305*<br />
(-2.43) (-2.50)<br />
State interviewed in 1984<br />
Victoria -0.061 -0.060<br />
(-0.51) (-0.50)<br />
Queensland 0.027 0.028<br />
(0.19) (0.20)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.306 -0.305*<br />
(-1.94) (-1.97)<br />
Western Australia/Tasmania -0.101 -0.102<br />
(-0.67) (-0.69)<br />
Education school overseas -0.095 -0.095<br />
(-0.33) (-0.33)<br />
Roman Catholic school -0.010 -0.009<br />
(-0.06) (-0.05)<br />
Private school 0.603* 0.604*<br />
(2.07) (2.08)<br />
Highest qualification<br />
Degree/diploma 0.513** 0.512**<br />
(3.22) (3.26)<br />
Apprenticeship 0.431* 0.432*<br />
(2.10) (2.11)<br />
O<strong>the</strong>r Post-School qualification 0.049 0.049<br />
(0.32) (0.32)<br />
Year 12 <strong>of</strong> school -0.063 -0.063<br />
(-0.40) (-0.41)<br />
Year 11 <strong>of</strong> school 0.357* 0.356*<br />
(2.18) (2.22)<br />
Year 9 <strong>of</strong> school or less -0.222 -0.222<br />
(-1.59) (-1.60)<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
-0.473** -0.473**
106<br />
(-4.19) (-4.20)<br />
Longest job by 1984 none 0.042 0.042<br />
(0.25) (0.25)<br />
< 1 year 0.190 0.190<br />
(1.40) (1.42)<br />
2 years 0.252 0.252<br />
(1.56) (1.57)<br />
3 years + 0.657** 0.657**<br />
(4.00) (4.01)<br />
Enter o<strong>the</strong>r govt prog -0.624** -0.623**<br />
(-4.74) (-4.90)<br />
Family background<br />
O<strong>the</strong>r city before aged 14 -0.218 -0.217<br />
(-1.61) (-1.65)<br />
Country town before aged 14 -0.001 -0.000<br />
(-0.01) -0.00<br />
Rural area before aged 14 -0.128 -0.127<br />
(-0.65) (-0.66)<br />
Overseas before aged 14 0.494 0.494<br />
(1.36) (1.37)<br />
Number <strong>of</strong> siblings -0.028 -0.028<br />
(-1.49) (-1.51)<br />
English good 0.403 0.403<br />
(1.91) (1.91)<br />
English poor 1.050** 1.050**<br />
(2.75) (2.75)<br />
Sexist -0.440* -0.440*<br />
(-2.35) (-2.35)<br />
Sexist*female 0.463 0.463<br />
(1.25) (1.25)<br />
Fa<strong>the</strong>rs occupation when resp. 14<br />
Fa<strong>the</strong>r not present when resp 14 -0.180 -0.178<br />
(-0.79) (-0.79)<br />
Labourer 0.134 0.135<br />
(0.55) (0.56)<br />
Plant operative 0.058 0.059<br />
(0.25) (0.26)<br />
Sales -0.049 -0.048<br />
(-0.18) (-0.18)<br />
Tradesperson -0.214 -0.213<br />
(-0.95) (-0.95)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.114 0.114<br />
(0.52) (0.53)<br />
Not employed 0.146 0.147<br />
(0.55) (0.56)<br />
Fa<strong>the</strong>r holds post-school qualification 0.141 0.142<br />
when resp 14<br />
(1.26) (1.28)<br />
Mo<strong>the</strong>rs occupation when resp. 14<br />
Mo<strong>the</strong>r not present when resp 14 -0.335 -0.335<br />
(-1.33) (-1.34)<br />
Labourer -0.151 -0.150<br />
(-0.64) (-0.64)<br />
Plant operative -0.576* -0.576*<br />
(-2.30) (-2.31)<br />
Sales -0.392 -0.391
107<br />
(-1.80) (-1.80)<br />
Tradesperson -0.229 -0.229<br />
(-0.70) (-0.70)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.217 .0217<br />
(0.97) (0.97)<br />
Not employed -0.087 -0.087<br />
(-0.49) (-0.49)<br />
Mo<strong>the</strong>r post-school qualification when -0.067 -0.067<br />
resp 14<br />
(-0.52) (-0.52)<br />
Religion brought up in<br />
Catholic 0.327* 0.327*<br />
(2.52) (2.56)<br />
Presbyterian 0.413 0.412<br />
(1.88) (1.92)<br />
Methodist 0.133 0.133<br />
(0.77) (0.77)<br />
O<strong>the</strong>r Christian -0.102 -0.102<br />
(-0.50) (-0.50)<br />
O<strong>the</strong>r religion -0.045 -0.045<br />
(-0.28) (-0.28)<br />
No religion 0.280 0.279<br />
(1.58) (1.62)<br />
rho -0.622 -0.626<br />
Observations 1283 1283<br />
Log likelihood -875.96 -875.96<br />
Wald chi 2 (degrees <strong>of</strong> freedom ) (118) 396.79<br />
Akaike Information Criterion 67 1.55<br />
Coefficient is reported with t statistic in brackets; * significant at 5%; ** significant at 1%<br />
NOTE 1: results Column 1 are from Table 4 and Table 5 pages 18-21 Richardson (1998). Base categories:<br />
European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest qualification<br />
year 10 at school, longest job by 1984 is 1 year, lived mostly in state capital city until respondent aged 14,<br />
English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r clerical worker when<br />
respondent aged 14, and religion brought up in is Anglican.<br />
67 AIC=(-2log likelihood + 2 P)/ N where N= number observations and P=number <strong>of</strong> parameters estimated.
108<br />
Table 3.1 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit<br />
Selection equation <strong>of</strong> <strong>the</strong><br />
bivariate probit analysis<br />
by Richardson (1998)<br />
Model <strong>of</strong> SYETP<br />
participation, 1986 survey<br />
Replication results<br />
data<br />
Age at 1984 survey -0.107** -0.106**<br />
(-3.16) (-3.17)<br />
Gender=female 0.088 0.088<br />
(0.71) (0.71)<br />
Married 1984 -0.855 -0.854<br />
(-1.52) (-1.53)<br />
Children 1984 0.465 0.464<br />
(0.78) (0.78)<br />
Children*female -0.296 -0.295<br />
(-0.37) (-0.37)<br />
Spouse employed 1984 0.498 0.496<br />
(0.81) (0.81)<br />
Aboriginal/Torres Strait Islander -0.451 -0.451<br />
(-1.01) (-1.01)<br />
O<strong>the</strong>r ethnic minority 0.081 0.081<br />
(0.33) (0.33)<br />
Work limited by health -0.633* -0.633*<br />
(-2.53) (-2.53)<br />
State interviewed in 1984<br />
Victoria 0.112 0.112<br />
(0.72) (0.72)<br />
Queensland -0.279 -0.280<br />
(-1.30) (-1.31)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.157 -0.157<br />
(-0.77) (-0.77)<br />
Western Australia/Tasmania 0.317 0.317<br />
(1.78) (1.78)<br />
CEP referrals 1984 0.144* 0.143*<br />
(1.97) (2.02)<br />
Education school overseas 0.078 0.078<br />
(0.22) (0.22)<br />
Roman Catholic school -0.310 -0.310<br />
(-1.30) (-1.30)<br />
Private school -0.636 -0.635<br />
(-1.38) (-1.39)<br />
Highest qualification in 1984<br />
Degree/diploma 0.120 0.119<br />
(0.51) (0.51)<br />
Apprenticeship -0.129 -0.129<br />
(-0.42) (-0.42)<br />
O<strong>the</strong>r Post-School qualification -0.036 -0.037<br />
(-0.14) (-0.14)<br />
Year 12 <strong>of</strong> school 0.433* 0.433*<br />
(2.46) (2.46)<br />
Year 11 <strong>of</strong> school 0.101 0.101<br />
(0.53) (0.54)<br />
Year 9 <strong>of</strong> school or less -0.074 -0.73<br />
Model <strong>of</strong> SYETP<br />
participation, 1986 survey<br />
data
109<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
(-0.35) (-0.35)<br />
0.487** 0.487**<br />
(2.92) (2.94)<br />
Longest job by 1984 none -0.348 -0.345<br />
(-1.34) (-1.35)<br />
< 1 year -0.020 -0.019<br />
(-0.11) (-0.10)<br />
2 years 0.173 0.173<br />
(0.80) (0.80)<br />
3 years + -0.326 -0.326<br />
(-1.26) (-1.26)<br />
Family background<br />
O<strong>the</strong>r city before aged 14 -0.244 -0.243<br />
(-1.48) (-1.48)<br />
Country town before aged 14 -0.473** -0.473**<br />
(-2.94) (-2.97)<br />
Rural area before aged 14 -0.466 -0.446<br />
(-1.69) (-1.69)<br />
Overseas before aged 14 -0.757 -0.757<br />
(-1.48) (-1.48)<br />
Number <strong>of</strong> siblings -0.011 -0.011<br />
(-0.70) (-0.70)<br />
English good -0.185 -0.186<br />
(-0.72) (-0.73)<br />
English poor -0.591 -0.592<br />
(-1.13) (-1.13)<br />
Sexist 0.317 0.318<br />
(1.22) (1.23)<br />
Sexist*female -0.903 -0.903<br />
(-1.38) (-1.39)<br />
Fa<strong>the</strong>rs occupation when resp. 14<br />
Fa<strong>the</strong>r not present when resp 14 -0.309 -0.309<br />
(-1.11) (-1.12)<br />
Labourer -0.263 -0.263<br />
(-0.84) (-0.85)<br />
Plant operative -0.267 -0.267<br />
(-0.96) (-0.96)<br />
Sales -0.086 -0.086<br />
(-0.26) (-0.26)<br />
Tradesperson -0.300 -0.300<br />
(-1.10) (-1.10)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.153 -0.153<br />
(-0.59) (-0.59)<br />
Not employed -0.457 -0.456<br />
(-1.29) (-1.29)<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14<br />
-0.315* -0.315*<br />
(-2.14) (-2.14)<br />
Mo<strong>the</strong>rs occupation when resp. 14<br />
Mo<strong>the</strong>r not present when resp 14 0.480 0.480<br />
(1.49) (1.49)<br />
Labourer 0.176 0.177<br />
(0.57) (0.57)<br />
Plant operative 0.697* 0.697*
110<br />
(2.26) (2.26)<br />
Sales 0.190 0.190<br />
(0.66) (0.66)<br />
Tradesperson 0.119 0.120<br />
(0.28) (0.29)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.247 -0.246<br />
(-0.85) (-0.84)<br />
Not employed 0.041 0.041<br />
(0.17) (0.18)<br />
Mo<strong>the</strong>r post-school qualification<br />
when resp 14<br />
0.266 0.266<br />
(1.66) (1.66)<br />
Religion brought up in<br />
Catholic 0.061 0.061<br />
(0.38) (0.37)<br />
Presbyterian 0.322 0.322<br />
(1.34) (1.34)<br />
Methodist 0.017 0.015<br />
(0.06) (0.06)<br />
O<strong>the</strong>r Christian 0.075 0.074<br />
(0.27) (0.27)<br />
O<strong>the</strong>r religion 0.176 0.176<br />
(0.80) (0.80)<br />
No religion 0.138 0.138<br />
(0.66) (0.67)<br />
Observations 1283 1283<br />
T statistics in brackets; * significant at 5%; ** significant at 1%<br />
NOTE 1: results Column 1 are sourced from Table 4 and Table 5 pages 18-21 Richardson (1998).<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, and religion brought up in is Anglican.
111<br />
4: Study 2 Propensity score matching for SYETP<br />
In this study, <strong>the</strong> semi-parametric method <strong>of</strong> propensity score matching (PSM) is applied<br />
to <strong>the</strong> same data. The aim is to answer <strong>the</strong> question <strong>of</strong> what would <strong>the</strong> result have been if<br />
this modelling method had been applied instead <strong>of</strong> <strong>the</strong> Heckman Bivariate Probit. It is<br />
also a consideration <strong>of</strong> relaxation <strong>of</strong> <strong>the</strong> parameterisation. In a sense this is <strong>the</strong>n a<br />
specification test <strong>of</strong> <strong>the</strong> modelling choice.<br />
The bivariate probit specification replicated in <strong>the</strong> earlier chapter is a parametric model.<br />
The choice <strong>of</strong> a parametric method requires full specification <strong>of</strong> <strong>the</strong> underlying model<br />
and relies on this set <strong>of</strong> assumptions being correct. It is <strong>of</strong> interest to consider <strong>the</strong><br />
importance <strong>of</strong> <strong>the</strong> modelling method to <strong>the</strong> outcome attained. This is especially true when<br />
considering <strong>the</strong> SYETP coefficient in <strong>the</strong> employment equation since this provides an<br />
estimate <strong>of</strong> <strong>the</strong> net employment impact <strong>of</strong> SYETP for those who participated.<br />
In order to facilitate comparison <strong>of</strong> <strong>the</strong> results, it is necessary to maintain <strong>the</strong> conditions<br />
as applied to <strong>the</strong> Heckman selection modelling. In this way, ceteris paribus, <strong>the</strong> effect <strong>of</strong><br />
using <strong>the</strong> different models can be isolated. Essentially, <strong>the</strong> data restrictions applied for <strong>the</strong><br />
replication <strong>of</strong> <strong>the</strong> Richardson (1998) results must be repeated. As far as possible, <strong>the</strong><br />
modelling variables used must also be <strong>the</strong> same. There are however limitations as to how<br />
similar <strong>the</strong> variables can be due to <strong>the</strong> different requirements <strong>of</strong> <strong>the</strong>se two methods. This<br />
limitation is addressed in detail within <strong>the</strong> chapter.<br />
In <strong>the</strong> first section <strong>of</strong> <strong>the</strong> study, fur<strong>the</strong>r consideration is given to <strong>the</strong> question <strong>of</strong> why it is<br />
useful to apply PSM to this data, and to <strong>the</strong> SYETP analysis. Having resolved <strong>the</strong><br />
potential usefulness <strong>of</strong> applying <strong>the</strong> PSM method, <strong>the</strong> <strong>the</strong>ory and methods <strong>of</strong> PSM for<br />
this study are presented. The results <strong>of</strong> <strong>the</strong> application <strong>of</strong> PSM follow; and a brief<br />
sensitivity analysis <strong>of</strong> <strong>the</strong> PSM protocol chosen is also implemented. The PSM result is<br />
<strong>the</strong>n compared to <strong>the</strong> formerly presented Heckman selection modelling, and discussed.
112<br />
4.1 Differences between <strong>the</strong> treatment and comparison group<br />
An important problem for evaluation <strong>of</strong> programme effects is whe<strong>the</strong>r <strong>the</strong> treated are<br />
compared to an adequate reference group. This is <strong>the</strong> issue <strong>of</strong> selection distortion, where<br />
those treated have a very different pr<strong>of</strong>ile to those in <strong>the</strong> comparison group. Selective<br />
recruitment onto subsidy would make <strong>the</strong> treated have a different pr<strong>of</strong>ile than <strong>the</strong> larger<br />
group <strong>of</strong> those eligible. Controls for selection distortion can be made on observable<br />
characteristics, such as those variables shown in Table 4.1. The Table 4.1 gives <strong>the</strong> mean<br />
and standard deviation from <strong>the</strong> mean for a set <strong>of</strong> characteristics, with column 1 and 2<br />
showing <strong>the</strong>se for <strong>the</strong> SYETP treated group, and columns 3 and 4 showing <strong>the</strong><br />
comparison group characteristics. The absolute difference between <strong>the</strong> means is in<br />
column 5, with <strong>the</strong> results <strong>of</strong> <strong>the</strong> t-test at one percent level <strong>of</strong> significance indicated by an<br />
asterisk to show statistically significant differences.<br />
Richardson (1998) p6 commented that <strong>the</strong> contrast between those who took part in<br />
SYETP and those in <strong>the</strong> comparison group was “striking”. Indeed, column 1 and column<br />
3 <strong>of</strong> Table 4.1, <strong>of</strong> which <strong>the</strong> means were also presented by Richardson (1998), shows that<br />
<strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> treatment and comparison groups differs strongly. In most cases, means<br />
were quite different for <strong>the</strong> SYETP relative to <strong>the</strong> comparisons, and in some cases <strong>the</strong><br />
variation from <strong>the</strong> mean, as represented by <strong>the</strong> standard deviation was also very different<br />
for each group. As Richardson (1998) highlights, SYETP participants had a different<br />
educational attainment pr<strong>of</strong>ile where post-school qualifications were less common, <strong>the</strong>y<br />
were about a year younger, and had poorer labour market experiences than those in <strong>the</strong><br />
comparison group. However, how different <strong>the</strong> SYETP group was from <strong>the</strong> comparisons<br />
is related more clearly here by <strong>the</strong> addition <strong>of</strong> <strong>the</strong> test <strong>of</strong> <strong>the</strong> statistical significance <strong>of</strong> <strong>the</strong><br />
mean difference. Almost all variables have a significant divergence between <strong>the</strong> means<br />
for <strong>the</strong> SYETP group and <strong>the</strong> comparisons. Consideration <strong>of</strong> <strong>the</strong> many statistically<br />
significant differences in Table 4.1 makes <strong>the</strong> selection distortion for SYETP in <strong>the</strong> ALS<br />
data apparent. The literature review also highlights o<strong>the</strong>r evidence that it was generally<br />
<strong>the</strong> case for SYETP entrants to be younger than <strong>the</strong> eligible group as a whole – mostly<br />
teenagers. Thus it is possible that <strong>the</strong> ALS sample reflects differences that existed<br />
between <strong>the</strong> SYETP and comparison populations.
113<br />
Propensity score matching provides a method <strong>of</strong> analysis that controls for <strong>the</strong> lack <strong>of</strong><br />
correspondence between <strong>the</strong> treatment and comparison group. As discussed in Chapter 1,<br />
evaluation methods seek to maximize <strong>the</strong> similarity <strong>of</strong> <strong>the</strong> comparison and treated groups,<br />
in order to return to a quasi-experimental situation where effects can be usefully<br />
attributed to <strong>the</strong> programme. The propensity score matching method is now fur<strong>the</strong>r<br />
discussed and applied. The outcomes <strong>of</strong> <strong>the</strong> propensity score matching are <strong>the</strong>n compared<br />
to that <strong>of</strong> <strong>the</strong> bivariate probit. The relevance <strong>of</strong> each approach to <strong>the</strong> case at hand is<br />
considered, and <strong>the</strong> result for <strong>the</strong> SYETP programme effect is discussed.
114<br />
Table 4.1 Difference between treatment group and comparison group<br />
SYETP<br />
mean<br />
s.d.<br />
Non- SYETP<br />
mean<br />
Female 43.3 0.50 41.0 0.49 2.3*<br />
Average age 1984 19.0 1.97 20.1 2.42 1.1*<br />
Aboriginal/Torres Strait Islander 1.0 0.10 3.1 0.17 2.1*<br />
O<strong>the</strong>r ethnic minority 7.7 0.27 7.9 0.27 0.2<br />
Married 1984 2.9 0.17 12.5 0.33 9.6*<br />
Spouse employed 1984 1.9 0.14 6.3 0.24 4.4*<br />
Children 1984 1.9 0.14 5.8 0.23 3.9*<br />
Highest qualification in 1984<br />
Degree/diploma 7.7 0.27 12.2 0.33 4.5*<br />
Apprenticeship 2.9 0.17 8.6 0.28 5.7*<br />
O<strong>the</strong>r post-school qualification 6.7 0.25 7.1 0.26 0.4<br />
Year 12 <strong>of</strong> school 23.1 0.42 13.9 0.35 9.2*<br />
Year 11 <strong>of</strong> school 17.3 0.38 13.6 0.34 3.7*<br />
Year 10 <strong>of</strong> school 31.7 0.47 31.5 0.46 0.2<br />
Year 9 <strong>of</strong> school 10.6 0.31 12.6 0.33 2.0*<br />
Parental background when resp. aged 14<br />
Fa<strong>the</strong>r postschool qualification 26.0 0.44 34.6 0.48 8.6*<br />
Mo<strong>the</strong>r postschool qualification 20.2 0.40 18.2 0.39 2.0*<br />
Fa<strong>the</strong>r manager, pr<strong>of</strong>essional, para-pr<strong>of</strong>essional 25.0 0.44 25.8 0.44 0.8<br />
Fa<strong>the</strong>r not employed 3.8 0.19 5.6 0.23 1.8*<br />
Fa<strong>the</strong>r not present 19.2 0.40 15.4 0.36 3.8*<br />
Mo<strong>the</strong>r manager, pr<strong>of</strong>essional, para-pr<strong>of</strong>essional 6.7 0.25 9.8 0.30 3.1*<br />
Mo<strong>the</strong>r not employed 48.1 0.50 55.3 0.50 7.2*<br />
Mo<strong>the</strong>r not present 8.7 0.28 5.0 0.22 3.7*<br />
Longest job ever held by 1984<br />
Never held a job 11.5 0.32 11.6 0.32 0.1<br />
< 1 year 55.8 0.49 40.1 0.49 15.7*<br />
1 year 13.5 0.34 13.5 0.34 0.0<br />
2 years 13.5 0.34 14.1 0.35 0.6<br />
3 years or more 5.8 0.23 19.8 0.40 14*<br />
Average pre-programme unemployment 69 19.0 1.97 20.1 2.42 7.5*<br />
Ever employed in 1986 70 86.5 0.34 72.9 0.44 13.6*<br />
Ever government programme 1986 71 14.4 0.35 10.7 0.31 3.7*<br />
Number <strong>of</strong> cases 104 1179<br />
NOTE: Column 5 shows <strong>the</strong> t statistic for hypo<strong>the</strong>sis that difference <strong>of</strong> mean for SYETP and comparison<br />
is zero, where * indicates is significant at <strong>the</strong> 1 percent level <strong>of</strong> significance.<br />
s.d.<br />
SYETP<br />
versus<br />
comparison<br />
absolute<br />
difference in<br />
means 68<br />
68 The statistic for any variable is <strong>the</strong> absolute value <strong>of</strong> <strong>the</strong> difference in means for SYETP and <strong>the</strong> control<br />
groups.<br />
69 Proportion <strong>of</strong> 1984 reference period to 3 June spent unemployed.<br />
70 Ever held a non-subsidised, non-government program job in <strong>the</strong> 1986 reference period, after <strong>the</strong> first 17<br />
weeks.<br />
71 Ever go on a government program, including SYETP, in <strong>the</strong> 1986 reference period.
115<br />
4.2 Propensity score matching methods<br />
The propensity score matching methods, expounded in Rosenbaum and Rubin (1983),<br />
Dehijia and Wahba (1998), Heckman, Ichimura and Todd (1997), Heckman, Ichimura<br />
and Todd (1998) and Imbens (2000) have more recently been applied in <strong>the</strong> evaluation<br />
literature. Lechner (2000, 2001) extended <strong>the</strong> propensity score matching methods to <strong>the</strong><br />
multi-treatment case. Using propensity score matching moves <strong>the</strong> emphasis away from<br />
specifying <strong>the</strong> selection bias towards more careful construction <strong>of</strong> <strong>the</strong> comparison group.<br />
It has been suggested that <strong>the</strong> key enhancement for evaluation allowed by this method is<br />
<strong>the</strong> comparison <strong>of</strong> comparable people (Heckman, Lalonde and Smith (1999): 2083). In<br />
order to do this, irrelevant comparison cases, which are not similar to <strong>the</strong> treated, are<br />
removed from <strong>the</strong> analysis.<br />
Matching methods find for each individual in <strong>the</strong> treated, at least one comparison group<br />
member with very similar pre-treatment characteristics. The differences in outcomes after<br />
<strong>the</strong> treatment are <strong>the</strong>n attributed to <strong>the</strong> programme. Recalling <strong>the</strong> evaluation problem<br />
discussed earlier, matching is subject initially to <strong>the</strong> same difficulty <strong>of</strong> all nonexperimental<br />
methods where assignment to treatment is non-random. However Rubin<br />
(1974) showed that matching balances <strong>the</strong> distributions <strong>of</strong> all pre-treatment<br />
characteristics that influence assignment to <strong>the</strong> treatment, and so gives an unbiased<br />
estimate <strong>of</strong> treatment on <strong>the</strong> treated, as long as all relevant similar pre-treatment<br />
characteristics (X) are controlled for, and <strong>the</strong> Conditional Independence Assumption<br />
(CIA) is invoked (fur<strong>the</strong>r explained below).<br />
Propensity score matching uses <strong>the</strong> propensity score to provide a single measure <strong>of</strong> <strong>the</strong><br />
set <strong>of</strong> characteristics (X) that influence <strong>the</strong> probability <strong>of</strong> participating and employment.<br />
Rosenbaum and Rubin (1983) established that if matching on a set <strong>of</strong> observed<br />
characteristics is valid, <strong>the</strong>n matching on <strong>the</strong> probability <strong>of</strong> selection into <strong>the</strong> programme<br />
conditional on <strong>the</strong>se characteristics, <strong>the</strong> propensity score, is also valid. Whereas matching<br />
on each characteristic leads to problems with dimensions, <strong>the</strong> propensity score reduces<br />
<strong>the</strong> problem to a single dimension.
116<br />
4.3 Theory underlying propensity score matching methods<br />
Matching methods are steeped in experimental and evaluation terminology, and <strong>the</strong><br />
following exposition which draws on <strong>the</strong> literature <strong>of</strong> Rosenbaum and Rubin (1983),<br />
Dehijia and Wahba (1998), Heckman, Ichimura and Todd (1997), Heckman, Ichimura<br />
and Todd (1998) reflects this.<br />
The impact <strong>of</strong> participating in <strong>the</strong> programme is defined as<br />
(1) ∆ = Y t – Y c<br />
where Y t is <strong>the</strong> potential outcome conditional on participation and Y c is <strong>the</strong> potential<br />
outcome conditional on non-participation. Following Rubin (1974), treatment assignment<br />
may be random given a set <strong>of</strong> characteristics or variables X that are unaffected by <strong>the</strong><br />
treatment. The potential values <strong>of</strong> X for <strong>the</strong> different treatment states coincide, and are<br />
thus exogenous. 72<br />
The parameter most commonly estimated is <strong>the</strong> mean impact <strong>of</strong> treatment on <strong>the</strong> treated:<br />
(2) θ = E(∆ | D = 1, X=x)<br />
= E(Y t – Y c | D = 1, X=x)<br />
= E(Y t | D = 1, X=x) - E(Y c | D = 1, X=x)<br />
where D=1 denotes treatment, D=0 denotes non-treatment and X is a set <strong>of</strong> conditioning<br />
variables. 73 The evaluation problem arises from <strong>the</strong> term E(Y c | D = 1, X=x). This is <strong>the</strong><br />
mean <strong>of</strong> <strong>the</strong> counterfactual which, since it is unobservable, must be identified and<br />
estimated. This issue in evaluation was earlier discussed in Chapter 1.<br />
The construction <strong>of</strong> a valid comparison group for matching techniques is based on <strong>the</strong><br />
Conditional Independence Assumption (CIA). Under CIA, potential outcomes Y t ,Y c are<br />
independent <strong>of</strong> <strong>the</strong> assignment to treatment, when conditional on all pre-treatment<br />
72 Hujer and Caliendo (2000) p9 explain that this is not exogeneity in <strong>the</strong> econometric sense. Heckman<br />
Ichimura and Todd (1997) p610 note that it is not necessary to assume that <strong>the</strong> expected value <strong>of</strong> <strong>the</strong> errors<br />
in <strong>the</strong> model <strong>of</strong> Y conditional on X are zero, so X can fail to be exogenous in <strong>the</strong> conventional sense <strong>of</strong> <strong>the</strong><br />
term.<br />
73 Population quantities are denoted by capital letters and sample quantities by lower-case letters.
117<br />
covariates that influence <strong>the</strong> assignment to treatment as well as potential outcomes. The<br />
precise form <strong>of</strong> CIA depends on <strong>the</strong> parameter being estimated. For <strong>the</strong> treatment on <strong>the</strong><br />
treated parameter, <strong>the</strong> CIA requires that, conditional on observable characteristics,<br />
potential non-treatment outcomes are independent <strong>of</strong> treatment participation.<br />
Formally, CIA can be written as<br />
(3) Y c ╨ D | X = x, ∀x∈ χ<br />
where ╨ denotes independence and χ denotes <strong>the</strong> part <strong>of</strong> <strong>the</strong> attribute space for which<br />
<strong>the</strong> treatment effect is defined. This assumption is sufficient for <strong>the</strong> treatment on <strong>the</strong><br />
treated parameter. The fuller CIA assumes that<br />
(3a) Y t ,Y c ╨ D | X.<br />
However, Heckman et al. (1997) show that for estimating <strong>the</strong> mean effect <strong>of</strong> treatment on<br />
<strong>the</strong> treated, CIA for Y c is sufficient. This is because inference <strong>of</strong> Y c for persons where<br />
D=1 is estimated from data on persons where D=0. Hence, after adjusting for observable<br />
differences, <strong>the</strong> mean <strong>of</strong> <strong>the</strong> no-treatment (potential) outcome is <strong>the</strong> same for those<br />
receiving treatment as for those not receiving treatment. This allows non-participants’<br />
outcomes to be used to infer participants’ counterfactual outcomes.<br />
However, this is only valid if <strong>the</strong>re are non-participants for all participants’ values <strong>of</strong> X<br />
(<strong>the</strong> support condition):<br />
(4) 0 < Pr ( D = 1 | X = x ) < 1<br />
where “Pr” is <strong>the</strong> probability <strong>of</strong> participating in <strong>the</strong> programme <strong>of</strong> treatment. This ensures<br />
that <strong>the</strong>re are no values <strong>of</strong> <strong>the</strong> characteristics in X for which <strong>the</strong> propensity score is zero<br />
or one. Heckman, Ichimura, Smith, and Todd (1998) show that failure to ensure common<br />
support will give biased estimates. Rosenbaum and Rubin (1983) showed that when this<br />
assumption <strong>of</strong> common support holds toge<strong>the</strong>r with <strong>the</strong> CIA, <strong>the</strong>n if treatment assignment<br />
is strongly ignorable when X is given, <strong>the</strong>n it is also strongly ignorable for any balancing<br />
score, such as <strong>the</strong> propensity score or probability <strong>of</strong> participating in <strong>the</strong> programme.
118<br />
The treatment parameter, θ, cannot be correctly identified for values <strong>of</strong> X which do not<br />
satisfy this support condition. Matching operates by constructing, for those participants<br />
with support, a counterfactual from <strong>the</strong> non-participants. There are a number <strong>of</strong> ways <strong>of</strong><br />
defining this counterfactual. 74 Once <strong>the</strong> counterfactuals are identified, <strong>the</strong> mean impact <strong>of</strong><br />
<strong>the</strong> programme can be estimated as <strong>the</strong> mean difference in <strong>the</strong> outcomes <strong>of</strong> <strong>the</strong> matched<br />
pairs.<br />
A refinement to <strong>the</strong> matching approach was introduced by Rosenbaum and Rubin (1983).<br />
Rosenbaum and Rubin (1983) suggest <strong>the</strong> use <strong>of</strong> balancing scores which are functions <strong>of</strong><br />
<strong>the</strong> relevant observed covariates (here termed X). If assumptions (3) and (4) hold <strong>the</strong>n<br />
Rosenbaum and Rubin (1983) show that <strong>the</strong> conditional independence assumption (CIA)<br />
extends to <strong>the</strong> use <strong>of</strong> <strong>the</strong> propensity score:<br />
(5) Y c ╨ D | P(X) = p(x), ∀x∈ χ<br />
where ( ╨ ) symbolises independence, P(X) is <strong>the</strong> propensity score more fully termed<br />
P(D=1| X), <strong>the</strong> probability <strong>of</strong> participating in <strong>the</strong> programme. The important advantage <strong>of</strong><br />
Rosenbaum and Rubin’s (1983) innovation is that <strong>the</strong> dimensionality <strong>of</strong> <strong>the</strong> match can be<br />
reduced to one. 75 Ra<strong>the</strong>r than matching on a vector <strong>of</strong> characteristics, it is possible to<br />
match on just <strong>the</strong> propensity score. The information from <strong>the</strong> vector <strong>of</strong> characteristics is<br />
condensed within <strong>the</strong> estimated probability representing <strong>the</strong> propensity score. Having<br />
done so, <strong>the</strong> mean impact <strong>of</strong> <strong>the</strong> programme is again estimated as <strong>the</strong> mean difference in<br />
<strong>the</strong> outcomes <strong>of</strong> <strong>the</strong> matched pairs:<br />
(6) E[Y c | D=1, P(X)=P(x)] = E[Y c | D=0, P(X)=P(x)] = E(Y c | P(X)=P(x)]<br />
So <strong>the</strong>n <strong>the</strong> average treatment effect can be written:<br />
74 See, for example, Heckman et al. (1997) for a comparison <strong>of</strong> alternative matching schemes.<br />
75 Matching on <strong>the</strong> propensity score removes <strong>the</strong> dimensionality problem if <strong>the</strong> propensity scores are<br />
estimated parametrically, and relies on parametric estimation <strong>of</strong> <strong>the</strong> probability <strong>of</strong> participation providing<br />
an advantage over parametric estimation <strong>of</strong> <strong>the</strong> outcome.
119<br />
(7) E(Y c | D=1) = E P(X) {E[Y c | P(X)=P(x)] , D=0| D=1}<br />
Where <strong>the</strong> outer expectation is taken over <strong>the</strong> distribution <strong>of</strong> <strong>the</strong> propensity score in <strong>the</strong><br />
treated population (Hujer and Caliendo (2000)). Equation 7 can be used for estimation.<br />
Thus, with an estimate <strong>of</strong> <strong>the</strong> propensity {P(X)} gained from data {P(x)}, <strong>the</strong>n <strong>the</strong><br />
evaluation <strong>of</strong> <strong>the</strong> programme can take place by pairing participants with non-participants<br />
that have <strong>the</strong> same propensity score. For participants and comparisons with <strong>the</strong> same<br />
propensity score <strong>the</strong> distributions <strong>of</strong> <strong>the</strong> covariates are <strong>the</strong> same, and so <strong>the</strong>y are balanced.<br />
The assumption required to identify <strong>the</strong> effect satisfactorily is generally termed CIA now.<br />
CIA is <strong>the</strong> term used in Lechner (2000). However <strong>the</strong> same assumption is also referred to<br />
as selection on observables in Heckman and Robb (1985), and as ignorable treatment<br />
assignment in Rosenbaum and Rubin (1983). Frölich (2001) p2 describes <strong>the</strong> validity <strong>of</strong><br />
CIA as <strong>the</strong> requirement that all variables that affect simultaneously <strong>the</strong> potential<br />
employment outcome and <strong>the</strong> probability to be sampled from <strong>the</strong> comparison instead <strong>of</strong><br />
<strong>the</strong> target population are observed and included in X.<br />
The estimate <strong>of</strong> <strong>the</strong> propensity is used to balance <strong>the</strong> distributions <strong>of</strong> <strong>the</strong> covariates across<br />
<strong>the</strong> treated and comparison groups. The empirical power <strong>of</strong> propensity score matching in<br />
reducing <strong>the</strong> problem <strong>of</strong> selection bias depends on <strong>the</strong> quality <strong>of</strong> <strong>the</strong> estimate <strong>of</strong> <strong>the</strong><br />
propensity score. Additional to <strong>the</strong>se needs, <strong>the</strong>re has to exist a comparison person with a<br />
propensity score very similar, but preferably equal, to that <strong>of</strong> each treated person. Thus<br />
although in <strong>the</strong>ory PSM can be powerful in estimating <strong>the</strong> effect <strong>of</strong> treatment on <strong>the</strong><br />
treated, <strong>the</strong> practical implementation <strong>of</strong> PSM is subject to common parametric issues <strong>of</strong><br />
how effectively <strong>the</strong> empirical estimation corresponds to <strong>the</strong> assumptions <strong>of</strong> <strong>the</strong> method.<br />
4.4 The propensity score matching protocol implemented<br />
There are several means <strong>of</strong> applying propensity score matching (PSM). One to one<br />
matching takes <strong>the</strong> outcome <strong>of</strong> <strong>the</strong> single most similar comparison unit, to provide a<br />
match to each treated unit. The comparison individual whose propensity score is <strong>the</strong><br />
closest in value to <strong>the</strong> score for a treated unit is selected as <strong>the</strong> ‘nearest neighbour’. This
120<br />
method is also known as <strong>the</strong> single nearest neighbour method, and can additionally be<br />
limited using <strong>the</strong> caliper or radius method. 76 The radius provides a bound on how far <strong>the</strong><br />
propensity <strong>of</strong> <strong>the</strong> nearest neighbour can be from <strong>the</strong> treated case. Selection <strong>of</strong> <strong>the</strong><br />
comparison can be performed with or without replacement. Allowing <strong>the</strong> pool <strong>of</strong><br />
comparison group members to be selected for a match to a treated unit more than once<br />
has been shown to improve <strong>the</strong> performance <strong>of</strong> <strong>the</strong> match (Dehija and Wahba (1998)).<br />
Also, not allowing replacement in nearest-neighbour-matching produces results which are<br />
not invariant to <strong>the</strong> order in which <strong>the</strong> data were sorted for matching (Friedlander et al.<br />
(1997): 1818). The choice <strong>of</strong> matching with or without replacement involves a trade<strong>of</strong>f<br />
between bias and variance. Matching with replacement tends to lowers bias, because on<br />
average <strong>the</strong> matches are closer, but increases <strong>the</strong> variance, because fewer separate<br />
observations are used to construct <strong>the</strong> counterfactual mean.<br />
In this study, two types <strong>of</strong> PSM are implemented. Firstly nearest-neighbour, within<br />
caliper with replacement PSM is applied, <strong>the</strong>n as a sensitivity analysis <strong>of</strong> <strong>the</strong> choice <strong>of</strong><br />
PSM protocol, all-in-radius 77 PSM. While kernel PSM 78 is an alternative, this is not used<br />
here as <strong>the</strong> choice <strong>of</strong> kernel and bandwidth is not straightforward, and little literature<br />
exists on how best to critically implement <strong>the</strong>se choices in <strong>the</strong> context <strong>of</strong> matching.<br />
The match is based upon observable characteristics. This is <strong>the</strong> key identifying<br />
assumption <strong>of</strong> matching known as <strong>the</strong> conditional independence assumption (CIA).<br />
Under CIA, when controlling for observable differences in <strong>the</strong> characteristics <strong>of</strong> <strong>the</strong><br />
programme participants and non-participants, <strong>the</strong> outcome in <strong>the</strong> absence <strong>of</strong> treatment is<br />
<strong>the</strong> same for each <strong>of</strong> <strong>the</strong>se groups. More generally, <strong>the</strong> conditional independence<br />
assumption assumes that <strong>the</strong> selection on unobservables does not affect outcomes.<br />
76 Caliper matching usually means single nearest neighbour matching with a maximum limit placed on <strong>the</strong><br />
distance between <strong>the</strong> treated observation and <strong>the</strong> corresponding nearest neighbour from <strong>the</strong> comparison<br />
group. Radius matching may also be used to indicate this, or using all <strong>the</strong> comparison group observations<br />
within a given radius to construct <strong>the</strong> estimated counterfactual, which we term all-in-radius. The literature<br />
is not clear in applying <strong>the</strong>se terms.<br />
77 Also termed all-in-caliper.<br />
78 Frölich suggests in monte carlo analysis that a variant <strong>of</strong> kernel matching is preferred to nearest<br />
neighbour, caliper and radius matching when using minimization <strong>of</strong> mean squared error as a criterion.
121<br />
For CIA to be plausible, a ‘rich’ dataset is needed as it is assumed that all <strong>the</strong> variables<br />
affecting participation and employment are observed. Under CIA, <strong>the</strong> distribution <strong>of</strong> <strong>the</strong><br />
counterfactual outcome for participants is <strong>the</strong> same as that <strong>of</strong> non-participants, and <strong>the</strong><br />
matching is <strong>the</strong>n analogous to creating a fictional experiment where conditional upon <strong>the</strong><br />
observed characteristics, <strong>the</strong> selection process is random. If CIA is satisfied, <strong>the</strong>n<br />
selection bias ceases to be an issue. Violation <strong>of</strong> <strong>the</strong> conditional independence<br />
assumption cannot usually be accurately tested in non-experimentally designed<br />
observational data. Rosenbaum (1984) points out that <strong>the</strong>ory about <strong>the</strong> causal mechanism<br />
being investigated can be <strong>the</strong> chief source, and that <strong>the</strong>ory toge<strong>the</strong>r with random<br />
assignment or experimental data can produce useful tests. Some tests are also outlined in<br />
Heckman and Hotz (1989), but <strong>the</strong>se are most commonly carried out with experimental<br />
data.<br />
The plausibility <strong>of</strong> matching depends <strong>the</strong>n on a rich set <strong>of</strong> conditioning variables<br />
(Heckman, Lalonde and Smith (1999): 1995). Specifically, <strong>the</strong> variables should allow<br />
control for all characteristics that will affect both <strong>the</strong> participation and outcomes jointly.<br />
In this case, <strong>the</strong> data used here provides what may be considered a suitable set <strong>of</strong><br />
explanatory variables which are defined prior to treatment, covering labour market<br />
history, family background, education and skills, and attitudes amongst o<strong>the</strong>rs. In order to<br />
enhance <strong>the</strong> comparability with <strong>the</strong> former analyses, <strong>the</strong> same variables are initially used<br />
to estimate <strong>the</strong> propensity score as were previously used in <strong>the</strong> SYETP participation<br />
equation. 79 In <strong>the</strong> later chapter 7, sensitivity analysis <strong>of</strong> this probit for <strong>the</strong> propensity<br />
score is performed, with some variables removed. This exploration is done in light <strong>of</strong><br />
Lechner (2000) and Lechner (2001). Once again, <strong>the</strong> probit model is used to fit <strong>the</strong><br />
probability <strong>of</strong> SYETP participation. Using <strong>the</strong> parametric model <strong>of</strong> <strong>the</strong> probability <strong>of</strong><br />
treatment moves <strong>the</strong> matching technique from non-parametric into a semi-parametric<br />
approach.<br />
79 Including <strong>the</strong> same variables in <strong>the</strong> propensity score as are included in <strong>the</strong> participation model from <strong>the</strong><br />
bivariate probit could be problematic, since <strong>the</strong> two models <strong>of</strong> probability <strong>of</strong> participation are designed with<br />
different intentions. In <strong>the</strong> bivariate probit, <strong>the</strong>re are exclusion restrictions to capture variation in <strong>the</strong><br />
probability <strong>of</strong> participation unrelated to outcomes, but a propensity score model does not usually include<br />
instruments – only variables that influence both employment and participation. However, in <strong>the</strong> broad<br />
context <strong>of</strong> this work, <strong>the</strong> exclusion restrictions, which appear to be weak, are <strong>of</strong> interest, and this issue is<br />
treated fur<strong>the</strong>r later.
122<br />
In finding a counterfactual match for <strong>the</strong> participant, it is possible that <strong>the</strong>re is no<br />
comparison group individual with a similar propensity score. This is termed <strong>the</strong> common<br />
support problem. The crux <strong>of</strong> common support is <strong>the</strong> identification and removal <strong>of</strong> treated<br />
individuals for which <strong>the</strong> propensity indicates no close match is available in <strong>the</strong><br />
comparison group. Imposing common support ensures that <strong>the</strong> observed characteristics <strong>of</strong><br />
<strong>the</strong> participants are close to that <strong>of</strong> <strong>the</strong> non-participants.<br />
4.5 Estimating <strong>the</strong> probability <strong>of</strong> participation for SYETP<br />
Initially, <strong>the</strong> main aim is to repeat <strong>the</strong> formulation <strong>of</strong> <strong>the</strong> Richardson (1998) specification<br />
as much as possible. In consideration <strong>of</strong> what variables to include in <strong>the</strong> propensity score<br />
estimation, it is <strong>the</strong>n deemed appropriate to apply <strong>the</strong> probit modelling <strong>of</strong> SYETP in as<br />
much as possible <strong>the</strong> same specification used by Richardson (1998).<br />
The results for <strong>the</strong> probit analysis, which is unweighted 80 , are shown in Table 4.2. The<br />
variables, which are all constructed using 1984 data, are: gender, age, marital status,<br />
children, partner’s employment, ethnicity, location, type <strong>of</strong> schooling, qualification level,<br />
longest job held, CEP referrals, proportion <strong>of</strong> time spent unemployed prior to June 1984,<br />
urban/rural area grew up in, number <strong>of</strong> siblings had, standard <strong>of</strong> spoken English, attitude<br />
towards women in work, parental occupation and whe<strong>the</strong>r parent held a post-school<br />
qualification, and religion brought up in. These are all <strong>the</strong> variables in <strong>the</strong> participation<br />
equation <strong>of</strong> <strong>the</strong> bivariate probit estimated in chapter 3.<br />
The variables in <strong>the</strong> model are central to <strong>the</strong> credibility <strong>of</strong> <strong>the</strong> CIA. In regard to labour<br />
market evaluation, pre-programme labour market history is considered <strong>the</strong> most<br />
important explanatory variable, and this is part <strong>of</strong> <strong>the</strong> model applied here. In addition, <strong>the</strong><br />
family background and attitudinal variables might be considered useful to help capture<br />
o<strong>the</strong>rwise unobservable characteristics such as motivation that could influence<br />
80 The aim here is to find equivalent estimates for <strong>the</strong> bivariate probit model <strong>of</strong> Richardson (1998) using<br />
PSM. Weighting is applied to both models in chapter 6.
123<br />
employment and programme participation. As such, it is considered that at least with<br />
regard to <strong>the</strong> breadth <strong>of</strong> characteristics controlled for, <strong>the</strong> CIA has been satisfied.<br />
Some measures <strong>of</strong> fit for <strong>the</strong> probit model as a whole are considered. However<br />
interpretation <strong>of</strong> such measures <strong>of</strong> fit is limited. Scalar measures <strong>of</strong> fit have a heuristic<br />
nature and <strong>the</strong>re is no convincing evidence that a model which maximizes <strong>the</strong> value <strong>of</strong> a<br />
given measure gives a model that is optimal in any sense o<strong>the</strong>r than <strong>the</strong> optimisation <strong>of</strong><br />
<strong>the</strong> particular measure <strong>of</strong> fit. However some <strong>of</strong> <strong>the</strong>se measures such as <strong>the</strong> AIC can be<br />
useful in comparing competing models, which is considered in later chapters. The Akaike<br />
Information Criterion (AIC) is a measure that can be used to compare models across<br />
different samples and non-nested models. All else being equal, <strong>the</strong> model with <strong>the</strong><br />
smaller AIC is judged to be better. Here, <strong>the</strong> value is 0.57. Tukey’s ‘goodness <strong>of</strong> fit’ test<br />
is applied to check whe<strong>the</strong>r <strong>the</strong> specification <strong>of</strong> <strong>the</strong> probit is suitable and <strong>the</strong> results<br />
indicate no problems. 81 The chi squared distributed likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis<br />
that all coefficients except <strong>the</strong> intercept are equal to zero is 107.57 and results in this<br />
hypo<strong>the</strong>sis being rejected. Mcfadden’s pseudo R 2 , sometimes known as <strong>the</strong> likelihood<br />
ratio index, also compares <strong>the</strong> model to a model with just an intercept. This tests <strong>the</strong> same<br />
hypo<strong>the</strong>sis and equals zero if <strong>the</strong> model is equivalent to <strong>the</strong> intercept model (but can<br />
never precisely equal one). Here <strong>the</strong> value is 0.149, and indicates <strong>the</strong> model is reasonable.<br />
A summary measure <strong>of</strong> <strong>the</strong> fit <strong>of</strong> <strong>the</strong> predictions shows that 92 per cent were correctly<br />
estimated. 82 Thus <strong>the</strong> predictive power <strong>of</strong> <strong>the</strong> model appears acceptable within <strong>the</strong><br />
context <strong>of</strong> <strong>the</strong> data. However, prediction rates are not ideal measures for assessing <strong>the</strong><br />
propensity score as if <strong>the</strong> prediction rate were perfect at one, <strong>the</strong>n <strong>the</strong>re would be no<br />
common support. As such, <strong>the</strong> value <strong>of</strong> <strong>the</strong>se measures for critically assessing <strong>the</strong><br />
propensity are low and judging <strong>the</strong> included variables, toge<strong>the</strong>r with <strong>the</strong> estimated model<br />
parameters, in <strong>the</strong> context <strong>of</strong> <strong>the</strong> <strong>the</strong>ory motivating <strong>the</strong> analysis is possibly more<br />
important, and examining <strong>the</strong> estimated propensity scores.<br />
81 A probit <strong>of</strong> SYETP is run on just <strong>the</strong> predictions, and <strong>the</strong> predictions squared, sourced from <strong>the</strong> model <strong>of</strong><br />
SYETP. If <strong>the</strong> model is specified correctly, <strong>the</strong> predictions squared should have no explanatory power – i.e.<br />
are statistically insignificant. The base reference is Tukey (1949). The results give a t value close to zero,<br />
which is clearly not significant.<br />
82 A fitted probability exceeding 0.5 is taken to indicate a predicted participation in SYETP, and <strong>the</strong>se<br />
predicted responses are compared to <strong>the</strong> actual participants/non-participants in SYETP to check which<br />
cases <strong>the</strong> model correctly predicted.
124<br />
It is interesting to look at <strong>the</strong> factors that affect <strong>the</strong> SYETP propensity, considering only<br />
statistically significant coefficients. Age had a negative impact on participation in SYETP,<br />
with older youths less likely to take part. Those with schooling to year 12, and those with<br />
longer unemployment spells were more likely to take part in SYETP. Health conditions<br />
that limited work reduced <strong>the</strong> chance <strong>of</strong> participation in SYETP. A family background <strong>of</strong><br />
living in a country town also reduced <strong>the</strong> likelihood <strong>of</strong> taking part in SYETP. Parental<br />
background had some influence, with those whose fa<strong>the</strong>rs had post school qualifications<br />
less likely to take part in SYETP, but those whose mo<strong>the</strong>r’s occupation had been a plant<br />
operative more likely to take part. The number <strong>of</strong> referrals by <strong>the</strong> CES to <strong>the</strong> Community<br />
Employment Programme, ano<strong>the</strong>r labour market programme available, has a positive<br />
influence on participation in SYETP.<br />
Table 4.2 Probit used to estimate propensity score for propensity score matching<br />
Participation in SYETP<br />
Gender=female 0.08<br />
(0.67)<br />
Age at 1984 survey -0.11<br />
(3.21)**<br />
Married 1984 -0.97<br />
(1.62)<br />
Children 1984 0.49<br />
(0.74)<br />
Children*female -0.33<br />
(0.39)<br />
Spouse employed 1984 0.60<br />
(0.91)<br />
Aboriginal/Torres Strait Islander -0.45<br />
(0.95)<br />
O<strong>the</strong>r ethnic minority 0.06<br />
State interviewed in 1984 (0.25)<br />
Victoria 0.11<br />
(0.73)<br />
Queensland -0.21<br />
(0.99)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.14<br />
(0.66)<br />
Western Australia/Tasmania 0.32<br />
(1.81)<br />
Education school overseas 0.14<br />
(0.38)<br />
Roman Catholic school -0.30<br />
(1.23)<br />
Private school -0.73
Highest qualification in 1984 (1.56)<br />
Degree/diploma 0.05<br />
(0.20)<br />
Apprenticeship -0.15<br />
(0.47)<br />
O<strong>the</strong>r Post-School qualification 0.07<br />
(0.28)<br />
Year 12 <strong>of</strong> school 0.40<br />
(2.25)*<br />
Year 11 <strong>of</strong> school 0.16<br />
(0.85)<br />
Year 9 <strong>of</strong> school or less -0.11<br />
(0.53)<br />
Longest job by 1984 none -0.42<br />
(1.68)<br />
< 1 year -0.04<br />
(0.24)<br />
2 years 0.15<br />
(0.70)<br />
3 years + -0.34<br />
(1.32)<br />
CEP referrals 1984 0.16<br />
(2.36)*<br />
duration <strong>of</strong> Pre-June 1984 unemployment 0.46<br />
(2.79)**<br />
Work limited by health -0.60<br />
Family background (2.32)*<br />
O<strong>the</strong>r city before aged 14 -0.25<br />
(1.49)<br />
Country town before aged 14 -0.43<br />
(2.76)**<br />
Rural area before aged 14 -0.46<br />
(1.70)<br />
Overseas before aged 14 -0.71<br />
(1.38)<br />
Number <strong>of</strong> siblings -0.01<br />
(0.83)<br />
English good -0.12<br />
(0.48)<br />
English poor -0.56<br />
(1.03)<br />
Sexist 0.25<br />
(0.95)<br />
Sexist*female -0.76<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.24)<br />
Fa<strong>the</strong>r not present when resp 14 -0.22<br />
(0.79)<br />
Labourer -0.16<br />
(0.52)<br />
Plant operative -0.21<br />
(0.75)<br />
Sales -0.10<br />
(0.30)<br />
Tradesperson -0.25<br />
(0.90)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.08<br />
125
126<br />
(0.33)<br />
Not employed -0.39<br />
(1.08)<br />
Fa<strong>the</strong>r holds post-school qualification when resp 14 -0.30<br />
Mo<strong>the</strong>rs occupation when resp. 14 (2.00)*<br />
Mo<strong>the</strong>r not present when resp 14 0.52<br />
(1.64)<br />
Labourer 0.14<br />
(0.45)<br />
Plant operative 0.65<br />
(2.10)*<br />
Sales 0.19<br />
(0.65)<br />
Tradesperson 0.09<br />
(0.21)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.23<br />
(0.76)<br />
Not employed 0.05<br />
(0.21)<br />
Mo<strong>the</strong>r post-school qualification when resp 14 0.25<br />
Religion brought up in (1.52)<br />
Catholic 0.06<br />
(0.38)<br />
Presbyterian 0.32<br />
(1.34)<br />
Methodist 0.12<br />
(0.46)<br />
O<strong>the</strong>r Christian 0.10<br />
(0.35)<br />
O<strong>the</strong>r religion 0.16<br />
(0.72)<br />
No religion 0.19<br />
(0.95)<br />
Constant 0.71<br />
(0.93)<br />
Observations 1283<br />
Log likelihood -307.19<br />
LR chi 2 (59) 83 107.57<br />
Mcfadden’s Pseudo R 2 84 0.149<br />
Akaike Information Criterion 85 0.57<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%<br />
83 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
84 This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(M full ) to a model with just <strong>the</strong> intercept (M intercept ). It is defined as R 2 = 1 – (log likelihood M full / log<br />
likelihood M intercept ) . The value <strong>of</strong> Mcfadden’s Pseudo R 2 increases as new variables are added.<br />
85 AIC=(-2log likelihood + 2 P)/ N where N= number observations and P=number <strong>of</strong> parameters estimated.
127<br />
4.6 Distribution <strong>of</strong> <strong>the</strong> propensity score<br />
The propensity score for each case is <strong>the</strong> predicted probability <strong>of</strong> taking part in SYETP,<br />
as estimated from <strong>the</strong> probit model. The estimated propensity score for <strong>the</strong> SYETP and<br />
<strong>the</strong> comparison group are shown in Figure 4.3 as histograms. Each histogram has 10 bins<br />
– this is <strong>the</strong>n comparable to most <strong>of</strong> <strong>the</strong> literature, such as Dehija and Wahba (1998,<br />
1999). The histograms show that <strong>the</strong> estimated probabilities <strong>of</strong> SYETP participation<br />
cover a limited area, however <strong>the</strong> histogram for <strong>the</strong> comparisons is misleading as it is so<br />
heavily weighted towards zero and has only 2 observations greater that 0.5, so that <strong>the</strong><br />
upper bar becomes invisible it is so thin. The treated have a modest but non-trivial<br />
density <strong>of</strong> observations greater than 0.5. Referring to <strong>the</strong> summary statistics in table 4.5<br />
shows <strong>the</strong> overlap between <strong>the</strong> treated and comparisons is restricted by <strong>the</strong> comparison<br />
group distribution which extends slightly beyond <strong>the</strong> treated propensities.<br />
Figure 4.4 shows <strong>the</strong> same distributions overlaid, and with a kernel smoothing applied to<br />
<strong>the</strong> distribution. The shapes <strong>of</strong> <strong>the</strong> distributions differ even where <strong>the</strong>y overlap. As would<br />
be expected, far more <strong>of</strong> <strong>the</strong> comparison group have values close to zero, with <strong>the</strong><br />
comparison distribution peaking sharply here and <strong>the</strong> main bulk transpiring at this point,<br />
before tailing <strong>of</strong>f rapidly. The comparison group more <strong>of</strong>ten has a propensity to<br />
participate in SYETP that is close to zero, i.e. no participation. The lower tail <strong>of</strong> <strong>the</strong><br />
comparison group always lies above that <strong>of</strong> <strong>the</strong> SYETP. For <strong>the</strong> SYETP group, <strong>the</strong><br />
distribution has a much lower peak, which is flatter and occurs between values 0.1 and<br />
0.2, and <strong>the</strong> upper tail falls away far more slowly. The upper tail <strong>of</strong> <strong>the</strong> SYETP lies above<br />
that <strong>of</strong> <strong>the</strong> comparison group where more SYETP cases have <strong>the</strong>se higher propensities to<br />
take part in SYETP, but <strong>the</strong> upper tails approach very similar values in <strong>the</strong> final stages <strong>of</strong><br />
<strong>the</strong> distributions. A good model for <strong>the</strong> propensity produces large differences <strong>of</strong> <strong>the</strong> mean<br />
predicted propensities across <strong>the</strong> treated and comparison groups [Larsson (2000)]. On this<br />
criterion, it appears that <strong>the</strong> propensity score distributions, and <strong>the</strong> underlying model, are<br />
acceptable. Table 4.5 shows <strong>the</strong> mean predicted propensity for SYETP is 0.17 while for<br />
<strong>the</strong> comparisons it is less than half <strong>of</strong> this at 0.07.
128<br />
4.7 Common support for <strong>the</strong> treated<br />
The first step is to discard from <strong>the</strong> treatment group those members whose propensity<br />
score is beyond <strong>the</strong> limits <strong>of</strong> that <strong>of</strong> <strong>the</strong> distribution <strong>of</strong> propensities <strong>of</strong> <strong>the</strong> comparison<br />
group. This form <strong>of</strong> <strong>the</strong> common support was used in <strong>the</strong> multiple treatments matching<br />
protocol <strong>of</strong> Gerfin and Lechner (2000) p22, where <strong>the</strong> minima and maxima <strong>of</strong> <strong>the</strong> treated<br />
propensities defined <strong>the</strong> common support region, in this case simplified by only one<br />
treatment and comparison group. Thus <strong>the</strong> propensity for <strong>the</strong> treated and comparisons<br />
only covers <strong>the</strong> same interval. This is <strong>the</strong> application <strong>of</strong> <strong>the</strong> rule <strong>of</strong> common support<br />
expressed in equation 4 <strong>of</strong> <strong>the</strong> <strong>the</strong>ory in section 4.3. There is support in <strong>the</strong> comparator<br />
group for <strong>the</strong> treatment group member if a propensity value between 0 and 1 exists<br />
amongst <strong>the</strong> comparisons for each propensity <strong>of</strong> <strong>the</strong> treatment group. Those individuals<br />
discarded for non-support have least similarity. The members <strong>of</strong> <strong>the</strong> treatment group are<br />
discarded for non-support if <strong>the</strong>ir propensity to participate in SYETP is lower than <strong>the</strong><br />
minimum score or higher than <strong>the</strong> maximum score <strong>of</strong> those in <strong>the</strong> comparison group.<br />
Those cases discarded for non-common support are not comparable, because for <strong>the</strong><br />
treated <strong>the</strong>re is no comparable non-participant.<br />
The distributional summaries in Table 4.5 give <strong>the</strong> four largest and smallest propensity<br />
scores observed in <strong>the</strong> treatment and comparison group. In this case, application <strong>of</strong> <strong>the</strong><br />
rule <strong>of</strong> common support has no effect on <strong>the</strong> treatment group, as <strong>the</strong> distribution for <strong>the</strong><br />
SYETP treatment propensities always lies within <strong>the</strong> bounds <strong>of</strong> <strong>the</strong> comparison group.<br />
The maximum comparison group propensity to take part in SYETP is 0.5458441, and <strong>the</strong><br />
minimum is 0.0000598. However, <strong>the</strong>re are just 2 cases in <strong>the</strong> comparison group with<br />
propensities greater than <strong>the</strong> maximum <strong>of</strong> <strong>the</strong> SYETP treatment group (<strong>the</strong>se are not<br />
clearly visible in <strong>the</strong> histogram as <strong>the</strong> bar is so thin due to low number <strong>of</strong> cases). For <strong>the</strong><br />
SYETP treatment group, <strong>the</strong> maximum comparison group propensity to take part in<br />
SYETP is 0.5258619 and <strong>the</strong> minimum is 0.0147686. In this case, <strong>the</strong>re are no SYETP<br />
participants for whom <strong>the</strong>re are not comparable non-participants.
129<br />
Figure 4.3 Histograms <strong>of</strong> estimated propensity score prior to matching<br />
1<br />
1<br />
Fra<br />
ctio<br />
n<br />
.5<br />
Fra<br />
ctio<br />
n<br />
.5<br />
0<br />
0<br />
0 .5 1<br />
104 observations, unweighted probit<br />
propensity score for treated (SYETP=1)<br />
0 .5 1<br />
1179 observations, unweighted probit<br />
propensity score for controls (SYETP=0)<br />
histograms <strong>of</strong> estimated propensity scores prior to matching<br />
Figure 4.4 Kernel Density <strong>of</strong> propensity scores distribution for Treated SYETP and<br />
untreated
130<br />
10<br />
Treated propensity scores,<br />
Untreated propensity scores,<br />
5<br />
0<br />
0 .2 .4 .6<br />
Pr(syetp)<br />
kernel densities <strong>of</strong> propensity scores <strong>of</strong> <strong>the</strong> treated vs untreated<br />
Note: Epanechnikov kernel.
131<br />
Table 4.5 Summary statistics for distribution <strong>of</strong> propensity scores<br />
Comparison group<br />
Percentiles Smallest<br />
1% .0004381 .0000598<br />
5% .0023942 .0000982<br />
10% .0056824 .0001258 Observations 1179<br />
25% .0174015 .0001335<br />
50% .0488829 Mean .073496<br />
Largest Std. Dev. .0784812<br />
75% .1021392 .4204476<br />
90% .1756042 .4249502 Variance .0061593<br />
95% .2366404 .5203477 Skewness 1.912722<br />
99% .3523099 .5458441 Kurtosis 7.518665<br />
SYETP group<br />
Percentiles Smallest<br />
1% .0224789 .0147686<br />
5% .0303421 .0224789<br />
10% .038564 .0227128 Observations 104<br />
25% .0867105 .0228779<br />
50% .1439536 Mean .1658229<br />
Largest Std. Dev. .1082047<br />
75% .2126023 .3920572<br />
90% .3091516 .4926768 Variance .0117083<br />
95% .3695493 .5044287 Skewness 1.04344<br />
99% .5044287 .5258619 Kurtosis 4.192943
132<br />
4.8 Results <strong>of</strong> Matching: one-to-one nearest-neighbour<br />
Matching is now performed. On a case-by-case basis, amongst <strong>the</strong> remaining comparison<br />
group members, members are identified that are similar to each individual <strong>of</strong> <strong>the</strong><br />
treatment group, i.e. <strong>the</strong>ir ‘match’. The basis for <strong>the</strong> match is <strong>the</strong> propensity score. The<br />
matching should produce a counterfactual for those in <strong>the</strong> treatment group. In this case,<br />
one-to-one nearest-neighbour within-caliper 86 matching with replacement is first<br />
applied. 87 For a pre-specified distance, <strong>the</strong> caliper width δ, <strong>the</strong> treated unit is matched to<br />
<strong>the</strong> comparison unit so that <strong>the</strong> nearest single neighbour criterion is satisfied within <strong>the</strong><br />
caliper,<br />
1) δ > | p i - p j | = min k ∈ { D=0} { | p i – p k | }<br />
Where<br />
δ = caliper width<br />
i = treated unit<br />
j = comparison unit<br />
p = propensity score p(x) = Probability { D=1 | X = x }<br />
x = set <strong>of</strong> pre-treatment characteristics<br />
D = indicator <strong>of</strong> treatment, 1= treatment, 0 = no treatment<br />
4.8.1 Discussion <strong>of</strong> <strong>the</strong> results<br />
The results <strong>of</strong> <strong>the</strong> matching are shown in Table 4.6. There is no general rule as to what<br />
caliper width is suitable. However <strong>the</strong> <strong>the</strong>ory underlying PSM indicates <strong>the</strong> caliper needs<br />
to provide a comparison person with a propensity score very similar but preferably equal<br />
to that <strong>of</strong> <strong>the</strong> treated persons. Thus a small caliper width is preferable, or so small <strong>the</strong><br />
propensity scores can be deemed roughly equal. Pagan and Ullah (1999) in discussing<br />
bandwidth selection for kernel density, suggest inspecting <strong>the</strong> data in order to choose a<br />
‘suitable’ width. It can observed from <strong>the</strong> summary <strong>of</strong> <strong>the</strong> propensity scores shown in<br />
86 Gu and Rosenbaum (1993) introduced this pair matching technique.<br />
87 The STATA ado file by Barbara Sianesi is used to implement this method (psmatch.ado).
133<br />
Table 4.5, that <strong>the</strong> variance <strong>of</strong> <strong>the</strong> propensity score <strong>of</strong> <strong>the</strong> SYETP group is about 0.012<br />
while for <strong>the</strong> comparison it is 0.006.<br />
It could be inferred that <strong>the</strong> smaller caliper allows for <strong>the</strong> best match, with <strong>the</strong> minimum<br />
limit being <strong>the</strong> need to find at least one case to match within <strong>the</strong> caliper. As <strong>the</strong><br />
comparison group is so much larger is size than <strong>the</strong> treated, <strong>the</strong>re is <strong>the</strong> possibility <strong>of</strong><br />
several matches being available within each caliper. With single-nearest neighbour<br />
matching, increasing <strong>the</strong> width <strong>of</strong> caliper simply widens <strong>the</strong> pool <strong>of</strong> comparisons that are<br />
available. The quality <strong>of</strong> <strong>the</strong> match might decrease as <strong>the</strong> pool is increased to <strong>the</strong> extent<br />
that more SYETP cases are <strong>the</strong>n matched to dissimilar comparison cases. This would be<br />
<strong>the</strong> case if it were difficult to match because <strong>the</strong> support space was ‘sparse’, or had<br />
‘sparse’ regions. However, once a single match has been accomplished within <strong>the</strong> caliper,<br />
that nearest-neighbour match does not alter by widening <strong>the</strong> caliper. On <strong>the</strong> o<strong>the</strong>r hand,<br />
narrowing <strong>the</strong> caliper could improve <strong>the</strong> match when <strong>the</strong> support space has sparse regions<br />
by making <strong>the</strong> nearest-neighbours matched nearer to each o<strong>the</strong>r, because it forces <strong>the</strong><br />
difference in <strong>the</strong>ir propensity to be smaller. This means that <strong>the</strong> caliper width can affect<br />
match quality, however <strong>the</strong>re is a cost. The key trade<strong>of</strong>f in choosing a caliper width is<br />
between match quality and <strong>the</strong> number <strong>of</strong> treated observations dropped because <strong>the</strong>y do<br />
not have a match inside <strong>the</strong> caliper.<br />
In order to better answer <strong>the</strong> question <strong>of</strong> what caliper width is suitable to apply, a variety<br />
<strong>of</strong> caliper widths was applied. The first column shows <strong>the</strong> smallest caliper width applied<br />
to <strong>the</strong> difference in <strong>the</strong> propensity used to find <strong>the</strong> matched treated and comparison cases,<br />
<strong>the</strong> caliper width 0.001. Each column to <strong>the</strong> right <strong>of</strong> column 1 uses a wider caliper:<br />
Column 2 shows <strong>the</strong> caliper width 0.005, column 3 shows 0.01 caliper width, column 4<br />
shows 0.02 caliper width and finally column 5 shows 0.05 caliper width. To assess <strong>the</strong><br />
matching, for each set <strong>of</strong> matching results is shown <strong>the</strong> matched mean difference, and <strong>the</strong><br />
t statistic <strong>of</strong> <strong>the</strong> statistical significance <strong>of</strong> <strong>the</strong> difference, <strong>the</strong> number <strong>of</strong> treated and<br />
comparison cases matched, <strong>the</strong> number <strong>of</strong> times comparison cases were used because we<br />
used matching with replacement, <strong>the</strong> mean and standard deviation <strong>of</strong> <strong>the</strong> difference in <strong>the</strong><br />
propensity scores and mean total bias statistic.
134<br />
In general, all <strong>the</strong> results pass <strong>the</strong> usual checks for performance <strong>of</strong> <strong>the</strong> matching. There is<br />
a reasonable, low total bias (<strong>the</strong> mean standardized bias statistic is defined fully in section<br />
4.9.2). The various calipers do not differ dramatically in terms <strong>of</strong> <strong>the</strong> mean standardized<br />
bias statistic. In general, <strong>the</strong> average difference in propensity was very low too, although<br />
this rises with greater caliper width as expected. The number <strong>of</strong> SYETP for which a<br />
match can be found within <strong>the</strong> caliper width chosen rises with caliper width as expected.<br />
The largest caliper width <strong>of</strong> 0.05 allows all 104 SYETP cases to be matched.<br />
Frölich et al. (1999) p41 point out that <strong>the</strong> advantage <strong>of</strong> replacement is clear as a<br />
matching algorithm that only uses a comparison case once can face difficulties when<br />
<strong>the</strong>re is very low density for <strong>the</strong> comparison group compared to <strong>the</strong> treatment group.<br />
Replacement is not problematic as long as <strong>the</strong>re is overlap in <strong>the</strong> propensity distributions.<br />
As shown in Figure 5, <strong>the</strong> propensity distributions overlap greatly, and <strong>the</strong>re is no<br />
common support problem, as discussed earlier no cases are lost in <strong>the</strong> imposition <strong>of</strong><br />
common support. The caliper also imposes a restriction that can act to impose <strong>the</strong><br />
common support condition.<br />
A drawback <strong>of</strong> <strong>the</strong> replacement method can arise if <strong>the</strong> same observation may be used too<br />
<strong>of</strong>ten. The result <strong>of</strong> over-reliance on single observations can be a great loss in precision.<br />
The number <strong>of</strong> times a comparison case is used with replacement is shown in Table 4.6 to<br />
help assess this. The percentage <strong>of</strong> matched comparison cases used more than once<br />
remains less than 10 per cent, for all calipers. For example, for caliper 0.001 only 6 <strong>of</strong> <strong>the</strong><br />
79 comparison cases are used more than once. All <strong>the</strong> mean differences in employment<br />
are significant, so efficiency is not considered problematic.
135<br />
Table 4.6 Matching results, Single nearest neighbour with replacement, within caliper<br />
Match<br />
with<br />
caliper<br />
width<br />
0.001<br />
Match<br />
with<br />
caliper<br />
width<br />
0.005<br />
Match<br />
with<br />
caliper<br />
width<br />
0.01<br />
Match<br />
with<br />
caliper<br />
width<br />
0.02<br />
Match<br />
with<br />
caliper<br />
width<br />
0.05<br />
Difference in employment 88 for 0.18 0.18 0.18 0.17 0.17<br />
matched treated and comparisons<br />
Student’s t statistic (2.74) (2.68) (2.68) (2.66) (2.63)<br />
Number <strong>of</strong> SYETP matched 87 101 102 103 104<br />
Number <strong>of</strong> comparison which satisfy 79 88 89 89 89<br />
<strong>the</strong> caliper rule<br />
Number <strong>of</strong> times used<br />
1 73 79 80 79 79<br />
2 5 8 8 9 8<br />
More than 2 1 1 1 1 2<br />
Mean difference in propensity score 0.0002 0.0005 0.0006 0.0007 0.001<br />
between single nearest neighbour<br />
matched treated and comparisons<br />
Standard deviation 0.0002 0.0009 0.004 0.002 0.003<br />
Mean Standardized bias calculation<br />
after matching<br />
9.66 9.90 9.74 9.45 9.58<br />
88 Ever employed in 1986 survey.
136<br />
4.8.2 Mean standardised bias statistic<br />
The distributions for <strong>the</strong> remaining comparisons when compared to <strong>the</strong> SYETP group are<br />
not perfectly balanced in terms <strong>of</strong> <strong>the</strong> separate independent variables after propensity<br />
score matching. This is because <strong>the</strong> propensity score allows some variables to be less<br />
well matched for a case, while o<strong>the</strong>rs variables are well matched, leading to an overall<br />
weighting scale across each <strong>of</strong> <strong>the</strong>se. A measure that summarises <strong>the</strong> overall balance is<br />
<strong>the</strong> standardized mean bias. For each variable included in <strong>the</strong> probit to estimate <strong>the</strong><br />
propensity score, <strong>the</strong> measure is calculated as:<br />
100*[ | µ treated - µ comparison | / √{( var treated + var comparison )/2} ]<br />
where µ is <strong>the</strong> mean and var is <strong>the</strong> variance observed for <strong>the</strong> covariates in <strong>the</strong> sample.<br />
The summary measure <strong>of</strong> <strong>the</strong> distribution <strong>of</strong> bias is found as <strong>the</strong> mean standardised bias<br />
across all <strong>the</strong> variables. 89 This measure can be used to gauge <strong>the</strong> matching performance<br />
(Lechner (2000)). It can be interpreted as bias as a percentage <strong>of</strong> <strong>the</strong> standard deviation.<br />
A smaller average bias is generally considered better. It is clear that a bias <strong>of</strong> zero is ideal,<br />
but <strong>the</strong> tolerance level for any non-zero standardized bias is not defined. Matching<br />
generally strongly improves mean bias. A poor match quality would indicate that <strong>the</strong><br />
matched groups are not genuinely balanced on <strong>the</strong> observed characteristics and that no<br />
outcome comparisons based on <strong>the</strong>se matches would be justified (Frölich et al. (2000):<br />
39). However <strong>the</strong>re is no clear gauge as to what level <strong>of</strong> standardized bias indicates a<br />
match is poor.<br />
Comparison with o<strong>the</strong>r studies, which report <strong>the</strong> standardized bias, gives ano<strong>the</strong>r gauge<br />
as to whe<strong>the</strong>r <strong>the</strong> levels are acceptable. Some recent studies that report balance measures<br />
are Sianesi (2001), Larsson (2000), Frölich et al. (2000), and Gerfin and Lechner (2000).<br />
Sianesi (2001) p47 appendix D found in Swedish administrative data analyses for 31,975<br />
cases a mean standardized bias <strong>of</strong> 0.86, but some variables were poor such as age at 2.86.<br />
This was a large data set, which enhances <strong>the</strong> ability to match. Larsson (2000) p27 and<br />
89 The use <strong>of</strong> <strong>the</strong> term bias here is <strong>the</strong> statistical sense, and refers to <strong>the</strong> differences in <strong>the</strong> conditioning<br />
variables used for matching and not bias in an estimator for a parameter.
137<br />
Table 6.3 using Swedish data with up to 2500 cases in matched samples, reported<br />
standardized bias ranging to 10 for individual variables, and reported a summary median,<br />
ra<strong>the</strong>r than <strong>the</strong> mean, <strong>of</strong> up to 6.5. Gerfin and Lechner(2000) p25 Table 7 in multiple<br />
treatment matching, using Swiss data for matched treatment samples <strong>of</strong> 395 to 6000<br />
amongst 16533 cases, found mean standardised bias ranging to 18.6 for some programme<br />
comparisons. They commented that <strong>the</strong>ir use <strong>of</strong> a greater number <strong>of</strong> variables to form <strong>the</strong><br />
propensity estimate led <strong>the</strong> quality <strong>of</strong> <strong>the</strong> match to fall, but still considered <strong>the</strong>se biases to<br />
be quite small. Frölich et al. (2000) p40 Table 6.4 in weighted Swedish data with 6287<br />
cases found mean standardised bias measures ranging to 8.5 and 15, but most were less<br />
than 10. It was advised that some caution be applied in interpreting matching results<br />
where <strong>the</strong> standardized bias was greater than 10. In this respect, <strong>the</strong> measure balance in<br />
this study is approaching <strong>the</strong> need for caution but remains less than 10, and compares<br />
reasonably well with o<strong>the</strong>r studies. Our data are <strong>of</strong> a much smaller size, and a fairly large<br />
number <strong>of</strong> variables are used in <strong>the</strong> propensity estimation, so <strong>the</strong> constraints to matching<br />
balance are greater. In light <strong>of</strong> this, <strong>the</strong> balance on <strong>the</strong> covariates represented by <strong>the</strong><br />
standardized bias indicates <strong>the</strong> matching has performed reasonably.<br />
Examination <strong>of</strong> standardized mean bias in detail can be found by referring to Appendix<br />
Table A2.1, which is deemed too large to be shown here. Rosenbaum and Rubin (1983)<br />
p51 point out that one great advantage for propensity score matching is that even<br />
variables that represent quite rare events can be accounted for where it would not have<br />
been possible to find a match using individual matching. An example <strong>of</strong> such a variable<br />
in our case is children, where none <strong>of</strong> <strong>the</strong> comparison matched were observed with<br />
children. The average standardized bias helpfully summarises into one figure <strong>the</strong><br />
variation between <strong>the</strong> treatment and comparison, but using only this measure may detract<br />
from quite poor individual bias measures for some <strong>of</strong> <strong>the</strong> individual variables.<br />
It is most useful to examine some <strong>of</strong> <strong>the</strong> variables that were found influential in <strong>the</strong> probit<br />
used to estimate <strong>the</strong> propensity score. Age has featured in <strong>the</strong> literature for SYETP as an<br />
important aspect <strong>of</strong> participation, and age had a strong statistically significant influence<br />
on SYETP participation in our propensity score probit estimation. The standardized bias
138<br />
calculation for age is very low, ranging from 2.34 for 0.001 caliper but only 0.92 for 0.05<br />
caliper. Prior to matching, <strong>the</strong> standardized bias figure for age was much worse at 49.85,<br />
which is a suitable example <strong>of</strong> <strong>the</strong> improved balance after <strong>the</strong> propensity score matching.<br />
Widening <strong>the</strong> caliper improved <strong>the</strong> balance score <strong>of</strong> this variable, which would be partly<br />
due to <strong>the</strong> number <strong>of</strong> times a comparison was used with replacement, and also partly due<br />
to <strong>the</strong> new comparison cases available for matching to additional SYETP cases as <strong>the</strong><br />
caliper was widened. Duration <strong>of</strong> pre-programme unemployment was also well balanced<br />
after matching, with a standardized bias <strong>of</strong> 2.58. This did not change with caliper size,<br />
but <strong>the</strong> sampling <strong>of</strong> <strong>the</strong> survey design for <strong>the</strong> data, where only those <strong>of</strong> specific<br />
unemployment duration formed <strong>the</strong> sample frame, would be <strong>the</strong> key factor in this.<br />
Qualification to year 12 was badly balanced, even after matching, with <strong>the</strong> means<br />
showing <strong>the</strong> SYETP group remaining more likely to have this. Frölich et al. (2000) point<br />
out that sometimes poor standardized bias can be due to small probabilities and hence<br />
small standard deviation used in standardizing <strong>the</strong> standard bias. In this study, <strong>the</strong> number<br />
<strong>of</strong> cases <strong>of</strong> SYETP is quite small, and <strong>the</strong> variables are not continuous, so this would<br />
contribute to <strong>the</strong> lower balance scores.<br />
One step to counter poor bias, i.e. balance <strong>of</strong> <strong>the</strong> covariates, is to re-specify <strong>the</strong><br />
propensity model (Larsson (2000): 54). This will be treated later in Chapter 7.<br />
A final consideration is how to choose between <strong>the</strong> matching results presented in Table<br />
4.6. The mean difference hardly varies in size, and is always significant, which means<br />
that <strong>the</strong> effectiveness <strong>of</strong> <strong>the</strong> match is <strong>the</strong> main criterion for selecting between <strong>the</strong> results.<br />
The model is identical, merely <strong>the</strong> precision <strong>of</strong> <strong>the</strong> match is adjusted in Table 4.6. The<br />
difference in propensity scores is smallest for <strong>the</strong> 0.001 caliper, indicating this gave <strong>the</strong><br />
closest match in this sense. But in terms <strong>of</strong> <strong>the</strong> standardized bias <strong>the</strong> resulting match is<br />
better for <strong>the</strong> calipers 0.02 and 0.05, although <strong>the</strong> difference in bias statistics is overall<br />
small. As <strong>the</strong> estimate is being selected for comparison with <strong>the</strong> earlier replication<br />
Heckman bivariate probit result <strong>of</strong> Chapter 3, it might be said that to make <strong>the</strong><br />
comparison most similar, all <strong>the</strong> SYETP cases should be included. This would favour <strong>the</strong><br />
result for 0.05 caliper. However, it is claimed that <strong>the</strong> main advantage <strong>of</strong> propensity score
139<br />
matching is <strong>the</strong> ability to weed out <strong>the</strong> incomparable cases, and find <strong>the</strong> treatment results<br />
for <strong>the</strong> most comparable treatment and comparison cases. In addition, <strong>the</strong> empirical<br />
power <strong>of</strong> propensity score matching in reducing <strong>the</strong> problem <strong>of</strong> selection bias is improved<br />
with a propensity score very similar but preferably equal to that <strong>of</strong> <strong>the</strong> treated persons.<br />
Also, <strong>the</strong> bivariate probit employed here is based on <strong>the</strong> assumption <strong>of</strong> a common effect,<br />
which makes <strong>the</strong> choice among propensity score models simpler because <strong>the</strong> loss <strong>of</strong><br />
treated observations due to lack <strong>of</strong> a good match does not affect <strong>the</strong> parameter being<br />
estimated by <strong>the</strong> propensity score matching (as opposed <strong>the</strong> scenario when heterogeneous<br />
effects are assumed). The 0.001 caliper result would <strong>the</strong>n be chosen as <strong>the</strong> result for<br />
comparison upon <strong>the</strong> basis that it had no great deterioration in performance but compared<br />
<strong>the</strong> most comparable cases. In light <strong>of</strong> this, <strong>the</strong> 0.001 caliper results are compared to <strong>the</strong><br />
bivariate probit in <strong>the</strong> discussion.<br />
4.9 Sensitivity analysis: all-in-Radius matching<br />
As a sensitivity analysis <strong>of</strong> <strong>the</strong> choice <strong>of</strong> matching algorithm, results for ‘all-in-radius’<br />
matching are shown in Table 4.7. ‘All in radius’ matching might achieve some gains in<br />
balance, as pair-matching forces <strong>the</strong> selection from within <strong>the</strong> caliper <strong>of</strong> only one<br />
comparison case. This sensitivity analysis is an exercise to examine <strong>the</strong> importance <strong>of</strong> <strong>the</strong><br />
choice <strong>of</strong> matching algorithm.<br />
Table 4.7 repeats <strong>the</strong> presentation <strong>of</strong> various calipers as used for <strong>the</strong> pair patching. The<br />
first column shows <strong>the</strong> smallest caliper width applied to <strong>the</strong> difference in <strong>the</strong> propensity<br />
used to find <strong>the</strong> matched treated and comparison cases, <strong>the</strong> caliper width 0.001. Each<br />
column to <strong>the</strong> right <strong>of</strong> column 1 uses a wider caliper: Column 2 shows <strong>the</strong> caliper width<br />
0.005, column 3 shows 0.01 caliper width, column 4 shows 0.02 caliper width and finally<br />
column 5 shows 0.05 caliper width. To assess <strong>the</strong> matching, for each set <strong>of</strong> matching<br />
results is shown <strong>the</strong> matched mean difference, and <strong>the</strong> F statistic <strong>of</strong> <strong>the</strong> statistical<br />
significance <strong>of</strong> <strong>the</strong> difference, <strong>the</strong> number <strong>of</strong> treat and comparison cases matched, <strong>the</strong><br />
number <strong>of</strong> comparison cases unmatched and <strong>the</strong> mean standardized bias from <strong>the</strong> match.
140<br />
As would be expected where <strong>the</strong> same caliper widths were used, <strong>the</strong> same numbers <strong>of</strong><br />
SYETP cases were matched. But <strong>the</strong> number <strong>of</strong> comparisons that could be matched from<br />
within <strong>the</strong> radius is much greater for every caliper, growing from 462 matched to <strong>the</strong> 87<br />
SYETP in <strong>the</strong> 0.001 caliper, to 920 comparisons matched to <strong>the</strong> 104 SYETP in <strong>the</strong> 0.05<br />
caliper. The good common support for <strong>the</strong> treated and <strong>the</strong> great size <strong>of</strong> <strong>the</strong> comparison<br />
group contribute to <strong>the</strong> large number <strong>of</strong> comparison that could match within each caliper.<br />
The standardised bias is smaller than found for <strong>the</strong> same caliper when pair matching was<br />
used, but not dramatically. Once again, all <strong>the</strong> mean differences in employment are<br />
statistically significant. The difference in employment varies in size more across <strong>the</strong><br />
different caliper widths than for pair matching. The difference in employment found for<br />
<strong>the</strong> 0.001 caliper varies from 0.18 for pair matching to 0.14 for <strong>the</strong> all-in-radius matching.<br />
However, for <strong>the</strong> 0.005 caliper <strong>the</strong> difference is very small, with 0.18 for pair matching to<br />
0.17 for <strong>the</strong> all-in-radius matching. The mean difference in employment for <strong>the</strong> 0.05<br />
caliper (0.15) is more similar to that for <strong>the</strong> 0.001 caliper (0.14). The difference in<br />
employment does not <strong>the</strong>n grow generally with <strong>the</strong> greater caliper. This suggests that<br />
similar propensity scores do not smoothly relate to similar employment outcomes.<br />
Comparing <strong>the</strong> estimate employment effects <strong>of</strong> SYETP in Table 4.8 to those <strong>of</strong> Table 4.7<br />
also gives variable results. At <strong>the</strong> caliper width <strong>of</strong> 0.001, ‘all in radius’ PSM gives a net<br />
employment effect <strong>of</strong> 0.14 percentage points while nearest neighbour PSM gives 0.18.<br />
However for calipers 0.005 to 0.02, <strong>the</strong> estimates are very similar in size, but diverging<br />
again for caliper width 0.05. Generally <strong>the</strong> ‘all in radius’ impact estimates are smaller.<br />
Overall, it is difficult to judge which PSM protocol is preferable. However, all-in-radius<br />
matching is not found used in <strong>the</strong> literature very <strong>of</strong>ten, and most literature apply pair<br />
matching, although some newer literature also applies kernel matching. From <strong>the</strong>se<br />
results, where <strong>the</strong> differences are usually slight, <strong>the</strong> gains to be had from all-in-radius<br />
matching appear to be small relative to pair matching for this data. As such, pairmatching<br />
is concluded to be <strong>the</strong> most useful algorithm and also conforms to <strong>the</strong> great<br />
majority <strong>of</strong> <strong>the</strong> literature. This is <strong>the</strong>n applied in <strong>the</strong> later analyses in following chapters.
141<br />
Table 4.7 Matching results, All-Within- caliper/Radius with replacement<br />
Match<br />
with<br />
caliper<br />
width<br />
0.001<br />
Match<br />
with<br />
caliper<br />
width<br />
0.005<br />
Match<br />
with<br />
caliper<br />
width<br />
0.01<br />
Match<br />
with<br />
caliper<br />
width<br />
0.02<br />
Match<br />
with<br />
caliper<br />
width<br />
0.05<br />
Difference in employment 90 for 0.14 0.17 0.17 0.16 0.15<br />
matched treated and comparisons<br />
F statistic 91 13.3 13.5 16.7 15.3 14.1<br />
df<br />
Number <strong>of</strong> SYETP matched 87 101 102 103 104<br />
Number <strong>of</strong> comparison which satisfy 462 886 917 918 920<br />
<strong>the</strong> caliper rule<br />
Number <strong>of</strong> comparisons unmatched 458 34 3 2 0<br />
Mean Standardized bias calculation 8.18 6.89 7.72 7.84 7.77<br />
after matching 92<br />
4.10 Discussion<br />
The PSM results are now compared to those found for <strong>the</strong> Heckman bivariate normal<br />
selection model in <strong>the</strong> earlier replication chapter. Earlier, <strong>the</strong> difficulty <strong>of</strong> which PSM<br />
result to choose was addressed. The 0.001 caliper result was chosen as <strong>the</strong> result for<br />
comparison upon <strong>the</strong> basis that it had no great deterioration in performance but covered<br />
all SYETP cases. This facilitates similarity <strong>of</strong> conditions with <strong>the</strong> Heckman result.<br />
We need to compare <strong>the</strong> marginal effect <strong>of</strong> SYETP in <strong>the</strong> Heckman probit to <strong>the</strong><br />
matching measure to make a common measure <strong>of</strong> <strong>the</strong> mean effect <strong>of</strong> treatment on <strong>the</strong><br />
treated. The Table 4.8 shown below contains <strong>the</strong>se estimates and <strong>the</strong>ir statistical<br />
significance. Both methods find a positive statistically significant effect on postprogramme<br />
employment from participation in SYETP. The difference between <strong>the</strong><br />
employment gain estimated from <strong>the</strong> two different methods is however quite large.<br />
Whereas for <strong>the</strong> Heckman bivariate probit <strong>the</strong> employment gain is closer to 30 percentage<br />
points, for PSM <strong>the</strong> employment gain is estimated as closer to 20 percentage points.<br />
90 Ever employed in 1986 survey.<br />
91 Degrees <strong>of</strong> freedom df=n-1, n=number <strong>of</strong> matched treated and untreated. The t statistic is not produced<br />
for this type <strong>of</strong> PSM.<br />
92 For all variables entering <strong>the</strong> probit <strong>of</strong> SYETP participation.
142<br />
Table 4.8 Employment effects <strong>of</strong> Heckman versus PSM<br />
Heckman<br />
PSM<br />
selection bivariate probit<br />
Employment effect 0.264 93 0.18<br />
T statistic (2.85)** (2.74)**<br />
PSM: one-to-one nearest-neighbour within-caliper (0.001) matching with replacement.<br />
A number <strong>of</strong> factors are thought to be behind <strong>the</strong> difference in <strong>the</strong> two estimates. The<br />
PSM argues that it controls better for differences in individual characteristics that arise in<br />
<strong>the</strong> SYETP and comparison group. The differences between <strong>the</strong> groups were found to be<br />
large, and so <strong>the</strong> difference in part may be due to this.<br />
The participation equation was identical for both modelling methods. The importance <strong>of</strong><br />
this is now examined.<br />
In <strong>the</strong> Heckman method, <strong>the</strong> participation equation is explicitly modelled to provide a<br />
variable that is <strong>the</strong>n used to control for <strong>the</strong> part <strong>of</strong> unobserved variation <strong>of</strong> <strong>the</strong><br />
employment equation that is correlated with <strong>the</strong> unobserved variation in <strong>the</strong> participation<br />
decision. There is some evidence to support <strong>the</strong> need to model accounting for selection<br />
bias in <strong>the</strong> SYETP process. For example, Stretton and Chapman (1990) p42 in discussing<br />
programme entry conclude that it “…does involve some selection bias as <strong>the</strong> CES selects<br />
those eligible clients who are judged to be ‘job ready with assistance´ for referral ...<br />
Employers <strong>the</strong>n select <strong>the</strong>ir subsidised employee from among a number <strong>of</strong> referrals made<br />
by <strong>the</strong> CES.” Although Stretton and Chapman (1990) were describing <strong>the</strong> selection for<br />
<strong>the</strong> later Jobstart programme, a very similar process existed for SYETP, as can be seen<br />
described in <strong>the</strong> earlier Chapter 2. In <strong>the</strong> SYETP administrative data it was found that<br />
gaining referrals and gaining employment were closely linked in a selection process<br />
[Wielgosz (1984), Aungles and Stewart (1986)]. As <strong>the</strong>re is no variable indicating <strong>the</strong><br />
administrative value <strong>of</strong> whe<strong>the</strong>r individuals were ‘job- ready-with-assistance’, <strong>the</strong>n this is<br />
93 For SYETP in <strong>the</strong> employment equation: dy/dx =0.0325919 and mean for SYETP is 0.081060. The mean<br />
SYETP translates to 8.1 per cent. The marginal effect calculated at <strong>the</strong> mean is <strong>the</strong>n 0.26418. This is<br />
interpreted as a 26 per cent increase in employment. Estimated using <strong>the</strong> mfx command in STATA7.0
143<br />
<strong>the</strong> unobserved component. If <strong>the</strong> variables included in <strong>the</strong> propensity score model and<br />
participation equation <strong>of</strong> <strong>the</strong> bivariate probit were used by <strong>the</strong> caseworkers in SYETP to<br />
determine that an applicant is ‘job-ready-with-assistance’, <strong>the</strong>n selection on<br />
unobservables should be negligible.<br />
For <strong>the</strong> exclusion restriction to identify <strong>the</strong> bivariate probit model, some variables<br />
included in <strong>the</strong> participation equation estimated are <strong>the</strong>n excluded from <strong>the</strong> employment<br />
equation. An important aspect <strong>of</strong> this estimation approach is <strong>the</strong> identification <strong>of</strong> a<br />
credible instrument for <strong>the</strong> exclusion restriction, in this case age and referrals to CEP.<br />
There is some support for this role for <strong>the</strong>se variables in <strong>the</strong> unweighted data, as later<br />
sensitivity analysis in Chapter 7 reports. Also, <strong>the</strong> results <strong>of</strong> estimation rest upon <strong>the</strong><br />
suitability <strong>of</strong> <strong>the</strong> underlying assumption <strong>of</strong> <strong>the</strong> bivariate normal distribution for <strong>the</strong> errors<br />
in <strong>the</strong> participation and employment equations. Finally, <strong>the</strong> specification <strong>of</strong> <strong>the</strong> model is<br />
assumed correct, with no variables left out to cause mis-specification bias.<br />
For <strong>the</strong> semi-parametric method <strong>of</strong> PSM, CIA is <strong>the</strong> critical underlying assumption.<br />
However, again <strong>the</strong> specification <strong>of</strong> <strong>the</strong> model for <strong>the</strong> propensity to participate in SYETP<br />
is assumed correct. The variables in <strong>the</strong> model are central to <strong>the</strong> credibility <strong>of</strong> <strong>the</strong> CIA<br />
assumption. Whe<strong>the</strong>r <strong>the</strong> CIA is met cannot be accurately tested, however <strong>the</strong> plausibility<br />
<strong>of</strong> CIA can be assessed.<br />
In regard to labour market evaluation, pre-programme labour market history is considered<br />
<strong>the</strong> most important explanatory variable, and this is part <strong>of</strong> <strong>the</strong> model applied here (see<br />
Ham and Lalonde (1996)). There are a large number <strong>of</strong> variables generally and it is<br />
considered that at least with regard to <strong>the</strong> breadth <strong>of</strong> individual characteristics controlled<br />
for, <strong>the</strong> CIA has been satisfied. However, although a wide range <strong>of</strong> individual<br />
characteristics has been included, it could be argued that some are lacking that could<br />
affect both participation and employment. Gerfin and Lechner (2000) had <strong>the</strong> long-term<br />
unemployment at <strong>the</strong> regional placement <strong>of</strong>fice, which here would correspond to <strong>the</strong> CES.<br />
This sort <strong>of</strong> variable is not available in <strong>the</strong> data however. A variable in <strong>the</strong> data and which
144<br />
might give an approximation measure <strong>of</strong> <strong>the</strong> functioning <strong>of</strong> <strong>the</strong> administration is CES<br />
referrals, and also CEP referrals which have been used in analysis.<br />
In this analysis, age and referrals to CEP might be <strong>the</strong> key variables under suspicion for<br />
violating <strong>the</strong> CIA rule for PSM. This is because <strong>the</strong>ir use for <strong>the</strong> exclusion in Richardson<br />
(1998) 94 was argued as due to <strong>the</strong>ir plausibly not affecting employment independently<br />
once SYETP participation was accounted for. Indeed, in later checks carried out in this<br />
study it was not possible to estimate <strong>the</strong> Heckman model if <strong>the</strong>se variables were included<br />
in <strong>the</strong> employment equation which might indicate misspecification. In later chapter 7, <strong>the</strong><br />
sensitivity <strong>of</strong> <strong>the</strong> PSM specification to CEP referrals is checked.<br />
Fur<strong>the</strong>rmore, in <strong>the</strong> application <strong>of</strong> both methods, casewise deletion was used for missing<br />
data (item non-response) on modelled variables. This may not be <strong>the</strong> most suitable<br />
treatment for missing data, as briefly discussed in <strong>the</strong> next chapter. However, as later<br />
discussion shows, casewise deletion is preferable because it assumes <strong>the</strong> correct standard<br />
error. As well, in light <strong>of</strong> <strong>the</strong> replication aims, it was necessary to repeat <strong>the</strong> modelling<br />
assumptions <strong>of</strong> Richardson (1998).<br />
However, at least one strong confounding factor to discussion comparing <strong>the</strong> results is <strong>the</strong><br />
issue <strong>of</strong> non-response to <strong>the</strong> surveys. The issue <strong>of</strong> attrition can introduce selection bias<br />
which would confound both Heckman and PSM methods unless it was treated. As a<br />
result, fur<strong>the</strong>r comparison <strong>of</strong> <strong>the</strong> results is deferred. This comparison is <strong>the</strong>n again treated<br />
in later chapters after consideration <strong>of</strong> <strong>the</strong> data problems and repairing <strong>the</strong> data to treat<br />
<strong>the</strong>m.<br />
94 This was checked by modelling <strong>the</strong> univariate probit for employment, where <strong>the</strong>se variables were not<br />
found to be statistically significant when included.
145<br />
5: Study 3 Attrition and non-response in <strong>the</strong> ALS<br />
In this chapter, data loss to <strong>the</strong> sample is examined for <strong>the</strong> <strong>Australian</strong> Longitudinal<br />
Survey data. The <strong>the</strong>ory relating to survey attrition is broadly summarized. Literature on<br />
<strong>the</strong> empirical aspects <strong>of</strong> attrition and effects on estimation are <strong>the</strong>n reviewed. Next, a<br />
practical means <strong>of</strong> investigating and treating attrition in <strong>the</strong> data is presented. Following<br />
this, attrition in <strong>the</strong> ALS is investigated in depth and <strong>the</strong> extent <strong>of</strong> sample reduction is<br />
considered. A univariate analysis <strong>of</strong> <strong>the</strong> scale <strong>of</strong> attrition effects is conducted. The<br />
different sources <strong>of</strong> sample loss are recognized separately. Initially <strong>the</strong> final sample and<br />
those lost from <strong>the</strong> sample are compared. Then, sample loss is examined within <strong>the</strong><br />
SYETP treatment and comparison groups. The non-response to <strong>the</strong> first survey and<br />
sampling design effects are examined to check <strong>the</strong>ir effects on analysis. These are<br />
accounted for with a weight that was supplied with <strong>the</strong> data. Multivariate analysis is <strong>the</strong>n<br />
conducted <strong>of</strong> <strong>the</strong> effects survey attrition on <strong>the</strong> participation model. Attrition is concluded<br />
to be a problem for <strong>the</strong> modelling. Weights are <strong>the</strong>n constructed to repair <strong>the</strong> data. These<br />
attrition weights are <strong>the</strong>n united with <strong>the</strong> sample design/non-response weights. These<br />
combined weights are used in subsequent chapters.<br />
5.1 Examining sample reduction in <strong>the</strong> ALS<br />
A particular concern for <strong>the</strong> analysis <strong>of</strong> longitudinal data is sample attrition. Attrition and<br />
survey non-response are a form <strong>of</strong> non-response usually termed ‘unit non-response’ by<br />
statisticians. The key worry is that attrition may lead to selective samples, which in turn<br />
may make interpretation <strong>of</strong> resulting estimates problematic. One potential problem for <strong>the</strong><br />
analysis in <strong>the</strong> earlier studies is <strong>the</strong> possibility <strong>of</strong> attrition bias due to sample reduction<br />
between <strong>the</strong> first sample survey in 1984 and <strong>the</strong> 1986 survey. An important reason for<br />
properly accounting for attrition is that ignoring attrition can lead to biased parameter<br />
estimates if <strong>the</strong> attrition is related to <strong>the</strong> behaviour being modelled.<br />
The issue <strong>of</strong> attrition was briefly examined in Richardson (1998) by comparing <strong>the</strong><br />
summary statistics <strong>of</strong> <strong>the</strong> final analysis sample (referred to as <strong>the</strong> ‘whole sample’), those
146<br />
<strong>of</strong> <strong>the</strong> SYETP treatment group, <strong>the</strong> non-SYETP comparison group, and <strong>of</strong> <strong>the</strong> ‘attrition’ 95 .<br />
It was concluded that <strong>the</strong> characteristics appeared broadly similar. As a result <strong>of</strong> this<br />
examination, attrition bias was treated as a minor problem.<br />
If attrition is random, <strong>the</strong>n it is not a major problem. Attrition is considered an important<br />
issue if it affects endogenous variables (Hsaio (1986): 203). The key endogenous<br />
variables analysed are observed employment in 1986, and participation in <strong>the</strong> SYETP<br />
programme. Attrition is also a problem if it is correlated with individual characteristics or<br />
with <strong>the</strong> impact <strong>of</strong> treatment conditional on characteristics (Heckman, Lalonde and Smith<br />
(1999): 1914). In light <strong>of</strong> <strong>the</strong> potential importance <strong>of</strong> attrition to <strong>the</strong> interpretation <strong>of</strong><br />
results, it seems sensible to perform fur<strong>the</strong>r investigation <strong>of</strong> <strong>the</strong> impact <strong>of</strong> attrition.<br />
The <strong>the</strong>oretical basis for <strong>the</strong> attrition modelling carried out here, and a review <strong>of</strong> previous<br />
studies <strong>of</strong> attrition is also included in this section. The review focuses on recent<br />
econometric work on attrition. This is followed by an empirical examination <strong>of</strong> <strong>the</strong><br />
sample reduction effects pertinent to <strong>the</strong> analysis. The assessment involves observing <strong>the</strong><br />
univariate effects, <strong>the</strong> effects on <strong>the</strong> model <strong>of</strong> interest, modelling attrition and<br />
constructing weights to account for attrition. At <strong>the</strong> same time, o<strong>the</strong>r data problems that<br />
warrant scrutiny are examined. These include initial non-response, toge<strong>the</strong>r with<br />
accounting for complex survey design, and missing data (item non-response).<br />
5.2 Theory <strong>of</strong> why attrition introduces bias<br />
Hsaio (1986) p198 details <strong>the</strong> issues relating to attrition in panel data concisely. At <strong>the</strong><br />
heart <strong>of</strong> <strong>the</strong> concern about attrition is <strong>the</strong> problem <strong>of</strong> missing data. If <strong>the</strong> data lost through<br />
attrition are missing completely at random [MCAR], or it is assumed that <strong>the</strong> underlying<br />
process <strong>of</strong> missing-ness is random in that it does not depend on <strong>the</strong> data values, <strong>the</strong>n <strong>the</strong><br />
common procedure is to focus on <strong>the</strong> panel <strong>of</strong> complete observations. This is also known<br />
as using <strong>the</strong> balanced panel, or complete case analysis. This is <strong>the</strong> path pursued by<br />
Richardson (1998). Dillman et al. (2002) point out that missing-ness with an MCAR<br />
mechanism is usually an unrealistically strong assumption when missing data do not<br />
95 See discussion Richardson (1998) p.5 and tables 1 and 2, p14 and p15.
147<br />
occur by design, because <strong>the</strong> missing-ness usually does depend on recorded values<br />
(Dillman et al. (2002): 17).<br />
The possibility that <strong>the</strong> data are missing for reasons <strong>of</strong> self-selection raises problems in<br />
econometric models <strong>of</strong> outcomes. In o<strong>the</strong>r words, attrition is most problematic when it is<br />
related to <strong>the</strong> endogenous variable. The data are <strong>the</strong>n not missing completely at random.<br />
If <strong>the</strong> probability <strong>of</strong> attrition is related to <strong>the</strong> outcomes, <strong>the</strong>n <strong>the</strong> estimates are biased and<br />
inconsistent.<br />
The 2 period model is as follows:<br />
(1) y it = β ‘ x it + v it<br />
(2) a i =1 if y i2 is observed,<br />
a i =0 if y i2 is not observed<br />
(3) a i =1 if a i * = γ y i2 + θ ‘x i2 + δ ‘ w i + ε i * ≥ 0<br />
where time T=2, v it is <strong>the</strong> error term, y it is <strong>the</strong> endogenous variable, a i is <strong>the</strong> attrition<br />
dummy, a i * is <strong>the</strong> latent variable for attrition, w i is <strong>the</strong> vector <strong>of</strong> variables that do not<br />
enter <strong>the</strong> conditional expectation <strong>of</strong> y but affect <strong>the</strong> probability <strong>of</strong> observing y, θ and δ are<br />
parameters, and ε i * are normally distributed. A common procedure for analysis where<br />
<strong>the</strong>re is attrition, is to use only those cases present in all surveys, also called a balanced<br />
panel. Attrition in <strong>the</strong> second period would mean that in a balanced panel, <strong>the</strong><br />
observations for which y i2 is missing are discarded.<br />
If <strong>the</strong> probability <strong>of</strong> observing y i2 varies with <strong>the</strong> value <strong>of</strong> attrition , as well as <strong>the</strong> value <strong>of</strong><br />
o<strong>the</strong>r variables, <strong>the</strong>n <strong>the</strong> probability <strong>of</strong> observing y i2 depends on <strong>the</strong> error term v i2 . The<br />
reduced form, where y i2 is substituted into equation (3), becomes:<br />
(4) a i * = (γ β ‘ + θ ‘)x i2 + δ ‘ w i + γ v i2 + ε i *<br />
This can be rewritten as:<br />
(5) a i * = π ‘x i2 + δ ‘ w i + γ v i2 + ε i<br />
and finally collecting terms in matrix notation:
148<br />
(6) a i * = d ‘ R i + ε i<br />
where d ‘ = (π ‘,δ ‘) ; R i = (x i2 , w i ) ; ε i = γ v i2 + ε i *<br />
If <strong>the</strong> v it are assumed to be normally distributed and <strong>the</strong> variance <strong>of</strong> ε i is one, and Φ(.) is<br />
<strong>the</strong> standard normal distribution <strong>the</strong>n <strong>the</strong> probability <strong>of</strong> no attrition is<br />
(7) Prob (a i = 1) = Φ(a ‘ R i )<br />
When using <strong>the</strong> balanced panel <strong>of</strong> complete observations, Hsaio (1986) p199 shows that<br />
<strong>the</strong> conditional expectation <strong>of</strong> y i2 is<br />
(8) E (y i2 | x i2 , a i =1 ) = β ‘ x i2 + σ 2 ε [φ (a ‘ R i ) / Φ (d ‘ R i )]<br />
Where σ 2 ε is <strong>the</strong> covariance between v i2 + ε i and φ (.) is <strong>the</strong> standard normal<br />
distribution. It is clear that this derivation is <strong>the</strong> same as <strong>the</strong> bivariate probit model<br />
presented earlier in Chapter 3, but applied to <strong>the</strong> case <strong>of</strong> attrition.<br />
In <strong>the</strong> <strong>Australian</strong> Longitudinal Survey, attrition is a strong issue for <strong>the</strong> analysis <strong>of</strong><br />
employment in 1986, as <strong>the</strong> employment in 1986 can never be known for those who are<br />
not interviewed in 1986 because <strong>the</strong>y are lost through attrition. This introduces <strong>the</strong><br />
problem <strong>of</strong> selection by virtue <strong>of</strong> survival, where <strong>the</strong> modelling <strong>of</strong> employment in 1986<br />
can only take place for <strong>the</strong> sample surviving from 1984 to 1986 (Hoem (1985): 260). The<br />
bias introduced by attrition is shown in equation 8. Unless σ 2 ε = 0 , <strong>the</strong>n estimates <strong>of</strong> β<br />
using <strong>the</strong> balanced panel <strong>of</strong> complete observations are biased and inconsistent.<br />
The first formal econometric model <strong>of</strong> attrition and labour market behaviour is usually<br />
attributed to Hausman and Wise (1979). There was a high attrition from <strong>the</strong> Gary Income<br />
Maintenance experiment and <strong>the</strong>y were concerned this would bias <strong>the</strong> experimental<br />
estimates because <strong>the</strong> attrition was likely related to income, which was <strong>the</strong>ir outcome <strong>of</strong><br />
interest. They found evidence <strong>of</strong> attrition bias in modelling attrition alongside <strong>the</strong> income<br />
equations, but concluded it had little effect on <strong>the</strong> experimental effect estimated. Ridder
149<br />
(1990) extended and improved <strong>the</strong> Hausman-Wise model. These models incorporate<br />
attrition into <strong>the</strong> behavioural model.<br />
Selection modelling is central to our model <strong>of</strong> intrinsic interest, bivariate modelling <strong>of</strong><br />
SYETP and employment outcomes. Accounting separately for attrition within this model,<br />
would require at least trivariate selection modelling. Recent work by Capellari (2001)<br />
uses <strong>the</strong> multivariate probability approach to account for endogenous panel attrition and<br />
selectivity in earnings, allowing estimation by application <strong>of</strong> simulated maximum<br />
likelihood techniques for <strong>the</strong> multi-dimensional integrals. However early examinations by<br />
Schmertmann (1994) found that <strong>the</strong> estimation approaches available for selectivity<br />
modelling <strong>of</strong> higher orders were not very good. The estimation suffers from <strong>the</strong> strict<br />
restrictions that need to be assumed for <strong>the</strong> bivariate correlations in <strong>the</strong> error distribution.<br />
It was concluded that in models with many selection criteria, or where sample sizes are<br />
small, <strong>the</strong> methods would not give good estimates. Fur<strong>the</strong>rmore, very strong assumptions<br />
are required about <strong>the</strong> correlations between response, SYETP participation and<br />
employment, and so on <strong>the</strong>se grounds this method <strong>of</strong> incorporating attrition with<br />
selection modelling is ruled out for this application. Fur<strong>the</strong>rmore, considering <strong>the</strong> small<br />
sample size we have available, combined with <strong>the</strong> strength <strong>of</strong> <strong>the</strong> assumptions required,<br />
this is beyond <strong>the</strong> scope <strong>of</strong> this paper.<br />
5.3 Empirical aspects <strong>of</strong> <strong>the</strong> effects <strong>of</strong> attrition on estimates<br />
Recent examinations <strong>of</strong> <strong>the</strong> empirical effects <strong>of</strong> attrition have raised <strong>the</strong> argument that <strong>the</strong><br />
<strong>the</strong>oretical potential for attrition bias does not always warrant concern. Fitzgerald et al.<br />
(1998a) concluded that <strong>the</strong> empirical existence and magnitude <strong>of</strong> attrition bias was not<br />
clearly related to <strong>the</strong> size <strong>of</strong> <strong>the</strong> attrition rate. They also concluded that attrition which is<br />
random in nature can result in no measurable bias arising in economic modelling, in<br />
accordance with earlier statistical literature (Fitzgerald et al. (1998a): 256).<br />
Fur<strong>the</strong>r recent studies on <strong>the</strong> evidence for <strong>the</strong> effects <strong>of</strong> attrition bias on estimates in<br />
labour market analysis make stronger claims. Lillard and Panis (1998) p456 argued that<br />
<strong>the</strong>ir results for <strong>the</strong> US panel data Michigan Panel Study on Income Dynamics (PSID)
150<br />
indicated that even significant attrition which is observed to be selective in nature does<br />
not introduce strong biases in estimation results. Falaris and Peters (1998) p 531 noted<br />
that effects <strong>of</strong> attrition on regression estimates in general is negligible, or only affects <strong>the</strong><br />
intercept. The overarching message from <strong>the</strong>se studies is that attrition bias, even when<br />
found to act selectively on observable characteristics, does not necessarily bias <strong>the</strong><br />
estimates <strong>of</strong> interest in modelling.<br />
O<strong>the</strong>r evidence also points to <strong>the</strong> importance <strong>of</strong> validating <strong>the</strong> extent <strong>of</strong> attrition bias<br />
effects on <strong>the</strong> estimates <strong>of</strong> interest. Alderman et al. (2000) apply <strong>the</strong> methods set out in<br />
Fitzgerald et al. (1998a), and find some outcomes are affected by attrition bias while<br />
o<strong>the</strong>rs are not. They find that univariate comparisons showing systematic attrition<br />
affecting particular variables <strong>of</strong> interest do not translate to <strong>the</strong>se variables being<br />
significant in a probit model predicting attrition. They warn that in <strong>the</strong> relations <strong>the</strong>y<br />
modelled attrition bias led to striking differences affecting <strong>the</strong> coefficients for some<br />
models <strong>of</strong> outcomes but not o<strong>the</strong>rs. They fur<strong>the</strong>r point out that <strong>the</strong>ir results indicate that<br />
attrition bias conclusions are not generalisable to all multivariate estimates or all data, but<br />
that <strong>the</strong> particular model, outcome <strong>of</strong> interest and data need to be assessed.<br />
5.4 Empirical attrition test and treatment<br />
Fitzgerald et al. (1998b) examined <strong>the</strong> Michigan Panel Study on Income Dynamics (PSID)<br />
<strong>of</strong> <strong>the</strong> US, and found attrition was highly selective, concentrated amongst those <strong>of</strong> lower<br />
socioeconomic status. Usefully, <strong>the</strong>y outlined <strong>the</strong> statistical framework for tests for<br />
attrition bias within an econometric context. In <strong>the</strong>ir model, a key distinction is drawn<br />
between attrition where selection is on observables as opposed to selection upon<br />
unobservables. The background for <strong>the</strong>ir approach is <strong>the</strong> selection bias modelling<br />
econometric literature, deriving chiefly from Heckman (1979). The earlier attrition study<br />
<strong>of</strong> <strong>the</strong> PSID by Becketti et al. (1988) is shown to be a close relative <strong>of</strong> <strong>the</strong> direct<br />
modelling <strong>of</strong> attrition proposed by Fitzgerald et al. (1998a). Their model is defined as<br />
follows:<br />
(9) Y = β 0 + β 1 X + ε
151<br />
(10) A* = δ 0 + δ 1 x +δ 2 z + ν<br />
(11) A =1 if A* ≥ 0 and A=0 if A*
152<br />
Richardson (1998) is <strong>the</strong>n extended here, by estimating <strong>the</strong> attrition <strong>of</strong> <strong>the</strong> <strong>Australian</strong><br />
Longitudinal Survey and constructing weights to reduce <strong>the</strong> resultant bias. This also<br />
serves as a test <strong>of</strong> <strong>the</strong> impact <strong>of</strong> attrition bias on <strong>the</strong> estimated results.<br />
5.5 Examining <strong>the</strong> role <strong>of</strong> attrition in <strong>the</strong> ALS and in modelling <strong>of</strong> <strong>the</strong> SYETP<br />
employment effect<br />
To assess <strong>the</strong> issue <strong>of</strong> attrition bias a set <strong>of</strong> analyses are conducted, drawing on <strong>the</strong> tests<br />
<strong>of</strong> Fitzgerald et al. (1998a). Firstly, <strong>the</strong> univariate effects <strong>of</strong> attrition upon means are<br />
examined. The Fitzgerald et al. (1998a) framework for <strong>the</strong> statistical analysis <strong>of</strong> attrition<br />
bias applied here focuses upon sample attrition which results in selection on observables.<br />
The case for attrition due to observables is examined for <strong>the</strong> ALS by investigating <strong>the</strong><br />
observed means. The effects <strong>of</strong> attrition upon <strong>the</strong> models <strong>of</strong> SYETP participation are also<br />
considered. The probability <strong>of</strong> attrition is <strong>the</strong>n modelled using <strong>the</strong> probit. Weighting<br />
effects upon <strong>the</strong> model <strong>of</strong> employment outcomes for <strong>the</strong> SYETP programme are<br />
additionally explored.<br />
5.5.1 Extent <strong>of</strong> sample reduction<br />
The ALS list sample was constructed from administrative records <strong>of</strong> persons aged 15-24<br />
on 1 September 1984. The sample was selected based on those registered with <strong>the</strong><br />
Commonwealth Employment Service <strong>of</strong>fices throughout Australia who were unemployed<br />
and seeking full-time work for three months or more at <strong>the</strong> time <strong>of</strong> selection in June 1984.<br />
The original sample had 2998 cases, and <strong>the</strong> first survey collected interviews from 2,403<br />
cases.<br />
Interviews for <strong>the</strong> first survey were conducted between September 1984 and November<br />
1984. The survey was repeated in each <strong>of</strong> <strong>the</strong> later years 1985, 1986 and 1987, following<br />
up <strong>the</strong> same people with whom interviews were conducted in 1984. In 1985 1,910<br />
interviews were conducted in <strong>the</strong> months June to October, in 1986 1,711 interviews were<br />
collected between September and November, and in 1987 1,518 interviews repeated
153<br />
again from September to November. Of prime interest here are <strong>the</strong> first three years <strong>of</strong><br />
survey observations to 1986.<br />
The ALS data finally used for analysis has a reasonable rate <strong>of</strong> sample reduction. Each<br />
year, fewer observations in ALS have survey outcomes available for analysis. The sample<br />
reduction due to attrition is shown in Table 5.1. The initial non-response rate for <strong>the</strong> first<br />
observations in 1984 is 20 per cent. Loss <strong>of</strong> sample observations between <strong>the</strong> first survey<br />
in1984 and <strong>the</strong> second survey in 1985, is 21 per cent. The attrition rate falls to 10 per cent<br />
between 1985 and 1986. By <strong>the</strong> third survey in 1986, only 57% <strong>of</strong> <strong>the</strong> original sample<br />
remains. Non-response was addressed in <strong>the</strong> survey design with heavy follow-up 96 . The<br />
final sample <strong>of</strong> 1283 cases used for analysis 97 arises due to analytical selection, where<br />
404 cases were discarded, amounting to 24 per cent <strong>of</strong> those who respond to all surveys<br />
to 1986. These different sources <strong>of</strong> sample reduction are now examined.<br />
Table 5.1 Sample Reduction for ALS List sample<br />
Number <strong>of</strong> cases<br />
% cases lost<br />
each stage<br />
Original sample 2998<br />
1984 survey interviews 2403 20 80<br />
1985 survey interviews 1910 21 64<br />
1986 survey interviews 1711 10 57<br />
1987 survey interviews 1518 11 51<br />
% cases<br />
remaining <strong>of</strong><br />
original<br />
sample<br />
A large part <strong>of</strong> <strong>the</strong> sample reduction in <strong>the</strong> analytical selection is due to missing data<br />
problems. Of those responding to <strong>the</strong> 1986 survey, that is, <strong>of</strong> those that do not attrit,<br />
fur<strong>the</strong>r sample reduction occurs for several reasons. As previously described from those<br />
interviewed in 1984, <strong>the</strong> final sample excludes those who were over 25 years at <strong>the</strong><br />
96 Interviewers were required to make 7 visits in person to <strong>the</strong> initial, 2 nd or 3 rd address <strong>of</strong> a mover, and<br />
were instructed to use a varying time <strong>of</strong> contact pattern, and supervisors would follow-up in attempts to<br />
convert refusals. Interpreters were used where necessary, and an introductory approach letter from <strong>the</strong><br />
Minister and <strong>the</strong> survey firm was sent in advance <strong>of</strong> surveying. All <strong>of</strong> <strong>the</strong>se took place before non-contact<br />
was declared.<br />
97 The sample is formed to match that <strong>of</strong> Richardson (1998) to facilitate replication and subsequent<br />
comparison <strong>of</strong> results.
154<br />
interview date, those who ever entered full-time education, or for whom responses were<br />
missing in 1984, 1985 or 1986 (Richardson (1998): 5). The first <strong>of</strong> <strong>the</strong>se restrictions is<br />
not dealt with here, as it prescribes <strong>the</strong> eligible set, but is returned to subsequently. The<br />
latter <strong>of</strong> <strong>the</strong>se has been dealt with in part, with reference to <strong>the</strong> natural attrition, which<br />
caused missing values. Apart from natural attrition, <strong>the</strong> set discarded for whom responses<br />
were missing in 1984, 1985 or 1986 also includes those cases for whom data on some <strong>of</strong><br />
<strong>the</strong> regressor variables is missing. In particular, when information about <strong>the</strong> family<br />
background is missing, or when <strong>the</strong> proportion <strong>of</strong> time in unemployment is missing, <strong>the</strong>se<br />
cases are dropped from <strong>the</strong> analysis set. Statisticians usually term this form <strong>of</strong> missing<br />
data ‘item non-response’.<br />
Greene (1991) points out <strong>the</strong>re are two main scenarios to consider, depending on why <strong>the</strong><br />
data on <strong>the</strong> regressors is missing. Imputation or discarding <strong>the</strong> cases is <strong>the</strong> usual approach<br />
to this type <strong>of</strong> missing data. Using <strong>the</strong> terminology <strong>of</strong> Griliches (1986) this type <strong>of</strong><br />
incomplete data is described as <strong>the</strong> ignorable case if when using <strong>the</strong> complete<br />
observations <strong>the</strong> data are not missing for self-selection reasons 98 . In <strong>the</strong> ignorable case,<br />
using only <strong>the</strong> complete observations is not problematic if efficiency related to <strong>the</strong><br />
variance is not important. The non-ignorable case, where <strong>the</strong> missing information is<br />
systematically related to <strong>the</strong> variable being modelled, is ano<strong>the</strong>r case <strong>of</strong> <strong>the</strong> sample<br />
selection problem.<br />
98 Ignorable non-response means <strong>the</strong> same as selection on observables.
155<br />
5.5.2 Univariate examination <strong>of</strong> sample reduction<br />
Table 5.2 gives a univariate analysis <strong>of</strong> <strong>the</strong> differences between <strong>the</strong> characteristics <strong>of</strong> <strong>the</strong><br />
final sample and those who are lost from <strong>the</strong> sample. This expands <strong>the</strong> information<br />
provided in Table 1 <strong>of</strong> Richardson (1998) by including <strong>the</strong> t-test <strong>of</strong> statistical significance<br />
for <strong>the</strong> difference between <strong>the</strong> mean <strong>of</strong> <strong>the</strong> remaining sample and those lost. Column 1<br />
shows <strong>the</strong> mean for <strong>the</strong> final sample, and corresponds to column 1 <strong>of</strong> Table 1, Richardson<br />
(1998). Allowing for rounding, <strong>the</strong> means are identical to those presented in Richardson.<br />
This is seen as a check that <strong>the</strong> final data matches that <strong>of</strong> Richardson (1998). The second<br />
column gives <strong>the</strong> standard deviation. Column 3 gives <strong>the</strong> mean <strong>of</strong> those lost from <strong>the</strong><br />
sample, with column 4 <strong>the</strong> standard deviation. The absolute difference in <strong>the</strong> mean is<br />
given in <strong>the</strong> fifth column, with <strong>the</strong> t statistic in <strong>the</strong> sixth column. The t test <strong>of</strong> <strong>the</strong><br />
hypo<strong>the</strong>sis that <strong>the</strong> difference in <strong>the</strong> means is zero, accommodating unequal variance in<br />
<strong>the</strong> samples, is shown in column 6, with significance levels <strong>of</strong> 10 per cent, 5 per cent and<br />
1 per cent indicated with 1, 2 and 3 stars in each case. A number <strong>of</strong> outcome and<br />
regressor variables are shown. Most <strong>of</strong> <strong>the</strong>se are useful variables in <strong>the</strong> modelling <strong>of</strong><br />
employment outcomes. All <strong>the</strong> variables are identified using <strong>the</strong> 1984 survey data, except<br />
‘ever employed in 1986’, and ‘ever on a government programme in 1986’, <strong>the</strong> first <strong>of</strong><br />
which is a dependent variable for analysis. SYETP participation is an important variable<br />
that is an outcome defined in <strong>the</strong> 1984 survey.<br />
Table 5.2 compares those remaining in <strong>the</strong> final sample to all those lost from <strong>the</strong> final<br />
sample. It is important to note that this does not differentiate between those cases lost due<br />
to non-response at surveys in 1985, 1986 and those cases dropped in <strong>the</strong> analytical<br />
selection process. The aim <strong>of</strong> initially examining <strong>the</strong> effect <strong>of</strong> sample loss on <strong>the</strong> means<br />
observed is not affected by this combination. The different sources <strong>of</strong> sample loss are<br />
subsequently dealt with separately in <strong>the</strong> fur<strong>the</strong>r analysis <strong>of</strong> attrition.<br />
The two variables ‘ever employed in 1986’, and ‘ever on a government programme in<br />
1986’ suffer from <strong>the</strong> ‘selection by survival’ problem. The parts <strong>of</strong> <strong>the</strong> sample lost due to
156<br />
natural attrition have missing values for <strong>the</strong>se cases, as <strong>the</strong>y are defined in <strong>the</strong> 1986<br />
survey. Accordingly, in <strong>the</strong> table <strong>the</strong>y are not presented since <strong>the</strong> overall mean would not<br />
be representative <strong>of</strong> <strong>the</strong> same observations as <strong>the</strong> rest <strong>of</strong> <strong>the</strong> table. Effects upon <strong>the</strong>se<br />
variables are dealt with in <strong>the</strong> later section where <strong>the</strong> source <strong>of</strong> data reduction is<br />
controlled for.<br />
Several characteristics in Table 5.2 exhibit statistically significant differences between<br />
<strong>the</strong> final sample and those lost from <strong>the</strong> sample: whe<strong>the</strong>r <strong>the</strong> partner was employed,<br />
schooling to year 9 and year 11, <strong>the</strong> length <strong>of</strong> <strong>the</strong> last job held, work limiting health<br />
problems, state <strong>of</strong> residence at time <strong>of</strong> interview, and <strong>the</strong> rural-urban nature <strong>of</strong> <strong>the</strong><br />
location. The SYETP treatment variable also shows evidence that it was affected by<br />
sample reduction. Thus both outcomes and regressors appear affected in <strong>the</strong> univariate<br />
analysis.<br />
It is interesting to examine how those lost from <strong>the</strong> sample compare to those remaining.<br />
Past studies, such as Hausman and Wise (1979) found patterns indicating particular strata<br />
<strong>of</strong> <strong>the</strong>ir dependent variable were not being observed, in <strong>the</strong>ir case parts <strong>of</strong> <strong>the</strong> earnings<br />
distribution were truncated, and certain subgroups more likely not to be surveyed.<br />
Fitzgerald et al. (1998a) also found lower socio-economic groups were more likely to be<br />
affected. Here, amongst those lost from <strong>the</strong> sample, when compared to those remaining,<br />
fewer had employed spouses in 1984, fewer had obtained only year 11 at school while<br />
more had year 9 <strong>of</strong> school, and fewer had health problems that affected <strong>the</strong>ir work. The<br />
length <strong>of</strong> <strong>the</strong> last job held differed, for more had held jobs <strong>of</strong> 1-2 years while less had<br />
held jobs <strong>of</strong> 2-3 years. Of those lost, fewer were first interviewed in South Australia or<br />
Nor<strong>the</strong>rn Territory and more in New South Wales or <strong>the</strong> <strong>Australian</strong> Capital Territory. The<br />
location <strong>of</strong> <strong>the</strong> interview was important, as amongst those lost fewer came from rural<br />
areas or country towns, and more were from cities. Fewer <strong>of</strong> those lost from <strong>the</strong> sample<br />
had taken part in SYETP.<br />
There is no clear picture <strong>of</strong> those lost being from lower socioeconomic groups or having<br />
a more disadvantaged labour market position. While <strong>the</strong> educational capital is slightly
157<br />
lower, <strong>the</strong> job lengths are only slightly shorter with <strong>the</strong> distribution <strong>of</strong> lengths only<br />
appearing bulked up in <strong>the</strong> middle, and <strong>the</strong>re are fewer with health problems, none <strong>of</strong><br />
which indicates a preponderance <strong>of</strong> observations at <strong>the</strong> lower end <strong>of</strong> <strong>the</strong> labour market.<br />
This mixed picture may arise due to <strong>the</strong> different sources <strong>of</strong> sample loss. Whe<strong>the</strong>r this<br />
can be resolved is dealt with again later by fur<strong>the</strong>r examining <strong>the</strong> differences by source <strong>of</strong><br />
sample loss.
158<br />
s.d. Those<br />
Table 5.2 Summary statistics for attrition effects in <strong>the</strong> final sample<br />
Mean and<br />
sample<br />
s.d.<br />
mean 99 sample<br />
standard deviation (s.d.) after all<br />
sample<br />
reduction<br />
lost<br />
from<br />
<strong>the</strong><br />
mean<br />
Difference in<br />
means<br />
(absolute)<br />
T –test<br />
SYETP 0.08 0.27 0.05 0.21 0.03 3.42***<br />
Female 0.41 0.49 0.40 0.49 0.01 0.43<br />
Average age 1984 19.99 2.41 20.02 2.38 0.03 0.30<br />
Aboriginal/Torres Strait 0.03 0.17 0.04 0.19 0.01 1.09<br />
Islander<br />
O<strong>the</strong>r ethnic minority 0.08 0.27 0.09 0.29 0.01 1.08<br />
Married 1984 0.12 0.32 0.10 0.30 0.02 1.21<br />
Spouse employed 1984 0.06 0.24 0.04 0.19 0.02 2.44***<br />
Children 1984 0.05 0.23 0.04 0.20 0.01 1.27<br />
Highest qualification in 1984<br />
Degree/diploma 0.12 0.32 0.11 0.31 0.01 1.03<br />
Apprenticeship 0.08 0.27 0.07 0.25 0.01 1.19<br />
O<strong>the</strong>r post-school<br />
0.07 0.26 0.07 0.26 0<br />
qualification<br />
Year 12 <strong>of</strong> school 0.15 0.35 0.17 0.37 0.02 1.35<br />
Year 11 <strong>of</strong> school 0.14 0.35 0.11 0.31 0.03 2.07**<br />
Year 10 <strong>of</strong> school 0.31 0.46 0.30 0.46 0.01 0.86<br />
Year 9 <strong>of</strong> school 0.12 0.33 0.15 0.36 0.03 2.04**<br />
Parental background<br />
when resp. aged 14<br />
Fa<strong>the</strong>r postschool<br />
qualification<br />
0.34 0.47 0.35 0.48 0.01 0.30<br />
Mo<strong>the</strong>r postschool<br />
0.18 0.39 0.20 0.40 0.02 0.82<br />
qualification<br />
Fa<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.26 0.44 0.24 0.43 0.02 1.02<br />
para-pr<strong>of</strong>essional<br />
Fa<strong>the</strong>r not employed 0.05 0.23 0.05 0.22 0.005 0.55<br />
Fa<strong>the</strong>r not present 0.16 0.36 0.17 0.38 0.01 0.97<br />
Mo<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.10 0.29 0.08 0.28 0.02 0.92<br />
para-pr<strong>of</strong>essional<br />
Mo<strong>the</strong>r not employed 0.55 0.50 0.55 0.50 0.006 0.30<br />
Mo<strong>the</strong>r not present 0.05 0.22 0.06 0.24 0.008 0.82<br />
Longest job ever held by 1984<br />
Never held a job 0.12 0.32 0.11 0.31 0.01 0.42<br />
< 1 year 0.42 0.49 0.44 0.50 0.02 0.92<br />
1 year 0.13 0.34 0.17 0.37 0.04 2.22**<br />
2 years 0.14 0.35 0.12 0.32 0.02 1.67*<br />
3 years or more 0.19 0.39 0.16 0.37 0.03 1.47<br />
Average unemployment 100 0.62 0.41 0.64 0.41 0.03 1.55<br />
Ever employed in 1986 101 0.74 0.44 Na Na Na Na<br />
Ever government programme<br />
1986 102 0.11 0.31 Na Na Na Na<br />
99 The mean <strong>of</strong> a 0/1 binary variable is <strong>the</strong> same as <strong>the</strong> proportion.<br />
100 Proportion <strong>of</strong> 1984 reference period to 3 June spent unemployed.<br />
101 Ever held a non-subsidised, non-government programme job in <strong>the</strong> 1986 reference period, after <strong>the</strong> first<br />
17 weeks.
159<br />
Work limited by health 1984 0.10 0.30 0.12 0.33 0.02 1.82*<br />
Interviewed location in 1984<br />
New South Wales /ACT 0.37 0.48 0.41 0.49 0.04 1.88*<br />
Victoria 0.24 0.43 0.24 0.42 0<br />
Queensland 0.14 0.34 0.15 0.35 0.01 0.66<br />
South Australia /Nor<strong>the</strong>rn 0.12 0.33 0.09 0.29 0.04 2.73***<br />
Territory<br />
Western Australia / Tasmania 0.13 0.33 0.12 0.32 0.01 0.88<br />
Capital city 0.47 0.50 0.52 0.50 0.05 2.43***<br />
O<strong>the</strong>r town 0.21 0.41 0.26 0.44 0.05 2.61***<br />
Country town 0.24 0.43 0.18 0.39 0.06 3.54***<br />
Rural area 0.08 0.27 0.04 0.20 0.04 3.79***<br />
Number <strong>of</strong> observations 1283 1085<br />
NOTES: Significance <strong>of</strong> t test at level 1% ***, 5%**, 10 % *.Student’s t tests <strong>of</strong> whe<strong>the</strong>r means are equal<br />
are performed with <strong>the</strong> assumption <strong>of</strong> unequal variances, n-1 degrees <strong>of</strong> freedom.<br />
Na = not applicable, <strong>the</strong>se statistics are always missing for natural attrition.<br />
102 Ever go on a government programme, including SYETP, in <strong>the</strong> 1986 reference period.
160<br />
5.5.3 Sample reduction by SYETP treatment group<br />
5.5.3.1 Comparing those lost in attrition to those who remain<br />
In <strong>the</strong> previous section it was found that fewer <strong>of</strong> those lost from <strong>the</strong> sample had taken<br />
part in SYETP. Table 5.3 again examines <strong>the</strong> means for attrition effects, but focusing<br />
within <strong>the</strong> SYETP treatment and comparison groups. This table once again expands on<br />
<strong>the</strong> information provided in Table 1 <strong>of</strong> Richardson (1998) by including <strong>the</strong> t-test <strong>of</strong><br />
statistical significance for <strong>the</strong> difference between <strong>the</strong> mean <strong>of</strong> <strong>the</strong> remaining sample and<br />
those lost. The small scale <strong>of</strong> <strong>the</strong> treatment group renders this an important means <strong>of</strong><br />
identifying statistically significant differences amongst <strong>the</strong> means. The wide difference in<br />
<strong>the</strong> size <strong>of</strong> <strong>the</strong> comparison and treatment groups, allows <strong>the</strong> fluctuations in <strong>the</strong><br />
comparison group to dominate as <strong>the</strong> main source <strong>of</strong> statistical differences.<br />
The first panel on <strong>the</strong> left <strong>of</strong> Table 5.3 shows <strong>the</strong> SYETP group, while <strong>the</strong> second panel<br />
on <strong>the</strong> right shows <strong>the</strong> comparison group. Within each panel, <strong>the</strong> same format as that <strong>of</strong><br />
Table 5.2 is followed, with <strong>the</strong> mean in <strong>the</strong> remaining sample in <strong>the</strong> first column, <strong>the</strong><br />
standard deviation in <strong>the</strong> second column, <strong>the</strong> mean for those lost in <strong>the</strong> third column, <strong>the</strong><br />
standard deviation in <strong>the</strong> fourth, <strong>the</strong> absolute difference between <strong>the</strong> means in <strong>the</strong> fifth<br />
column and <strong>the</strong> t statistic in <strong>the</strong> sixth column. Column 1 in <strong>the</strong> first panel corresponds to<br />
Column 2 <strong>of</strong> Table 1 <strong>of</strong> Richardson (1998) p14, while Column 2 in <strong>the</strong> second panel<br />
reflects <strong>the</strong> same information as Column 3 <strong>of</strong> Table 1 <strong>of</strong> Richardson (1998). Generally,<br />
all statistics are in agreement, which is seen as a fur<strong>the</strong>r check that <strong>the</strong> same final sample<br />
as Richardson (1998) has been achieved.<br />
The SYETP treatment group shows an important difference on two key characteristics,<br />
<strong>the</strong> proportion <strong>of</strong> time spent unemployed in <strong>the</strong> 1984 reference period, and <strong>the</strong> share with<br />
year 12 schooling. Those lost had a much smaller proportion <strong>of</strong> <strong>the</strong>ir time spent<br />
unemployed, almost half <strong>of</strong> that for those remaining in <strong>the</strong> sample. In addition, far fewer<br />
had <strong>the</strong> year 12 leaving certificate than those left in <strong>the</strong> sample.
161<br />
The characteristics <strong>of</strong> <strong>the</strong> SYETP participant pr<strong>of</strong>ile affected by <strong>the</strong> sample reduction are<br />
quite different to those among <strong>the</strong> comparison group. Amongst <strong>the</strong> comparison group,<br />
fewer <strong>of</strong> those lost had employed partners. As well, <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> schooling was different,<br />
more <strong>of</strong> those lost had year 12 and year 9 schooling, but fewer had year 11. The pattern<br />
<strong>of</strong> job lengths held by those <strong>of</strong> <strong>the</strong> comparisons lost was different to those remaining:<br />
more had held jobs <strong>of</strong> one to 2 years, but less had held jobs <strong>of</strong> three years or more. The<br />
locations <strong>of</strong> those lost also differ from <strong>the</strong> locations <strong>of</strong> those who remain in <strong>the</strong><br />
comparison group, with more in New South Wales and <strong>Australian</strong> Capital Territory, and<br />
fewer in South Australia or Nor<strong>the</strong>rn Territory, less came from rural areas or country<br />
towns, and more were from cities. It is conspicuous that <strong>the</strong> location <strong>of</strong> those lost within<br />
<strong>the</strong> comparison group reflects <strong>the</strong> overall pattern observed earlier in Table 5.2. The bulk<br />
<strong>of</strong> observations being in <strong>the</strong> comparison group enforces <strong>the</strong> influence <strong>of</strong> this group’s<br />
pr<strong>of</strong>ile upon <strong>the</strong> overall statistics in Table 2.<br />
The variation in <strong>the</strong> effects within <strong>the</strong> SYETP treatment and comparison groups gives<br />
rise to concerns that SYETP participation as measured in <strong>the</strong> ALS data used may be<br />
affected by <strong>the</strong> data problems due to sample reduction. The former analysis <strong>of</strong> <strong>the</strong> overall<br />
effects <strong>of</strong> sample reduction also showed evidence <strong>of</strong> SYETP being affected, as well as<br />
potential regressor variables. The source <strong>of</strong> sample reduction is now controlled for in<br />
fur<strong>the</strong>r dissection <strong>of</strong> <strong>the</strong> univariate effects.
162<br />
Table 5.3 Summary statistics <strong>of</strong> attrition effects, for comparison and treated SYETP<br />
SYETP<br />
Non-SYETP<br />
Mean and<br />
SYETP s.d. SYETP s.d. Difference T –test Non- s.d. Non- s.d. Difference T –test<br />
standard deviation (s.d.) sample<br />
after<br />
attrition<br />
mean<br />
lost to<br />
attrition<br />
mean<br />
in means<br />
SYETP<br />
sample<br />
after<br />
attrition<br />
mean<br />
SYETP<br />
lost to<br />
attrition<br />
mean<br />
in means<br />
Panel 1 Panel 2<br />
(1) (2) (3) (4) (5) (6) (1) (2) (3) (4) (5) (6)<br />
Female 0.43 0.50 0.33 0.48 0.10 1.20 0.41 0.49 0.41 0.49 0<br />
Average age 1984 19.03 1.97 19.16 2.41 0.13 0.33 20.07 2.42 20.06 2.37 0.01 0.12<br />
Aboriginal/Torres Strait 0.01 0.10 0.00 0.00 0.01 0.43 0.03 0.17 0.04 0.20 0.01 1.04<br />
Islander<br />
O<strong>the</strong>r ethnic minority 0.08 0.27 0.06 0.24 0.02 0.81 0.08 0.27 0.09 0.29 0.01 1.17<br />
Married 1984 0.03 0.17 0.06 0.24 0.03 0.81 0.12 0.33 0.10 0.30 0.02 1.57<br />
Spouse employed 1984 0.02 0.14 0.00 0.00 0.02 1.42 0.06 0.24 0.04 0.20 0.02 2.48***<br />
Children 1984 0.02 0.14 0.06 0.24 0.04 1.10 0.06 0.23 0.04 0.20 0.02 1.63*<br />
Highest qualification in 1984<br />
Degree/diploma 0.08 0.27 0.06 0.24 0.02 0.43 0.12 0.33 0.11 0.31 0.01 1.09<br />
Apprenticeship 0.03 0.17 0.02 0.14 0.01 0.36 0.09 0.28 0.07 0.26 0.02 1.32<br />
O<strong>the</strong>r post-school qualification 0.07 0.25 0.04 0.20 0.03 0.76 0.07 0.26 0.07 0.26 0<br />
Year 12 <strong>of</strong> school 0.23 0.42 0.12 0.33 0.11 1.84* 0.14 0.35 0.17 0.38 0.03 1.95**<br />
Year 11 <strong>of</strong> school 0.17 0.38 0.14 0.35 0.03 0.58 0.14 0.34 0.11 0.31 0.03 1.90*<br />
Year 10 <strong>of</strong> school 0.32 0.47 0.45 0.50 0.13 1.59 0.31 0.46 0.29 0.45 0.02 1.20<br />
Year 9 <strong>of</strong> school 0.11 0.31 0.16 0.37 0.05 0.86 0.13 0.33 0.15 0.36 0.02 1.85*<br />
Parental background<br />
when resp. aged 14<br />
Fa<strong>the</strong>r postschool qualification 0.26 0.44 0.21 0.41 0.05 0.70 0.35 0.48 0.35 0.48 0<br />
Mo<strong>the</strong>r postschool qualification 0.20 0.40 0.18 0.39 0.02 0.34 0.18 0.39 0.20 0.40 0.02 0.95<br />
Fa<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.25 0.44 0.18 0.39 0.07 1.07 0.26 0.44 0.24 0.43 0.02 0.85<br />
para-pr<strong>of</strong>essional<br />
Fa<strong>the</strong>r not employed 0.04 0.19 0.04 0.20 0 0.06 0.23 0.05 0.22 0.01 0.63<br />
Fa<strong>the</strong>r not present 0.19 0.40 0.16 0.37 0.03 0.55 0.15 0.36 0.17 0.38 0.02 1.19<br />
Mo<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.07 0.25 0.06 0.24 0.02 0.20 0.10 0.30 0.09 0.28 0.01 0.98
163<br />
para-pr<strong>of</strong>essional<br />
Mo<strong>the</strong>r not employed 0.48 0.50 0.61 0.49 0.13 1.50 0.55 0.50 0.55 0.50 0<br />
Mo<strong>the</strong>r not present 0.09 0.28 0.06 0.24 0.03 0.64 0.05 0.22 0.06 0.24 0.01 1.11<br />
Longest job ever held by 1984<br />
Never held a job 0.12 0.32 0.08 0.27 0.04 0.75 0.12 0.32 0.11 0.32 0.01 0.30<br />
< 1 year 0.56 0.50 0.63 0.49 0.07 0.83 0.41 0.49 0.43 0.50 0.02 1.02<br />
1 year 0.13 0.34 0.18 0.39 0.04 0.66 0.13 0.34 0.17 0.37 0.04 2.12**<br />
2 years 0.13 0.34 0.06 0.24 0.08 1.60 0.14 0.35 0.12 0.33 0.02 1.45<br />
3 years or more 0.06 0.23 0.06 0.24 0 0.20 0.40 0.17 0.38 0.03 1.77*<br />
Average unemployment 103 0.68 0.37 0.80 0.32 0.12 2.08** 0.61 0.41 0.63 0.41 0.02 1.38<br />
Ever employed in 1986 104 0.87 0.34 Na Na Na Na 0.73 0.44 Na Na Na Na<br />
Ever government programme 0.14 0.35 Na Na Na Na 0.11 0.31 Na Na Na Na<br />
1986 105<br />
Work limited by health 1984 0.04 0.19 0.06 0.24 0.02 0.53 0.11 0.31 0.13 0.33 0.02 1.57<br />
Interviewed location in 1984<br />
New South Wales /ACT 0.31 0.46 0.29 0.46 0.02 0.17 0.38 0.49 0.42 0.49 0.04 1.81*<br />
Victoria 0.27 0.45 0.24 0.43 0.03 0.43 0.23 0.42 0.24 0.42 0<br />
Queensland 0.09 0.28 0.20 0.40 0.11 1.75* 0.14 0.35 0.15 0.35 0.003 0.17<br />
South Australia /Nor<strong>the</strong>rn 0.13 0.34 0.12 0.33 0.01 0.30 0.12 0.33 0.09 0.28 0.03 2.69**<br />
Territory<br />
Western Australia / Tasmania 0.20 0.40 0.16 0.37 0.04 0.69 0.12 0.33 0.11 0.32 0.01 0.53<br />
Capital city 0.58 0.50 0.55 0.50 0.03 0.33 0.46 0.50 0.52 0.50 0.06 2.73**<br />
O<strong>the</strong>r town 0.16 0.37 0.25 0.44 0.09 1.28 0.21 0.41 0.26 0.44 0.05 2.30**<br />
Country town 0.21 0.41 0.12 0.33 0.09 1.54 0.24 0.43 0.18 0.39 0.06 3.38***<br />
Rural area 0.05 0.21 0.08 0.27 0.03 0.70 0.08 0.27 0.04 0.20 0.04 4.10***<br />
Number <strong>of</strong> observations 104 51 1179 1034<br />
Significance <strong>of</strong> Student’s t test at level 1% ***, 5%**, 10 % *. T tests <strong>of</strong> whe<strong>the</strong>r means are equal are performed with <strong>the</strong> assumption <strong>of</strong> unequal variances, n-1<br />
degrees <strong>of</strong> freedom.<br />
103 This is <strong>the</strong> proportion <strong>of</strong> <strong>the</strong> 1984 reference period to 3 June that was spent unemployed.<br />
104 Ever held a non-subsidised, non-government programme job in <strong>the</strong> 1986 reference period, after <strong>the</strong> first 17 weeks.<br />
105 Ever go on a government programme, including SYETP, in <strong>the</strong> 1986 reference period.
164<br />
5.5.3.2 The effect <strong>of</strong> sample reduction on <strong>the</strong> difference between SYETP and comparison<br />
groups<br />
Overall, in <strong>the</strong> previous section it can be seen that <strong>the</strong> effect <strong>of</strong> <strong>the</strong> sample reduction is<br />
not equal on <strong>the</strong> treatment and <strong>the</strong> comparison groups’ pr<strong>of</strong>ile for <strong>the</strong>se characteristics<br />
examined. This is crucial as <strong>the</strong> aim <strong>of</strong> later modelling <strong>of</strong> <strong>the</strong> treatment effect is <strong>the</strong><br />
comparison <strong>of</strong> <strong>the</strong> treatment and comparison groups. The effect <strong>of</strong> sample reduction on<br />
<strong>the</strong> difference in pr<strong>of</strong>iles <strong>of</strong> <strong>the</strong> treatment and comparison groups is now explored. This<br />
provides a different perspective on how <strong>the</strong> data reduction impacts on <strong>the</strong> analysis.<br />
The contrast between <strong>the</strong> treatment and comparison group is summarised in <strong>the</strong> absolute<br />
mean difference that is shown in Table 5.4. The first column <strong>of</strong> Table 5.4 gives <strong>the</strong> presample-<br />
reduction differential while <strong>the</strong> second column is <strong>the</strong> post-sample-reduction<br />
statistic. Statistically significant differences between <strong>the</strong>se means are shown with a star<br />
indicating a one percent level <strong>of</strong> significance. Almost all <strong>the</strong> differences are statistically<br />
significant. However <strong>the</strong> differences between <strong>the</strong> treatment and comparison prior to<br />
sample reduction are not <strong>the</strong> same as those present after sample reduction.<br />
The lack <strong>of</strong> correspondence between <strong>the</strong> treatment and comparison group before sample<br />
reduction is quite great. Table 5.4 shows <strong>the</strong> pr<strong>of</strong>ile prior to sample reduction to be<br />
markedly different for <strong>the</strong> treatment and comparison groups for certain characteristics,<br />
and <strong>the</strong> scale <strong>of</strong> contrast can be illustrated by considering <strong>the</strong> figures underlying column 1<br />
<strong>of</strong> Table 5.4. Far fewer <strong>of</strong> those who enter <strong>the</strong> SYETP were married in 1984 (3.9 per cent)<br />
than amongst <strong>the</strong> comparison group (11.7 per cent). The educational pr<strong>of</strong>ile differs for<br />
those on SYETP compared to <strong>the</strong> comparison group. Fewer on SYETP have post-school<br />
qualifications (15.5 per cent) than in <strong>the</strong> comparison group (26.7 per cent), while more <strong>of</strong><br />
those on SYETP have schooling <strong>of</strong> year 10 (36.1 per cent), year 11 (16.1 per cent) and 12<br />
(19.4 per cent) than do those in <strong>the</strong> comparison group (at 29.9 per cent, 12.1 per cent and<br />
15.1 per cent respectively). The work history pr<strong>of</strong>ile <strong>of</strong> those who entered SYETP is also<br />
quite different to that <strong>of</strong> <strong>the</strong> comparison group. The share <strong>of</strong> those on SYETP with work<br />
experience <strong>of</strong> less than a year is 58.1 per cent, which contrasts strongly with <strong>the</strong>
165<br />
comparison group where <strong>the</strong> share is only 41.5 per cent. Those on SYETP spent more<br />
time prior to June 1984 unemployed (72.4 per cent) than did those in <strong>the</strong> comparison<br />
group (62 per cent).<br />
Comparison <strong>of</strong> columns1 and 2 <strong>of</strong> Table 5.4 shows <strong>the</strong> impact <strong>of</strong> <strong>the</strong> sample reduction on<br />
<strong>the</strong> difference between <strong>the</strong> treatment and comparison group characteristics. The effects <strong>of</strong><br />
<strong>the</strong> sample reduction lead some characteristics <strong>of</strong> <strong>the</strong> treated and comparison pr<strong>of</strong>iles to<br />
diverge fur<strong>the</strong>r, while <strong>the</strong> effect <strong>of</strong> <strong>the</strong> sample reduction on o<strong>the</strong>r features make <strong>the</strong><br />
treated and comparison groups more similar in pr<strong>of</strong>ile. The change in gender composition<br />
<strong>of</strong> <strong>the</strong> SYETP group results in raising <strong>the</strong> difference between <strong>the</strong> treated and <strong>the</strong><br />
comparison gender composition. Raising <strong>the</strong> share <strong>of</strong> those with year 12 in <strong>the</strong> SYETP<br />
leads to a marked increase in <strong>the</strong> difference between <strong>the</strong> treated and comparison for this<br />
aspect. The reduction in <strong>the</strong> proportion whose mo<strong>the</strong>r was not employed amongst <strong>the</strong><br />
SYETP, also leads to a heightened difference between <strong>the</strong> treated and <strong>the</strong> control for this<br />
feature. On <strong>the</strong> o<strong>the</strong>r hand, <strong>the</strong> fall in <strong>the</strong> share <strong>of</strong> those with year 10 in <strong>the</strong> SYETP results<br />
in <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> treated and <strong>the</strong> comparison groups being very similar for this. For<br />
some o<strong>the</strong>r traits, <strong>the</strong> overall effect is a reduction in <strong>the</strong> divergence between <strong>the</strong> treated<br />
and comparison, <strong>the</strong>se include <strong>the</strong> proportion that held a job <strong>of</strong> less than one year, <strong>the</strong><br />
average proportion <strong>of</strong> time spent in unemployment prior to June 1984, <strong>the</strong> share observed<br />
with employment in 1986 and <strong>the</strong> share observed in a government programme in 1986.
166<br />
Table 5.4 Difference between treatment group and comparison group before and after<br />
sample reduction<br />
SYETP vs. comparison<br />
Before<br />
Sample reduction Absolute<br />
difference in means<br />
Female 0.8 2.3***<br />
Average age 1984 1.0*** 1.1***<br />
Aboriginal/Torres Strait Islander 2.5*** 2.1***<br />
O<strong>the</strong>r ethnic minority 1.6*** 0.2<br />
Married 1984 7.9*** 9.6***<br />
Spouse employed 1984 4.0*** 4.4***<br />
Children 1984 1.9*** 3.9***<br />
Highest qualification in 1984<br />
Degree/diploma 4.4*** 4.5***<br />
Apprenticeship 5.4*** 5.7***<br />
O<strong>the</strong>r post-school qualification 1.4*** 0.4<br />
Year 12 <strong>of</strong> school 4.3*** 9.2***<br />
Year 11 <strong>of</strong> school 4.0*** 3.7***<br />
Year 10 <strong>of</strong> school 6.2*** 0.2<br />
Year 9 <strong>of</strong> school 1.5*** 2.0***<br />
Parental background<br />
when resp. aged 14<br />
Fa<strong>the</strong>r postschool qualification 10.0*** 8.6***<br />
Mo<strong>the</strong>r postschool qualification 0.9 2.0***<br />
Fa<strong>the</strong>r manager, pr<strong>of</strong>essional, para-pr<strong>of</strong>essional 2.1 0.8<br />
Fa<strong>the</strong>r not employed 1.3*** 1.8***<br />
Fa<strong>the</strong>r not present 2.0*** 3.8***<br />
Mo<strong>the</strong>r manager, pr<strong>of</strong>essional, para-pr<strong>of</strong>essional 2.6 3.1***<br />
Mo<strong>the</strong>r not employed 2.1*** 7.2***<br />
Mo<strong>the</strong>r not present 2.3*** 3.7***<br />
Longest job ever held by 1984<br />
Never held a job 1.0 0.1<br />
< 1 year 16.6*** 15.7***<br />
1 year 0.2 0<br />
2 years 2.3*** 0.6<br />
3 years or more 13.2*** 14***<br />
Average unemployment 107 10.4*** 7.5***<br />
Ever employed in 1986 108 15.8*** 13.6***<br />
Ever government programme 1986 109 4.8*** 3.7***<br />
SYETP vs. comparison After<br />
Sample reduction Absolute<br />
difference in means 106<br />
106 The statistic for any variable is <strong>the</strong> absolute value <strong>of</strong> <strong>the</strong> difference in means for <strong>the</strong> treatment and<br />
control groups. *** indicates <strong>the</strong> T statistic, for <strong>the</strong> hypo<strong>the</strong>sis that <strong>the</strong> difference in mean in SYETP and<br />
<strong>the</strong> control group is zero, is significant at <strong>the</strong> 1 percent level <strong>of</strong> significance.<br />
107 Proportion <strong>of</strong> 1984 reference period to 3 June spent unemployed.<br />
108 Ever held a non-subsidised, non-government-program job in <strong>the</strong> 1986 reference period, after <strong>the</strong> first 17<br />
weeks.<br />
109 Ever go on a government programme, including SYETP, in <strong>the</strong> 1986 reference period.
167<br />
5.5.4 Attrition: natural attrition or analytical sample reduction?<br />
Table 5.5 draws a distinction between <strong>the</strong> sources <strong>of</strong> sample reduction. Sample reduction<br />
resulting from attrition, where <strong>the</strong> respondent did not take part in later surveys, is here<br />
termed natural attrition. As outlined earlier, fur<strong>the</strong>r sample reduction through analytical<br />
selection also takes place before <strong>the</strong> final sample used for analysis is reached. Zabel<br />
(1998) points out such analytical selection can in <strong>the</strong>ory introduce biases <strong>of</strong> a similar<br />
nature to attrition bias.<br />
Indeed, Table 5.5 shows that both forms <strong>of</strong> sample reduction result in statistically<br />
significant differences to <strong>the</strong> means observed <strong>of</strong> those remaining in <strong>the</strong> final sample. This<br />
hints that both sources <strong>of</strong> sample loss may be suspected <strong>of</strong> altering <strong>the</strong> sample<br />
characteristics. Also, <strong>the</strong> pattern <strong>of</strong> differences varies by <strong>the</strong> form <strong>of</strong> sample reduction<br />
being natural attrition or analytical selection. This signals <strong>the</strong> added value <strong>of</strong> treating each<br />
source separately in order to allow for differential impacts.<br />
The first two columns <strong>of</strong> Table 5.5 show <strong>the</strong> mean and standard deviation in <strong>the</strong> final<br />
sample, to which <strong>the</strong> two sources <strong>of</strong> sample reduction are compared. The next four<br />
columns make up <strong>the</strong> first panel <strong>of</strong> <strong>the</strong> table which relates to <strong>the</strong> sample lost due to<br />
natural attrition, and <strong>the</strong>y show <strong>the</strong> mean, standard deviation, absolute difference in <strong>the</strong><br />
means and <strong>the</strong> t statistic, in a similar fashion to <strong>the</strong> former tables. The second panel gives<br />
<strong>the</strong> statistics for those lost though analytical selection. Only differences that are<br />
statistically significant at normally adopted confidence levels are discussed.<br />
The outcome variables ‘ever employed in 1986’ and ‘ever in a government programme in<br />
1986’ are not observable for those lost through natural attrition. It is useful to compare<br />
<strong>the</strong> statistically significant differences in <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> those remaining in <strong>the</strong> sample to<br />
those lost to natural attrition. Of those not interviewed again: fewer had taken part in<br />
SYETP, more had year nine <strong>of</strong> school but less had year 11, more had held jobs <strong>of</strong> 1-2<br />
years while less had held jobs <strong>of</strong> 2-3 years, <strong>the</strong> proportion <strong>of</strong> unemployment in 1984 had
168<br />
been higher, and <strong>the</strong> location <strong>of</strong> <strong>the</strong> interview differed to those remaining with fewer<br />
from South Australia and <strong>the</strong> Nor<strong>the</strong>rn Territory, Western Australia and Tasmania, more<br />
from cities and fewer from country towns and rural areas.<br />
The group <strong>of</strong> observations lost due to analytical selection had different features from both<br />
those remaining and from those lost by natural attrition. For those lost for analytical<br />
reasons, <strong>the</strong> means <strong>of</strong> more characteristics were affected than for those lost through<br />
natural attrition. Compared to those remaining, fewer <strong>of</strong> those lost were married, had<br />
employed spouses, apprenticeships, year 10 <strong>of</strong> school, mo<strong>the</strong>rs in <strong>the</strong> managerial<br />
occupations, or had held a job for 3 years or more, while on <strong>the</strong> o<strong>the</strong>r hand more had<br />
work limiting health problems and year 12 <strong>of</strong> school. The key outcome variable,<br />
employment in 1986, was significantly lower for those lost by analytical selection.<br />
The effects <strong>of</strong> analytical selection reinforced those <strong>of</strong> natural attrition for three variables,<br />
SYETP treatment, South Australia and <strong>the</strong> Nor<strong>the</strong>rn Territory and rural areas. For each <strong>of</strong><br />
<strong>the</strong>se aspects, fewer cases were observed for those lost by natural attrition. The process <strong>of</strong><br />
analytical selection worked in <strong>the</strong> same direction as that <strong>of</strong> natural attrition, and so <strong>of</strong><br />
those lost, fewer had taken part in SYETP, fewer were from South Australia and <strong>the</strong><br />
Nor<strong>the</strong>rn Territory and fewer were from rural areas.<br />
There is a weak suggestion that among those lost due to natural attrition, <strong>the</strong> average<br />
proportion <strong>of</strong> unemployment in 1984 is lower, and <strong>of</strong> those dropped by analytical<br />
selection fewer were observed in employment in 1986. However, it is considered that this<br />
is not sufficient evidence to support <strong>the</strong> hypo<strong>the</strong>sis that key subgroups are<br />
disproportionately affected in terms <strong>of</strong> <strong>the</strong> dependent variable labour market status. This<br />
question is readdressed in <strong>the</strong> multivariate analysis that follows.<br />
Drawing toge<strong>the</strong>r <strong>the</strong> findings <strong>of</strong> <strong>the</strong> univariate analysis <strong>of</strong> sample reduction, it can be<br />
said that pr<strong>of</strong>iles <strong>of</strong> some important regressor variables, and <strong>the</strong> key treatment variable<br />
SYETP, show evidence <strong>of</strong> being statistically different after sample reduction. Segregation<br />
by <strong>the</strong> two types <strong>of</strong> sample reduction, shows natural attrition and analytical selection are
169<br />
both sources that impose changes in <strong>the</strong> pr<strong>of</strong>iles, and <strong>the</strong>y have differing impacts.<br />
However, univariate analysis cannot account for <strong>the</strong> correlations between <strong>the</strong>se variables<br />
and so <strong>the</strong> t statistics do not represent independent effects. As such, it is maintained that<br />
<strong>the</strong>re is as yet no strong indication <strong>of</strong> serious problems arising from sample reduction.<br />
Fur<strong>the</strong>r tests accounting for effects <strong>of</strong> sample reduction in a multivariate context are now<br />
applied. As <strong>the</strong> pattern <strong>of</strong> differences varies by <strong>the</strong> form <strong>of</strong> sample reduction being<br />
natural attrition or analytical selection, it seems that it will be important to maintain <strong>the</strong><br />
breakdown <strong>of</strong> sample reduction by source in fur<strong>the</strong>r examinations. Accordingly, <strong>the</strong><br />
modelling <strong>of</strong> sample reduction, and any weights arising, takes separate account <strong>of</strong> natural<br />
attrition and analytical selection.
170<br />
Table 5.5 Summary statistics for attrition effects by source <strong>of</strong> sample loss<br />
Mean and<br />
standard deviation (s.d.)<br />
sample<br />
after<br />
attrition<br />
mean 110<br />
(1)<br />
s.d.<br />
(2)<br />
Those<br />
lost<br />
through<br />
natural<br />
attrition<br />
mean<br />
s.d.<br />
Difference<br />
in means 111<br />
(absolute)<br />
T –test<br />
Those<br />
lost<br />
through<br />
analytical<br />
selection<br />
mean<br />
s.d.<br />
Differenc<br />
e in<br />
means<br />
(absolute)<br />
syetp 0.08 0.27 0.05 0.22 0.03 2.91** 0.04 0.21 0.04 2.85**<br />
Female 0.41 0.49 0.41 0.49 0 0.40 0.49 0.01 0.49<br />
Average age 1984 19.99 2.41 20.13 2.39 0.16 1.30 19.82 2.36 0.17 1.25<br />
Aboriginal/Torres Strait 0.03 0.17 0.04 0.19 0.01 0.82 0.04 0.20 0.01 0.92<br />
Islander<br />
O<strong>the</strong>r ethnic minority 0.08 0.27 0.09 0.29 0.01 1.03 0.09 0.29 0.01 0.65<br />
Married 1984 0.12 0.32 0.12 0.33 0 0.07 0.25 0.05 3.27***<br />
Spouse employed 1984 0.06 0.24 0.05 0.21 0.01 1.02 0.02 0.14 0.04 4.12***<br />
Children 1984 0.05 0.23 0.04 0.21 0.01 1.03 0.04 0.20 0.01 1.05<br />
Highest qualification in 1984<br />
Degree/diploma 0.12 0.32 0.11 0.32 0.01 0.45 0.09 0.29 0.03 1.43<br />
Apprenticeship 0.08 0.27 0.07 0.26 0.01 0.49 0.06 0.23 0.02 1.74*<br />
O<strong>the</strong>r post-school<br />
0.07 0.26 0.06 0.25 0.01 0.53 0.08 0.27 0.01 0.70<br />
qualification<br />
Year 12 <strong>of</strong> school 0.15 0.35 0.14 0.34 0.01 0.70 0.22 0.41 0.07 3.22***<br />
Year 11 <strong>of</strong> school 0.14 0.35 0.11 0.31 0.03 1.86* 0.11 0.31 0.03 1.49<br />
Year 10 <strong>of</strong> school 0.31 0.46 0.32 0.47 0.01 0.04 0.27 0.44 0.04 1.76*<br />
Year 9 <strong>of</strong> school 0.12 0.33 0.18 0.38 0.06 2.98*** 0.12 0.32 0<br />
Parental background<br />
when resp. aged 14<br />
Fa<strong>the</strong>r postschool<br />
qualification<br />
0.34 0.47 0.34 0.47 0 0.36 0.48 0.02 0.67<br />
Mo<strong>the</strong>r postschool<br />
0.18 0.39 0.21 0.41 0.03 1.26 0.18 0.38 0<br />
qualification<br />
Fa<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.26 0.44 0.24 0.43 0.02 1.01 0.24 0.43 0.02 0.57<br />
T –test<br />
110 The mean <strong>of</strong> a 0/1 binary variable is <strong>the</strong> same as <strong>the</strong> proportion.<br />
111 The difference is calculated between <strong>the</strong> mean <strong>of</strong> those lost from <strong>the</strong> sample and <strong>the</strong> mean <strong>of</strong> those remaining after attrition.
171<br />
para-pr<strong>of</strong>essional<br />
Fa<strong>the</strong>r not employed 0.05 0.23 0.05 0.22 0 0.05 0.22 0<br />
Fa<strong>the</strong>r not present 0.16 0.36 0.18 0.38 0.02 1.13 0.16 0.37 0<br />
Mo<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.10 0.29 0.10 0.30 0 0.06 0.24 0.04 2.14**<br />
para-pr<strong>of</strong>essional<br />
Mo<strong>the</strong>r not employed 0.55 0.50 0.55 0.50 0 0.56 0.50 0.01 0.43<br />
Mo<strong>the</strong>r not present 0.05 0.22 0.07 0.25 0.02 1.39 0.05 0.21 0<br />
Longest job ever held by 1984<br />
Never held a job 0.12 0.32 0.10 0.30 0.02 1.22 0.13 0.34 0.01 0.79<br />
< 1 year 0.42 0.49 0.43 0.50 0.01 0.46 0.45 0.50 0.03 1.13<br />
1 year 0.13 0.34 0.19 0.39 0.06 2.99** 0.13 0.34 0<br />
2 years 0.14 0.35 0.11 0.31 0.03 2.21** 0.14 0.34 0<br />
3 years or more 0.19 0.39 0.17 0.38 0.02 0.68 0.15 0.35 0.04 1.98**<br />
Average unemployment 112 0.62 0.41 0.66 0.40 0.04 2.36** 0.61 0.42 0.01 0.29<br />
Ever employed in 1986 113 0.74 0.44 Na Na Na Na 0.60 0.49 0.14 5.18***<br />
Ever government programme 0.11 0.31 Na Na Na Na 0.10 0.31 0.01 0.73<br />
1986 114<br />
Work limited by health 1984 0.10 0.30 0.12 0.32 0.02 1.28 0.13 0.34 0.03 1.67*<br />
Interviewed location in 1984<br />
New South Wales /ACT 0.37 0.48 0.41 0.49 0.04 1.72* 0.41 0.49 0.03 1.22<br />
Victoria 0.24 0.43 0.24 0.43 0 0.23 0.42 0.01 0.28<br />
Queensland 0.14 0.34 0.16 0.37 0.02 1.46 0.12 0.33 0.02 0.88<br />
South Australia /Nor<strong>the</strong>rn 0.12 0.33 0.09 0.29 0.03 2.29** 0.09 0.28 0.03 2.22**<br />
Territory<br />
Western Australia / Tasmania 0.13 0.33 0.09 0.29 0.04 2.38*** 0.15 0.36 0.02 1.31<br />
Capital city 0.47 0.50 0.54 0.50 0.07 2.78*** 0.50 0.50 0.03 0.82<br />
O<strong>the</strong>r town 0.21 0.41 0.27 0.44 0.06 2.82** 0.24 0.42 0.03 1.06<br />
Country town 0.24 0.43 0.16 0.37 0.08 4.47*** 0.22 0.41 0.02 0.86<br />
Rural area 0.08 0.27 0.04 0.19 0.04 3.97*** 0.05 0.22 0.03 2.16**<br />
Number <strong>of</strong> observations 1283 681 404<br />
Significance <strong>of</strong> t test at level 1% ***, 5%**, 10 % *. Student’s t tests <strong>of</strong> whe<strong>the</strong>r means are equal<br />
112 The proportion <strong>of</strong> 1984 reference period to 3 June spent unemployed.<br />
113 Ever held a non-subsidised, non-government programme job in <strong>the</strong> 1986 reference period, after <strong>the</strong> first 17 weeks.<br />
114 Ever go on a government programme, including SYETP, in <strong>the</strong> 1986 reference period.
172<br />
5.6 Accounting for non-response and sample design<br />
In <strong>the</strong> analysis so far, only sample reduction subsequent to <strong>the</strong> 1984 survey has been<br />
considered. It is however pertinent to ensure that <strong>the</strong> survey design and initial nonresponse<br />
in <strong>the</strong> 1984 survey is also accounted for. As <strong>the</strong> application <strong>of</strong> weights is a key<br />
factor in <strong>the</strong> subsequent analysis, <strong>the</strong> survey weight is introduced here. Although <strong>the</strong>re is<br />
some disagreement about <strong>the</strong> quality and general applicability <strong>of</strong> survey weights, it is<br />
now generally accepted in <strong>the</strong> statistical community that <strong>the</strong> sampling design and nonresponse<br />
for a survey should be accounted for in order to obtain estimates that reference<br />
<strong>the</strong> population. As <strong>the</strong> ALS was not a simple random sample 115 , <strong>the</strong>n it is important that<br />
<strong>the</strong> survey weights are applied.<br />
There is a selection/response weight provided with <strong>the</strong> data 116 , which weights <strong>the</strong> 1984<br />
data back <strong>the</strong> population: persons aged 15-24 on 1 September 1984, registered with <strong>the</strong><br />
Commonwealth Employment Service <strong>of</strong>fices throughout Australia who were unemployed<br />
and seeking full-time work for three months or more at <strong>the</strong> time <strong>of</strong> selection, 30 June<br />
1984 117 . To construct <strong>the</strong> weight for non-response and selection provided with <strong>the</strong> data,<br />
administrative data was used to weight back to <strong>the</strong> population. The weight is complex,<br />
adjusting for unequal probability <strong>of</strong> selection, non-response using administrative data on<br />
specific categories, and benchmarking adjustment to known totals 118 to reduce <strong>the</strong><br />
standard errors and produce estimates comparable with o<strong>the</strong>r sources. Foreshadowing<br />
115 It was a stratified cluster sample. Those interested in <strong>the</strong> sample design details should refer to<br />
Kronenburg et al. (1985). Non-computerisation led to a number <strong>of</strong> issues in <strong>the</strong> ‘list’ sample which affected<br />
<strong>the</strong> sample design, as <strong>the</strong> records that made up <strong>the</strong> sample frame were in fact paper card records stored at<br />
each CES <strong>of</strong>fice. The key problem was double-counting and multiple registrations at several CES.<br />
Stratification was also not always straightforward – for <strong>the</strong> Nor<strong>the</strong>rn Territory, in Alice Springs almost all<br />
<strong>the</strong> unemployed youths were Aboriginal (716 <strong>of</strong> 886), a much different proportion to all o<strong>the</strong>r locations and<br />
so only those in Darwin were chosen to represent <strong>the</strong> Nor<strong>the</strong>rn Territory (Kronenburg et al. (1985): 7).<br />
116 The variable in 1984 is named ‘Weight1’.<br />
117 In fact, <strong>the</strong> sample was <strong>of</strong> 2 stages – (1) selection <strong>of</strong> CES <strong>of</strong>fices based on <strong>the</strong> persons aged 15-24 on 1<br />
September 1984, registered with <strong>the</strong> CES <strong>of</strong>fices throughout Australia who were unemployed and seeking<br />
full-time work for three months or more at 31 March. (2) samples <strong>of</strong> 50 persons [persons aged 15-24 on 1<br />
September 1984, registered with <strong>the</strong> CES <strong>of</strong>fices throughout Australia who were unemployed and seeking<br />
full-time work for three months or more] selected from each <strong>of</strong>fice on 30 June 1984. The March and June<br />
numbers in scope <strong>of</strong> <strong>the</strong> survey were different, leading to differing selection probabilities. (Mcrae et al.<br />
(1985): 23).<br />
118 The benchmark data came from CES population counts for each age*sex*duration <strong>of</strong> registration<br />
category within <strong>the</strong> geographic selection strata (Mcrae et al. (1985: 24).
173<br />
work in later sections, this supplied weight does not take account <strong>of</strong> fur<strong>the</strong>r non-response<br />
beyond <strong>the</strong> 1984 survey, and so additional weights will be developed to adjust for <strong>the</strong><br />
non-response to <strong>the</strong> 1985 and 1986 surveys.<br />
Mcrae et al. (1985) examine <strong>the</strong> importance <strong>of</strong> each <strong>of</strong> <strong>the</strong> available variables 119 to <strong>the</strong><br />
response rate in 1984. In univariate analysis <strong>of</strong> response rates, <strong>the</strong>y found that 15-19 year<br />
olds were more likely to have been interviewed in 1984. Amongst this age group females<br />
had a higher response than males, but for 20-24 year olds <strong>the</strong>re was no gender difference<br />
in response. The response rate was higher for those who had been unemployed longer.<br />
Movers had a much lower response rate than non-movers, because <strong>the</strong>y could be located<br />
to interview less <strong>of</strong>ten. Certain states, Queensland and <strong>the</strong> Nor<strong>the</strong>rn Territory, had lower<br />
response rates. Logistic regression analysis fur<strong>the</strong>r examined <strong>the</strong> response, removing <strong>the</strong><br />
correlation between <strong>the</strong>se variables and showing <strong>the</strong> direct association with <strong>the</strong> response<br />
rate. This analysis identified mobility to be <strong>the</strong> chief variable related to non-response, but<br />
age, gender, duration <strong>of</strong> registration and state <strong>of</strong> registration had independent effects as<br />
well. Fur<strong>the</strong>r analysis found that <strong>the</strong> strong differential response effect for movers and<br />
non-movers affected labour force variables measured in <strong>the</strong> 1984 survey. A final analysis<br />
compared ALS estimates to corresponding published data from <strong>the</strong> <strong>Australian</strong> Bureau <strong>of</strong><br />
Statistics, which gave similar results or showed readily explainable differences. These<br />
results were interpreted as giving fur<strong>the</strong>r confidence to <strong>the</strong> validity <strong>of</strong> ALS estimates.<br />
Comparison <strong>of</strong> <strong>the</strong> adjusted and unadjusted pr<strong>of</strong>iles gives an indication <strong>of</strong> <strong>the</strong> specific<br />
features that are most strongly affected by survey design, selection and non-response.<br />
Table 5.6 shows <strong>the</strong> effect on <strong>the</strong> data pr<strong>of</strong>ile <strong>of</strong> adjusting for selection and response for<br />
<strong>the</strong> 1984 survey respondents. In <strong>the</strong> first column is shown <strong>the</strong> unadjusted pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong><br />
data for certain key characteristics, while <strong>the</strong> second column shows how <strong>the</strong> weighted<br />
sample pr<strong>of</strong>ile appears. It can be seen that <strong>the</strong>re is only a very slight effect on <strong>the</strong> pr<strong>of</strong>ile.<br />
119 Mcrae et al. (1985) p10: CES records held information on <strong>the</strong> age group (15-19 or 20-24), gender,<br />
duration <strong>of</strong> unemployment (measured by <strong>the</strong> time from <strong>the</strong> date <strong>of</strong> registration with <strong>the</strong> Commonwealth<br />
Employment Service [CES] to <strong>the</strong> date <strong>of</strong> sample selection, 30 June 1984; categories <strong>of</strong> less than 3 months,<br />
between 3-9 months, more than 9 months CES registration), State or Territory <strong>of</strong> CES registration, country<br />
<strong>of</strong> birth (Australia or overseas), marital status (married/defacto or unmarried), whe<strong>the</strong>r <strong>the</strong> address <strong>the</strong><br />
person lived at when interviewed was <strong>the</strong> same as <strong>the</strong> address <strong>of</strong> CES registration.
174<br />
The greatest effect <strong>of</strong> <strong>the</strong> weight on <strong>the</strong> pr<strong>of</strong>ile is for those interviewed in Queensland in<br />
1984. It seems that slightly lower response or selection was associated with individuals in<br />
Queensland.
175<br />
Table 5.6: Effect <strong>of</strong> selection/response weight on 1984 survey respondents<br />
Mean 1984 unadjusted pr<strong>of</strong>ile 1984 weighted to sampled<br />
population pr<strong>of</strong>ile<br />
Age at 1984 survey 20.00 19.88<br />
Gender=female 0.41 0.39<br />
Work limited by health 1984 0.11 0.11<br />
Aboriginal/Torres Strait Islander 0.03 0.04<br />
O<strong>the</strong>r ethnic minority 0.08 0.08<br />
Married 1984 0.11 0.11<br />
State interviewed in 1984<br />
Victoria 0.24 0.24<br />
Queensland 0.14 0.16<br />
South Australia/Nor<strong>the</strong>rn Territory 0.11 0.10<br />
Western Australia/Tasmania 0.12 0.11<br />
New South Wales/<strong>Australian</strong> Capital Territory 0.39 0.39<br />
Area <strong>of</strong> residence in 1984<br />
Capital city 0.49 0.49<br />
O<strong>the</strong>r city 0.23 0.23<br />
Country town 0.21 0.22<br />
Rural area 0.06 0.06<br />
Highest qualification in 1984<br />
Degree/diploma 0.11 0.11<br />
Apprenticeship 0.08 0.07<br />
O<strong>the</strong>r post-school qualification 0.07 0.07<br />
Year 12 <strong>of</strong> school 0.16 0.15<br />
Year 11 <strong>of</strong> school 0.13 0.13<br />
Year 10 <strong>of</strong> school 0.31 0.31<br />
Year 9 <strong>of</strong> school 0.14 0.14<br />
Average recorded Pre-June 1984 unemployment 120 0.63 0.62<br />
Observations 2368 2368<br />
NOTE 1: for a 0-1 indicator variable, <strong>the</strong> mean gives <strong>the</strong> proportion. For example, <strong>the</strong> mean in <strong>the</strong><br />
population, column 1, for gender=female is <strong>the</strong> proportion <strong>of</strong> females in <strong>the</strong> population.<br />
120 Average recorded Pre-June 1984 unemployment is missing for 27 observations, and has a base <strong>of</strong> 2341.
176<br />
5.6.1 Survey design and non-response effects on modelling<br />
Mcrae et al. (1985) found that <strong>the</strong> strong differential response effect for movers and nonmovers<br />
affected labour force variables measured in <strong>the</strong> 1984 survey. They pointed out<br />
that models and estimates <strong>of</strong> such variables might be biased by <strong>the</strong> non-response, unless<br />
<strong>the</strong> weights were applied. Movers, who had much lower chance <strong>of</strong> interview, were more<br />
likely than non-movers to be older than 20, married and in employment 121 , and had<br />
briefer CES registration. The definition <strong>of</strong> <strong>the</strong> SYETP treatment group relies on 1984<br />
survey data. The eligibility criteria for SYETP, and <strong>the</strong> o<strong>the</strong>r evidence 122 that <strong>the</strong>y were<br />
more likely to be teenagers, hints that SYETP participants would more commonly have<br />
been non-movers, and have had a higher response rate. In light <strong>of</strong> this, <strong>the</strong> SYETP<br />
participation model is first examined, by exploring <strong>the</strong> effects <strong>of</strong> <strong>the</strong> survey weights. The<br />
aim is to study <strong>the</strong> influence <strong>of</strong> non-response, and so check whe<strong>the</strong>r <strong>the</strong> mobility, age,<br />
gender, duration <strong>of</strong> registration and state <strong>of</strong> registration were related to <strong>the</strong> variable <strong>of</strong><br />
interest here, SYETP participation.<br />
5.6.1.1 Analytical selection<br />
Before continuing, <strong>the</strong> analytical selection from <strong>the</strong> full data is examined. Recall <strong>the</strong><br />
limits <strong>of</strong> <strong>the</strong> sample: <strong>the</strong> exclusion from those interviewed in 1984 <strong>of</strong> those who were<br />
over 25 years at <strong>the</strong> interview date, those in full-time education, or for whom responses<br />
were missing in 1984, 1985 or 1986 (Richardson (1998): 5). For <strong>the</strong> purposes <strong>of</strong><br />
evaluation analysis <strong>of</strong> SYETP, only those who were aged less than 25 and not in fulltime<br />
education in <strong>the</strong> 1984 survey interview can be <strong>of</strong> interest due to <strong>the</strong> eligibility restrictions<br />
for SYETP.<br />
These are discarded at different stages <strong>of</strong> <strong>the</strong> analysis however. Age was observed in <strong>the</strong><br />
administrative data, and is accounted for in <strong>the</strong> non-response weight and so it is<br />
121 Movers were also less likely to be unemployed, but had <strong>the</strong> same proportions not in <strong>the</strong> labour force as<br />
non-movers.<br />
122 See Chapter 2, for example <strong>the</strong> results <strong>of</strong> Hoy and Lampe (1982) covered in Section 2.2.6.2.
177<br />
appropriate to discard <strong>the</strong>se from <strong>the</strong> analysis prior to examining attrition. As a result,<br />
cases not meeting <strong>the</strong>se age restrictions can be excluded from analysis <strong>of</strong> attrition. In all<br />
35 cases are affected by this restriction. Due to <strong>the</strong> very short amount <strong>of</strong> time between <strong>the</strong><br />
age at sample selection date and <strong>the</strong> interviews, this is unlikely to be very different to <strong>the</strong><br />
population. 123 However, those ‘not in fulltime education in <strong>the</strong> 1984 survey’ can only be<br />
identified after <strong>the</strong> non-response stage, and so <strong>the</strong>y must be left in for <strong>the</strong> analysis <strong>of</strong><br />
attrition from 1984 which is used to construct weights. In this way, <strong>the</strong> representation <strong>of</strong><br />
<strong>the</strong> 1984 survey is maintained in modelling attrition to 1986. Those in fulltime education<br />
in <strong>the</strong> 1984 survey must be discarded in <strong>the</strong> Heckman bivariate probit and PSM<br />
modelling <strong>of</strong> <strong>the</strong> treatment effects <strong>of</strong> SYETP. If <strong>the</strong>y are not discarded in <strong>the</strong> Heckman<br />
bivariate probit and PSM modelling <strong>of</strong> <strong>the</strong> treatment effects <strong>of</strong> SYETP <strong>the</strong> data for <strong>the</strong><br />
treatment and comparison do not represent <strong>the</strong> eligible set. 124 This is because <strong>the</strong><br />
evaluation is attempting to identify treatment on <strong>the</strong> treated by constructing a comparison<br />
group from those eligible but not treated.<br />
As can be seen in <strong>the</strong> Table 5.7 below, 287 cases are dropped because <strong>of</strong> <strong>the</strong> analytical<br />
restriction excluding cases in fulltime education, and 1400 <strong>of</strong> those who responded to all<br />
waves to 1986 remain, after which ano<strong>the</strong>r 117 cases are dropped because <strong>the</strong>y have<br />
missing information.<br />
Table 5.7 How <strong>the</strong> observations reduce to 1283 from those who respond to <strong>the</strong> 1986<br />
survey<br />
Not missing information Missing information<br />
Selection <strong>of</strong> cases 0 1 Total<br />
Not selected: in fulltime<br />
education in 1984<br />
Selected: not in fulltime<br />
education 1984<br />
264 23 287<br />
1283 117 1400<br />
Total 1547 140 1687 125<br />
123 Age at 1 September 1984 was used for selection, while interviews were conducted between September<br />
1984 and November 1984. The full sample falls from 2403 to 2368.<br />
124 For an example <strong>of</strong> selecting <strong>the</strong> eligible set see Frölich et al. (2000) p55.<br />
125 Note: remember 35 have been dropped who were older than 24 years at <strong>the</strong> 1984 survey.
178<br />
Table 5.5a draws a distinction between <strong>the</strong> sources <strong>of</strong> sample reduction amongst those<br />
dropped for <strong>the</strong> analysis –<strong>the</strong> 140 cases dropped for having missing information on<br />
variables used in analysis and <strong>the</strong> 264 cases dropped for o<strong>the</strong>r analytical reasons. 126 Table<br />
5.5a shows <strong>the</strong> means, standard deviation, absolute difference and t test <strong>of</strong> <strong>the</strong><br />
significance <strong>of</strong> <strong>the</strong> difference from zero, using <strong>the</strong> same layout as for Table 5.5. The first<br />
two columns <strong>of</strong> Table 5.5a show <strong>the</strong> mean and standard deviation in <strong>the</strong> final sample, to<br />
which <strong>the</strong> two sources <strong>of</strong> sample reduction are compared in <strong>the</strong> t-test <strong>of</strong> <strong>the</strong> significance<br />
<strong>of</strong> <strong>the</strong> difference from zero. The next four columns make up <strong>the</strong> first panel <strong>of</strong> <strong>the</strong> table<br />
which relates to <strong>the</strong> sample lost due to missing information, and <strong>the</strong>y show <strong>the</strong> mean,<br />
standard deviation, absolute difference in <strong>the</strong> means and <strong>the</strong> t statistic, in a similar fashion<br />
to <strong>the</strong> former tables. The second panel gives <strong>the</strong> statistics for those lost though fur<strong>the</strong>r<br />
analytical selection. Only differences that are statistically significant at normally adopted<br />
confidence levels are discussed.<br />
Amongst those cases dropped because certain variables used in analysis are missing<br />
information, it is first noticeable that case-wise deletion in this manner leads to <strong>the</strong> loss <strong>of</strong><br />
information from <strong>the</strong> whole case. For example ‘fa<strong>the</strong>r post-school qualification’ is<br />
present for 79 <strong>of</strong> <strong>the</strong> 140 cases dropped, but while <strong>the</strong> missing information for ‘fa<strong>the</strong>r<br />
post-school qualification’ was <strong>the</strong> reason for <strong>the</strong> 61 cases being dropped, <strong>the</strong>se extra 79<br />
cases are dropped due to o<strong>the</strong>r variables having missing information. It is <strong>the</strong>n very much<br />
an ‘all or nothing strategy’, where more information is lost than is actually missing.<br />
Column 5 <strong>of</strong> Table 5.5a indicates <strong>the</strong> number <strong>of</strong> cases that were missing information, at<br />
most 140. It can be seen from Table 5.5a that amongst those where all 140 cases had<br />
missing information for <strong>the</strong> variable in question, that for some <strong>of</strong> <strong>the</strong> variables <strong>the</strong><br />
missing had a different pr<strong>of</strong>ile to those remaining in <strong>the</strong> final sample. As a result, those<br />
dropped for case-wise deletion to treat missing information had undertaken less SYETP<br />
treatment, more <strong>of</strong>ten had children, less <strong>of</strong>ten had highest qualification <strong>of</strong> year 12 or year<br />
11, were much less <strong>of</strong>ten in employment in 1986, had more health problems affecting<br />
work, and came less <strong>of</strong>ten from Victoria and Queensland and more <strong>of</strong>ten from Western<br />
126 There are 404 cases dropped for analytical reasons, <strong>of</strong> which <strong>the</strong>re are 140 cases where at least one <strong>of</strong><br />
<strong>the</strong> variables used in <strong>the</strong> analysis is missing for a case, <strong>the</strong>n a fur<strong>the</strong>r 264 cases are dropped for o<strong>the</strong>r<br />
analytical reasons.
179<br />
Australia/Tasmania. Amongst those variables where fewer than 140 cases remained after<br />
<strong>the</strong> missing data were accounted for, <strong>the</strong> variables for which a different pr<strong>of</strong>ile was<br />
observed were parental background where <strong>the</strong> mo<strong>the</strong>r had a post-school qualification,<br />
where <strong>the</strong> fa<strong>the</strong>r or mo<strong>the</strong>r was manager, pr<strong>of</strong>essional or para-pr<strong>of</strong>essional. It can be seen<br />
that <strong>the</strong> pattern <strong>of</strong> statistical differences is different for those dropped for item nonresponse<br />
in comparison to those lost through attrition shown in table 5.5. Thus different<br />
processes might be behind <strong>the</strong>se distinct sources <strong>of</strong> missing data.<br />
Panel 2 <strong>of</strong> Table 5.5a shows a different pattern <strong>of</strong> statistical differences for those 264<br />
cases dropped for o<strong>the</strong>r analytical reasons. It should be noted that amongst <strong>the</strong>se, all cases<br />
are observed for all variables. The variables for which those dropped had a different<br />
pr<strong>of</strong>ile to those remaining in <strong>the</strong> final sample were SYETP treatment, those married in<br />
1984, partner employed, children, those with apprenticeship, year 12, year 10 or year 9 as<br />
<strong>the</strong>ir highest qualification, those who had never held a job and those who had held a job<br />
for 3 years or more, those in employment in 1986, and those in South Australia/Nor<strong>the</strong>rn<br />
Territory and Western Australia /Tasmania, and all locations o<strong>the</strong>r than Capital City. The<br />
differing pr<strong>of</strong>iles meant that those dropped for o<strong>the</strong>r analytical reasons had less <strong>of</strong>ten had<br />
SYETP treatment, were less <strong>of</strong>ten married or with children, had less <strong>of</strong>ten highest<br />
qualifications <strong>of</strong> apprenticeship, year 10 or year 9 and more <strong>of</strong>ten year 12, had more <strong>of</strong>ten<br />
never held a job, and less <strong>of</strong>ten had a job for 3 years or more, were less <strong>of</strong>ten in<br />
employment in 1986, lived less <strong>of</strong>ten in South Australia/Nor<strong>the</strong>rn Territory and Western<br />
Australia /Tasmania, less <strong>of</strong>ten lived in country towns and rural areas and more <strong>of</strong>ten in<br />
‘o<strong>the</strong>r towns’. The pattern <strong>of</strong> pr<strong>of</strong>ile differences is distinct from that found for those lost<br />
through attrition shown in Table 5.5.<br />
Both groups <strong>of</strong> cases dropped for analysis had different pr<strong>of</strong>iles for <strong>the</strong> key variables used<br />
to model SYETP treatment and employment in 1986. This indicates that as for attrition,<br />
fur<strong>the</strong>r examination <strong>of</strong> <strong>the</strong> impact on analysis <strong>of</strong> <strong>the</strong> dropping <strong>of</strong> <strong>the</strong>se cases is warranted.<br />
In <strong>the</strong> analysis so far, <strong>the</strong>se cases have been treated in <strong>the</strong> same way, and so dropped for<br />
<strong>the</strong> analysis. Before continuing it is worth readdressing <strong>the</strong> reasons why <strong>the</strong>se cases were<br />
dropped.
180<br />
The first group <strong>of</strong> cases dropped for analysis, were dropped as a treatment for missing<br />
information on any <strong>of</strong> <strong>the</strong> variables in analysis. The following variables have missing<br />
information once attrition is accounted for: parental occupation, parental qualifications,<br />
and unemployment spell prior to 3 June 1984. Of <strong>the</strong>se, <strong>the</strong> unemployment spell<br />
information is generally considered <strong>the</strong> most important for evaluation analysis. However<br />
<strong>the</strong> parental background variables have <strong>the</strong> greatest extent <strong>of</strong> missing information. 127<br />
Alternatives to dropping <strong>the</strong>se cases include not using those variables where <strong>the</strong>re is<br />
missing information, and inference or imputation <strong>of</strong> <strong>the</strong> missing values. This is briefly<br />
considered in <strong>the</strong> next section. With regard to dropping <strong>the</strong>se variables from <strong>the</strong> analysis:<br />
chiefly <strong>the</strong> unemployment spell would be deemed critical to modelling <strong>of</strong> participation<br />
and employment, and so dropping this variable would introduce misspecification bias,<br />
while <strong>the</strong> o<strong>the</strong>rs may be potentially useful but could possibly be dropped. As dropping<br />
<strong>the</strong>se variables alters <strong>the</strong> specification <strong>of</strong> <strong>the</strong> model, this is considered later in Chapter 7.<br />
The second group <strong>of</strong> cases dropped for analysis described <strong>the</strong>mselves as those in fulltime<br />
education at <strong>the</strong> start <strong>of</strong> <strong>the</strong> reference period for <strong>the</strong> evaluation. The central reason<br />
for selecting cases not in fulltime education arises when defining <strong>the</strong> eligible group for<br />
<strong>the</strong> evaluation. For this rationale, those who are in full-time education at <strong>the</strong> point <strong>of</strong><br />
eligibility must be excluded. 128 Retaining <strong>the</strong>se cases would contaminate <strong>the</strong> comparison<br />
group with cases incomparable to those in <strong>the</strong> SYETP treatment group, since <strong>the</strong>y would<br />
not have been eligible to participate in SYETP. In <strong>the</strong> context <strong>of</strong> propensity score<br />
matching, individuals enrolled in fulltime education have a known probability <strong>of</strong> SYETP<br />
participation equal to zero, and could not make good matches for any SYETP treatment<br />
group members, who necessarily have non-zero probabilities <strong>of</strong> SYETP participation. As<br />
this is survey data, and <strong>the</strong> records are based on recall <strong>the</strong>re is <strong>the</strong> potential that some part<br />
<strong>of</strong> this may be subject to recall error, but it is not possible to fur<strong>the</strong>r account for this.<br />
127 It should also be noted that it is quite plausible that respondents did not know or could not recall <strong>the</strong>ir<br />
parents’ occupation or qualification at <strong>the</strong> time <strong>the</strong> respondent was 14.<br />
128 The data here has eligibility set at 3 months unemployment due to <strong>the</strong> sample selection at June 1984,<br />
and also <strong>the</strong> reference period for <strong>the</strong> SYETP treatment variable starts on 3 June 1984. Those dropped<br />
indicated <strong>the</strong>y were in full-time education in <strong>the</strong> period prior to June 1984, and not unemployment, which<br />
makes <strong>the</strong>m ineligible for SYETP.
181<br />
Table 5.5a Summary statistics by source <strong>of</strong> sample loss due to analytical selection<br />
Mean and<br />
standard deviation (s.d.)<br />
sample<br />
after all<br />
sample<br />
loss and<br />
attrition<br />
mean 129<br />
(1)<br />
s.d.<br />
(2)<br />
Those<br />
dropped<br />
due to<br />
missing<br />
mean<br />
s.d.<br />
No.<br />
cases for<br />
that<br />
variable<br />
Difference in<br />
means 130<br />
(absolute)<br />
T –test<br />
Those<br />
lost –<br />
dropped<br />
for o<strong>the</strong>r<br />
analytical<br />
selection<br />
mean<br />
s.d.<br />
Difference<br />
in means<br />
(absolute)<br />
syetp 0.08 0.27 0.04 0.19 140 0.04 2.03** 0.05 0.21 0.04 2.38**<br />
Female 0.41 0.49 0.39 0.49 140 0.005 0.12 0.39 0.49 0.02 0.55<br />
Average age 1984 19.99 2.41 20.05 2.46 140 0.02 0.10 19.74 2.33 0.25 1.56<br />
Aboriginal/Torres Strait 0.03 0.17 0.06 0.23 140 0.01 0.74 0.04 0.19 0.008 0.65<br />
Islander<br />
O<strong>the</strong>r ethnic minority 0.08 0.27 0.10 0.30 140 0.007 0.28 0.09 0.29 0.01 0.63<br />
Married 1984 0.12 0.32 0.15 0.36 140 0.04 1.25 0.02 0.14 0.10 7.97***<br />
Spouse employed 1984 0.06 0.24 0.07 0.25 140 0.002 0.10 0.00 0.00 0.06 8.99***<br />
Children 1984 0.05 0.23 0.11 0.31 140 0.05 1.73* 0.01 0.11 0.04 4.74***<br />
Highest qualification in 1984<br />
Degree/diploma 0.12 0.32 0.09 0.28 140 0.03 1.29 0.10 0.30 0.02 0.98<br />
Apprenticeship 0.08 0.27 0.09 0.29 140 0.01 0.46 0.04 0.19 0.04 3.08***<br />
O<strong>the</strong>r post-school<br />
0.07 0.26 0.09 0.28 140 0.07 0.26 0.001 0.06<br />
qualification<br />
Year 12 <strong>of</strong> school 0.15 0.35 0.09 0.29 140 0.05 1.70* 0.28 0.45 0.14 4.66***<br />
Year 11 <strong>of</strong> school 0.14 0.35 0.08 0.27 140 0.05 1.74* 0.12 0.33 0.02 0.79<br />
Year 10 <strong>of</strong> school 0.31 0.46 0.34 0.47 140 0.007 0.16 0.24 0.43 0.07 2.46**<br />
Year 9 <strong>of</strong> school 0.12 0.33 0.20 0.40 140 0.05 1.40 0.09 0.28 0.04 1.91*<br />
Parental background<br />
when resp. aged 14<br />
Fa<strong>the</strong>r postschool<br />
qualification<br />
Mo<strong>the</strong>r postschool<br />
qualification<br />
0.34 0.47 0.40 0.49 79 0.07 1.15 0.34 0.48 0.006 0.18<br />
0.18 0.39 0.15 0.35 65 0.08 1.88* 0.19 0.40 0.01 0.38<br />
T –test<br />
129 The mean <strong>of</strong> a 0/1 binary variable is <strong>the</strong> same as <strong>the</strong> proportion.<br />
130 The difference is calculated between <strong>the</strong> mean <strong>of</strong> those lost from <strong>the</strong> sample and <strong>the</strong> mean <strong>of</strong> those remaining after attrition.
182<br />
Fa<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.26 0.44 0.17 0.37 135 0.11 3.30*** 0.29 0.46 0.03 1.13<br />
para-pr<strong>of</strong>essional<br />
Fa<strong>the</strong>r not employed 0.05 0.23 0.06 0.24 135 0.003 0.13 0.05 0.22 0.005 0.36<br />
Fa<strong>the</strong>r not present 0.16 0.36 0.13 0.34 140 0.04 1.22 0.19 0.39 0.03 1.08<br />
Mo<strong>the</strong>r manager, pr<strong>of</strong>essional, 0.10 0.29 0.07 0.26 113 0.06 3.13*** 0.08 0.27 0.02 1.10<br />
para-pr<strong>of</strong>essional<br />
Mo<strong>the</strong>r not employed 0.55 0.50 0.59 0.49 113 0.06 1.32 0.54 0.50 0.009 0.27<br />
Mo<strong>the</strong>r not present 0.05 0.22 0.05 0.21 140 0.02 1.58 0.06 0.23 0.004 0.24<br />
Longest job ever held by 1984<br />
Never held a job 0.12 0.32 0.09 0.28 140 0.09 1.20 0.16 0.36 0.04 1.63*<br />
< 1 year 0.42 0.49 0.47 0.50 140 0.06 1.29 0.44 0.50 0.02 0.55<br />
1 year 0.13 0.34 0.15 0.35 140 0.01 0.46 0.14 0.35 0.005 0.23<br />
2 years 0.14 0.35 0.14 0.35 140 0.02 0.71 0.12 0.33 0.02 0.88<br />
3 years or more 0.19 0.39 0.15 0.36 140 0.04 1.15 0.14 0.35 0.04 1.78*<br />
Average unemployment 131 0.62 0.41 0.67 0.40 125 0.05 1.42 0.58 0.42 0.04 1.27<br />
Ever employed in 1986 132 0.74 0.44 0.37 0.48 140 0.17 3.87*** 0.61 0.49 0.13 3.91***<br />
Ever government programme 0.11 0.31 0.07 0.26 140 0.004 0.15 0.10 0.30 0.01 0.56<br />
1986 133<br />
Work limited by health 1984 0.10 0.30 0.16 0.37 140 0.09 2.53*** 0.10 0.30 0.003 0.12<br />
Interviewed location in 1984<br />
New South Wales /ACT 0.37 0.48 0.44 0.50 140 0.04 0.91 0.41 0.49 0.03 0.94<br />
Victoria 0.24 0.43 0.15 0.35 140 0.09 2.94*** 0.28 0.45 0.04 1.32<br />
Queensland 0.14 0.34 0.09 0.29 140 0.10 4.83*** 0.16 0.37 0.03 1.01<br />
South Australia /Nor<strong>the</strong>rn 0.12 0.33 0.10 0.30 140 0.02 0.60 0.08 0.27 0.05 2.57***<br />
Territory<br />
Western Australia / Tasmania 0.13 0.33 0.22 0.42 140 0.17 4.18*** 0.08 0.27 0.05 2.49***<br />
Capital city 0.47 0.50 0.47 0.50 140 0.04 0.80 0.53 0.50 0.05 1.62<br />
O<strong>the</strong>r town 0.21 0.41 0.20 0.40 140 0.02 0.69 0.26 0.44 0.05 1.76*<br />
Country town 0.24 0.43 0.27 0.45 140 0.07 1.62 0.17 0.38 0.07 2.54***<br />
Rural area 0.08 0.27 0.06 0.24 140 0.007 0.28 0.04 0.19 0.04 2.87***<br />
Number <strong>of</strong> observations 1283 140 264<br />
Significance <strong>of</strong> t test at level 1% ***, 5%**, 10 % *. Student’s t tests <strong>of</strong> whe<strong>the</strong>r means are equal<br />
131 The proportion <strong>of</strong> 1984 reference period to 3 June spent unemployed.<br />
132 Ever held a non-subsidised, non-government programme job in <strong>the</strong> 1986 reference period, after <strong>the</strong> first 17 weeks.<br />
133 Ever go on a government programme, including SYETP, in <strong>the</strong> 1986 reference period.
183<br />
5.6.1.2 Effects <strong>of</strong> <strong>the</strong> non-response to 1984 Survey on <strong>the</strong> participation model<br />
Table 5.8 Column 1 shows <strong>the</strong> probit results for SYETP participation, for all observations<br />
in <strong>the</strong> 1984 survey, where no weight has been applied. Column 2 shows <strong>the</strong> probit results<br />
for SYETP participation for all observations in <strong>the</strong> 1984 survey, weighted with <strong>the</strong> survey<br />
weight (for non-response and survey design), discussed above. The results are changed<br />
slightly by <strong>the</strong> application <strong>of</strong> <strong>the</strong> weight. The variables that are statistically significant<br />
would normally be focused on, and which variables are significant is affected by applying<br />
<strong>the</strong> survey weight. Some variables become insignificant when <strong>the</strong> weight is applied, such<br />
as no job held before 1984. Some variables become significant when <strong>the</strong> weight is used,<br />
such as married in 1984, attended a private school, and longest job held before 1984 was<br />
3 years or more.<br />
The effects <strong>of</strong> accounting for <strong>the</strong> survey design do alter <strong>the</strong> interpretation <strong>of</strong> <strong>the</strong> results. In<br />
light <strong>of</strong> <strong>the</strong> analysis <strong>of</strong> Mcrae et al. (1985), <strong>the</strong> SYETP models and estimates <strong>of</strong> such<br />
variables appear to be biased by <strong>the</strong> non-response, unless <strong>the</strong> weights are applied.<br />
Pfefferman (1993) p321 points out that when <strong>the</strong> sample is selected by simple random<br />
sampling, <strong>the</strong> model holding for <strong>the</strong> sample data is <strong>the</strong> same as <strong>the</strong> model holding in <strong>the</strong><br />
population, however with complex survey sampling designs, <strong>the</strong> two models can differ.<br />
The ignorability <strong>of</strong> a complex sampling design for regression analysis depends not only<br />
on <strong>the</strong> sample design but also <strong>the</strong> model and parameters <strong>of</strong> interest, so that if all design<br />
variables are included <strong>the</strong>n <strong>the</strong> sampling is ignorable for <strong>the</strong> modelling. Because <strong>the</strong><br />
design variables are not in <strong>the</strong> data, <strong>the</strong> design information that was used to construct <strong>the</strong><br />
weights in <strong>the</strong> ALS data cannot be incorporated into <strong>the</strong> modelling without using <strong>the</strong><br />
weights formed using <strong>the</strong> design information. The sampling and non-response weights<br />
<strong>the</strong>n act as surrogates for this information set. Incorporating <strong>the</strong> weights can account for<br />
<strong>the</strong> differences between <strong>the</strong> complexly designed sample, and <strong>the</strong> population.
184<br />
Table 5.8 Probit <strong>of</strong> SYETP participation, showing sample reduction effects<br />
Before sample<br />
reduction,<br />
unweighted<br />
Before sample<br />
reduction, with<br />
response weight<br />
After sample<br />
reduction,<br />
unweighted<br />
After sample<br />
reduction, with<br />
response weight<br />
(1) (2) (3) (4)<br />
syetp syetp syetp syetp<br />
Gender=female -0.02 -0.03 0.08 0.06<br />
(0.16) (0.33) (0.66) (0.48)<br />
Age at 1984 survey -0.08 -0.06 -0.11 -0.08<br />
(3.21)** (2.41)* (3.21)** (2.86)**<br />
Married 1984 -0.59 -0.68 -0.97 -1.09<br />
(1.78) (2.66)** (1.62) (4.20)**<br />
Children 1984 0.48 0.38 0.49 0.33<br />
(1.17) (1.14) (0.73) (0.82)<br />
Children*female -0.17 0.16 -0.30 0.03<br />
(0.30) (0.35) (0.36) (0.06)<br />
Spouse employed 1984 0.15 0.20 0.59 0.70<br />
(0.34) (0.56) (0.91) (1.93)<br />
Aboriginal/Torres Strait Islander -0.73 -0.70 -0.45 -0.37<br />
(1.78) (1.73) (0.96) (0.78)<br />
O<strong>the</strong>r ethnic minority 0.07 -0.03 0.06 0.00<br />
State interviewed in 1984 (0.35) (0.16) (0.24) (0.01)<br />
Victoria 0.13 0.15 0.12 0.11<br />
(1.12) (1.17) (0.75) (0.70)<br />
Queensland -0.05 -0.01 -0.20 -0.13<br />
(0.35) (0.07) (0.97) (0.66)<br />
South Australia/Nor<strong>the</strong>rn Territory 0.12 0.13 -0.12 -0.11<br />
(0.78) (0.82) (0.61) (0.59)<br />
Western Australia/Tasmania 0.31 0.35 0.32 0.35<br />
(2.24)* (2.44)* (1.79) (2.02)*<br />
Education school overseas 0.07 0.24 0.14 0.38<br />
(0.28) (0.90) (0.40) (1.07)<br />
Roman Catholic school -0.38 -0.40 -0.31 -0.27<br />
(2.07)* (2.21)* (1.28) (1.15)<br />
Private school -0.75 -1.01 -0.72 -0.97<br />
Highest qualification in 1984 (1.85) (2.56)* (1.54) (2.32)*<br />
Degree/diploma -0.11 -0.12 0.05 -0.01<br />
(0.62) (0.65) (0.21) (0.03)<br />
Apprenticeship -0.33 -0.29 -0.14 -0.12<br />
(1.32) (1.22) (0.43) (0.41)<br />
O<strong>the</strong>r Post-School qualification -0.09 -0.00 0.08 0.05<br />
(0.44) (0.01) (0.32) (0.19)<br />
Year 12 <strong>of</strong> school 0.15 0.23 0.41 0.41<br />
(1.12) (1.60) (2.27)* (2.29)*<br />
Year 11 <strong>of</strong> school 0.06 0.10 0.16 0.22<br />
(0.45) (0.73) (0.85) (1.14)<br />
Year 9 <strong>of</strong> school or less -0.19 -0.10 -0.14 -0.02<br />
(1.30) (0.63) (0.68) (0.09)<br />
Longest job by 1984 none -0.42 -0.34 -0.41 -0.31<br />
(2.20)* (1.83) (1.62) (1.33)<br />
< 1 year -0.04 -0.04 -0.04 -0.12<br />
(0.30) (0.32) (0.22) (0.67)<br />
2 years -0.03 -0.07 0.15 0.04<br />
(0.18) (0.39) (0.66) (0.17)
185<br />
3 years + -0.35 -0.47 -0.34 -0.51<br />
(1.82) (2.58)** (1.30) (2.12)*<br />
CEP referrals 1984 0.08 0.08 0.15 0.12<br />
(1.66) (1.48) (2.27)* (1.64)<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
0.40 0.30 0.47 0.34<br />
(3.24)** (2.64)** (2.85)** (2.46)*<br />
Work limited by health -0.50 -0.60 -0.59 -0.65<br />
Family background (2.69)** (3.33)** (2.30)* (2.84)**<br />
O<strong>the</strong>r city before aged 14 -0.11 -0.21 -0.25 -0.40<br />
(0.89) (1.69) (1.51) (2.40)*<br />
Country town before aged 14 -0.32 -0.36 -0.43 -0.51<br />
(2.62)** (2.83)** (2.78)** (3.32)**<br />
Rural area before aged 14 -0.28 -0.39 -0.46 -0.39<br />
(1.38) (1.86) (1.70) (1.53)<br />
Overseas before aged 14 -0.73 -0.66 -0.69 -0.62<br />
(1.59) (1.58) (1.36) (1.32)<br />
Number <strong>of</strong> siblings 0.01 0.01 0.01 0.02<br />
(0.58) (0.60) (0.42) (0.56)<br />
English good -0.13 -0.18 -0.11 -0.14<br />
(0.68) (1.03) (0.43) (0.56)<br />
English poor -0.68 -0.74 -0.60 -0.73<br />
(1.50) (1.50) (1.11) (1.18)<br />
Sexist 0.11 0.19 0.25 0.31<br />
(0.59) (0.97) (0.95) (1.12)<br />
Sexist*female -0.66 -0.76 -0.76 -0.88<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.24) (1.67) (1.25) (1.60)<br />
Fa<strong>the</strong>r not present when resp 14 -0.21 -0.24 -0.22 -0.22<br />
(1.02) (1.16) (0.82) (0.81)<br />
Labourer -0.06 0.00 -0.17 -0.07<br />
(0.28) (0.02) (0.58) (0.24)<br />
Plant operative -0.13 -0.09 -0.22 -0.13<br />
(0.65) (0.47) (0.79) (0.51)<br />
Sales -0.19 -0.27 -0.09 -0.25<br />
(0.70) (1.00) (0.27) (0.71)<br />
Tradesperson -0.05 -0.05 -0.25 -0.27<br />
(0.24) (0.27) (0.91) (1.03)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.04 -0.08 -0.09 -0.19<br />
(0.21) (0.44) (0.35) (0.74)<br />
Not employed -0.31 -0.23 -0.41 -0.47<br />
(1.13) (0.90) (1.15) (1.42)<br />
Fa<strong>the</strong>r holds post-school<br />
-0.29 -0.26 -0.29 -0.22<br />
qualification when resp 14<br />
Mo<strong>the</strong>rs occupation when resp. 14 (2.60)** (2.24)* (1.97)* (1.61)<br />
Mo<strong>the</strong>r not present when resp 14 0.34 0.29 0.47 0.37<br />
(1.39) (1.25) (1.48) (1.24)<br />
Labourer 0.12 0.08 0.13 -0.04<br />
(0.55) (0.38) (0.43) (0.14)<br />
Plant operative 0.38 0.31 0.65 0.52<br />
(1.62) (1.30) (2.10)* (1.69)<br />
Sales 0.18 0.26 0.20 0.25<br />
(0.79) (1.12) (0.67) (0.89)<br />
Tradesperson 0.02 -0.02 0.11 0.13<br />
(0.05) (0.06) (0.25) (0.31)<br />
Manager/pr<strong>of</strong>essional/para- -0.14 -0.19 -0.23 -0.32
186<br />
pr<strong>of</strong>essional<br />
(0.60) (0.83) (0.78) (1.11)<br />
Not employed 0.08 -0.00 0.03 -0.10<br />
(0.46) (0.03) (0.13) (0.45)<br />
Mo<strong>the</strong>r post-school qualification 0.21 0.19 0.25 0.24<br />
when resp 14<br />
Religion brought up in (1.64) (1.45) (1.55) (1.40)<br />
Catholic 0.16 0.12 0.05 -0.02<br />
(1.27) (0.97) (0.32) (0.11)<br />
Presbyterian 0.28 0.21 0.32 0.25<br />
(1.46) (1.05) (1.31) (1.05)<br />
Methodist 0.12 0.24 0.11 0.23<br />
(0.60) (1.11) (0.44) (0.96)<br />
O<strong>the</strong>r Christian -0.06 -0.11 0.09 0.10<br />
(0.26) (0.52) (0.34) (0.36)<br />
O<strong>the</strong>r religion 0.16 0.22 0.14 0.13<br />
(0.94) (1.16) (0.63) (0.54)<br />
No religion 0.14 0.17 0.17 0.16<br />
(0.94) (1.08) (0.84) (0.83)<br />
missing fa<strong>the</strong>rs qualifications -0.49 -0.55<br />
(1.60) (1.69)<br />
missing Mo<strong>the</strong>rs qualifications 0.01 0.10<br />
(0.02) (0.42)<br />
missing proportion <strong>of</strong> time spent -0.07 -0.08<br />
unemployed<br />
(0.12) (0.16)<br />
missing number <strong>of</strong> siblings 0.20 0.17<br />
(0.79) (0.68)<br />
Constant -0.01 -0.33 0.66 0.39<br />
(0.03) (0.60) (0.86) (0.59)<br />
Observations 2368 2368 1283 1283<br />
Log likelihood -503.74 -500.54 -307.52 -307.63<br />
LR chi 2 (59) 134 137.31 150.58 106.91 126.23<br />
Mcfadden’s Pseudo R 2 135<br />
0.1199 0.1284 0.1481 0.1553<br />
Akaike Information Criterion 0.48 0.48 0.57 0.57<br />
Coefficient with t statistic in brackets. Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%, **<br />
significant at 1%. Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT,<br />
government school, highest qualification in 1984 year 10 at school, longest job by 1984 is 1 year, lived<br />
mostly in state capital city until respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when<br />
respondent aged 14, mo<strong>the</strong>r clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
Notes: Variables miss 1 (mo<strong>the</strong>rs occupation missing) and miss 2 (fa<strong>the</strong>rs occupation missing) predict<br />
failure perfectly, and cannot be added to <strong>the</strong> regression.<br />
134 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
135 This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(M full ) to a model with just <strong>the</strong> intercept (M intercept ). It is defined as R 2 = 1 – (log likelihood M full / log<br />
likelihood M intercept ) . The value <strong>of</strong> Mcfadden’s Pseudo R 2 increases as new variables are added.
187<br />
5.7 Multivariate analysis <strong>of</strong> effects <strong>of</strong> sample reduction<br />
The univariate evidence on <strong>the</strong> sample reduction effects treated earlier suggested that<br />
SYETP participants and non-participants were differentially affected. Accordingly, <strong>the</strong><br />
model <strong>of</strong> SYETP is also examined in light <strong>of</strong> <strong>the</strong> fur<strong>the</strong>r sample reduction after <strong>the</strong> 1984<br />
survey. The estimation is repeated on both samples, all available in 1984 and those cases<br />
available after sample reduction. This has <strong>the</strong> benefit <strong>of</strong> revealing whe<strong>the</strong>r <strong>the</strong> estimation<br />
would have looked different using <strong>the</strong> entire sample available for <strong>the</strong> 1984 survey.<br />
5.7.1 Accounting for item non-response<br />
As previously pointed out, <strong>the</strong>re are some cases who respond at <strong>the</strong> 1984 survey for<br />
whom certain regressor variables have missing information. To best regain <strong>the</strong> lost<br />
information <strong>of</strong> <strong>the</strong> incomplete observations, <strong>the</strong> modified zero order regression is usually<br />
adopted (Greene (1991): 288). The missing cases are filled with zeros, and <strong>the</strong> regression<br />
includes for each instance a dummy variable which takes <strong>the</strong> value <strong>of</strong> 1 for missing<br />
observations and 0 for complete cases.<br />
Recent work by King et al. (2001) examines <strong>the</strong> role <strong>of</strong> missing data in <strong>the</strong> explanatory<br />
variables. King et al. (2001) examines <strong>the</strong> validity <strong>of</strong> this form <strong>of</strong> imputation (mean<br />
substitution), as compared to deleting <strong>the</strong> observations where missing data occurs, or an<br />
alternative form <strong>of</strong> multiple imputation algorithm proposed. The algorithm outperforms<br />
both more commonly used alternatives. In particular it is pointed out that mean<br />
substitution gives standard errors that are too small because it assumes in estimation that<br />
<strong>the</strong> substituted values are known with <strong>the</strong> same certainty as <strong>the</strong> observed values (King et<br />
al. (2001): 66). On <strong>the</strong> o<strong>the</strong>r hand, dropping those cases with missing covariates (or<br />
listwise deletion) gives <strong>the</strong> correct standard error, although estimates do suffer <strong>the</strong><br />
problems <strong>of</strong> bias/inefficiency as recounted in our discussion. It is commented that for this<br />
reason, deletion can be a preferable approach.<br />
In light <strong>of</strong> this, Appendix Table A2.2 shows <strong>the</strong> effect <strong>of</strong> applying each <strong>of</strong> <strong>the</strong> mean<br />
substitution and casewise deletion approaches. In order to detract from <strong>the</strong> issue <strong>of</strong> <strong>the</strong>
188<br />
weighting, both <strong>the</strong> weighted (columns 2 and 4) and unweighted (columns 1 and 3)<br />
results are shown for each missing data approach. Of main interest is <strong>the</strong> comparison <strong>of</strong><br />
<strong>the</strong> estimates using each missing data approach, thus column 1 versus column 3, or<br />
column 2 versus column 4. Discussion <strong>of</strong> Table A2.2 generalises <strong>the</strong> results to<br />
comparison between <strong>the</strong> first panel (using mean substitution) and <strong>the</strong> second panel<br />
(deletion <strong>of</strong> cases with missing data – this gives <strong>the</strong> smaller base <strong>of</strong> 2150 cases where<br />
1984 survey information has no information missing on any explanatory variables) by<br />
discussing <strong>the</strong> weighted results. Moving from mean substitution to <strong>the</strong> deletion approach<br />
for <strong>the</strong> weighted data does change which variables are statistically significant, as <strong>the</strong> t-<br />
statistic size changes – for example ’o<strong>the</strong>r city before aged 14’ becomes statistically<br />
significant, and ‘age in 1984’ becomes insignificant. It also leads to a change in <strong>the</strong> size<br />
<strong>of</strong> coefficients for statistically significant variables – for example <strong>the</strong> coefficient for<br />
‘proportion <strong>of</strong> pre-June unemployment’ falls, and <strong>the</strong> t-statistic falls. In a probit, <strong>the</strong><br />
coefficient size is not clearly interpretable, so <strong>the</strong> interpretation <strong>of</strong> a positive influence <strong>of</strong><br />
this variable on participation in SYETP is not changed by <strong>the</strong> various missing data<br />
approaches, however <strong>the</strong> calculated marginal effect would be affected. When <strong>the</strong> variable<br />
is not statistically significant, changing <strong>the</strong> missing data approach can also lead to change<br />
in <strong>the</strong> sign, such as for ‘children 1984’. Although not discussed in detail here, it can be<br />
seen that substantively important changes in <strong>the</strong> estimates arise depending on each<br />
approach used. The chief variation is in which variables are statistically significant, so<br />
that choice <strong>of</strong> treatment <strong>of</strong> missing data affects which coefficients are interpreted as<br />
statistically significant. The trade<strong>of</strong>f between bias and efficiency and using more<br />
information affects <strong>the</strong> interpretation <strong>of</strong> results. In light <strong>of</strong> this, it may be worth pursuing<br />
<strong>the</strong> application <strong>of</strong> <strong>the</strong> imputation algorithm <strong>of</strong> King et al. (2001) in future research, which<br />
is argued to outperform <strong>the</strong> mean substitution and deletion methods. However, only <strong>the</strong><br />
more commonly accepted approaches are dealt with here.<br />
The effects <strong>of</strong> a set <strong>of</strong> dummy variables for mean imputation is shown in <strong>the</strong> first 2<br />
columns <strong>of</strong> Table 5.8. Missing information on <strong>the</strong> parental occupation predicts failure<br />
perfectly, <strong>the</strong> problem <strong>of</strong> collinearity, and <strong>the</strong>se variables must be dropped from <strong>the</strong><br />
regression to enable estimation. The missing information for parental qualifications,
189<br />
proportion <strong>of</strong> time spent unemployed and number <strong>of</strong> siblings is controlled for using <strong>the</strong><br />
dummies. The estimation on <strong>the</strong> data where those cases with missing information in <strong>the</strong>se<br />
variables are dropped is given in Appendix Table A2.3. The results in columns one and<br />
two are slightly different to that <strong>of</strong> Table 5.8. Of course <strong>the</strong> number <strong>of</strong> observations in<br />
columns one and two <strong>of</strong> Appendix Table A2.3 are lower at 2150 because <strong>the</strong> observations<br />
with missing information are dropped, whereas in Table 5.8 <strong>the</strong>y are 2368. The Akaike<br />
Information Criterion does not vary much in size between <strong>the</strong> models, and so does not<br />
assist much in model selection here (because <strong>the</strong> sample and variables change between<br />
<strong>the</strong> models, this fit measure is more relevant).The arguments <strong>of</strong> King et al. (2001) suggest<br />
that dropping those cases, casewise deletion, gives <strong>the</strong> correct standard error, although<br />
estimates do suffer <strong>the</strong> problems <strong>of</strong> bias. It is <strong>the</strong>n a subjective choice as to whe<strong>the</strong>r <strong>the</strong><br />
analyst prefers to trade-<strong>of</strong>f bias, however correct standard error estimation is essential if<br />
<strong>the</strong> statistical significance <strong>of</strong> <strong>the</strong> coefficients is important to analysis. In light <strong>of</strong> this, it is<br />
deemed more useful to apply casewise deletion than mean imputation dummies.<br />
5.7.2 Sample reduction effects on model <strong>of</strong> SYETP participation<br />
Columns 3 and 4 <strong>of</strong> Table 5.8 give <strong>the</strong> probit results for SYETP participation for <strong>the</strong> final<br />
data set after sample reduction. Column 3 136 shows <strong>the</strong> unweighted results, and column 4<br />
shows <strong>the</strong> results weighted with <strong>the</strong> survey weight. 137 As for <strong>the</strong> whole sample discussed<br />
earlier, <strong>the</strong> variables that are statistically significant alter with <strong>the</strong> use <strong>of</strong> <strong>the</strong> weight. The<br />
variables that gain significance when using <strong>the</strong> weight are married in 1984, attended a<br />
private school, interviewed in Western Australia/ Tasmania, longest job held before 1984<br />
was 3 years or more, mostly lived in a city until aged 14. The variables that lose<br />
statistical significance are CEP referrals in 1984, fa<strong>the</strong>r held a post-school qualification<br />
when respondent aged 14, mo<strong>the</strong>r worked as plant operative when respondent aged 14. A<br />
worrying change is <strong>the</strong> loss <strong>of</strong> statistical significance for CEP referrals in 1984. This is<br />
fur<strong>the</strong>r discussed later in <strong>the</strong> modelling <strong>of</strong> <strong>the</strong> treatment effect <strong>of</strong> SYETP, because this is<br />
a key element <strong>of</strong> <strong>the</strong> identifying restriction in <strong>the</strong> bivariate probit <strong>of</strong> employment<br />
136 The results in column 3 are equivalent to <strong>the</strong> univariate probit estimated in Richardson (1998).<br />
137 Note that no account has been made <strong>of</strong> sample reduction from <strong>the</strong> 1984 survey in this weight. This is<br />
treated next.
190<br />
outcomes and SYETP participation. These changes to <strong>the</strong> estimation are interpreted as<br />
signifying <strong>the</strong> importance <strong>of</strong> accounting for <strong>the</strong> survey design and non-response in <strong>the</strong><br />
data,<br />
Finally, Table 5.8 examines <strong>the</strong> effects <strong>of</strong> applying <strong>the</strong> SYETP model to <strong>the</strong> full sample<br />
and <strong>the</strong> sample after sample reduction. This is a fur<strong>the</strong>r examination <strong>of</strong> <strong>the</strong> sample<br />
reduction effects upon <strong>the</strong> desired model <strong>of</strong> SYETP participation. The importance <strong>of</strong> <strong>the</strong><br />
sampling weight has been shown. The results weighted by <strong>the</strong> survey weight are<br />
discussed first, in a comparison <strong>of</strong> columns 2 and 4, followed by a comparison <strong>of</strong> <strong>the</strong> unweighted<br />
data in columns 1 and 3. The non-response weighting is not fully appropriate<br />
for <strong>the</strong> final sample as it does not include fur<strong>the</strong>r effects <strong>of</strong> attrition, which are yet to be<br />
constructed, however it is presented here to allow <strong>the</strong> opportunity to disregard <strong>the</strong> share<br />
due to non-response weight effects when comparing before and after sample reduction.<br />
The table shows that whe<strong>the</strong>r or not <strong>the</strong> non-response weight was used, <strong>the</strong> results are<br />
affected by <strong>the</strong> use <strong>of</strong> only <strong>the</strong> reduced sample compared to all those cases available<br />
before sample reduction. In using <strong>the</strong> full sample compared to <strong>the</strong> reduced sample, two<br />
important types <strong>of</strong> changes are found that would alter interpretation <strong>of</strong> <strong>the</strong> estimates:<br />
changes in sign, from positive coefficient to negative and vice versa, and changes in<br />
which variables are statistically significant. These changes occur whe<strong>the</strong>r <strong>the</strong> weight is<br />
used or not, however <strong>the</strong> incidence is different in <strong>the</strong> weighted data to that in <strong>the</strong> unweighted<br />
data. The changes in sign occur only for variables that were not statistically<br />
significant. No change in significance is associated with a change in sign.<br />
The results weighted by <strong>the</strong> survey weight change sign for gender, o<strong>the</strong>r ethnicity, South<br />
Australia/Nor<strong>the</strong>rn Territory, fa<strong>the</strong>rs occupation was labourer, mo<strong>the</strong>rs occupation was<br />
labourer or tradesperson, and religion brought up in was Catholic or o<strong>the</strong>r Christian. The<br />
variables that change statistical significance, ei<strong>the</strong>r falling out <strong>of</strong> statistical significance or<br />
becoming significant, are Roman Catholic schooling, highest qualification was schooling<br />
to year 12, mostly lived in a city to age 14, and fa<strong>the</strong>r held a post school qualification<br />
when respondent was 14. The unweighted results change sign for gender, South<br />
Australia/Nor<strong>the</strong>rn Territory, highest qualification in 1984 is degree or diploma or o<strong>the</strong>r
191<br />
post-school qualification, and longest job held by 1984 is 2 to 3 years. The variables that<br />
change statistical significance, are interviewed in Western Australia/ Tasmania in 1984,<br />
Roman Catholic schooling, highest qualification was schooling to year 12, no job held by<br />
1984, CEP referrals in 1984, mo<strong>the</strong>rs occupation was plant operative, and brought up in<br />
o<strong>the</strong>r Christian religion.<br />
Sample reduction is found to affect <strong>the</strong> results <strong>of</strong> modelling SYETP quite substantially.<br />
Comparison between <strong>the</strong> full data and reduced data estimates shows significant<br />
differences between <strong>the</strong> results <strong>of</strong> <strong>the</strong> models, and <strong>the</strong>se are interpreted as representative<br />
<strong>of</strong> <strong>the</strong> presence <strong>of</strong> attrition bias. This supports <strong>the</strong> <strong>the</strong>oretical position that bias is<br />
introduced to <strong>the</strong> estimates. It also concurs with <strong>the</strong> empirical material <strong>of</strong> Alderman et al.<br />
(2000) discussed earlier where similar impacts on modelling were also found. The next<br />
section deals with multivariate modelling <strong>of</strong> <strong>the</strong> sample reduction. Weights are <strong>the</strong>n<br />
produced to account for <strong>the</strong> sample reduction.
192<br />
5.8 Model <strong>of</strong> <strong>the</strong> probability <strong>of</strong> attrition<br />
To construct <strong>the</strong> weight for non-response and selection that was provided with <strong>the</strong> data,<br />
administrative data was used to weight back to <strong>the</strong> population. Only survey data is<br />
available to be applied to estimates <strong>of</strong> <strong>the</strong> sample attrition weight. As <strong>the</strong> attrition only<br />
occurs after <strong>the</strong> 1984 survey, it is likely <strong>the</strong> survey information is <strong>the</strong> most relevant for<br />
describing <strong>the</strong> characteristics <strong>of</strong> <strong>the</strong> respondents, as it is ga<strong>the</strong>red at a more recent<br />
juncture while <strong>the</strong> administrative data would be more dated. One useful piece <strong>of</strong> relevant<br />
information that is not available in <strong>the</strong> data would be survey collection information about<br />
movers. This variable had a central role in describing non-response. However,<br />
unavailability means <strong>the</strong> importance <strong>of</strong> this item to attrition cannot be assessed. The<br />
variables selected for explaining attrition are chosen from amongst those variables<br />
measured in <strong>the</strong> 1984 survey. The model is weighted by <strong>the</strong> initial nonresponse/design/selection<br />
weight in order to represent <strong>the</strong> 1984 data, so that <strong>the</strong> estimates<br />
which will be used for <strong>the</strong> attrition weights will only account for <strong>the</strong> fur<strong>the</strong>r non-response<br />
to <strong>the</strong> later surveys to 1986. In this following model, it is assumed that <strong>the</strong> same process<br />
as that <strong>of</strong> attrition to 1986 defines attrition to <strong>the</strong> 1985 survey. 138<br />
The results <strong>of</strong> estimation for <strong>the</strong> probit model <strong>of</strong> attrition are shown in column 1 <strong>of</strong> table<br />
5.9. Those who were interviewed in South Australia and <strong>the</strong> Nor<strong>the</strong>rn Territory, Western<br />
Australia and Tasmania, were more likely to also be interviewed in later years to 1986.<br />
Additionally, those in country towns and rural areas, and those with a highest<br />
qualification to year 12, had a higher tendency to be interviewed again in later years.<br />
Those with longer unemployment prior to June 1984 were less likely to be surveyed<br />
again in later years, as were those who were missing <strong>the</strong> unemployment variable, had<br />
held longer jobs in <strong>the</strong> past, and those who were married. Participants in SYETP were<br />
more likely to respond to later surveys.<br />
138 In exploration <strong>of</strong> this, results for which are not shown, separate modelling <strong>of</strong> <strong>the</strong> attrition to each year<br />
was not found to have different estimated models, with <strong>the</strong> same coefficients statistically significant and <strong>of</strong><br />
similar magnitude, and so common modelling <strong>of</strong> <strong>the</strong> attrition was found appropriate.
193<br />
Generally, those variables found to be statistically significant in <strong>the</strong> multivariate model <strong>of</strong><br />
attrition reflect those pointed out by <strong>the</strong> univariate analysis <strong>of</strong> difference in mean arising<br />
from attrition (see section5.5.4 and Table 5.5).<br />
As a summary measure <strong>of</strong> fit for <strong>the</strong> probit model, <strong>the</strong> predictions <strong>of</strong> <strong>the</strong> model are<br />
examined. A fitted probability exceeding 0.5 is taken to indicate a predicted response to<br />
<strong>the</strong> survey. When <strong>the</strong>se predicted responses are compared to <strong>the</strong> actual interviews in 1986<br />
<strong>the</strong> model correctly predicted <strong>the</strong> attrition to 1986 for 72 per cent <strong>of</strong> <strong>the</strong> 1984<br />
observations. The o<strong>the</strong>r measures <strong>of</strong> fit indicate no serious problems.<br />
These weights are now introduced into a single final weight that can be used for<br />
modelling. This combined weight could <strong>the</strong>n be used to reflect <strong>the</strong> total effect due to nonresponse,<br />
survey design, and survey attrition.
194<br />
Table 5.9: Probit results used for construction <strong>of</strong> attrition weights<br />
Model <strong>of</strong> Response to<br />
1986 survey 139<br />
syetp syetp 0.25<br />
(2.05)*<br />
age84 Age at 1984 survey -0.03<br />
(1.71)<br />
female Gender=female -0.05<br />
(0.83)<br />
roatsi Aboriginal/Torres Strait Islander -0.06<br />
(0.40)<br />
rones O<strong>the</strong>r ethnic minority -0.00<br />
(0.04)<br />
health84 Work limited by health 0.02<br />
(0.17)<br />
mar84 Married 1984 -0.36<br />
State interviewed in 1984 (2.66)**<br />
vic Victoria 0.05<br />
(0.62)<br />
qld Queensland -0.03<br />
(0.41)<br />
sant South Australia/Nor<strong>the</strong>rn Territory 0.26<br />
(2.58)**<br />
watas Western Australia/Tasmania 0.26<br />
Area <strong>of</strong> residence in 1984 (2.62)**<br />
oc o<strong>the</strong>r town 0.02<br />
(0.33)<br />
ct Country town 0.40<br />
(5.10)**<br />
ra Rural area 0.54<br />
Highest qualification in 1984 (4.06)**<br />
hqdd84 Degree/diploma 0.15<br />
(1.46)<br />
hqtra84 Apprenticeship 0.06<br />
(0.54)<br />
hqoth84 O<strong>the</strong>r Post-School qualification 0.16<br />
(1.37)<br />
hq12_84 Year 12 <strong>of</strong> school 0.23<br />
(2.55)*<br />
hq11_84 Year 11 <strong>of</strong> school 0.16<br />
(1.60)<br />
hq9_84 Year 9 <strong>of</strong> school or less -0.16<br />
(1.86)<br />
upropjn duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
-0.16<br />
(2.20)*<br />
child84 Children in 1984 0.45<br />
(2.78)**<br />
spemp Spouse employed in 1984 0.30<br />
(1.71)<br />
139 Dependent variable is response to 1986 survey, with value 1 if respondent at survey in 1986, zero<br />
o<strong>the</strong>rwise.
195<br />
longj0 Longest job by 1984 < 1 year -0.08<br />
(0.63)<br />
longj1 1
196<br />
5.9 Attrition Weights from <strong>the</strong> model<br />
As a check on <strong>the</strong> performance <strong>of</strong> <strong>the</strong> weights, <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> a number <strong>of</strong> explanatory<br />
variables is considered. The application <strong>of</strong> <strong>the</strong> weights constructed is shown in Table 5.10.<br />
The 1984 survey data is shown in <strong>the</strong> first column, while <strong>the</strong> 1986 survey data is shown<br />
in <strong>the</strong> second column. The constructed attrition weights should approximately reproduce<br />
<strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> 1984 data for variables significantly affected by attrition.<br />
Applying <strong>the</strong> weights from <strong>the</strong> probability model <strong>of</strong> attrition yields <strong>the</strong> pr<strong>of</strong>ile in column<br />
3 <strong>of</strong> Table 5.10. Column 3 shows that <strong>the</strong> attrition weight has restored <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong><br />
1984 survey quite well.<br />
The final two columns in Table 5.10 examine <strong>the</strong> combination <strong>of</strong> <strong>the</strong> attrition weight with<br />
<strong>the</strong> response/selection weight. The fourth column shows <strong>the</strong> 1984 survey weighted to<br />
restore <strong>the</strong> sampled population pr<strong>of</strong>ile. The final column shows <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> 1986<br />
data restored to <strong>the</strong> sampled population, where <strong>the</strong> response/selection weight has been<br />
multiplied by <strong>the</strong> attrition weight. The pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> 1986 data in column five is now<br />
more similar to that <strong>of</strong> <strong>the</strong> population. Thus <strong>the</strong> weights constructed have performed <strong>the</strong><br />
desired repair to <strong>the</strong> dataset so that it better reflects <strong>the</strong> original sample eligible for<br />
SYETP. This combined weight is to be used in <strong>the</strong> analysis <strong>of</strong> <strong>the</strong> treatment effects <strong>of</strong><br />
SYETP, to restore <strong>the</strong> sampled population characteristics where <strong>the</strong> ‘1986 panel’ data is<br />
used. This is performed in <strong>the</strong> next study.<br />
The summary statistics describing <strong>the</strong> distribution <strong>of</strong> <strong>the</strong> weights for <strong>the</strong> 1283 cases used<br />
in analysis is shown in <strong>the</strong> Appendix 2 Table A2.4.<br />
5.10 Conclusions<br />
Attrition was found to have a substantial effect on participation in SYETP, one <strong>of</strong> <strong>the</strong><br />
chief variables for modelling. Non-response and sampling design were investigated, and<br />
also found to have important effects on modelling, and so <strong>the</strong> weight to account for this<br />
was adopted. Item non-response was considered and <strong>the</strong> casewise deletion approach
197<br />
adopted in order to maintain correct standard errors in estimation. Analytical sample<br />
reduction was found to suitable for <strong>the</strong> analysis when it reflected <strong>the</strong> eligibility criteria for<br />
<strong>the</strong> programme. Weights were constructed to treat attrition, which were combined with<br />
<strong>the</strong> non-response and sampling design weight. The combined weight is to be used for<br />
analysis in later chapters.
198<br />
Table 5.10: Performance <strong>of</strong> attrition weight as measured by unweighted and weighted<br />
pr<strong>of</strong>ile<br />
mean 1984<br />
unadjusted<br />
pr<strong>of</strong>ile<br />
1986<br />
unadjusted<br />
pr<strong>of</strong>ile<br />
1986<br />
weighted<br />
to 1984<br />
pr<strong>of</strong>ile<br />
1984<br />
weighted to<br />
sampled<br />
population<br />
pr<strong>of</strong>ile<br />
1986<br />
adjusted to<br />
sampled<br />
population<br />
Age at 1984 survey 20.13 19.95 20.01 19.88 19.92<br />
Gender=female 0.41 0.41 0.41 0.39 0.39<br />
Work limited by health 0.12 0.11 0.11 0.11 0.11<br />
Aboriginal/Torres Strait 0.03 0.03 0.03 0.04 0.03<br />
Islander<br />
O<strong>the</strong>r ethnic minority 0.08 0.08 0.08 0.08 0.08<br />
Married 1984 0.12 0.10 0.11 0.11 0.11<br />
State interviewed in 1984<br />
Victoria 0.24 0.24 0.24 0.24 0.24<br />
Queensland 0.16 0.13 0.14 0.16 0.16<br />
South Australia/Nor<strong>the</strong>rn 0.11 0.11 0.11 0.10 0.10<br />
Territory<br />
Western Australia/Tasmania 0.12 0.13 0.12 0.11 0.11<br />
New South Wales/<strong>Australian</strong> 0.39 0.38 0.39 0.39 0.39<br />
Capital Territory<br />
Area <strong>of</strong> residence in 1984<br />
Capital city 0.49 0.47 0.49 0.49 0.49<br />
O<strong>the</strong>r city 0.23 0.22 0.23 0.23 0.23<br />
Country town 0.21 0.24 0.21 0.22 0.22<br />
Rural area 0.06 0.07 0.06 0.06 0.06<br />
Highest qualification in 1984<br />
Degree/diploma 0.11 0.11 0.11 0.11 0.11<br />
Apprenticeship 0.08 0.08 0.08 0.07 0.08<br />
O<strong>the</strong>r post-school<br />
0.07 0.07 0.07 0.07 0.07<br />
qualification<br />
Year 12 <strong>of</strong> school 0.16 0.16 0.16 0.15 0.16<br />
Year 11 <strong>of</strong> school 0.13 0.13 0.13 0.13 0.13<br />
Year 10 <strong>of</strong> school 0.31 0.30 0.31 0.31 0.31<br />
Year 9 <strong>of</strong> school 0.14 0.12 0.14 0.14 0.14<br />
Average recorded Pre-June<br />
1984 unemployment 142 0.63 0.61 0.62 0.62 0.61<br />
observations 2368 1687 1687 2368 1687<br />
NOTE 1: for a 0-1 indicator variable, <strong>the</strong> mean gives <strong>the</strong> proportion. For example, <strong>the</strong> mean in <strong>the</strong><br />
population, column 1, for gender=female is <strong>the</strong> proportion <strong>of</strong> females in <strong>the</strong> population.<br />
142 Average recorded Pre-June 1984 unemployment is missing for 27 observations, and has a base <strong>of</strong> 2341.
199<br />
6: Study 4 Weighting to counteract attrition and nonresponse<br />
in ALS<br />
In this chapter <strong>the</strong> combined weights are applied, which were constructed in <strong>the</strong> previous<br />
chapter to repair <strong>the</strong> data for loss due to attrition, non-response and to account for <strong>the</strong><br />
survey design. First <strong>the</strong> weights are applied to <strong>the</strong> Heckman bivariate probit modelling,<br />
and <strong>the</strong> effects discussed. Then <strong>the</strong> weights are applied to <strong>the</strong> propensity score matching<br />
(PSM). The application <strong>of</strong> weights to PSM is more complex, and <strong>the</strong> full protocol is first<br />
presented and <strong>the</strong>n <strong>the</strong> weighted results are discussed. Finally, <strong>the</strong> employment effects<br />
found from <strong>the</strong> Heckman and PSM models in <strong>the</strong> repaired data are contrasted with those<br />
formerly found. The discussion focuses on <strong>the</strong> limitations to each application.<br />
6.1 Results <strong>of</strong> weighting Heckman bivariate probit<br />
The weights accounting for attrition, non-response and survey design were constructed in<br />
<strong>the</strong> previous section. Table 6.1 shows <strong>the</strong> effect <strong>of</strong> applying this sample reduction weight<br />
to <strong>the</strong> bivariate probit <strong>of</strong> employment and SYETP participation. The first column shows<br />
<strong>the</strong> unweighted replication results to better enable comparison while <strong>the</strong> final column<br />
gives <strong>the</strong> results for <strong>the</strong> regression weighted for sample reduction and survey design.<br />
Attention is focused on <strong>the</strong> treatment effect, as indicated by <strong>the</strong> SYETP variable.<br />
An important change brought in by accounting for <strong>the</strong> sample reduction is <strong>the</strong> loss <strong>of</strong><br />
statistical significance for CEP referrals in 1984 in <strong>the</strong> SYETP participation equation. It<br />
is now less significant, although it would pass <strong>the</strong> t test at <strong>the</strong> significance level <strong>of</strong> 10 per<br />
cent. In <strong>the</strong> modelling <strong>of</strong> <strong>the</strong> treatment effect <strong>of</strong> SYETP, this is a key element <strong>of</strong> <strong>the</strong><br />
identifying restriction in <strong>the</strong> bivariate probit. Along with this change, <strong>the</strong> SYETP<br />
treatment dummy loses size and statistical significance in <strong>the</strong> employment equation. The<br />
change introduced by weighting for sample reduction strongly affects <strong>the</strong> interpretation<br />
<strong>of</strong> SYETP treatment effect. Given <strong>the</strong> coincident change in <strong>the</strong> CEP referrals in <strong>the</strong><br />
participation equation, and it’s role in <strong>the</strong> identifying restrictions for <strong>the</strong> bivariate probit,<br />
it is highly likely that adjusting for <strong>the</strong> selection effects due to attrition <strong>of</strong> <strong>the</strong> sample, has
200<br />
reduced <strong>the</strong> need to account for selection into SYETP, with subsequent impacts on <strong>the</strong><br />
treatment effect measured. This potential problem, where attrition seemed related to<br />
SYETP participation, was hinted at in earlier analysis <strong>of</strong> <strong>the</strong> sample reduction effects on<br />
<strong>the</strong> univariate probit <strong>of</strong> SYETP participation. 143<br />
The Wald test <strong>of</strong> significance <strong>of</strong> <strong>the</strong> selection correction also gives a value which in <strong>the</strong><br />
chi square test that correlation is zero, leads to <strong>the</strong> conclusion <strong>of</strong> failing to reject <strong>the</strong> null<br />
hypo<strong>the</strong>sis. This might suggest that a simple probit <strong>of</strong> employment might be an<br />
acceptable modelling format, when <strong>the</strong> attrition has been accounted for with weighting.<br />
The basic univariate probit results, similarly weighted and specified, are shown in <strong>the</strong><br />
Appendix Tables A2.5a and A2.5b. In <strong>the</strong> basic probit analysis <strong>of</strong> employment, <strong>the</strong><br />
employment effect <strong>of</strong> SYETP is positive i.e. marginal effect <strong>of</strong> 13 percentage points, and<br />
statistically significant. This is slightly higher than <strong>the</strong> marginal effect <strong>of</strong> SYETP in <strong>the</strong><br />
bivariate probit, which gives a 10 percentage point gain in employment (see Table 6.9<br />
later).<br />
O<strong>the</strong>r differences also occur in comparison <strong>of</strong> <strong>the</strong> weighted to <strong>the</strong> former unweighted<br />
Heckman bivariate probit results. A number <strong>of</strong> variables formerly statistically<br />
insignificant, are statistically significant in <strong>the</strong> weighted equation; <strong>the</strong>se include: being <strong>of</strong><br />
o<strong>the</strong>r ethnic minority, holding highest qualification <strong>of</strong> up to year 9 schooling, having<br />
good English, no religion and Presbyterian religion. O<strong>the</strong>r variables lose statistical<br />
significance once weighting is applied: holding highest qualification <strong>of</strong> apprentice,<br />
mo<strong>the</strong>r’s occupation <strong>of</strong> plant operative, and negative attitude to women in work. Similar<br />
effects are also evident in <strong>the</strong> participation equation, but are not detailed. Following<br />
Fitzgerald et al. (1998a), <strong>the</strong> interpretation <strong>of</strong> <strong>the</strong>se changes with <strong>the</strong> introduction <strong>of</strong><br />
attrition weighting is that sample reduction lead to bias in <strong>the</strong> estimates. Accounting for<br />
<strong>the</strong> sample reduction with weights reduces <strong>the</strong> bias.<br />
143 See section 5.8 on attrition and <strong>the</strong> discussion <strong>of</strong> <strong>the</strong> Probit Model <strong>of</strong> SYETP and effects <strong>of</strong> sample<br />
reduction.
201<br />
Table 6.1, part A Employment equation from bivariate probit<br />
Replication results Replicated equation<br />
weighted with<br />
combined weight<br />
Model <strong>of</strong> ever employed<br />
in 1986 survey<br />
Dropping cases with attrition, missing<br />
information or not eligible for SYETP<br />
Observations 1283 1283<br />
SYETP 1.596** 0.933<br />
(2.85) (0.79)<br />
Gender=Female -0.397** -0.463**<br />
(-4.04) (-4.48)<br />
married -0.54 0.033<br />
(-0.28) (0.14)<br />
children -0.238 -0.395<br />
(-0.99) (-1.47)<br />
Children*female -1.211** -1.271**<br />
(-3.80) (-3.75)<br />
Spouse employed 1984 0.542* 0.554*<br />
(2.41) (2.25)<br />
Aboriginal/Torres Strait Islander -0.271 -0.344<br />
(-1.18) (-1.38)<br />
O<strong>the</strong>r ethnic minority -0.318 -0.373*<br />
(-1.66) (-1.99)<br />
Victoria -0.060 -0.126<br />
(-0.50) (-1.00)<br />
Queensland 0.028 0.048<br />
(0.20) (0.32)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.305* -0.488**<br />
(-1.97) (-3.03)<br />
Western Australia/Tasmania -0.102 -0.111<br />
(-0.69) (-0.63)<br />
Education school overseas -0.095 -0.208<br />
(-0.33) (-0.62)<br />
Roman Catholic school -0.009 -0.020<br />
(-0.05) (-0.11)<br />
Private school 0.604* 0.623*<br />
(2.08) (1.97)<br />
Degree/diploma 0.512** 0.512**<br />
(3.26) (3.13)<br />
Apprenticeship 0.432* 0.371<br />
(2.11) (1.74)<br />
O<strong>the</strong>r Post-School qualification 0.049 0.096<br />
(0.32) (0.53)<br />
Year 12 <strong>of</strong> school -0.063 -0.075<br />
(-0.41) (-0.44)<br />
Year 11 <strong>of</strong> school 0.356* 0.394*<br />
(2.22) (2.38)<br />
Year 9 <strong>of</strong> school or less -0.222 -0.291*<br />
(-1.60) (-2.03)<br />
Longest job by 1984 none 0.042 -0.033<br />
(0.25) (-0.19)<br />
< 1 year 0.190 0.225<br />
(1.42) (1.64)<br />
2 years 0.252 0.222<br />
(1.57) (1.34)<br />
3 years + 0.657** 0.572**<br />
(4.01) (3.13)<br />
Enter o<strong>the</strong>r govt prog -0.623** -0.730**<br />
(-4.90) (-5.33)
202<br />
duration <strong>of</strong> Pre-June 1984 unemployment -0.473** -0.442**<br />
(-4.20) (-3.55)<br />
Work limited by health -0.305* -0.341*<br />
(-2.50) (-2.53)<br />
O<strong>the</strong>r city before aged 14 -0.217 -0.274<br />
(-1.65) (-1.83)<br />
Country town before aged 14 -0.000 -0.086<br />
-0.00 (-0.59)<br />
Rural area before aged 14 -0.127 -0.412<br />
(-0.66) (-1.91)<br />
Overseas before aged 14 0.494 0.400<br />
(1.37) (1.06)<br />
Number <strong>of</strong> siblings -0.028 -0.035<br />
(-1.51) (-1.68)<br />
English good 0.403 0.430*<br />
(1.91) (2.08)<br />
English poor 1.050** 1.021**<br />
(2.75) (2.58)<br />
Sexist -0.440* -0.395<br />
(-2.35) (-1.93)<br />
Sexist*female 0.463 0.596<br />
(1.25) (1.55)<br />
Fa<strong>the</strong>r not present when resp 14 -0.178 -0.200<br />
(-0.79) (-0.89)<br />
Fa<strong>the</strong>r Labourer 0.135 0.180<br />
(0.56) (0.75)<br />
Fa<strong>the</strong>r Plant operative 0.059 0.152<br />
(0.26) (0.68)<br />
Fa<strong>the</strong>r Sales -0.048 0.05<br />
(-0.18) (0.18)<br />
Fa<strong>the</strong>r Tradesperson -0.213 0.241<br />
(-0.95) (-0.72)<br />
Fa<strong>the</strong>r Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
0.114 0.241<br />
(0.53) (1.12)<br />
Fa<strong>the</strong>r Not employed 0.147 0.087<br />
(0.56) (0.33)<br />
Fa<strong>the</strong>r holds post-school qualification<br />
when resp 14<br />
0.142 0.089<br />
(1.28) (0.75)<br />
Mo<strong>the</strong>r not present when resp 14 -0.335 -0.169<br />
(-1.34) (-0.65)<br />
Mo<strong>the</strong>r Labourer -0.150 -0.149<br />
(-0.64) (-0.60)<br />
Mo<strong>the</strong>r Plant operative -0.576* -0.503<br />
(-2.31) (-1.80)<br />
Mo<strong>the</strong>r Sales -0.391 -0.399<br />
(-1.80) (-1.75)<br />
Mo<strong>the</strong>r Tradesperson -0.229 -0.329<br />
(-0.70) (-0.97)<br />
Mo<strong>the</strong>r Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
.0217 0.066<br />
(0.97) (0.28)<br />
Mo<strong>the</strong>r Not employed -0.087 -0.07<br />
(-0.49) (-0.37)<br />
Mo<strong>the</strong>r post-school qualification when<br />
resp 14<br />
-0.067 -0.012<br />
(-0.52) (-0.08)<br />
Catholic 0.327* 0.430**<br />
(2.56) (3.18)<br />
Presbyterian 0.412 0.504*<br />
(1.92) (2.24)
203<br />
Methodist 0.133 0.261<br />
(0.77) (1.24)<br />
O<strong>the</strong>r Christian -0.102 -0.107<br />
(-0.50) (-0.51)<br />
O<strong>the</strong>r religion -0.045 0.009<br />
(-0.28) (0.05)<br />
No religion 0.279 0.412*<br />
(1.62) (2.30)<br />
rho -0.626 -0.206<br />
Wald test <strong>of</strong> Rho=0 (chi2 (1) statistic) 0.09<br />
Observations 1283 1283<br />
Log likelihood -875.96 -858.74<br />
LR chi 2 (degrees <strong>of</strong> freedom) 144 (118) 379.79 (118) 429.85<br />
Akaike Information Criterion 1.55 1.52<br />
Coefficient is reported with Student’s t statistic in brackets; * significant at 5%; ** significant at 1%<br />
NOTE 1: Results in Column 1 are from Table 4 and Table 5 pages 18-21 Richardson (1998). Base categories: European<br />
ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest qualification year 10 at school, longest<br />
job by 1984 is 1 year, lived mostly in state capital city until respondent aged 14, English is first language, fa<strong>the</strong>r clerical<br />
worker when respondent aged 14, mo<strong>the</strong>r clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
144 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.
204<br />
Table 6.1, Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit<br />
Replication results Replicated<br />
equation with<br />
combined weight<br />
Model <strong>of</strong> SYETP participation, 1986<br />
survey data<br />
Dropping cases with attrition, missing<br />
information or not eligible for SYETP<br />
Observations 1283 1283<br />
Gender=female 0.088 0.067<br />
(0.71) (0.53)<br />
Age at 1984 survey -0.106** -0.081*<br />
(-3.17) (-2.78)<br />
Married 1984 -0.854 -1.008**<br />
(-1.53) (-3.57)<br />
Children 1984 0.464 0.347<br />
(0.78) (0.86)<br />
Children*female -0.295 -0.097<br />
(-0.37) (-0.16)<br />
Spouse employed 1984 0.496 0.689<br />
(0.81) (1.80)<br />
Aboriginal/Torres Strait Islander -0.451 -0.381<br />
(-1.01) (-0.79)<br />
O<strong>the</strong>r ethnic minority 0.081 0.009<br />
(0.33) (0.03)<br />
Victoria 0.112 0.088<br />
(0.72) (0.57)<br />
Queensland -0.280 -0.137<br />
(-1.31) (-0.66)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.157 -0.119<br />
(-0.77) (-0.63)<br />
Western Australia/Tasmania 0.317 0.381*<br />
(1.78) (2.20)<br />
Education school overseas 0.078 0.358<br />
(0.22) (0.89)<br />
Roman Catholic school -0.310 -0.255<br />
(-1.30) (-1.10)<br />
Private school -0.635 -0.910*<br />
(-1.39) (-2.19)<br />
Degree/diploma 0.119 0.001<br />
(0.51) (0.00)<br />
Apprenticeship -0.129 -0.110<br />
(-0.42) (-0.38)<br />
O<strong>the</strong>r Post-School qualification -0.037 0.025<br />
(-0.14) (0.08)<br />
Year 12 <strong>of</strong> school 0.433* 0.402*<br />
(2.46) (2.21)<br />
Year 11 <strong>of</strong> school 0.101 0.187<br />
(0.54) (0.93)<br />
Year 9 <strong>of</strong> school or less -0.73 0.015<br />
(-0.35) (0.07)<br />
Longest job by 1984 none -0.345 -0.294<br />
(-1.35) (-1.08)<br />
< 1 year -0.019 -0.068<br />
(-0.10) (-0.34)<br />
2 years 0.173 0.065<br />
(0.80) (0.29)<br />
3 years + -0.326 -0.491<br />
(-1.26) (-1.99)*
205<br />
CEP referrals 1984 0.143* 0.128<br />
(2.02) (1.78) 145<br />
Proportion <strong>of</strong> time Pre-June 1984 spent in 0.487** 0.335*<br />
unemployment<br />
(2.94) (2.40)<br />
Work limited by health -0.633* -0.673**<br />
(-2.53) (-2.90)<br />
O<strong>the</strong>r city before aged 14 -0.243 -0.395*<br />
(-1.48) (-2.34)<br />
Country town before aged 14 -0.473** -0.521**<br />
(-2.97) (-3.28)<br />
Rural area before aged 14 -0.446 -0.412<br />
(-1.69) (-1.59)<br />
Overseas before aged 14 -0.757 -0.607<br />
(-1.48) (-1.26)<br />
Number <strong>of</strong> siblings -0.011 -0.010<br />
(-0.70) (-0.89)<br />
English good -0.186 -0.162<br />
(-0.73) (-0.61)<br />
English poor -0.592 -0.677<br />
(-1.13) (-1.09)<br />
Sexist 0.318 0.373<br />
(1.23) (1.22)<br />
Sexist*female -0.903 -0.892<br />
(-1.39) (-1.57)<br />
Fa<strong>the</strong>r not present when resp 14 -0.309 -0.241<br />
(-1.12) (-0.82)<br />
Fa<strong>the</strong>r Labourer -0.263 -0.084<br />
(-0.85) (-0.27)<br />
Fa<strong>the</strong>r Plant operative -0.267 -0.141<br />
(-0.96) (-0.51)<br />
Fa<strong>the</strong>r Sales -0.086 -0.217<br />
(-0.26) (-0.62)<br />
Fa<strong>the</strong>r Tradesperson -0.300 -0.280<br />
Fa<strong>the</strong>r Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
(-1.10) (-1.05)<br />
-0.153 -0.182<br />
(-0.59) (-0.70)<br />
Fa<strong>the</strong>r Not employed -0.456 -0.435<br />
(-1.29) (-1.33)<br />
Fa<strong>the</strong>r holds post-school qualification<br />
when resp 14<br />
-0.315* -0.229<br />
(-2.14) (-1.60)<br />
Mo<strong>the</strong>r not present when resp 14 0.480 0.368<br />
(1.49) (1.22)<br />
Mo<strong>the</strong>r Labourer 0.177 -0.034<br />
(0.57) (-0.11)<br />
Mo<strong>the</strong>r Plant operative 0.697* 0.575*<br />
(2.26) (1.81)<br />
Mo<strong>the</strong>r Sales 0.190 0.216<br />
(0.66) (0.77)<br />
Mo<strong>the</strong>r Tradesperson 0.120 0.197<br />
(0.29) (0.46)<br />
Mo<strong>the</strong>r manager/ pr<strong>of</strong> /para-pr<strong>of</strong>essional -0.246 -0.320<br />
(-0.84) (-1.11)<br />
Mo<strong>the</strong>r Not employed 0.041 -0.124<br />
(0.18) (-0.59)<br />
Mo<strong>the</strong>r post-school qualification when 0.266 0.218<br />
145 A t-test probability <strong>of</strong> 0.076, so this is statistically significant for a test set at <strong>the</strong> 10 per cent level <strong>of</strong><br />
significance.
206<br />
resp 14<br />
(1.66) (1.27)<br />
Catholic 0.061 -0.006<br />
(0.37) (-0.03)<br />
Presbyterian 0.322 0.321<br />
(1.34) (1.33)<br />
Methodist 0.015 0.262<br />
(0.06) (1.01)<br />
O<strong>the</strong>r Christian 0.074 0.057<br />
(0.27) (0.21)<br />
O<strong>the</strong>r religion 0.176 0.142<br />
(0.80) (0.58)<br />
No religion 0.138 0.145<br />
(0.67) (0.74)<br />
Observations 1283 1283<br />
Student’s t statistics in brackets; * significant at 5%; ** significant at 1%<br />
Weighted with combined weight for attrition, non-response and survey design.<br />
NOTE 1: Results Column 1 are sourced from Table 4 and Table 5 pages 18-21 Richardson (1998).<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, religion brought up in is Anglican.
207<br />
6.2 Results <strong>of</strong> weighting <strong>the</strong> PSM<br />
Weighting <strong>the</strong> Heckman bivariate probit is a relatively simple task <strong>of</strong> ensuring <strong>the</strong><br />
weights are used in estimating <strong>the</strong> model. In contrast, weighting <strong>the</strong> PSM requires a little<br />
more attention. Frölich et al. (2001) is one <strong>of</strong> <strong>the</strong> few applications <strong>of</strong> PSM found where<br />
weights have been applied to reflect <strong>the</strong> selective nature <strong>of</strong> <strong>the</strong> sample surveyed. As <strong>the</strong><br />
application <strong>of</strong> weights in PSM is not generally treated in <strong>the</strong> literature, details <strong>of</strong> <strong>the</strong><br />
approach are made clear. 146<br />
6.2.1 Weighting protocol<br />
In order to correctly adjust <strong>the</strong> PSM for <strong>the</strong> attrition, two steps need to be taken. The<br />
weighting protocol steps are shown in Table 6.2. The probit model <strong>of</strong> SYETP used to<br />
estimate <strong>the</strong> propensity scores, must be weighted with <strong>the</strong> attrition weight. The common<br />
support for <strong>the</strong> weighted propensity needs to be checked. Once again, <strong>the</strong> minimum and<br />
maximum <strong>of</strong> <strong>the</strong> weighted SYETP propensity scores are used to define <strong>the</strong> common<br />
support. Once <strong>the</strong> matches have been found for <strong>the</strong>se estimated propensity scores, <strong>the</strong>n<br />
<strong>the</strong> final match needs to be re-weighted. The balance <strong>of</strong> <strong>the</strong> treatment group in <strong>the</strong><br />
population is <strong>the</strong> only consideration. The weights that formerly applied to <strong>the</strong> comparison<br />
cases are <strong>the</strong>n discarded. Instead, <strong>the</strong> weight that applies to <strong>the</strong> SYETP case to which<br />
<strong>the</strong>y are matched is donated to <strong>the</strong> comparison case. In this way, <strong>the</strong> final match is<br />
weighted using only <strong>the</strong> weights <strong>of</strong> <strong>the</strong> treated. This is done so that only <strong>the</strong><br />
characteristics <strong>of</strong> <strong>the</strong> treated have influence, as <strong>the</strong> parameter <strong>of</strong> interest is <strong>the</strong> impact <strong>of</strong><br />
treatment on <strong>the</strong> treated. One o<strong>the</strong>r important step also applies when weighting. In<br />
assessing <strong>the</strong> standardized bias for <strong>the</strong> performance <strong>of</strong> <strong>the</strong> match, weighted mean and<br />
variance statistics must be used to construct <strong>the</strong> standardized bias.<br />
Table 6.2 Weighting protocol steps<br />
• Weight <strong>the</strong> propensity probit<br />
• Weight <strong>the</strong> final match, using only <strong>the</strong> weight <strong>of</strong> <strong>the</strong> treated; where comparison<br />
case used with replacement, sum <strong>the</strong> applicable treated case weights.<br />
146 I am grateful to Michael Lechner who in response to an email query made clear <strong>the</strong> steps in <strong>the</strong> protocol.
208<br />
6.2.2 Effects <strong>of</strong> weighting PSM<br />
The first effect <strong>of</strong> <strong>the</strong> weighting was to produce slightly different propensity score<br />
distributions for <strong>the</strong> SYETP and comparison than <strong>the</strong> unweighted estimations. The weight<br />
used is <strong>the</strong> combined weight developed in Chapter 5. The results for <strong>the</strong> estimation <strong>of</strong> <strong>the</strong><br />
weighted probit used to create <strong>the</strong> propensity scores are shown in Table 6.3. The<br />
application <strong>of</strong> <strong>the</strong> weight produces no real change to <strong>the</strong> fit <strong>of</strong> <strong>the</strong> model. The model is<br />
estimated to have fit 147 <strong>of</strong> 92 per cent, which indicates that by this measure it has <strong>the</strong><br />
same predictive power as <strong>the</strong> unweighted model. The Wald chi squared and pseudo R<br />
squared indicate no problems with <strong>the</strong> fit <strong>of</strong> <strong>the</strong> model and are only slightly large than for<br />
<strong>the</strong> unweighted model. The Akaike Information Criterion is very similar to that <strong>of</strong> <strong>the</strong><br />
unweighted model. More coefficients are statistically significant in <strong>the</strong> weighted model<br />
than was <strong>the</strong> case for <strong>the</strong> unweighted model <strong>of</strong> <strong>the</strong> propensity. A number <strong>of</strong> variables<br />
formerly statistically insignificant, are statistically significant in <strong>the</strong> weighted equation;<br />
<strong>the</strong>se include: marital status, partner in employment, <strong>the</strong> location <strong>of</strong> Western<br />
Australia/Tasmania, private schooling, past job held for more than 3 years, and<br />
background living in a non-Capital city. The CEP referral variable which was significant<br />
in <strong>the</strong> unweighted propensity, is not statistically significant in <strong>the</strong> weighted propensity<br />
model. The direction <strong>of</strong> <strong>the</strong> coefficients remains plausible in <strong>the</strong> weighted propensity. It<br />
is reasonable to expect that for SYETP age has a negative influence, additionally negative<br />
influences on SYETP participation are from being married, or having private schooling,<br />
past job held for more than 3 years, poor health, and background living in a non-Capital<br />
city or country town. Longer spells <strong>of</strong> unemployment retain a positive impact on<br />
participation, with fur<strong>the</strong>r positive influences from partner in employment, school-leavers<br />
with highest qualification <strong>of</strong> year 12, living in Western Australia (which evidence in<br />
chapter 2 shows is <strong>the</strong> state had <strong>the</strong> highest usage <strong>of</strong> SYETP).<br />
147 A fitted probability exceeding 0.5 is taken to indicate a predicted response to <strong>the</strong> survey; <strong>the</strong>se predicted<br />
responses are compared to <strong>the</strong> actual participants/non-participants in SYETP to check which cases <strong>the</strong><br />
model correctly predicted.
209<br />
The distribution <strong>of</strong> <strong>the</strong> propensity scores is shown using <strong>the</strong> histogram in Figure 6.4;<br />
kernel density plots <strong>of</strong> <strong>the</strong> propensities before matching are shown in Figure 6.6. The<br />
statistics summarizing <strong>the</strong> distribution <strong>of</strong> <strong>the</strong> new propensity scores are also shown in<br />
Table 6.5 for both <strong>the</strong> comparison group and SYETP participants. The weighted<br />
propensity distributions, when compared to <strong>the</strong> unweighted shown previously in Figures<br />
4.3, 4.4 and Table 4.5, do not appear very much altered by <strong>the</strong> application <strong>of</strong> <strong>the</strong> weights.<br />
However, <strong>the</strong> distributions have had slight changes. The SYETP propensities have grown<br />
slightly at <strong>the</strong> upper end, with <strong>the</strong> largest cases greater in size than when unweighted,<br />
while at <strong>the</strong> lower end <strong>of</strong> <strong>the</strong> distribution <strong>the</strong> smallest cases have shrunk in size. The<br />
median is roughly <strong>the</strong> same as when unweighted, but <strong>the</strong> mean propensity is now slightly<br />
larger. For <strong>the</strong> comparison group <strong>of</strong> nonparticipants, it is also true that <strong>the</strong> tails have been<br />
slightly altered, with <strong>the</strong> largest cases greater in size than when unweighted, and <strong>the</strong><br />
smallest cases smaller than when unweighted. However, <strong>the</strong> mean and median <strong>of</strong> <strong>the</strong><br />
comparisons are largely unaffected by <strong>the</strong> introduction <strong>of</strong> weighting.<br />
Comparison with <strong>the</strong> former unweighted propensity scores shows only very small change.<br />
The propensities still have good properties, in that <strong>the</strong> distributions overlap, and <strong>the</strong> range<br />
<strong>of</strong> <strong>the</strong> distributions gives a probability roughly from zero up to 0.5 for both treated and<br />
comparisons. The kernel density mapping doesn’t look too different to that <strong>of</strong> <strong>the</strong><br />
unweighted propensity model. A result <strong>of</strong> <strong>the</strong> new weighted propensity scores is that now<br />
<strong>the</strong> application <strong>of</strong> <strong>the</strong> common support condition leads to one case being dropped from<br />
<strong>the</strong> SYETP group, <strong>the</strong> SYETP case with <strong>the</strong> largest propensity score value. The<br />
implication is that for that SYETP case, <strong>the</strong>re was no comparison case which had a<br />
propensity score this high. Indeed <strong>the</strong> distributions shown in <strong>the</strong> histogram and kernel<br />
density plot indicate most comparison cases had near-zero propensity values.<br />
The matching results are shown in <strong>the</strong> Table 6.7, where <strong>the</strong> PSM implemented was single<br />
nearest –neighbour –with replacement, within-caliper, with <strong>the</strong> additional protocol <strong>of</strong><br />
weighting <strong>the</strong> propensity with survey weights for attrition. Comparison with <strong>the</strong> former<br />
results shows that <strong>the</strong> weighting has led to much smaller results for <strong>the</strong> employment<br />
effect. The statistical significance <strong>of</strong> <strong>the</strong> employment effects is also low. Although
210<br />
reasonable for smaller calipers, for <strong>the</strong> wider calipers <strong>of</strong> 0.01, 0.02 and 0.05, <strong>the</strong><br />
statistical significance falls <strong>of</strong>f dramatically. This would indicate that applying a wider<br />
caliper was far more consequential in <strong>the</strong> weighted data. Compared to <strong>the</strong> former<br />
unweighted results, <strong>the</strong>re is a much greater variation in <strong>the</strong> size <strong>of</strong> employment effect<br />
estimated. However, none <strong>of</strong> <strong>the</strong> new PSM results for <strong>the</strong> employment effect have<br />
statistical significance at normal test levels. To an extent this would be due to <strong>the</strong> use <strong>of</strong><br />
weights as <strong>the</strong> variance estimate is <strong>the</strong>n less efficient. When <strong>the</strong> weights are used, <strong>the</strong><br />
more conservative estimate <strong>of</strong> <strong>the</strong> variance is used in order to account for <strong>the</strong> fact that it<br />
is assumed <strong>the</strong> missing data follows <strong>the</strong> same pattern as that <strong>of</strong> <strong>the</strong> non-missing data.<br />
The effectiveness <strong>of</strong> <strong>the</strong> matching as assessed by <strong>the</strong> standardized bias measure gives a<br />
slightly poorer performance than for <strong>the</strong> former unweighted results. The weighted<br />
matches have standardized mean biases which are between 11 and 12 in magnitude,<br />
whereas before <strong>the</strong>y were between 9 and 10. This change in size is very slight. As<br />
discussed in <strong>the</strong> preceding chapter, this would indicate that caution should be applied in<br />
accepting <strong>the</strong> performance <strong>of</strong> <strong>the</strong> match. The mean difference in <strong>the</strong> propensity scores<br />
after matching is very small, which would indicate close matching as a small mean<br />
difference is consistent with good matches. The number <strong>of</strong> comparison group<br />
observations used for <strong>the</strong> matching has slightly risen in <strong>the</strong> weighted results, compared to<br />
<strong>the</strong> unweighted, which would indicate slightly less reliance on a few comparison cases.<br />
However, taken as a whole, <strong>the</strong> effectiveness <strong>of</strong> <strong>the</strong> weighted match is not too different<br />
from that <strong>of</strong> <strong>the</strong> unweighted matching.<br />
A graphical summary <strong>of</strong> <strong>the</strong> matched propensity scores for <strong>the</strong> 0.001 caliper is shown in<br />
Figure 6.8. Although a kernel density plot can only give a rough guide, <strong>the</strong> distributions<br />
<strong>of</strong> <strong>the</strong> matched propensity scores overlap very well. It would seem that at <strong>the</strong> upper and<br />
lower extremes, <strong>the</strong> overlap is better, but in <strong>the</strong> middle regions <strong>of</strong> <strong>the</strong> distributions <strong>the</strong>re<br />
are two gaps. In <strong>the</strong> first gap, at lower propensities, <strong>the</strong> untreated lie outside <strong>the</strong> SYETP,<br />
while in <strong>the</strong> second gap, <strong>the</strong> SYETP lie outside <strong>the</strong> untreated. In <strong>the</strong>se gaps, <strong>the</strong><br />
difference in <strong>the</strong> propensity scores <strong>of</strong> <strong>the</strong> treated and comparison would be greater, and in<br />
<strong>the</strong>se gap regions <strong>the</strong> match is slightly poorer.
211<br />
Table 6.3 Weighted probit used to estimate propensity score for propensity score<br />
matching<br />
Variable<br />
Model <strong>of</strong><br />
SYETP<br />
participation<br />
Weighted for<br />
Attrition<br />
non-response<br />
and survey<br />
design<br />
syetp<br />
Gender=female female 0.07<br />
(0.52)<br />
Age at 1984 survey age84 -0.08<br />
(2.87)**<br />
Married 1984 mar84 -1.03<br />
(3.91)**<br />
Children 1984 child84 0.34<br />
(0.85)<br />
Children*female ch84fem -0.09<br />
(0.14)<br />
Spouse employed 1984 spemp84 0.72<br />
(1.96)*<br />
Aboriginal/Torres Strait Islander roatsi -0.37<br />
(0.78)<br />
O<strong>the</strong>r ethnic minority rones -0.00<br />
State interviewed in 1984 (0.00)<br />
Victoria vic 0.09<br />
(0.58)<br />
Queensland qld -0.12<br />
(0.62)<br />
South Australia/Nor<strong>the</strong>rn Territory sant -0.12<br />
(0.63)<br />
Western Australia/Tasmania watas 0.38<br />
(2.18)*<br />
Education school overseas lsos 0.38<br />
(1.04)<br />
Roman Catholic school lsrcs -0.25<br />
(1.09)<br />
Private school lspriv -0.93<br />
Highest qualification in 1984 (2.25)*<br />
Degree/diploma hqdd84 -0.02<br />
(0.11)<br />
Apprenticeship hqtra84 -0.11<br />
(0.38)<br />
O<strong>the</strong>r Post-School qualification hqoth84 0.06<br />
(0.24)<br />
Year 12 <strong>of</strong> school hq12_84 0.39<br />
(2.19)*<br />
Year 11 <strong>of</strong> school hq11_84 0.20<br />
(1.05)<br />
Year 9 <strong>of</strong> school or less hq9_84 -0.00
(0.02)<br />
Longest job by 1984 none longjno -0.32<br />
(1.37)<br />
< 1 year longj0 -0.08<br />
(0.47)<br />
2 years longj2 0.05<br />
(0.25)<br />
3 years + longj3p -0.50<br />
(2.13)*<br />
CEP referrals 1984 cepref84 0.13<br />
(1.75)<br />
duration <strong>of</strong> Pre-June 1984 unemployment upropjn 0.33<br />
(2.40)*<br />
Work limited by health health84 -0.66<br />
Family background (2.91)**<br />
O<strong>the</strong>r city before aged 14 oc14 -0.40<br />
(2.39)*<br />
Country town before aged 14 ct14 -0.51<br />
(3.34)**<br />
Rural area before aged 14 ra14 -0.42<br />
(1.61)<br />
Overseas before aged 14 os14 -0.60<br />
(1.25)<br />
Number <strong>of</strong> siblings nsib2 -0.01<br />
(0.91)<br />
English good egood -0.14<br />
(0.56)<br />
English poor epoor -0.66<br />
(1.06)<br />
Sexist ksink5p 0.35<br />
(1.26)<br />
Sexist*female ks5pfem -0.85<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.54)<br />
Fa<strong>the</strong>r not present when resp 14 fnpres14 -0.22<br />
(0.79)<br />
Labourer flabo -0.06<br />
(0.19)<br />
Plant operative fplan -0.12<br />
(0.46)<br />
Sales fsale -0.23<br />
(0.66)<br />
Tradesperson ftrad -0.27<br />
(1.02)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional fmpp -0.17<br />
(0.66)<br />
Not employed fnemp -0.42<br />
(1.31)<br />
Fa<strong>the</strong>r holds post-school qualification when resp 14 fpsq -0.22<br />
Mo<strong>the</strong>rs occupation when resp. 14 (1.59)<br />
Mo<strong>the</strong>r not present when resp 14 mnpres14 0.37<br />
(1.24)<br />
Labourer mlabo -0.05<br />
(0.16)<br />
Plant operative mplan 0.56<br />
(1.80)<br />
Sales msale 0.21<br />
212
213<br />
(0.76)<br />
Tradesperson mtrad 0.20<br />
(0.46)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional mmpp -0.31<br />
(1.10)<br />
Not employed mnemp -0.12<br />
(0.57)<br />
Mo<strong>the</strong>r post-school qualification when resp 14 mpsq 0.22<br />
Religion brought up in (1.26)<br />
Catholic cath -0.01<br />
(0.05)<br />
Presbyterian pres 0.31<br />
(1.29)<br />
Methodist meth 0.29<br />
(1.19)<br />
O<strong>the</strong>r Christian othx 0.06<br />
(0.21)<br />
O<strong>the</strong>r religion othrel 0.14<br />
(0.56)<br />
No religion norel 0.16<br />
(0.84)<br />
Constant Constant 0.44<br />
(0.65)<br />
Observations Observations 1283<br />
Log likelihood -303.41<br />
Wald chi 2 (59) 148 131.99<br />
Mcfadden’s Pseudo R 2 149 0.1572<br />
Akaike Information Criterion 0.56<br />
Coefficient with robust t statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%<br />
Weighted with <strong>the</strong> combination weights developed in Chapter 5.<br />
148 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
149 This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(M full ) to a model with just <strong>the</strong> intercept (M intercept ). It is defined as R 2 = 1 – (log likelihood M full / log<br />
likelihood M intercept ) . The value <strong>of</strong> Mcfadden’s Pseudo R 2 increases as new variables are added.
214<br />
1<br />
1<br />
Fra<br />
ctio<br />
n<br />
.5<br />
Fra<br />
ctio<br />
n<br />
.5<br />
0<br />
0<br />
0 .5 1<br />
104 observations, unweighted probit<br />
propensity score for treated(SYETP=1)<br />
0 .5 1<br />
1179 observations, unweighted probit<br />
propensity score for control(SYETP=0)<br />
with combined weight<br />
histograms <strong>of</strong> weighted estimated propensity scores prior to matching<br />
Figure 6.4 Propensity histogram with combined weight for attrition, non-response and<br />
survey design
215<br />
Table 6.5 Summary statistics for distribution <strong>of</strong> weighted propensity scores, comparison<br />
group and SYETP<br />
SYETP<br />
Comparison<br />
Percentiles Smallest<br />
1% 0.011806 0.008913<br />
5% 0.025983 0.011806<br />
10% 0.037321 0.014426 Obs 104<br />
25% 0.080427 0.018845 Sum <strong>of</strong> Wgt. 104<br />
50% 0.143067 Mean 0.167949<br />
Largest Std. Dev. 0.119193<br />
75% 0.22751 0.443881<br />
90% 0.30302 0.525781 Variance 0.014207<br />
95% 0.409976 0.565907 Skewness 1.225549<br />
99% 0.565907 0.591027 Kurtosis 4.833109<br />
Percentiles Smallest<br />
1% 0.000322 1.08E-05<br />
5% 0.002353 1.53E-05<br />
10% 0.005012 4.85E-05 Obs 1179<br />
25% 0.016947 5.74E-05 Sum <strong>of</strong> Wgt. 1179<br />
50% 0.047517 Mean 0.074077<br />
Largest Std. Dev. 0.080501<br />
75% 0.104098 0.455594<br />
90% 0.177439 0.473631 Variance 0.00648<br />
95% 0.242457 0.491513 Skewness 2.004687<br />
99% 0.376687 0.572013 Kurtosis 8.090501<br />
Weighted with combined weight for attrition, non-response and survey design.
216<br />
10<br />
Treated propensity scores,<br />
Untreated propensity scores,<br />
5<br />
0<br />
0 .2 .4 .6<br />
attrition weight<br />
kernel densities <strong>of</strong> weightd propensity scores, treated vs untreated<br />
Figure 6.6 Kernel density plot <strong>of</strong> attrition weighted propensity scores, before matching<br />
Note: Epanechnikov kernel.
217<br />
Table 6.7 Matching results, single nearest neighbour with replacement, within caliper,<br />
weighting <strong>the</strong> propensity with combined weights for attrition, non-response and design<br />
Match<br />
with<br />
caliper<br />
width<br />
0.001<br />
Match<br />
with<br />
caliper<br />
width<br />
0.005<br />
Match<br />
with<br />
caliper<br />
width<br />
0.01<br />
Match<br />
with<br />
caliper<br />
width<br />
0.02<br />
Match<br />
with<br />
caliper<br />
width<br />
0.05<br />
Difference in employment 150 for 0.08 0.06 0.02 0.02 0.02<br />
matched treated and comparisons<br />
T statistic 1.57 151 1.19 0.14 0.14 0.14<br />
Number <strong>of</strong> SYETP matched 87 99 102 102 103<br />
Number <strong>of</strong> comparison which satisfy 80 89 91 91 92<br />
<strong>the</strong> caliper rule<br />
Number <strong>of</strong> times used<br />
1 73 80 81 81 82<br />
2 7 8 9 9 9<br />
More than 2 1 1 1 1<br />
Mean difference in propensity score 0.0002 0.0004 0.0005 0.0005 0.0009<br />
between single nearest neighbour<br />
matched treated and comparisons<br />
Standard deviation 0.0002 0.0006 0.0012 0.0012 0.0035<br />
Mean bias 11.04 11.85 11.73 11.73 11.84<br />
Common support – 1 case dropped. Weighting protocol: weight propensity, weight <strong>the</strong> match using <strong>the</strong><br />
treated weight only. The weighted mean bias is calculated using svymean in Stata.<br />
150 Ever employed in 1986 survey.<br />
151 Probability <strong>of</strong> (0.22) for acceptance <strong>of</strong> <strong>the</strong> null hypo<strong>the</strong>sis.
218<br />
6<br />
Treated propensity scores<br />
Untreated propensity scores,<br />
4<br />
2<br />
0<br />
0 .1 .2 .3 .4<br />
combined weight, after matching<br />
kernel densities <strong>of</strong> weighted propensity scores, treated vs untreated<br />
Figure 6.8 Propensity Distribution after matching, 0.001 Caliper<br />
Note: Epanechnikov kernel.
219<br />
6.3 Discussion<br />
The comparison <strong>of</strong> <strong>the</strong> employment effects <strong>of</strong> Heckman versus PSM is shown in<br />
Table 6.9. The former unweighted results are in <strong>the</strong> first columns, to facilitate comparison.<br />
The weighted results are shown in <strong>the</strong> last two columns, shaded.<br />
Comparison between <strong>the</strong> weighted and unweighted results provides an additional<br />
indicator for possible attrition bias. D’Agostino and Rubin (2000) consider <strong>the</strong> issue <strong>of</strong><br />
missing data in item non-response, and note that most propensity score methods are based<br />
on complete data. They suggest that missingness needs to be controlled in PSM. The fall<br />
in <strong>the</strong> employment gain estimated under PSM once attrition is accounted for, is in line<br />
with <strong>the</strong> need to account for <strong>the</strong> missing data that attrition introduces. The significant<br />
differences between <strong>the</strong> results <strong>of</strong> <strong>the</strong> weighted and unweighted models are interpreted as<br />
indicative <strong>of</strong> <strong>the</strong> presence <strong>of</strong> attrition bias. In a general comparison with <strong>the</strong> earlier<br />
unweighted results, it is noticeable that <strong>the</strong> weighted results are much smaller in size,<br />
although still positive, and are also <strong>of</strong> much lower statistical significance. Whereas before<br />
<strong>the</strong> weighting was applied, <strong>the</strong> Heckman and PSM gave very different sizes for <strong>the</strong><br />
employment effect, once weighted to account for attrition <strong>the</strong> effects gained are very<br />
similar in size. This would indicate that under <strong>the</strong> different modelling assumptions <strong>of</strong><br />
PSM and <strong>the</strong> Heckman bivariate probit, similar results for <strong>the</strong> employment effect are<br />
gained once selection due to attrition is accounted for.<br />
The PSM result is significant at <strong>the</strong> 22 per cent level <strong>of</strong> significance, which is outside<br />
normal test bounds. However, <strong>the</strong> variance estimates in <strong>the</strong> weighted estimates are also<br />
far more conservative than for <strong>the</strong> unweighted. 152 If <strong>the</strong> unweighted Heckman probit<br />
results had used <strong>the</strong> more conservative estimate for variance, <strong>the</strong> t statistic for <strong>the</strong><br />
employment effect falls to 1.49. This is <strong>the</strong>n similar in size to <strong>the</strong> weighted PSM, and<br />
also gives about a 22 per cent level <strong>of</strong> significance. Under <strong>the</strong> more conservative variance<br />
estimates, nei<strong>the</strong>r Heckman probit gives statistically significant results at normal test<br />
sizes.<br />
152 See chapter 7 sensitivity analysis for a fuller discussion <strong>of</strong> <strong>the</strong> variance estimates used.
220<br />
As both <strong>the</strong> Heckman bivariate probit and PSM are not statistically significant when<br />
weighted, <strong>the</strong>re would seem to be a substantial loss <strong>of</strong> efficiency in <strong>the</strong> application <strong>of</strong> <strong>the</strong><br />
weights.<br />
Table 6.9 Employment effects <strong>of</strong> Heckman versus PSM<br />
unweighted<br />
Weighted for attrition<br />
Heckman PSM<br />
Heckman PSM<br />
selection bivariate<br />
probit<br />
selection<br />
bivariate<br />
probit<br />
Employment effect 0.26 153 0.18 0.10 154 0.08<br />
t statistic (2.85)** (2.74)** (0.79) (1.50) 155<br />
PSM: one-to-one nearest-neighbour within-caliper (0.001) matching with replacement.<br />
The key assumption where weighting is applied is that <strong>the</strong> non-response has a missing at<br />
random [MAR] process. Violation <strong>of</strong> this assumption would invalidate <strong>the</strong> results. It<br />
should be noted that not treating for non-response, as in <strong>the</strong> unweighted results, adopts<br />
<strong>the</strong> stricter missing completely at random [MCAR] assumption. Thus, weighting <strong>the</strong> data<br />
is a relaxation <strong>of</strong> this assumption. Part <strong>of</strong> <strong>the</strong> MAR assumption for <strong>the</strong> weighting<br />
treatment <strong>of</strong> attrition and non-response is that <strong>the</strong> observed variables used for modelling<br />
can accurately represent <strong>the</strong> non-response and <strong>the</strong>re are no unobserved variables<br />
involved. 156<br />
This assumption is added to <strong>the</strong> modelling assumptions applicable to each <strong>of</strong> <strong>the</strong><br />
Heckman and PSM models, as discussed earlier. In <strong>the</strong> Heckman bivariate probit model,<br />
<strong>the</strong> unobservable information is assumed to be correlated suitably with <strong>the</strong> observed<br />
variables used, in order to solve <strong>the</strong> selection problem. For <strong>the</strong> PSM, CIA assumes that all<br />
observable variables that jointly influence employment and participation are both in <strong>the</strong><br />
data and in <strong>the</strong> model. Unobserved factors, such as lack <strong>of</strong> job search effort, could affect<br />
153 For SYETP in <strong>the</strong> employment equation: dy/dx =0.0325919 and mean for SYETP is 0.081060. The<br />
mean SYETP translates to 8.1 per cent. The marginal effect calculated at <strong>the</strong> mean is <strong>the</strong>n 0.26418. This is<br />
interpreted as a 26 per cent increase in employment. Estimated using <strong>the</strong> mfx command in STATA7.0.<br />
154 For SYETP in <strong>the</strong> employment equation: dy/dx =0.0126509 and mean for SYETP is 0.080747. The<br />
mean SYETP translates to 8.1 per cent. The marginal effect calculated at <strong>the</strong> mean is <strong>the</strong>n 0.10215. This is<br />
interpreted as a 10 per cent increase in employment. Estimated using <strong>the</strong> mfx command in STATA7.0.<br />
155 Probability <strong>of</strong> (0.22).<br />
156 In o<strong>the</strong>r words, MAR implies <strong>the</strong> absence <strong>of</strong> ‘attrition on unobservables’.
221<br />
<strong>the</strong> selection into SYETP and <strong>the</strong> employment outcome jointly, invalidating <strong>the</strong> CIA<br />
(Frölich et al. (2000): 29). A key change to <strong>the</strong> Heckman bivariate probit results was that<br />
<strong>the</strong> test <strong>of</strong> <strong>the</strong> rho correlation factor in <strong>the</strong> selection modelling failed to reject that it was<br />
zero whereas in <strong>the</strong> former analysis this hypo<strong>the</strong>sis was rejected. Fur<strong>the</strong>rmore, in <strong>the</strong><br />
application <strong>of</strong> both methods, and for both <strong>the</strong> weighted and unweighted analyses,<br />
casewise deletion was used for missing data (when <strong>the</strong>re was item non-response) on<br />
modelled variables. This may not be <strong>the</strong> most suitable treatment for missing data, as was<br />
discussed in <strong>the</strong> previous chapter.<br />
All <strong>of</strong> <strong>the</strong>se considerations limit <strong>the</strong> validity <strong>of</strong> <strong>the</strong> results. Sensitivity <strong>of</strong> some <strong>of</strong> <strong>the</strong>se<br />
modelling assumptions is addressed in <strong>the</strong> next chapter.
222<br />
7: Study 5 Sensitivity analysis<br />
In this chapter, <strong>the</strong> specifications modelled so far are fur<strong>the</strong>r investigated. Firstly,<br />
specification <strong>of</strong> <strong>the</strong> Heckman bivariate probit is examined. As a part <strong>of</strong> this, a potential<br />
source <strong>of</strong> confounding for <strong>the</strong> results is initially considered. Then <strong>the</strong> reliance <strong>of</strong> <strong>the</strong><br />
exclusion restriction on <strong>the</strong> variable CEP referrals is explored. These analyses test some<br />
<strong>of</strong> <strong>the</strong> limitations <strong>of</strong> <strong>the</strong> Heckman bivariate probit modelling. The robustness <strong>of</strong> <strong>the</strong><br />
propensity score matching (PSM) is <strong>the</strong>n looked at by varying <strong>the</strong> specification <strong>of</strong> <strong>the</strong><br />
probit used to estimate <strong>the</strong> propensity. This explores <strong>the</strong> importance <strong>of</strong> <strong>the</strong><br />
parameterisation <strong>of</strong> <strong>the</strong> propensity to <strong>the</strong> measured employment outcome.<br />
7.1 Sensitivity <strong>of</strong> Heckman specification<br />
The Heckman modelling in Chapter 6, where weighting for attrition was applied, showed<br />
a big fall in <strong>the</strong> size <strong>of</strong> <strong>the</strong> SYETP coefficient and a dramatic loss <strong>of</strong> it’s statistical<br />
significance. Although <strong>the</strong> size <strong>of</strong> <strong>the</strong> probit coefficient is not directly interpretable, it<br />
does impact on <strong>the</strong> marginal effect calculable from <strong>the</strong> coefficient as a measure <strong>of</strong> <strong>the</strong><br />
employment impact <strong>of</strong> SYETP. Mostly <strong>the</strong> impact <strong>of</strong> weighting was not found to affect<br />
o<strong>the</strong>r coefficients, and where o<strong>the</strong>r coefficients were affected, <strong>the</strong> change was not so<br />
dramatic. The first sensitivity check examines a possible source <strong>of</strong> this effect.<br />
7.1.1 Potential heteroskedasticity<br />
It is <strong>of</strong> interest to briefly consider <strong>the</strong> possibility <strong>of</strong> one <strong>of</strong> <strong>the</strong> modelling assumptions<br />
being violated. One way <strong>of</strong> allowing for this is through considering <strong>the</strong> Huber-White<br />
variance estimation for <strong>the</strong> equation. The Huber-White estimates <strong>of</strong> variance 157 provide a<br />
more conservative estimate that can allow for potential heteroskedasticity <strong>of</strong> undefined<br />
form amongst <strong>the</strong> residuals. Homoskedasticity is <strong>the</strong> standard assumption in modelling,<br />
such that <strong>the</strong> disturbances have <strong>the</strong> same variance. The variance <strong>of</strong> each disturbance term<br />
conditional for <strong>the</strong> chosen values <strong>of</strong> <strong>the</strong> explanatory variables is constant.<br />
157 Also known as <strong>the</strong> ‘sandwich estimator <strong>of</strong> variance’, or ‘robust estimator <strong>of</strong> variance’, key recent<br />
development papers are White (1980) and White (1982).
223<br />
Heteroskedasticity is a violation <strong>of</strong> this assumption. Heteroskedasticity can introduce a<br />
form <strong>of</strong> mis-specification if <strong>the</strong> modelling assumes homoskedasticity. The worth <strong>of</strong><br />
allowing <strong>the</strong> heteroskedasticity to be <strong>of</strong> undefined form is that <strong>the</strong> form it takes is usually<br />
unknown. Fur<strong>the</strong>rmore, as Gujurati (1988) p326 points out, due to data problems <strong>the</strong><br />
investigation <strong>of</strong> heteroskedasticity is <strong>of</strong>ten sheer speculation. The possibility <strong>of</strong><br />
heteroskedasticity in cross-sectional or longitudinal data, involving heterogeneous units is<br />
very high however.<br />
The Huber-White variance estimates can be interpreted as trading <strong>of</strong>f efficient estimation<br />
<strong>of</strong> <strong>the</strong> variance-covariance matrix for estimates that accommodate a degree <strong>of</strong> slack and<br />
so potentially allow for <strong>the</strong> issue <strong>of</strong> minor mis-specification. The point estimates <strong>of</strong><br />
coefficients are unaffected. The replication results applying <strong>the</strong> Huber-White variance<br />
estimation are shown in Column 3 <strong>of</strong> Table 7.1. The first two columns show <strong>the</strong><br />
Richardson (1998) results and <strong>the</strong> earlier replication estimation to better enable<br />
comparison. The t-statistics for some variables are quite different, smaller in size,<br />
reflecting more cautious estimates <strong>of</strong> error. The impact is not even across variables,<br />
which suggests some variables might be more affected by heteroskedasticity than o<strong>the</strong>rs.<br />
The SYETP treatment variable loses statistical significance under <strong>the</strong> Huber-White<br />
variance estimates, with <strong>the</strong> significance level falling to 14 per cent from 1 per cent. The<br />
conventional levels <strong>of</strong> statistical significance are not now met, and <strong>the</strong> 95 per cent<br />
confidence interval for <strong>the</strong> SYETP coefficient now ranges from –0.50 to 3.69 whereas<br />
before it was 0.50 to 2.69. O<strong>the</strong>r variables generally were not affected strongly in this<br />
way.<br />
The results above indicate that <strong>the</strong> assumption <strong>of</strong> homoskedasticity may be violated in <strong>the</strong><br />
data. Given that <strong>the</strong> survey design used a clustered sample, one potential source <strong>of</strong><br />
heteroskedasticity may be due to spatial correlation. 158 Maddala (1983) points out that<br />
little work studies <strong>the</strong> effects <strong>of</strong> violations <strong>of</strong> <strong>the</strong> different assumptions for this type <strong>of</strong><br />
model. For <strong>the</strong> general class <strong>of</strong> limited dependent variable models, he reports that<br />
findings show <strong>the</strong> estimates are nei<strong>the</strong>r consistent nor efficient when <strong>the</strong>re is<br />
158 As <strong>the</strong> clusters are not known in <strong>the</strong> data, this cannot be examined.
224<br />
heteroskedasticity <strong>of</strong> <strong>the</strong> residuals (Maddala (1983): 178). Yatchew and Griliches (1985)<br />
examine <strong>the</strong> effect <strong>of</strong> heteroscedastic errors on <strong>the</strong> probit model. They found that<br />
heteroscedastic errors led <strong>the</strong> usual estimators to be inconsistent but that for small<br />
departures from homoskedasticity <strong>the</strong> parameter vector is simply rescaled if <strong>the</strong> variance<br />
<strong>of</strong> <strong>the</strong> residual is uncorrelated with <strong>the</strong> explanatory variable (Yatchew and Griliches<br />
(1985): 135). They reported that numerical simulations in <strong>the</strong>ir earlier work showed that<br />
biases tended to increase as <strong>the</strong> heteroskedasticity increased, and <strong>the</strong> biases also tended to<br />
be larger when <strong>the</strong> explanatory variable was correlated with <strong>the</strong> heteroskedasticity<br />
(Yatchew and Griliches (1985): 138).<br />
Applying this interpretation to <strong>the</strong>se results, indicates that SYETP may be correlated with<br />
heteroskedasticity <strong>of</strong> <strong>the</strong> variance introducing some bias. One source <strong>of</strong><br />
heteroskedasticity that involves SYETP may be that <strong>the</strong> group <strong>of</strong> SYETP placements are<br />
very heterogeneous units relative to <strong>the</strong> comparison group. The heteroskedasticity would<br />
seem to be small however. The solutions for heteroskedasticity usually rely on specifying<br />
<strong>the</strong> form it takes, which in this case we are not prepared to do. The Huber-White<br />
estimates <strong>of</strong>fer an alternative that roughly adjusts for this problem. These more<br />
conservative estimates <strong>of</strong> variance are applied when weighting is used, and to some<br />
extent this introduces <strong>the</strong> lower statistical significance <strong>of</strong> <strong>the</strong> SYETP employment effect.<br />
O<strong>the</strong>r implications <strong>of</strong> heteroskedasticity, although <strong>the</strong>se may be broad, are not fur<strong>the</strong>r<br />
accounted for here as <strong>the</strong>y are beyond <strong>the</strong> scope <strong>of</strong> this work.
225<br />
Table 7.1, Part A Employment equation from bivariate probit, showing effect <strong>of</strong> standard<br />
error estimate<br />
Employment<br />
equation<br />
from<br />
bivariate<br />
probit <strong>of</strong><br />
Richardson<br />
(1998)<br />
Replication<br />
results<br />
Model <strong>of</strong> ever employed<br />
in 1986 survey<br />
Replication<br />
results<br />
with<br />
Huber-<br />
White<br />
estimates<br />
<strong>of</strong><br />
Standard<br />
error<br />
SYETP 1.590* 1.596** 1.596<br />
(2.45) (2.85) (1.49)<br />
Gender=Female -0.398** -0.397** -0.397**<br />
(-3.99) (-4.04) (-3.74)<br />
Married -0.055 -0.054 -0.054<br />
(-0.28) (-0.28) (-0.26)<br />
Children -0.238 -0.238 -0.238<br />
(-0.98) (-0.99) (-0.93)<br />
Children*female -1.212** -1.211** -1.211**<br />
(-3.79) (-3.80) (-3.79)<br />
Spouse employed 1984 0.542* 0.542* 0.542*<br />
(2.40) (2.41) (2.29)<br />
Aboriginal/Torres Strait Islander -0.272 -0.271 -0.271<br />
(-1.17) (-1.18) (-1.08)<br />
O<strong>the</strong>r ethnic minority -0.318 -0.318 -0.318<br />
(-1.66) (-1.66) (-1.78)<br />
Work limited by health -0.306* -0.305* -0.305*<br />
(-2.43) (-2.50) (-1.96)<br />
State interviewed in 1984<br />
Victoria -0.061 -0.060 -0.060<br />
(-0.51) (-0.50) (-0.53)<br />
Queensland 0.027 0.028 0.028<br />
(0.19) (0.20) (0.19)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.306 -0.305* -0.305<br />
(-1.94) (-1.97) (-1.70)<br />
Western Australia/Tasmania -0.101 -0.102 -0.102<br />
(-0.67) (-0.69) (-0.60)<br />
Education school overseas -0.095 -0.095 -0.095<br />
(-0.33) (-0.33) (-0.31)<br />
Roman Catholic school -0.010 -0.009 -0.009<br />
(-0.06) (-0.05) (-0.05)<br />
Private school 0.603* 0.604* 0.604*<br />
(2.07) (2.08) (2.07)<br />
Highest qualification<br />
Degree/diploma 0.513** 0.512** 0.512**<br />
(3.22) (3.26) (2.95)<br />
Apprenticeship 0.431* 0.432* 0.432*<br />
(2.10) (2.11) (2.27)<br />
O<strong>the</strong>r Post-School qualification 0.049 0.049 0.049
226<br />
(0.32) (0.32) (0.32)<br />
Year 12 <strong>of</strong> school -0.063 -0.063 -0.063<br />
(-0.40) (-0.41) (-0.37)<br />
Year 11 <strong>of</strong> school 0.357* 0.356* 0.356*<br />
(2.18) (2.22) (1.98)<br />
Year 9 <strong>of</strong> school or less -0.222 -0.222 -0.222<br />
(-1.59) (-1.60) (-1.49)<br />
Duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
-0.473** -0.473** -0.473**<br />
(-4.19) (-4.20) (-4.22)<br />
Longest job by 1984 none 0.042 0.042 0.042<br />
(0.25) (0.25) (0.26)<br />
< 1 year 0.190 0.190 0.190<br />
(1.40) (1.42) (1.29)<br />
2 years 0.252 0.252 0.252<br />
(1.56) (1.57) (1.53)<br />
3 years + 0.657** 0.657** 0.657**<br />
(4.00) (4.01) (4.16)<br />
Enter o<strong>the</strong>r govt prog -0.624** -0.623** -0.623**<br />
(-4.74) (-4.90) (-3.78)<br />
Family background<br />
O<strong>the</strong>r city before aged 14 -0.218 -0.217 -0.217<br />
(-1.61) (-1.65) (-1.39)<br />
Country town before aged 14 -0.001 -0.000 -0.000<br />
(-0.01) -0.00 -0.00<br />
Rural area before aged 14 -0.128 -0.127 -0.127<br />
(-0.65) (-0.66) (-0.53)<br />
Overseas before aged 14 0.494 0.494 0.494<br />
(1.36) (1.37) (1.34)<br />
Number <strong>of</strong> siblings -0.028 -0.028 -0.028<br />
(-1.49) (-1.51) (-1.61)<br />
English good 0.403 0.403 0.403*<br />
(1.91) (1.91) (2.12)<br />
English poor 1.050** 1.050** 1.050**<br />
(2.75) (2.75) (2.92)<br />
Sexist -0.440* -0.440* -0.440*<br />
(-2.35) (-2.35) (-2.33)<br />
Sexist*female 0.463 0.463 0.463<br />
(1.25) (1.25) (1.30)<br />
Fa<strong>the</strong>rs occupation when resp. 14<br />
Fa<strong>the</strong>r not present when resp 14 -0.180 -0.178 -0.178<br />
(-0.79) (-0.79) (-0.79)<br />
Labourer 0.134 0.135 0.135<br />
(0.55) (0.56) (0.59)<br />
Plant operative 0.058 0.059 0.059<br />
(0.25) (0.26) (0.28)<br />
Sales -0.049 -0.048 -0.048<br />
(-0.18) (-0.18) (-0.19)<br />
Tradesperson -0.214 -0.213 -0.213<br />
(-0.95) (-0.95) (-0.93)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.114 0.114 0.114<br />
(0.52) (0.53) (0.57)<br />
Not employed 0.146 0.147 0.147<br />
(0.55) (0.56) (0.58)<br />
Fa<strong>the</strong>r holds post-school qualification<br />
when resp 14<br />
0.141 0.142 0.142
227<br />
(1.26) (1.28) (1.16)<br />
Mo<strong>the</strong>rs occupation when resp. 14<br />
Mo<strong>the</strong>r not present when resp 14 -0.335 -0.335 -0.335<br />
(-1.33) (-1.34) (-1.27)<br />
Labourer -0.151 -0.150 -0.150<br />
(-0.64) (-0.64) (-0.67)<br />
Plant operative -0.576* -0.576* -0.576*<br />
(-2.30) (-2.31) (-2.29)<br />
Sales -0.392 -0.391 -0.391<br />
(-1.80) (-1.80) (-1.87)<br />
Tradesperson -0.229 -0.229 -0.229<br />
(-0.70) (-0.70) (-0.74)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.217 .0217 .0217<br />
(0.97) (0.97) (1.02)<br />
Not employed -0.087 -0.087 -0.087<br />
(-0.49) (-0.49) (-0.52)<br />
Mo<strong>the</strong>r post-school qualification when -0.067 -0.067 -0.067<br />
resp 14<br />
(-0.52) (-0.52) (-0.52)<br />
Religion brought up in<br />
Catholic 0.327* 0.327* 0.327*<br />
(2.52) (2.56) (2.29)<br />
Presbyterian 0.413 0.412 0.412<br />
(1.88) (1.92) (1.63)<br />
Methodist 0.133 0.133 0.133<br />
(0.77) (0.77) (0.72)<br />
O<strong>the</strong>r Christian -0.102 -0.102 -0.102<br />
(-0.50) (-0.50) (-0.53)<br />
O<strong>the</strong>r religion -0.045 -0.045 -0.045<br />
(-0.28) (-0.28) (-0.29)<br />
No religion 0.280 0.279 0.279<br />
(1.58) (1.62) (1.32)<br />
rho -0.622 -0.626 -0.626<br />
Observations 1283 1283 1283<br />
Log likelihood -875.96 -875.96 -875.96<br />
Coefficient is reported with Student’s t statistic in brackets; * significant at 5%; ** significant at 1%<br />
NOTE: results Column 1 are from Table 4 and Table 5 pages 18-21 Richardson (1998). Base categories:<br />
European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest qualification<br />
year 10 at school, longest job by 1984 is 1 year, lived mostly in state capital city until respondent aged 14,<br />
English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r clerical worker when<br />
respondent aged 14, religion brought up in is Anglican.
228<br />
Table 7.1, Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit, showing effect<br />
<strong>of</strong> standard error estimate<br />
Selection<br />
equation <strong>of</strong><br />
<strong>the</strong> bivariate<br />
probit<br />
analysis by<br />
Richardson<br />
(1998)<br />
Replication<br />
results<br />
Replication<br />
results<br />
with<br />
Huber/White<br />
estimates <strong>of</strong><br />
Standard<br />
error<br />
Model <strong>of</strong> SYETP participation, 1986 survey<br />
data<br />
Age at 1984 survey -0.107** -0.106** -0.106**<br />
(-3.16) (-3.17) (-3.38)<br />
Gender=female 0.088 0.088 0.088<br />
(0.71) (0.71) (0.77)<br />
Married 1984 -0.855 -0.854 -0.854**<br />
(-1.52) (-1.53) (-2.16)<br />
Children 1984 0.465 0.464 0.464<br />
(0.78) (0.78) (1.21)<br />
Children*female -0.296 -0.295 -0.295<br />
(-0.37) (-0.37) (-0.52)<br />
Spouse employed 1984 0.498 0.496 0.496<br />
(0.81) (0.81) (1.26)<br />
Aboriginal/Torres Strait Islander -0.451 -0.451 -0.451<br />
(-1.01) (-1.01) (-1.03)<br />
O<strong>the</strong>r ethnic minority 0.081 0.081 0.081<br />
(0.33) (0.33) (0.33)<br />
Work limited by health -0.633* -0.633* -0.633**<br />
(-2.53) (-2.53) (-2.80)<br />
State interviewed in 1984<br />
Victoria 0.112 0.112 0.112<br />
(0.72) (0.72) (0.77)<br />
Queensland -0.279 -0.280 -0.280<br />
(-1.30) (-1.31) (-1.25)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.157 -0.157 -0.157<br />
(-0.77) (-0.77) (-0.82)<br />
Western Australia/Tasmania 0.317 0.317 0.317<br />
(1.78) (1.78) (1.93)<br />
CEP referrals 1984 0.144* 0.143* 0.143<br />
(1.97) (2.02) (1.52)<br />
Education school overseas 0.078 0.078 0.078<br />
(0.22) (0.22) (0.23)<br />
Roman Catholic school -0.310 -0.310 -0.310<br />
(-1.30) (-1.30) (-1.39)<br />
Private school -0.636 -0.635 -0.635<br />
(-1.38) (-1.39) (-1.53)<br />
Highest qualification in 1984<br />
Degree/diploma 0.120 0.119 0.119<br />
(0.51) (0.51) (0.48)<br />
Apprenticeship -0.129 -0.129 -0.129<br />
(-0.42) (-0.42) (-0.46)
229<br />
O<strong>the</strong>r Post-School qualification -0.036 -0.037 -0.037<br />
(-0.14) (-0.14) (-0.13)<br />
Year 12 <strong>of</strong> school 0.433* 0.433* 0.433*<br />
(2.46) (2.46) (2.54)<br />
Year 11 <strong>of</strong> school 0.101 0.101 0.101<br />
(0.53) (0.54) (0.47)<br />
Year 9 <strong>of</strong> school or less -0.074 -0.73 -0.73<br />
(-0.35) (-0.35) (-0.36)<br />
Duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
0.487** 0.487** 0.487**<br />
(2.92) (2.94) (3.14)<br />
Longest job by 1984 none -0.348 -0.345 -0.345<br />
(-1.34) (-1.35) (-1.32)<br />
< 1 year -0.020 -0.019 -0.019<br />
(-0.11) (-0.10) (-0.11)<br />
2 years 0.173 0.173 0.173<br />
(0.80) (0.80) (0.83)<br />
3 years + -0.326 -0.326 -0.326<br />
(-1.26) (-1.26) (-1.37)<br />
Family background<br />
O<strong>the</strong>r city before aged 14 -0.244 -0.243 -0.243<br />
(-1.48) (-1.48) (-1.54)<br />
Country town before aged 14 -0.473** -0.473** -0.473**<br />
(-2.94) (-2.97) (-2.73)<br />
Rural area before aged 14 -0.466 -0.446 -0.446<br />
(-1.69) (-1.69) (-1.74)<br />
Overseas before aged 14 -0.757 -0.757 -0.757<br />
(-1.48) (-1.48) (-1.55)<br />
Number <strong>of</strong> siblings -0.011 -0.011 -0.011<br />
(-0.70) (-0.70) (-0.97)<br />
English good -0.185 -0.186 -0.186<br />
(-0.72) (-0.73) (-0.68)<br />
English poor -0.591 -0.592 -0.592<br />
(-1.13) (-1.13) (-1.04)<br />
Sexist 0.317 0.318 0.318<br />
(1.22) (1.23) (1.16)<br />
Sexist*female -0.903 -0.903 -0.903<br />
(-1.38) (-1.39) (-1.58)<br />
Fa<strong>the</strong>rs occupation when resp. 14<br />
Fa<strong>the</strong>r not present when resp 14 -0.309 -0.309 -0.309<br />
(-1.11) (-1.12) (-1.05)<br />
Labourer -0.263 -0.263 -0.263<br />
(-0.84) (-0.85) (-0.77)<br />
Plant operative -0.267 -0.267 -0.267<br />
(-0.96) (-0.96) (-0.98)<br />
Sales -0.086 -0.086 -0.086<br />
(-0.26) (-0.26) (-0.27)<br />
Tradesperson -0.300 -0.300 -0.300<br />
(-1.10) (-1.10) (-1.14)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.153 -0.153 -0.153<br />
(-0.59) (-0.59) (-0.57)<br />
Not employed -0.457 -0.456 -0.456<br />
(-1.29) (-1.29) (-1.45)<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14<br />
-0.315* -0.315* -0.315*
230<br />
(-2.14) (-2.14) (-2.41)<br />
Mo<strong>the</strong>rs occupation when resp. 14<br />
Mo<strong>the</strong>r not present when resp 14 0.480 0.480 0.480<br />
(1.49) (1.49) (1.55)<br />
Labourer 0.176 0.177 0.177<br />
(0.57) (0.57) (0.59)<br />
Plant operative 0.697* 0.697* 0.697*<br />
(2.26) (2.26) (2.30)<br />
Sales 0.190 0.190 0.190<br />
(0.66) (0.66) (0.71)<br />
Tradesperson 0.119 0.120 0.120<br />
(0.28) (0.29) (0.32)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.247 -0.246 -0.246<br />
(-0.85) (-0.84) (-0.90)<br />
Not employed 0.041 0.041 0.041<br />
Mo<strong>the</strong>r post-school qualification<br />
when resp 14<br />
(0.17) (0.18) (0.20)<br />
0.266 0.266 0.266<br />
(1.66) (1.66) (1.70)<br />
Religion brought up in<br />
Catholic 0.061 0.061 0.061<br />
(0.38) (0.37) (0.39)<br />
Presbyterian 0.322 0.322 0.322<br />
(1.34) (1.34) (1.46)<br />
Methodist 0.017 0.015 0.015<br />
(0.06) (0.06) (0.05)<br />
O<strong>the</strong>r Christian 0.075 0.074 0.074<br />
(0.27) (0.27) (0.27)<br />
O<strong>the</strong>r religion 0.176 0.176 0.176<br />
(0.80) (0.80) (0.77)<br />
No religion 0.138 0.138 0.138<br />
(0.66) (0.67) (0.69)<br />
Observations 1283 1283 1283<br />
Student’s t statistics in brackets; * significant at 5%; ** significant at 1%<br />
NOTE 1: results Column 1 are sourced from Table 4 and Table 5 pages 18-21 Richardson (1998).<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, religion brought up in is Anglican.
231<br />
7.1.2 Exclusion restriction in <strong>the</strong> Heckman model<br />
The weighted results for <strong>the</strong> Heckman model are very different to those <strong>of</strong> <strong>the</strong><br />
unweighted, especially for <strong>the</strong> size and significance <strong>of</strong> <strong>the</strong> employment effect on SYETP.<br />
As section 5.6.1.1 found, <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> SYETP treatment varied amongst those lost to<br />
selection and attrition, and so <strong>the</strong> weights that treat this would rebalance to restore <strong>the</strong><br />
SYETP pr<strong>of</strong>ile. Thus <strong>the</strong> weighted specifications are likely to differ from <strong>the</strong> unweighted<br />
as a result <strong>of</strong> this. Some fur<strong>the</strong>r potential sources for changes to <strong>the</strong> employment effect<br />
are now investigated. As well, <strong>the</strong> robustness <strong>of</strong> <strong>the</strong> Heckman bivariate probit modelling<br />
is explored with sensitivity analysis <strong>of</strong> <strong>the</strong> specification.<br />
One <strong>of</strong> <strong>the</strong> chief considerations <strong>of</strong> <strong>the</strong> Heckman bivariate probit modelling is <strong>the</strong><br />
exclusion restriction. As was noted, in <strong>the</strong> weighted Heckman results <strong>the</strong> key variable for<br />
<strong>the</strong> restriction has no independent effect in <strong>the</strong> participation modelling, whereas in <strong>the</strong><br />
unweighted model it had been strongly statistically significant and large in size. In<br />
addition, in <strong>the</strong> weighted modelling, <strong>the</strong> hypo<strong>the</strong>sis tests <strong>of</strong> <strong>the</strong> correlation due to<br />
selection fail to reject <strong>the</strong> null hypo<strong>the</strong>sis that no selection is present. The sensitivity<br />
analysis here examines <strong>the</strong> role <strong>of</strong> <strong>the</strong> variables chosen for <strong>the</strong> exclusion restriction, and<br />
<strong>the</strong> validity <strong>of</strong> <strong>the</strong> restriction in <strong>the</strong> weighted and unweighted Heckman model.<br />
The two key variables forming <strong>the</strong> exclusion restriction are age and number <strong>of</strong> CES<br />
referrals to <strong>the</strong> CEP programme. Tables A2.6, A2.7, A2.8 in <strong>the</strong> appendix show <strong>the</strong> effect<br />
on <strong>the</strong> model <strong>of</strong> changing <strong>the</strong> specification by including <strong>the</strong>se in <strong>the</strong> employment<br />
equation. The specification remains identical to that previously estimated, but for <strong>the</strong><br />
change. Table 7.2 shows a summary <strong>of</strong> key aspects <strong>of</strong> <strong>the</strong> changes to <strong>the</strong> specification.<br />
The first two columns show <strong>the</strong> original results, unweighted and weighted. The table is<br />
<strong>the</strong>n formed <strong>of</strong> a fur<strong>the</strong>r three panels, with two columns in each panel, unweighted and<br />
weighted, where <strong>the</strong> exclusion restriction is tested by inclusion <strong>of</strong> <strong>the</strong> variables in <strong>the</strong><br />
employment equation. The first panel <strong>of</strong> new results shows inclusion <strong>of</strong> age in <strong>the</strong><br />
employment equation, <strong>the</strong> second panel gives results when CEP referrals are included in
232<br />
<strong>the</strong> employment equation, and <strong>the</strong> final panel considers <strong>the</strong> effect <strong>of</strong> including both age<br />
and CEP referrals in <strong>the</strong> employment equation as well as <strong>the</strong> participation equation.<br />
Table A2.6 in <strong>the</strong> appendix shows <strong>the</strong> effect on <strong>the</strong> weighted model <strong>of</strong> changing <strong>the</strong><br />
specification by including age in <strong>the</strong> employment equation. The first column in <strong>the</strong> first<br />
new results panels in Table 7.2 gives <strong>the</strong> lack <strong>of</strong> results for <strong>the</strong> unweighted equation – it<br />
proved impossible to estimate <strong>the</strong> unweighted equation with age in <strong>the</strong> employment<br />
equation. This is interpreted as indicative that because <strong>the</strong> selection arising from data loss<br />
is unaccounted for, <strong>the</strong>re is misspecification when age is included in <strong>the</strong> unweighted<br />
equation. In <strong>the</strong> second column <strong>of</strong> this panel is shown <strong>the</strong> weighted Heckman bivariate<br />
probit results where age is in <strong>the</strong> employment equation and participation equation, but<br />
CEP referrals remains as an exclusion restriction. The weighted specification is estimable.<br />
However, age does not have a reasonably sized coefficient nor is it statistically significant<br />
in <strong>the</strong> employment equation. The size <strong>of</strong> <strong>the</strong> SYETP coefficient is slightly affected by<br />
this alteration to <strong>the</strong> specification – with <strong>the</strong> size increased, although remaining<br />
statistically insignificant at conventional test levels.<br />
Now, <strong>the</strong> same form <strong>of</strong> specification change is shown in panel 2 <strong>of</strong> Table 7.2, with CEP<br />
referrals dropped from <strong>the</strong> exclusion restriction and included in <strong>the</strong> employment equation<br />
as well as <strong>the</strong> participation equation. Again, Table A2.7 in <strong>the</strong> appendix shows <strong>the</strong> effect<br />
on <strong>the</strong> weighted model <strong>of</strong> changing <strong>the</strong> specification by including CEP referrals in <strong>the</strong><br />
employment equation. The first column <strong>of</strong> <strong>the</strong> second panel again shows no results for<br />
<strong>the</strong> unweighted model, as again this specification proved impossible to estimate for <strong>the</strong><br />
unweighted data. In <strong>the</strong> second column is shown <strong>the</strong> weighted Heckman results where<br />
CEP referrals are in <strong>the</strong> employment equation and participation equation, but age remains<br />
as an exclusion restriction entered only in <strong>the</strong> participation equation. The consequences<br />
are similar to those found for age – <strong>the</strong> equation is estimable, and <strong>the</strong> coefficient <strong>of</strong> CEP<br />
referrals is not <strong>of</strong> reasonable size and it is not statistically significant, but <strong>the</strong> change to<br />
specification raises <strong>the</strong> size <strong>of</strong> <strong>the</strong> SYETP coefficient although <strong>the</strong>re is no gain in<br />
statistical significance.
233<br />
Finally, <strong>the</strong> third panel <strong>of</strong> new results in Table 7.2 considers reintroduction <strong>of</strong> both age<br />
and CEP referrals to <strong>the</strong> employment equation, with no exclusion restriction operation.<br />
Once more, Table A2.8 in <strong>the</strong> appendix shows <strong>the</strong> effect on <strong>the</strong> weighted model <strong>of</strong><br />
changing <strong>the</strong> specification by including both age and CEP referrals in <strong>the</strong> employment<br />
equation. The results are very similar to those already discussed. The unweighted<br />
equation is inestimable in this specification, and <strong>the</strong> same interpretation is given as before.<br />
Again, although introducing <strong>the</strong>se variables to <strong>the</strong> employment specification raises <strong>the</strong><br />
size <strong>of</strong> <strong>the</strong> SYETP coefficient it does not gain statistical significance, and nei<strong>the</strong>r variable<br />
age or CEP referrals is <strong>of</strong> significant size or statistical significance.<br />
Overall, <strong>the</strong> results shown for varying <strong>the</strong> specification <strong>of</strong> <strong>the</strong> exclusion restriction in both<br />
<strong>the</strong> weighted and unweighted data indicate a number <strong>of</strong> helpful conclusions.<br />
In <strong>the</strong> weighted data, <strong>the</strong> alteration <strong>of</strong> <strong>the</strong> exclusion restriction shows that age was not<br />
particularly important to <strong>the</strong> exclusion restriction, but that it had no independent effect in<br />
<strong>the</strong> employment equation specification. However, including CEP referrals is likely to be a<br />
mis-specification as it has no independent effect in <strong>the</strong> equation and also dramatically<br />
reduces <strong>the</strong> SYETP coefficient. On <strong>the</strong> o<strong>the</strong>r hand, in <strong>the</strong> unweighted data, age and CEP<br />
referrals provided a useful exclusion restriction, and accounted for selection in <strong>the</strong> data.<br />
Once selection due to sample reduction from non-response, sample attrition, and sample<br />
design effects was accounted for using <strong>the</strong> weights, <strong>the</strong> selection problem in <strong>the</strong> data<br />
appears to have been reduced. The hypo<strong>the</strong>sis tests indicate that <strong>the</strong> weighted equation<br />
does not have evidence for selection. It is fur<strong>the</strong>r concluded that <strong>the</strong> original Heckman<br />
specification was not greatly improved by any <strong>of</strong> <strong>the</strong>se specification changes, as<br />
evidenced by <strong>the</strong> only slightly lower AIC. The goodness <strong>of</strong> fit assessment shows not<br />
strong basis for preferring one model over <strong>the</strong> o<strong>the</strong>r. The SYETP coefficient increased in<br />
size slightly but <strong>the</strong>re was no improvement in statistical significance.<br />
There is a suggestion that <strong>the</strong> unweighted model is particularly sensitive to <strong>the</strong> variables<br />
in <strong>the</strong> exclusion restriction, since only when <strong>the</strong>y are excluded is <strong>the</strong> equation estimable.<br />
The weighted equation is more robust to changes in <strong>the</strong>se variables, but it is posited that
234<br />
this is mostly because <strong>the</strong>re is no substantive selection to account for once weighted, as<br />
evidenced by <strong>the</strong> hypo<strong>the</strong>sis tests. This interpretation suggests that <strong>the</strong> sample attrition,<br />
which affected <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> characteristics including SYETP, may have acted as a form<br />
<strong>of</strong> selection that <strong>the</strong> bivariate probit incorporated via selection into <strong>the</strong> program. As<br />
section 5.6.1.1 showed, SYETP participation was associated with <strong>the</strong> data loss. As <strong>the</strong><br />
bivariate probit historically has been associated with applications where <strong>the</strong> data loss<br />
from non-response has been accounted for in this way, this is not unreasonable. The<br />
serious difficulties in obtaining an estimable specification for <strong>the</strong> bivariate probit are<br />
highlighted by <strong>the</strong> failure <strong>of</strong> estimation for each <strong>of</strong> <strong>the</strong> alternative unweighted<br />
specifications put forward. This indicates that <strong>the</strong> search for a suitable estimable<br />
specification amongst plausible alternatives can be arduous. This can be a difficulty for<br />
instrumental variables models, <strong>of</strong> which <strong>the</strong> Heckman bivariate probit is an example due<br />
to <strong>the</strong> use <strong>of</strong> <strong>the</strong> exclusion restriction.
235<br />
Table 7.2 summary <strong>of</strong> changes to exclusion restriction <strong>of</strong> Heckman specification<br />
Model <strong>of</strong> ever<br />
employed<br />
in 1986 survey<br />
Original results Panel 1 Panel 2 Panel3<br />
Add age to employment Add CEP referrals to Add both age and CEP<br />
equation<br />
employment<br />
referrals to employment<br />
Replication<br />
results<br />
Replicated<br />
equation<br />
weighted<br />
for all<br />
attrition<br />
No weights<br />
weighted<br />
for all<br />
attrition<br />
equation<br />
No weights<br />
weighted<br />
for all<br />
attrition<br />
equation<br />
No weights<br />
weighted<br />
for all<br />
attrition<br />
Observations 1283 1283 1283 1283 1283 1283 1283 1283<br />
SYETP 1.596** 0.933 Not 1.06 Not 0.32 Not 0.38<br />
estimable<br />
estimable<br />
estimable<br />
(2.85) (0.79) (0.77) (0.23) (0.25)<br />
Rho=-1 Rho=-1 Rho=-1<br />
Age coefficient<br />
0.01 0.00<br />
in employment<br />
equation<br />
(0.30) (0.17)<br />
CEP referral<br />
-0.03 -0.03<br />
coefficient in<br />
employment<br />
equation<br />
(-0.39) (0.40)<br />
Estimated rho -0.75 -0.21 -0.28 0.14 0.11<br />
Wald test <strong>of</strong> 0.34 0.08 0.11 0.41 0.02<br />
Rho=0 (chi2<br />
(1) statistic)<br />
Log likelihood -875.96 -858.74 -858.68 -858.64 -858.55<br />
LR chi 2 (df) 159 (118)<br />
396.79<br />
(118)<br />
429.85<br />
(119)<br />
443.03<br />
(119)<br />
436.99<br />
(120)<br />
457.81<br />
Akaike 1.55 1.52 1.52 1.52 1.53<br />
Information<br />
Criterion<br />
Coefficient is reported with absolute value <strong>of</strong> t statistic in brackets.<br />
159 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom (df) <strong>of</strong><br />
this chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.
236<br />
7.2 Varying <strong>the</strong> Propensity Score specification, effects on <strong>the</strong> match<br />
The specification <strong>of</strong> <strong>the</strong> propensity score estimated by probit is vital to <strong>the</strong> propensity<br />
score matching outcome. Some variation to this specification is now examined. In this<br />
section, <strong>the</strong> variable CEP referrals is excluded from <strong>the</strong> model for <strong>the</strong> propensity score,<br />
and <strong>the</strong> effect on <strong>the</strong> matching outcome examined. Additionally, a more concise<br />
specification for <strong>the</strong> probit is estimated, using only those variables thought to have most<br />
plausible economic influence on SYETP participation and employment. The combined<br />
weighting for sample reduction due to attrition and non-response and survey design is<br />
applied in all models, with <strong>the</strong> weighting protocol for <strong>the</strong> propensity score matching as<br />
shown in <strong>the</strong> earlier chapter.<br />
7.2.1 Propensity score matching and <strong>the</strong> effect <strong>of</strong> excluding CEP referrals<br />
Recall that <strong>the</strong> originally estimated propensity specification was designed to be as similar<br />
as possible to <strong>the</strong> Heckman modelling <strong>of</strong> Richardson (1998), as <strong>the</strong> aim had been to<br />
discover what <strong>the</strong> result would have been if propensity score matching had been used. It<br />
was earlier noted, that at least one variable in <strong>the</strong> propensity score <strong>the</strong>n estimated might<br />
not satisfy <strong>the</strong> requirement that it influence both employment and participation in SYETP:<br />
namely CEP referrals.<br />
Table 7.3 shows <strong>the</strong> full results for <strong>the</strong> weighted probit when <strong>the</strong> variable CEP referrals is<br />
excluded from <strong>the</strong> model. The first column shows <strong>the</strong> former results with <strong>the</strong> original<br />
specification, while <strong>the</strong> second column is for <strong>the</strong> specification when CEP referrals is<br />
excluded. The predicted fit <strong>of</strong> <strong>the</strong> model, calculated as before 160 , shows 85 per cent <strong>of</strong><br />
cases are predicted correctly. Thus <strong>the</strong> predictive power <strong>of</strong> <strong>the</strong> model remains acceptable,<br />
although lower than for <strong>the</strong> former weighted specification. O<strong>the</strong>r scalar fit measures<br />
presented all indicate <strong>the</strong> new model to be similarly acceptable as <strong>the</strong> former model. Most<br />
changes to <strong>the</strong> coefficients and t statistics are very slight. Only <strong>the</strong> variable private<br />
160 A fitted probability exceeding 0.5 is taken to indicate a predicted response to <strong>the</strong> survey; <strong>the</strong>se predicted<br />
responses are compared to <strong>the</strong> actual participants/non-participants in SYETP to check which cases <strong>the</strong><br />
model correctly predicted.
237<br />
schooling that was statistically significant in <strong>the</strong> former model changes with a move to<br />
non-significance at conventional levels when CEP referrals is excluded.<br />
The distribution <strong>of</strong> <strong>the</strong> estimated propensity score prior to matching is shown in Table 7.4.<br />
It is useful to compare this distribution to that found for <strong>the</strong> weighted results using <strong>the</strong><br />
original specification (see Table 6.5 in <strong>the</strong> previous chapter). The distribution <strong>of</strong> <strong>the</strong><br />
estimated propensity for those in <strong>the</strong> SYETP treatment group appears only a little<br />
changed by <strong>the</strong> altered specification. In <strong>the</strong> SYETP treatment group, <strong>the</strong> limits <strong>of</strong> <strong>the</strong><br />
distribution are very similar to those for <strong>the</strong> former specification, with only very small<br />
adjustments to <strong>the</strong> size <strong>of</strong> <strong>the</strong> largest and smallest observed propensities. The mean<br />
propensity for <strong>the</strong> SYETP treatment group is also very similar; however <strong>the</strong> median had<br />
fallen in size from roughly 0.143 to 0.135, which would indicate a slight shift down in <strong>the</strong><br />
central peak <strong>of</strong> <strong>the</strong> distribution. The standard deviation <strong>of</strong> <strong>the</strong> propensity for <strong>the</strong> SYETP<br />
treatment group is hardly changed by <strong>the</strong> new specification. The propensity score<br />
distribution for <strong>the</strong> comparison group is also only changed slightly overall. The greatest<br />
effect appears for <strong>the</strong> lower tail <strong>of</strong> <strong>the</strong> distribution for <strong>the</strong> comparisons, where <strong>the</strong><br />
smallest estimated propensities have greater size in <strong>the</strong> altered specification. The mean<br />
propensity, median and standard deviation for <strong>the</strong> comparisons remains very similar to<br />
that found for <strong>the</strong> former specification.
238<br />
Table 7.3 Weighted Probit used to estimate propensity score for propensity score<br />
matching, exclude CEP referrals<br />
Coefficient, (t statistic) Original specification Original specification,<br />
Don’t include cep referrals<br />
Model <strong>of</strong> SYETP participation Model <strong>of</strong> SYETP participation<br />
Gender=female 0.07 0.06<br />
(0.52) (0.51)<br />
Age at 1984 survey -0.08 -0.08<br />
(2.87)** (2.90)**<br />
Married 1984 -1.03 -1.04<br />
(3.91)** (4.02)**<br />
Children 1984 0.34 0.32<br />
(0.85) (0.81)<br />
Children*female -0.09 -0.09<br />
(0.14) (0.16)<br />
Spouse employed 1984 0.72 0.74<br />
(1.96)* (2.04)*<br />
Aboriginal/Torres Strait Islander -0.37 -0.35<br />
(0.78) (0.78)<br />
O<strong>the</strong>r ethnic minority -0.00 0.00<br />
State interviewed in 1984 (0.00) (0.02)<br />
Victoria 0.09 0.07<br />
(0.58) (0.46)<br />
Queensland -0.12 -0.13<br />
(0.62) (0.68)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.12 -0.11<br />
(0.63) (0.57)<br />
Western Australia/Tasmania 0.38 0.35<br />
(2.18)* (2.05)*<br />
Education school overseas 0.38 0.38<br />
(1.04) (1.06)<br />
Roman Catholic school -0.25 -0.26<br />
(1.09) (1.12)<br />
Private school -0.93 -0.83<br />
Highest qualification in 1984 (2.25)* (1.83)<br />
Degree/diploma -0.02 -0.04<br />
(0.11) (0.17)<br />
Apprenticeship -0.11 -0.12<br />
(0.38) (0.41)<br />
O<strong>the</strong>r Post-School qualification 0.06 0.06<br />
(0.24) (0.24)<br />
Year 12 <strong>of</strong> school 0.39 0.38<br />
(2.19)* (2.14)*<br />
Year 11 <strong>of</strong> school 0.20 0.19<br />
(1.05) (0.99)<br />
Year 9 <strong>of</strong> school or less -0.00 0.00<br />
(0.02) (0.01)<br />
Longest job by 1984 none -0.32 -0.31<br />
(1.37) (1.32)<br />
< 1 year -0.08 -0.09<br />
(0.47) (0.52)<br />
2 years 0.05 0.05
239<br />
(0.25) (0.21)<br />
3 years + -0.50 -0.52<br />
(2.13)* (2.24)*<br />
CEP referrals 0.13<br />
(1.75)<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
0.33 0.33<br />
(2.40)* (2.42)*<br />
Work limited by health -0.66 -0.64<br />
Family background (2.91)** (2.76)**<br />
O<strong>the</strong>r city before aged 14 -0.40 -0.38<br />
(2.39)* (2.33)*<br />
Country town before aged 14 -0.51 -0.51<br />
(3.34)** (3.35)**<br />
Rural area before aged 14 -0.42 -0.41<br />
(1.61) (1.57)<br />
Overseas before aged 14 -0.60 -0.61<br />
(1.25) (1.28)<br />
Number <strong>of</strong> siblings -0.01 -0.01<br />
(0.91) (0.65)<br />
English good -0.14 -0.11<br />
(0.56) (0.44)<br />
English poor -0.66 -0.68<br />
(1.06) (1.12)<br />
Sexist 0.35 0.31<br />
(1.26) (1.14)<br />
Sexist*female -0.85 -0.84<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.54) (1.53)<br />
Fa<strong>the</strong>r not present when resp 14 -0.22 -0.20<br />
(0.79) (0.73)<br />
Labourer -0.06 -0.06<br />
(0.19) (0.21)<br />
Plant operative -0.12 -0.13<br />
(0.46) (0.49)<br />
Sales -0.23 -0.23<br />
(0.66) (0.66)<br />
Tradesperson -0.27 -0.26<br />
(1.02) (1.01)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.17 -0.17<br />
(0.66) (0.66)<br />
Not employed -0.42 -0.43<br />
(1.31) (1.34)<br />
Fa<strong>the</strong>r holds post-school<br />
-0.22 -0.21<br />
qualification when resp 14<br />
Mo<strong>the</strong>rs occupation when resp. 14 (1.59) (1.55)<br />
Mo<strong>the</strong>r not present when resp 14 0.37 0.37<br />
(1.24) (1.25)<br />
Labourer -0.05 -0.03<br />
(0.16) (0.12)<br />
Plant operative 0.56 0.55<br />
(1.80) (1.79)<br />
Sales 0.21 0.25<br />
(0.76) (0.90)<br />
Tradesperson 0.20 0.26<br />
(0.46) (0.58)
240<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.31 -0.31<br />
(1.10) (1.09)<br />
Not employed -0.12 -0.11<br />
(0.57) (0.55)<br />
Mo<strong>the</strong>r post-school qualification 0.22 0.21<br />
when resp 14<br />
Religion brought up in (1.26) (1.21)<br />
Catholic -0.01 0.00<br />
(0.05) (0.03)<br />
Presbyterian 0.31 0.29<br />
(1.29) (1.20)<br />
Methodist 0.29 0.28<br />
(1.19) (1.18)<br />
O<strong>the</strong>r Christian 0.06 0.04<br />
(0.21) (0.15)<br />
O<strong>the</strong>r religion 0.14 0.13<br />
(0.56) (0.53)<br />
No religion 0.16 0.13<br />
(0.84) (0.72)<br />
Constant 0.44 0.47<br />
(0.65) (0.70)<br />
Observations 1283 1283<br />
Log likelihood -303.41 -304.97<br />
LR chi 2 (59) 161 131.99 118.83<br />
Mcfadden’s Pseudo R 2 162<br />
0.1572 0.1528<br />
Akaike Information Criterion 0.56 0.57<br />
Robust z-statistics in paren<strong>the</strong>ses* significant at 5%; ** significant at 1%<br />
161 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
162<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(Mfull) to a model with just <strong>the</strong> intercept (Mintercept). It is defined as R2 = 1 – (log likelihood Mfull / log<br />
likelihood Mintercept). The value <strong>of</strong> Mcfadden’s Pseudo R2 increases as new variables are added.
241<br />
Table 7.4 Summary <strong>of</strong> distribution <strong>of</strong> propensity, exclude CEP referrals<br />
Distribution <strong>of</strong> estimated propensity for SYETP treatment group<br />
Percentiles Smallest<br />
1% .0096799 .0080714<br />
5% .0209195 .0096799<br />
10% .0391096 .011608 Obs 104<br />
25% .0818465 .0178796 Sum <strong>of</strong> Wgt. 104<br />
50% .1352024 Mean .1653081<br />
Largest Std. Dev. .1161794<br />
75% .2308325 .4278012<br />
90% .3124363 .456956 Variance .0134977<br />
95% .4151957 .4614879 Skewness 1.059129<br />
99% .4614879 .6016818 Kurtosis 4.17753<br />
Distribution <strong>of</strong> estimated propensity for comparison group<br />
Percentiles Smallest<br />
1% .0003765 .0000136<br />
5% .0025803 .0000267<br />
10% .005414 .0000539 Obs 1179<br />
25% .0176855 .0000611 Sum <strong>of</strong> Wgt. 1179<br />
50% .0492093 Mean .0743632<br />
Largest Std. Dev. .0793953<br />
75% .1044401 .4422829<br />
90% .1755594 .4788667 Variance .0063036<br />
95% .2357188 .4821222 Skewness 2.031504<br />
99% .3915129 .5711446 Kurtosis 8.35458<br />
The new matching results are shown in column 2 <strong>of</strong> Table 7.5. To better facilitate<br />
comparison, <strong>the</strong> former results with CEP referrals included are shown in <strong>the</strong> first column.<br />
The matching results are now compared, to show <strong>the</strong> effect <strong>of</strong> removing CEP referrals<br />
from <strong>the</strong> specification. The employment effect falls slightly in size, from eight percentage<br />
points to six, and statistical significance drops very low. The number <strong>of</strong> common support<br />
cases discarded from <strong>the</strong> SYETP treatment group does not change. However <strong>the</strong> number<br />
<strong>of</strong> SYETP matched falls slightly, and <strong>the</strong> number <strong>of</strong> comparison cases used to match to<br />
<strong>the</strong>m also falls, with more comparison cases used with replacement than before. The<br />
mean difference in <strong>the</strong> propensity scores after matching is <strong>of</strong> <strong>the</strong> same magnitude as<br />
formerly, however <strong>the</strong> standard deviation is now slightly larger. The mean standardized<br />
bias has risen from 11.04 to 16.15. Overall, <strong>the</strong> efficiency <strong>of</strong> <strong>the</strong> match is judged to be<br />
poorer than before.
242<br />
The magnitude <strong>of</strong> <strong>the</strong> employment effect falls, but only slightly. The dramatic changes to<br />
<strong>the</strong> statistical significance <strong>of</strong> <strong>the</strong> employment effect when CEP referrals is dropped might<br />
be attributed to <strong>the</strong> relatively large fall in <strong>the</strong> number <strong>of</strong> comparison cases used to match<br />
against <strong>the</strong> SYETP. This has a role in reducing <strong>the</strong> efficiency that in turn causes <strong>the</strong> fall<br />
in statistical significance. However, fur<strong>the</strong>r discussion <strong>of</strong> <strong>the</strong> results is now deferred until<br />
after <strong>the</strong> estimation <strong>of</strong> a new specification.<br />
Table 7.5 Matching results, Single nearest neighbour with replacement, within caliper,<br />
weighting <strong>the</strong> propensity with combined weights: vary specification<br />
Match with<br />
caliper width<br />
0.001<br />
Original<br />
specification<br />
Match with<br />
caliper width<br />
0.001<br />
Original<br />
specification,<br />
Don’t include<br />
cep referrals<br />
Match with<br />
caliper width<br />
0.001<br />
New reduced<br />
specification<br />
Difference in employment 163 for 0.08 0.06 0.18<br />
matched treated and comparisons<br />
t statistic 1.57 164 0.53 4.72**<br />
Total SYETP available 104 104 109<br />
Number <strong>of</strong> SYETP matched 87 85 104<br />
% SYETP matched 84% 82% 95%<br />
Number <strong>of</strong> comparison which satisfy 80 75 93<br />
<strong>the</strong> caliper rule<br />
Number <strong>of</strong> times used<br />
1 73 68 83<br />
2 7 5 9<br />
More than 2 2 1<br />
Mean difference in propensity score 0.0002 0.0002 0.0002<br />
between single nearest neighbour<br />
matched treated and comparisons<br />
Standard deviation 0.0002 0.0003 0.0002<br />
Mean standardized bias 11.04 16.15 14.48<br />
Common support cases dropped from<br />
syetp before matching<br />
1 1 0<br />
Number <strong>of</strong> observations 1283 1283 1389<br />
Weighting protocol: weight propensity, weight <strong>the</strong> match using <strong>the</strong> treated weight only. The weighted mean<br />
bias is calculated using svymean in Stata. * significant at 10 % l.o.s., ** 5% l.o.s.<br />
163 Ever employed in 1986 survey.<br />
164 Probability <strong>of</strong> (0.22).
243<br />
7.2.2 Propensity score matching and <strong>the</strong> effect <strong>of</strong> reduced specification<br />
Augurzky and Schmidt (2001) argue that although all available variables that rule <strong>the</strong><br />
selection process are usually included in <strong>the</strong> participation equation, it may be better to<br />
remove variables <strong>of</strong> lesser importance from <strong>the</strong> specification. Their analysis showed that<br />
only including variables that were strongly significant could help combat <strong>the</strong> unnecessary<br />
effort <strong>of</strong> trying to balance <strong>the</strong>se variables, which <strong>the</strong>y found came at <strong>the</strong> expense <strong>of</strong><br />
balance <strong>of</strong> <strong>the</strong> most relevant variables. In light <strong>of</strong> this, variables with low significance, or<br />
with only a <strong>the</strong>oretically minor role in employment, are candidates for exclusion from <strong>the</strong><br />
probit for participation. Heckman, Ichimura and Todd (1997) also suggest that it is useful<br />
to compare <strong>the</strong> matching results from a reduced model <strong>of</strong> participation to that <strong>of</strong> <strong>the</strong> fuller<br />
model, in order to better assess <strong>the</strong> different approach to modelling entailed.<br />
In light <strong>of</strong> this a reduced specification <strong>of</strong> <strong>the</strong> probit for <strong>the</strong> propensity is proposed. The<br />
new matching results are shown in column 3 <strong>of</strong> Table 7.5, already shown. Only those<br />
variables with a potentially strong role in both employment and programme participation<br />
are included in <strong>the</strong> probit. Labour market outcomes for programmes are most <strong>of</strong>ten based<br />
on gender, age and unemployment experience which are included in <strong>the</strong> reduced model.<br />
The human capital effects <strong>of</strong> education, represented by highest qualification, and work<br />
experience are added. Marital status, children and partner’s labour market behaviour are<br />
included as variables that usually influence labour supply. Health, ethnicity and location<br />
are o<strong>the</strong>r factors commonly entered as constraints to labour supply or demand. The<br />
variation by location (state) was found to strongly influence SYETP participation [see<br />
section 2.2.6]. The rural/urban location is based on background prior to programme entry,<br />
retained from <strong>the</strong> former specification, because it is highly related to location in 1984 and<br />
later surveys. But this variable might act as a better instrument for <strong>the</strong> influences <strong>of</strong><br />
personal background on labour market behaviour because location for young people is<br />
mostly due to parental choice. All <strong>of</strong> <strong>the</strong>se variables form a subgroup <strong>of</strong> <strong>the</strong> fuller original<br />
specification. Note that CEP referrals is not included. Only one variable that was<br />
formerly statistically significant in <strong>the</strong> full model was excluded: nature <strong>of</strong> schooling<br />
(private, government, overseas, Roman Catholic). All o<strong>the</strong>r variables excluded from <strong>the</strong><br />
reduced model were not statistically significant in <strong>the</strong> estimated probit.
244<br />
Table 7.6 shows <strong>the</strong> results <strong>of</strong> <strong>the</strong> new specification <strong>of</strong> <strong>the</strong> probit including only <strong>the</strong>se<br />
variables. The estimated coefficients that are statistically significant are compared to<br />
those in <strong>the</strong> original specification. Age and marital status remain statistically significant<br />
at conventional levels; however partner’s employment loses statistical significance. The<br />
location <strong>of</strong> Western Australia/Tasmania retains a positive influence on participation. This<br />
is consistent with <strong>the</strong> administrative data that showed Western Australia to have a much<br />
higher SYETP placement rate than o<strong>the</strong>r states [see section 2.2.6]. Qualifications lose<br />
<strong>the</strong>ir impact on SYETP participation in <strong>the</strong> reduced form model, whereas in <strong>the</strong> original<br />
model year 12 schooling had a positive impact. A longer qualifying spell <strong>of</strong><br />
unemployment still has a positive impact, and past experience <strong>of</strong> a job held for more than<br />
3 years still has a negative impact on SYETP participation. O<strong>the</strong>r city or country town<br />
kept <strong>the</strong>ir negative impact on SYETP participation in <strong>the</strong> reduced model. The only key<br />
change for variables included in both <strong>the</strong> full and reduced models is that partner’s<br />
employment and highest qualification <strong>of</strong> year 12 lose statistical significance in <strong>the</strong><br />
reduced form.<br />
The predictive power <strong>of</strong> <strong>the</strong> probit is 92 per cent. This is <strong>the</strong> same power as <strong>the</strong> original<br />
model. Thus <strong>the</strong> excluded variables do not change <strong>the</strong> predictive performance <strong>of</strong> <strong>the</strong><br />
probit. This measure <strong>of</strong> predictive power is only very rough however. O<strong>the</strong>r measures <strong>of</strong><br />
fit indicate <strong>the</strong> reduced specification to be slightly better, with <strong>the</strong> AIC lower at 0.53<br />
compared to <strong>the</strong> former models values <strong>of</strong> 0.56 and 0.57 (note <strong>the</strong> sample size has<br />
increased as certain variables dropped had missing cases).<br />
The propensity scores, for both SYETP participants and comparisons, from <strong>the</strong> new<br />
specification are shown in Table 7.7. These are compared to those found earlier [see<br />
section 6.2.2]. This change to <strong>the</strong> specification has quite a strong impact on <strong>the</strong> range <strong>of</strong><br />
<strong>the</strong> distribution. Figure 7.8 shows <strong>the</strong> histograms for <strong>the</strong> propensities estimated. Both<br />
propensities for SYETP and <strong>the</strong> comparison group now have a range less than 0.4. The<br />
distributions have been shifted towards zero. The upper tails have generally disappeared,<br />
with <strong>the</strong> remainder <strong>of</strong> <strong>the</strong> distribution shaped similarly to <strong>the</strong> distribution from <strong>the</strong>
245<br />
original model, but with <strong>the</strong> peaks at lower points in <strong>the</strong> distribution. The central peak <strong>of</strong><br />
<strong>the</strong> SYETP propensities has been changed most. The mean <strong>of</strong> <strong>the</strong> propensities for <strong>the</strong><br />
treatment SYETP has been shifted much lower down than for <strong>the</strong> original specification,<br />
while <strong>the</strong> mean <strong>of</strong> <strong>the</strong> comparisons has hardly changed. Figure 7.9 shows <strong>the</strong> kernel<br />
densities <strong>of</strong> <strong>the</strong> SYETP and comparison distribution <strong>of</strong> propensities overlaid. The peak <strong>of</strong><br />
<strong>the</strong> SYETP is raised higher than before. This is likely due to <strong>the</strong> bunching effect <strong>of</strong><br />
reducing <strong>the</strong> range. The peak <strong>of</strong> <strong>the</strong> distribution <strong>of</strong> propensities for <strong>the</strong> comparison is<br />
lower than before, but <strong>the</strong> upper tail is slightly fatter, so <strong>the</strong> reduction in range for <strong>the</strong><br />
comparisons has a more diffuse impact. The change to <strong>the</strong> specification has brought <strong>the</strong><br />
peaks <strong>of</strong> <strong>the</strong> SYETP and comparisons closer in height than before. The different locations<br />
<strong>of</strong> <strong>the</strong> propensity peaks for <strong>the</strong> SYETP and comparisons indicate that <strong>the</strong> participation<br />
model is still producing an adequate model that distinguishes participation.<br />
The effects <strong>of</strong> <strong>the</strong> new specification on <strong>the</strong> matching results are shown in Table 7.5. The<br />
number <strong>of</strong> SYETP matched is raised to 104. No cases have been dropped due to<br />
application <strong>of</strong> <strong>the</strong> common support rule. The number <strong>of</strong> cases <strong>of</strong> comparisons matched to<br />
<strong>the</strong> SYETP is higher than before at 93 cases, while more are also used with replacement.<br />
The mean difference in propensities is no different to that found before. 165 The mean<br />
standardised bias after matching is now worse than before however, as it has risen from<br />
11.04 to 14.48. However, in comparison to <strong>the</strong> o<strong>the</strong>r specification change, where CEP<br />
referrals was excluded from <strong>the</strong> original model, shown in column 2 Table 7.5, <strong>the</strong> bias<br />
has improved. As <strong>the</strong> bias was much worse where CEP referrals was excluded than <strong>the</strong><br />
original model, CEP referrals seems to have helped balance <strong>the</strong> bias.<br />
D’Agostino and Rubin (2000) consider <strong>the</strong> issue <strong>of</strong> missing data in item non-response,<br />
and note that most propensity score methods are based on complete data. They suggest<br />
that missing-ness needs to be controlled in PSM. This suggests <strong>the</strong> new specification may<br />
also benefit from <strong>the</strong> reduction in missing data due to item non-response, which may help<br />
explain <strong>the</strong> great gain in size and statistical significance <strong>of</strong> <strong>the</strong> measured employment<br />
effect. In <strong>the</strong> new specification, only <strong>the</strong> proportion <strong>of</strong> time unemployed contains missing<br />
165 Although rounding hides that <strong>the</strong> mean is very slightly smaller at 0.00016.
246<br />
information, whereas in <strong>the</strong> old specification family background variables contributed to<br />
a large number <strong>of</strong> item non-response cases. This raises <strong>the</strong> number <strong>of</strong> observations usable<br />
in <strong>the</strong> new specification. The number <strong>of</strong> SYETP cases rises slightly as a result <strong>of</strong> <strong>the</strong><br />
change in specification, as <strong>the</strong> item non-response is resolved. The percentage <strong>of</strong> SYETP<br />
matched rises from 85 per cent in <strong>the</strong> old specification to 95 per cent in <strong>the</strong> new<br />
specification.<br />
7.2.3 Fur<strong>the</strong>r discussion and conclusions<br />
Lechner (2001) p23 also examined <strong>the</strong> effects on propensity score matching <strong>of</strong> reduction<br />
in <strong>the</strong> covariate set, and <strong>the</strong> results were found to be substantially different, but not<br />
monotone, so that <strong>the</strong> conclusion was that it is not necessarily better to control for more<br />
variables. Rosenbaum and Rubin (1985b) p115 noted that in practice nearest neighbour<br />
matching algorithms are subject to <strong>the</strong> trade-<strong>of</strong>f between finding matches for all treated<br />
units and obtaining matched treated-comparison pairs that are extremely similar. The<br />
failure to match all treated units can give incomplete matching bias, while inexact pair<br />
matching can result from greater differences in matched pairs. They advocated <strong>the</strong> use <strong>of</strong><br />
complete matching in order to avoid incomplete matching bias. It was noted that<br />
incomplete matching can lead to loss <strong>of</strong> efficiency.<br />
In comparing <strong>the</strong> original specification to <strong>the</strong> reduced specification, it appears that<br />
reduction <strong>of</strong> <strong>the</strong> incomplete matching bias leads to a dramatic gain in <strong>the</strong> employment<br />
effect <strong>of</strong> SYETP, and <strong>the</strong> statistical significance is much greater. The percentage <strong>of</strong><br />
SYETP matched rises from 85 per cent in <strong>the</strong> old specification to 95 per cent in <strong>the</strong> new<br />
specification. These results are in accordance with <strong>the</strong> bias and inefficiency arising from<br />
incomplete matching, referred to by Rosenbaum and Rubin (1985b). However, some <strong>of</strong><br />
this is also attributable to effects <strong>of</strong> missing data on PSM suggested in D’Agostino and<br />
Rubin (2000). At <strong>the</strong> same time as <strong>the</strong>se gains, <strong>the</strong> standardized bias is not much worse,<br />
an indication that <strong>the</strong> incomplete matching bias is <strong>of</strong> a lesser degree. As <strong>the</strong> reduced<br />
specification had fewer variables, it appears that trying to balance <strong>the</strong> included variables<br />
is very difficult with <strong>the</strong>se data. It is possible that this might be interpreted as evidence<br />
that selection on unobservables is more consistent with <strong>the</strong> data. If this belief is
247<br />
maintained, <strong>the</strong>n <strong>the</strong> Heckman bivariate probit modelling results give a modelling result<br />
in harmony with <strong>the</strong> role <strong>of</strong> unobservables in SYETP participation. But while this may be<br />
true, <strong>the</strong> form which <strong>the</strong> unobservables take is also highly restricted in <strong>the</strong> Heckman<br />
bivariate probit modelling, and <strong>the</strong> reasonableness <strong>of</strong> this must in turn be considered.<br />
Rosenbaum and Rubin (1985b) p 109 also note that <strong>the</strong> assumption <strong>of</strong> strongly ignorable<br />
treatment (CIA) needs to be consistent with <strong>the</strong> data and <strong>the</strong> causal mechanism, through<br />
which <strong>the</strong> SYETP treatment is thought to operate to produce employment effects, and<br />
without which <strong>the</strong> matching results are biased. CEP referrals is questioned as potentially<br />
violating <strong>the</strong> CIA requirements, because <strong>of</strong> <strong>the</strong> use <strong>of</strong> this variable for <strong>the</strong> exclusion in<br />
Richardson (1998). 166 It was argued earlier that CEP referrals credibly did not affect<br />
employment independently once SYETP participation was accounted for. The effects <strong>of</strong><br />
<strong>the</strong> CEP referrals for <strong>the</strong> replicated Heckman modelling in <strong>the</strong> same data have been<br />
shown to be strongly diminished by accounting for attrition with weighting. The<br />
<strong>the</strong>oretical view against <strong>the</strong> inclusion <strong>of</strong> CEP referrals in <strong>the</strong> weighted model for <strong>the</strong><br />
propensity can however also be argued on <strong>the</strong> basis <strong>of</strong> <strong>the</strong> evidence <strong>of</strong> Wielgosz (1984)<br />
and Aungles and Stewart (1986), discussed in section 2.2.6. If this conviction is upheld,<br />
<strong>the</strong>n <strong>the</strong> sensitivity results where CEP referrals are excluded, or <strong>the</strong> reduced specification,<br />
provide matching results consistent with this view.<br />
In conclusion, <strong>the</strong> sensitivity analysis shows that <strong>the</strong> employment effects found are<br />
subject to <strong>the</strong> specifications and modelling assumptions adopted. This was true for both<br />
<strong>the</strong> Heckman and PSM modelling approaches. In an overview <strong>of</strong> <strong>the</strong> various results<br />
gained which disregards <strong>the</strong> magnitude <strong>of</strong> point estimates, it can be determined that no<br />
negative employment effects were found for SYETP, even in sensitivity analysis.<br />
Friedlander et al. (1997) p1819 note that in debates about non-experimental evaluations<br />
<strong>the</strong>re is no clear-cut way <strong>of</strong> determining which modelling assumption is valid. Within <strong>the</strong><br />
range <strong>of</strong> alternatives explored, and as can be determined within <strong>the</strong> limits <strong>of</strong> <strong>the</strong> ALS data,<br />
<strong>the</strong> direction <strong>of</strong> employment effect is robustly positive. Statistical significance in this<br />
166 This was checked by modelling <strong>the</strong> univariate probit for employment, where <strong>the</strong>se variables were not<br />
found significant.
248<br />
analysis was however strongly affected by <strong>the</strong> small sample size and reductions to<br />
efficiency from various assumptions. As such <strong>the</strong> sensitivity analysis indicates that it<br />
would be useful to provide a form <strong>of</strong> ‘confidence interval’ for model selection in<br />
evaluation results, showing <strong>the</strong> variation employment effects are subject to for changes to<br />
key modelling assumptions. Provision <strong>of</strong> such a confidence interval can allow for <strong>the</strong><br />
statistical modelling uncertainty, without which evidence for <strong>the</strong> employment effects can<br />
be masked. This can be particularly helpful when <strong>the</strong>re is no clear evidence favouring one<br />
model and it’s assumptions over <strong>the</strong> o<strong>the</strong>r. In final consideration <strong>of</strong> <strong>the</strong> evidence for<br />
SYETP found in this analysis, <strong>the</strong> conclusions <strong>of</strong> Heckman et al. (1999) are deferred to:<br />
“…every estimator relies on identifying assumptions about <strong>the</strong><br />
outcome and participation processes. When a particular estimator<br />
is applied to data, where those assumptions fail to hold, bias<br />
results. This bias can be substantial. When different estimators are<br />
applied to <strong>the</strong> same data, <strong>the</strong> estimates <strong>the</strong>y produce will vary<br />
because at most one set <strong>of</strong> underlying assumptions is consistent<br />
with <strong>the</strong> data. Only if <strong>the</strong>re is no problem <strong>of</strong> selection bias would<br />
all estimators identify <strong>the</strong> same parameter.” (Heckman et al.<br />
(1999): 2007)<br />
As <strong>the</strong> limitations <strong>of</strong> <strong>the</strong> non-experimental data used here bound <strong>the</strong> extent <strong>of</strong> conclusions,<br />
it is perhaps <strong>the</strong> evaluation policy which needs to be addressed. However, debating <strong>the</strong><br />
role <strong>of</strong> experimentation in social policy is beyond <strong>the</strong> scope <strong>of</strong> this study, and inevitably<br />
becomes a political and ethical decision.
249<br />
Table 7.6 Weighted Probit used to estimate propensity score for propensity score<br />
matching, new specification<br />
New specification<br />
syetp<br />
Gender=female 0.02<br />
(0.17)<br />
Age at 1984 survey -0.07<br />
(2.52)*<br />
Married 1984 -0.91<br />
(2.91)**<br />
Children 1984 0.43<br />
(1.20)<br />
Spouse employed 1984 0.63<br />
(1.52)<br />
Aboriginal/Torres Strait Islander -0.48<br />
(1.07)<br />
O<strong>the</strong>r ethnic minority -0.08<br />
(0.37)<br />
Victoria at 1984 survey 0.21<br />
(1.41)<br />
Queensland at 1984 survey 0.01<br />
South Australia/Nor<strong>the</strong>rn Territory at<br />
1984 survey<br />
(0.04)<br />
-0.03<br />
(0.17)<br />
0.35<br />
Western Australia/Tasmania at 1984<br />
survey<br />
Highest qualification in 1984 (2.10)*<br />
Degree/diploma -0.16<br />
(0.74)<br />
Apprenticeship -0.23<br />
(0.78)<br />
O<strong>the</strong>r Post-School qualification -0.03<br />
(0.11)<br />
Year 12 <strong>of</strong> school 0.29<br />
(1.74)<br />
Year 11 <strong>of</strong> school 0.15<br />
(0.80)<br />
Year 9 <strong>of</strong> school or less -0.06<br />
(0.27)<br />
Longest job by 1984 none -0.44<br />
(1.87)<br />
< 1 year -0.16<br />
(0.89)<br />
2 years 0.01<br />
(0.05)<br />
3 years + -0.62<br />
(2.79)**<br />
Proportion <strong>of</strong> time Pre-June 1984 spent<br />
in unemployment<br />
0.34<br />
(2.43)*<br />
Work limited by health -0.64<br />
(2.84)**<br />
O<strong>the</strong>r city before aged 14 -0.28
250<br />
(1.75)<br />
Country town before aged 14 -0.47<br />
(3.16)**<br />
Rural area before aged 14 -0.52<br />
(1.97)*<br />
Overseas before aged 14 -0.43<br />
(0.84)<br />
Constant 0.09<br />
(0.15)<br />
Observations 1389<br />
Log likelihood -341.55<br />
LR chi 2 (27) 167 80.04<br />
Mcfadden’s Pseudo R 2 168<br />
0.1029<br />
Akaike Information Criterion 0.53<br />
Robust z-statistics in paren<strong>the</strong>ses* significant at 5%; ** significant at 1%<br />
167 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
168<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(M full ) to a model with just <strong>the</strong> intercept (M intercept ). It is defined as R 2 = 1 – (log likelihood M full / log<br />
likelihood M intercept ) . The value <strong>of</strong> Mcfadden’s Pseudo R 2 increases as new variables are added.
251<br />
Table 7.7 Summary <strong>of</strong> distribution <strong>of</strong> propensity, new specification<br />
Distribution <strong>of</strong> estimated propensity for SYETP treatment group<br />
Percentiles Smallest<br />
1% .0206492 .0041307<br />
5% .037368 .0206492<br />
10% .0438178 .0270285 Obs 109<br />
25% .0682665 .0334976 Sum <strong>of</strong> Wgt. 109<br />
50% .1295373 Mean .1298514<br />
Largest Std. Dev. .0704821<br />
75% .1679348 .280174<br />
90% .2276545 .2934951 Variance .0049677<br />
95% .2737046 .2991401 Skewness .4874995<br />
99% .2991401 .3021699 Kurtosis 2.585342<br />
Distribution <strong>of</strong> estimated propensity for comparison group<br />
Percentiles Smallest<br />
1% .000886 .0002578<br />
5% .0043483 .0003094<br />
10% .0096206 .0003152 Obs 1280<br />
25% .0250092 .0003735 Sum <strong>of</strong> Wgt. 1280<br />
50% .059004 Mean .0743514<br />
Largest Std. Dev. .0624978<br />
75% .1078141 .3332027<br />
90% .1600628 .3395213 Variance .003906<br />
95% .1981402 .3596305 Skewness 1.224251<br />
99% .2763645 .3648924 Kurtosis 4.590139<br />
1<br />
1<br />
Fraction<br />
.5<br />
Fraction<br />
.5<br />
0<br />
0<br />
0 .5 1<br />
104 observations, unweighted probit<br />
propensity score for treated(SYETP=1)<br />
0 .5 1<br />
1179 observations, unweighted probit<br />
propensity score for control(SYETP=0)<br />
histograms <strong>of</strong> estimated propensity scores prior to matching<br />
Figure 7.8 Histograms <strong>of</strong> <strong>the</strong> propensity scores for <strong>the</strong> new specification.
252<br />
Treated propensity scores, Epan Untreated propensity scores, Ep<br />
8<br />
6<br />
4<br />
2<br />
0<br />
0 .1 .2 .3 .4<br />
Pr(syetp)<br />
kernel densities <strong>of</strong> propensity scores <strong>of</strong> <strong>the</strong> treated vs untreated<br />
Figure 7.9 Kernel densities <strong>of</strong> <strong>the</strong> propensities estimated for new specification
253<br />
8: Summary and Conclusions<br />
The underlying motivation for this work is <strong>the</strong> desire for a closer examination <strong>of</strong> <strong>the</strong><br />
<strong>Australian</strong> wage subsidy SYETP. The main <strong>the</strong>oretical rationale for wage subsidies is<br />
that <strong>the</strong>y stimulate a gain in employment. However <strong>the</strong> review <strong>of</strong> <strong>the</strong> <strong>the</strong>oretical models<br />
<strong>of</strong> wage subsidies shows <strong>the</strong>y cannot provide unambiguous pro<strong>of</strong> <strong>of</strong> an increase in<br />
employment. The importance <strong>of</strong> empirical evaluation <strong>of</strong> wage subsidies in part stems<br />
from this uncertainty. The central question remains: do wage subsidies improve<br />
employment prospects for <strong>the</strong> individual?<br />
Microeconomic evaluation evidence can demonstrate whe<strong>the</strong>r SYETP has led to a gain in<br />
employment for participants. While <strong>the</strong> main evaluation interest is <strong>the</strong> programme<br />
influence on job entry, <strong>the</strong> processes <strong>of</strong> participation and job entry can involve both<br />
observable and unobservable components that can introduce selection bias. An evaluation<br />
<strong>of</strong> <strong>the</strong> program effect needs to account for this selection. Two evaluation methods are<br />
explored here – <strong>the</strong> Heckman selection bivariate probit model, and matching methods, in<br />
particular propensity score matching. Both identify a parameter corresponding to <strong>the</strong><br />
mean effect <strong>of</strong> treatment on <strong>the</strong> treated, which can be used to decide whe<strong>the</strong>r <strong>the</strong><br />
programme leads to employment gains. However each method uses different assumptions<br />
to achieve this. Selection on unobservables is assumed by <strong>the</strong> Heckman bivariate probit,<br />
while selection on observables is assumed by matching methods.<br />
The review <strong>of</strong> recent overseas evidence for wage subsidies shows a variety <strong>of</strong> positive<br />
and negative impacts. Two key <strong>the</strong>mes are identified. The empirical ambiguity as to<br />
employment gains from wage subsidies remains unresolved. Contributing to this, in many<br />
studies <strong>the</strong> non-experimental evaluation methods insufficiently test <strong>the</strong> assumptions,<br />
alternative methods, and data issues. It was noted that meta-analysis might be more<br />
satisfactory in accounting for <strong>the</strong> deviations in evaluation evidence, since it was hard to
254<br />
separate <strong>the</strong>se from <strong>the</strong> wide dissimilarities in <strong>the</strong> differing data, methods, environments,<br />
subsidy conditions, and o<strong>the</strong>r sources <strong>of</strong> variation.<br />
The appraisal <strong>of</strong> micro-evaluation evidence in Australia also shows <strong>the</strong> empirical<br />
evidence for wage subsidies is not well established and suffers from similar deficiencies.<br />
SYETP existed with a broad remit from 1976 to 1985. In <strong>the</strong> early 1980’s, <strong>the</strong> ‘Adult<br />
<strong>Wage</strong> <strong>Subsidy</strong> Scheme’ (AWSS), and ‘General Training Assistance On-Job-Training’<br />
(GTA-OTJ) briefly coexisted with SYETP, and <strong>the</strong>n from 1985 to 1996, Jobstart replaced<br />
SYETP. As well as being short-lived, both AWSS and GTA-OTJ were very small in<br />
scale, with very little expenditure and few placements and GTA-OTJ was uniquely<br />
discretionary in targeting making <strong>the</strong> eligible group difficult to define. The AWSS was<br />
never evaluated and GTA-OTJ was only evaluated relative to o<strong>the</strong>r programs. The more<br />
recent Jobstart was evaluated several times, mostly using exact matching but with very<br />
limited variables, and data on comparisons drawn from surveys but no accounting for<br />
non-response. Before 1980, evaluation <strong>of</strong> SYETP was not published and possibly did not<br />
take place. Several evaluations <strong>of</strong> SYETP undertaken by Stretton (1982, 1984), Baker<br />
(1984), Rao and Jones (1984), and Richardson (1998) were appraised in detail. All found<br />
positive effects <strong>of</strong> SYETP on employment. All used survey data, but although most<br />
discussed <strong>the</strong> potential for non-response and even modelled it, none acceptably accounted<br />
for it in <strong>the</strong>ir evaluation estimations. Some evaluated SYETP relative to o<strong>the</strong>r programs,<br />
such as GTA-OTJ and o<strong>the</strong>r training programs instead <strong>of</strong> a non-participant comparison<br />
group, although <strong>the</strong> eligible groups do not always clearly overlap. Various forms <strong>of</strong><br />
regression modelling, such as logit, probit or ordered probit, were used, however only<br />
Richardson (1998) accounted for selection using <strong>the</strong> Heckman selection model <strong>of</strong> SYETP<br />
participation and employment. The inadequacies <strong>of</strong> past analyses <strong>of</strong> SYETP contributed<br />
three <strong>the</strong>mes to address: suitable modelling <strong>of</strong> selection to account for <strong>the</strong> influence <strong>of</strong><br />
observables or unobservables, dealing with non-response in <strong>the</strong> observational data, and<br />
appropriate control for <strong>the</strong> differences between <strong>the</strong> SYETP and comparison groups.<br />
The evidence for <strong>the</strong> economic environment and operation <strong>of</strong> SYETP provided a useful<br />
understanding <strong>of</strong> <strong>the</strong> issues affecting <strong>the</strong> evaluation <strong>of</strong> SYETP. During <strong>the</strong> long period
255<br />
over which it ran, <strong>the</strong> review makes it clear SYETP underwent repeated adjustment, with<br />
<strong>the</strong> eligible groups, amount and length <strong>of</strong> subsidy, political setting, and economic<br />
conditions constantly changing. This gave perspective into <strong>the</strong> dynamic aspects <strong>of</strong> labour<br />
market programmes, which is <strong>of</strong>ten ignored. Micro-evaluations are essentially static, so it<br />
is important to maintain awareness that <strong>the</strong>y provide only a snapshot <strong>of</strong> <strong>the</strong> employment<br />
effects relevant to a particular macroeconomic setting.<br />
The summary <strong>of</strong> <strong>the</strong> economic context during <strong>the</strong> period <strong>of</strong> our empirical studies, 1984-<br />
1986 provided characterisation <strong>of</strong> <strong>the</strong> <strong>Australian</strong> youth labour market that SYETP<br />
addressed. <strong>Youth</strong> minimum wages were institutionally set by <strong>the</strong> award wage system, and<br />
were lower relative to adults. <strong>Youth</strong>s earned about 50-60 per cent <strong>of</strong> adult wages.<br />
Although youth unemployment was high relative to adults, this was countered by rising<br />
school participation, and employment. The regional and rural/urban contrasts in Australia<br />
contributed to wide variation in <strong>the</strong> incidence opportunities and unemployment. The CES<br />
administered SYETP, but faced a limited supply <strong>of</strong> registered vacancies that were<br />
generally for lower-skilled occupations.<br />
O<strong>the</strong>r factors, which have been reviewed here, would also have had a role in affecting <strong>the</strong><br />
functioning <strong>of</strong> <strong>the</strong> SYETP subsidy, including lower relative youth labour costs, elastic<br />
demand for youths relative to adults, and substitutability <strong>of</strong> formal education for on <strong>the</strong><br />
job training in human capital. There were minimum wages, which would give<br />
institutionalised inefficiency <strong>of</strong> <strong>the</strong> labour market. Finally, <strong>the</strong> value <strong>of</strong> SYETP payments<br />
to employers was greater than <strong>the</strong> unemployment benefits to <strong>the</strong> eligible unemployed<br />
youths, and in turn participants received Award wages that were greater than benefit<br />
payments (usually substantially greater). The <strong>the</strong>ory <strong>of</strong> wage subsidies outlined in<br />
Chapter 1 indicates how <strong>the</strong>se might influence <strong>the</strong> potential for employment gains from a<br />
wage subsidy targeted at youths. Toge<strong>the</strong>r, <strong>the</strong>se features favour <strong>the</strong> suggestion that<br />
SYETP might have given employment gains to those eligible. However, it is impossible<br />
in this context to determine whe<strong>the</strong>r <strong>the</strong>se features were caused by SYETP, or were<br />
merely <strong>the</strong> backdrop influencing SYETP operation.
256<br />
A series <strong>of</strong> empirical studies was presented, forming an evaluation <strong>of</strong> <strong>the</strong> employment<br />
gains attributable to SYETP. The central aims <strong>of</strong> <strong>the</strong> analysis were threefold: to examine<br />
<strong>the</strong> results <strong>of</strong> <strong>the</strong> Heckman selection modelling against those <strong>of</strong> propensity score<br />
matching, and so relax <strong>the</strong> specification from fully parameterised to semi-parametric; to<br />
account for attrition and non-response in <strong>the</strong> data; and to compare <strong>the</strong> employment<br />
impact <strong>of</strong> SYETP under <strong>the</strong> different modelling assumptions. The analysis has produced<br />
new results for <strong>the</strong> employment gains <strong>of</strong> SYETP. The new results account for potential<br />
bias due to survey attrition, and provide more evidence as to <strong>the</strong> robustness <strong>of</strong> <strong>the</strong><br />
employment gains to varying <strong>the</strong> modelling assumptions. The new evaluation evidence<br />
for SYETP thus improves <strong>the</strong> knowledge relating to <strong>the</strong> employment gains for SYETP.<br />
The Richardson (1998) evaluation using <strong>the</strong> Heckman selection probit was first replicated<br />
successfully. The propensity score matching (PSM) methods were applied using <strong>the</strong><br />
nearest-neighbour-within-caliper-with-replacement protocol. The choice <strong>of</strong> caliper width<br />
was explored by <strong>the</strong> application <strong>of</strong> several plausible caliper widths. To examine <strong>the</strong><br />
sensitivity <strong>of</strong> <strong>the</strong> results to <strong>the</strong> choice <strong>of</strong> Propensity Score Matching protocol, all-incaliper<br />
matching was also applied, and it was concluded that <strong>the</strong> results were not strongly<br />
affected by <strong>the</strong> choice <strong>of</strong> matching protocol. The PSM results reduce <strong>the</strong> size and<br />
significance <strong>of</strong> <strong>the</strong> employment effect found when compared to <strong>the</strong> Heckman modelling.<br />
It was concluded that potential attrition bias might affect <strong>the</strong> results by introducing<br />
selection bias.<br />
Sample reduction in <strong>the</strong> <strong>Australian</strong> Longitudinal Survey data was explored. The effects <strong>of</strong><br />
survey design, initial non-response, and subsequent panel attrition were examined and<br />
found to have substantial impacts on <strong>the</strong> participant and comparison group characteristics,<br />
introducing potential bias. This was <strong>the</strong>n accounted for, and <strong>the</strong> impact on evaluation<br />
assessed. Weights were constructed to deal with attrition, non-response and complex<br />
survey design in <strong>the</strong> ALS data. The weighted results were found to be smaller and to have<br />
low statistical significance. Accounting for attrition in both <strong>the</strong> Heckman bivariate probit<br />
and PSM models reduces <strong>the</strong> size <strong>of</strong> <strong>the</strong> employment effect found, and results have low<br />
significance. However, after accounting for attrition, <strong>the</strong> employment effect found using
257<br />
<strong>the</strong> Heckman and PSM methods were very similar. It is concluded this is due to <strong>the</strong><br />
removal <strong>of</strong> selection effects due to attrition. Sensitivity analysis explores alternative<br />
specifications for each model. The employment effects for <strong>the</strong> models were found to be<br />
susceptible to <strong>the</strong> specification changes.<br />
In light <strong>of</strong> past research for SYETP, this analysis provides a number <strong>of</strong> improvements.<br />
Propensity score matching is applied for <strong>the</strong> first time to <strong>the</strong> evaluation data. No past<br />
wage subsidy evaluation in Australia has applied this method, although exact matching<br />
has been applied. This evaluation <strong>the</strong>n accounts for more potentially influential variables<br />
in <strong>the</strong> use <strong>of</strong> matching methods, accommodated within <strong>the</strong> propensity score matching.<br />
Selection due to survey attrition is accounted for with weights, at <strong>the</strong> same time as<br />
selection modelling to account for selection due to participation using <strong>the</strong> Heckman<br />
bivariate probit. Past SYETP analyses were only able to deal at most with one <strong>of</strong> <strong>the</strong>se<br />
sources <strong>of</strong> selection, as shown in section 2. The role <strong>of</strong> attrition and data loss in<br />
Propensity Score Matching was also dealt with. Few papers account for sample reduction<br />
in Propensity Score Matching. The analysis <strong>of</strong> PSM and Heckman bivariate probit with<br />
weighting for attrition shows that both <strong>the</strong>se methods produce strongly different measures<br />
<strong>of</strong> employment gains once attrition is accounted for. The dependence <strong>of</strong> <strong>the</strong> evaluation<br />
strategy on <strong>the</strong> assumption <strong>of</strong> selection relating to observable or unobservable variables is<br />
relaxed by considering employment gains in both <strong>the</strong> Heckman and PSM methods.<br />
The value <strong>of</strong> replication has been validated in this case. Having <strong>the</strong> original data, <strong>the</strong><br />
replication <strong>of</strong> <strong>the</strong> sample and methods proved highly informative, revealing <strong>the</strong> important<br />
role <strong>of</strong> sample reduction, through attrition and sample selection. This finding concurs<br />
with <strong>the</strong> literature, where Smith and Todd (2000, 2003), Dehijia and Wahba (1998, 1999)<br />
also found key influences on <strong>the</strong> results arose from <strong>the</strong> data sample and approaches to<br />
data issues. The conclusions accordingly advocate <strong>the</strong> adoption <strong>of</strong> replication as an<br />
important initial point for research.<br />
Comparing <strong>the</strong> different models gives increased external validity to <strong>the</strong> research findings<br />
for SYETP employment gains. SYETP had positive employment impacts in all past
258<br />
evaluations, as shown in <strong>the</strong> review, although most <strong>of</strong> <strong>the</strong> results can only be considered<br />
with strong caveats, due to <strong>the</strong> inadequate methods employed. At least partially, due to<br />
<strong>the</strong> varying timeframes for <strong>the</strong> analyses, this gives evidence that SYETP gave<br />
employment gains in a dynamic context.<br />
In all variations, <strong>the</strong> research here found positive impacts <strong>of</strong> SYETP, even if some were<br />
not statistically significant, a difficulty influenced by <strong>the</strong> small sample and efficiency<br />
losses from some <strong>of</strong> <strong>the</strong> methods. Regardless <strong>of</strong> whe<strong>the</strong>r selection was based on <strong>the</strong><br />
assumption <strong>of</strong> observables or unobservables, <strong>the</strong> impact remained positive. This<br />
contributes evidence <strong>of</strong> <strong>the</strong> robustness <strong>of</strong> <strong>the</strong> positive effect on employment for SYETP<br />
participants.<br />
The sensitivity <strong>of</strong> <strong>the</strong> specifications <strong>of</strong> both <strong>the</strong> bivariate probit and <strong>the</strong> propensity score<br />
used for <strong>the</strong> PSM were explored. In <strong>the</strong> bivariate probit this centered around <strong>the</strong><br />
exclusion restriction, and in <strong>the</strong> propensity score matching <strong>the</strong> implications <strong>of</strong> <strong>the</strong><br />
exclusion restriction for <strong>the</strong> comparative analysis were investigated. In <strong>the</strong> case <strong>of</strong> <strong>the</strong><br />
bivariate probit, comparisons <strong>of</strong> <strong>the</strong> weighted and unweighted results found that <strong>the</strong><br />
specification has great bearing on <strong>the</strong> results. An interpretation advanced here is that<br />
because sample attrition was confounded with <strong>the</strong> SYETP program treatment, <strong>the</strong><br />
bivariate probit accounted for this with <strong>the</strong> modelling <strong>of</strong> <strong>the</strong> selection into <strong>the</strong> program.<br />
Hence, <strong>the</strong> strong differences in <strong>the</strong> outcomes <strong>of</strong> <strong>the</strong> weighted and unweighted analyses.<br />
There were serious difficulties in estimating alternative specifications in <strong>the</strong> unweighted<br />
data, indicating that <strong>the</strong> search for a plausible specification could be problematic. In <strong>the</strong><br />
PSM, in line with recent contributions to <strong>the</strong> matching literature, <strong>the</strong> variants examined<br />
<strong>the</strong> restriction <strong>of</strong> <strong>the</strong> propensity to those variables expected on empirical or <strong>the</strong>oretical<br />
grounds to have significant relations with both SYETP participation and employment.<br />
The substantial changes to <strong>the</strong> results provide a useful illustration <strong>of</strong> <strong>the</strong> importance <strong>of</strong><br />
<strong>the</strong> modelling assumptions, and indicate that <strong>the</strong> assumptions involved in matching are<br />
not simple. The results show that <strong>the</strong> statistical modelling uncertainty can be difficult to<br />
resolve without clear evidence <strong>of</strong> a strong basis for favouring one model over ano<strong>the</strong>r,<br />
unless <strong>the</strong> goodness <strong>of</strong> fit assessment undertaken provides obvious resolution. It is
259<br />
posited that that such sensitivity analysis, comparing bivariate probit and PSM where <strong>the</strong><br />
data can lend support to both models, should be provided to enhance evaluation results<br />
and provide a form <strong>of</strong> confidence interval for model selection, and so account for <strong>the</strong><br />
statistical modelling uncertainty.<br />
This exploration has confirmed some relationships suggested in <strong>the</strong> literature and<br />
revealed some new insights into SYETP and wage subsidy employment effects. The<br />
research challenges orthodox evaluation methods by applying not one potentially<br />
appropriate selection approach, and underlying assumption about observables and<br />
unobservables, but both. The benefits are informed overview <strong>of</strong> <strong>the</strong> role <strong>of</strong> <strong>the</strong><br />
assumption to <strong>the</strong> evaluation outcome. To <strong>the</strong> extent that <strong>the</strong> data allows both methods to<br />
be attempted, such as sufficiently extensive data on characteristics <strong>of</strong> employment and<br />
participation, as well as variables that can act as instruments, this allows <strong>the</strong> conclusion<br />
that this approach should be adopted wherever possible. Deliberative attempts were made<br />
to constructively account for data and modelling issues. The constraints <strong>of</strong> <strong>the</strong> nonexperimental<br />
data limit <strong>the</strong> conclusions for SYETP, as without a benchmark from<br />
experimental data, it is not possible to fur<strong>the</strong>r choose between <strong>the</strong> model <strong>of</strong> <strong>the</strong> Heckman<br />
and PSM methods. Although unsatisfying to a small extent, <strong>the</strong> remaining limitations <strong>of</strong><br />
this research can inform fur<strong>the</strong>r work on <strong>the</strong> evaluation <strong>of</strong> labour market programmes<br />
such as SYETP.
260<br />
Appendix 1 Data appendix<br />
Table 1 Description <strong>of</strong> <strong>the</strong> data<br />
Description Derivation and details <strong>of</strong> construction Variable<br />
name in<br />
data<br />
SYETP participation<br />
indicator<br />
Derived from <strong>the</strong> weekly calendar files for <strong>the</strong> 4 survey years, which<br />
cover <strong>the</strong> survey ‘reference period’. Dummy identifier with code 1 for<br />
all those that have a spell on SYETP in <strong>the</strong> 1984 survey reference<br />
period after 3 June 1984 or in 1985 survey reference period up until <strong>the</strong><br />
1985 interview date. All spells in <strong>the</strong> job calendar which are a job<br />
where <strong>the</strong>re is a wage received were asked if <strong>the</strong>y got <strong>the</strong> job through a<br />
government programme (SYETP, <strong>the</strong>n later Jobstart), up to a<br />
maximum <strong>of</strong> 4 spells in each year had this information collected<br />
although up to 8 spells in a year had start and end dates. The reference<br />
period in <strong>the</strong> 1984 survey started 1 January 1984 OR <strong>the</strong> week in<br />
which entered <strong>the</strong> labour force if later. Reference periods in <strong>the</strong> 3<br />
surveys 1985-1987 started a week after <strong>the</strong> interview in <strong>the</strong> former<br />
survey year, and ended in <strong>the</strong> week <strong>of</strong> <strong>the</strong> current interview. The<br />
calendars correspond to roughly covering a yearly period. The<br />
treatment group, described by <strong>the</strong> SYETP variable, has differing<br />
programme participation start dates, as described by Richardson (1998)<br />
p5.<br />
SYETP<br />
Sex indicator Sex dummy, with values <strong>of</strong> 0 for male, 1 for female. female<br />
Marital status<br />
indicator, 1984<br />
survey<br />
Dummy with value <strong>of</strong> 1 if married or de facto, zero o<strong>the</strong>rwise. Mar84<br />
State initial interview<br />
conducted in 1984<br />
Ethnicity indicator<br />
English language<br />
skills indicator<br />
Main area <strong>of</strong><br />
residence before<br />
respondent age 14<br />
Main area <strong>of</strong><br />
residence 1984<br />
survey<br />
Time varying marital status dummy, takes <strong>the</strong> value for marital status<br />
for each survey year, such as 1985 or 1986. Same construction as for<br />
mar84.<br />
Set <strong>of</strong> dummies. For each state dummy, value <strong>of</strong> 1 for <strong>the</strong> state. New<br />
South Wales/ACT, nswact; Victoria, vic; Queensland, qld; South<br />
Australia/Nor<strong>the</strong>rn Territories, sant; Western Australia/Tasmania,<br />
watas;<br />
Ethnic origin, equals stated ethnic origin if born Australia; o<strong>the</strong>rwise<br />
Asian if respondent or ei<strong>the</strong>r parent born Asia, O<strong>the</strong>r if not Asian and<br />
respondent or ei<strong>the</strong>r parent born Middle East, Pacific Oceania or o<strong>the</strong>r<br />
(principally Africa, Latin America), European if not Asian nor O<strong>the</strong>r.<br />
Set <strong>of</strong> 4 indicator dummies: aboriginal/Torres Strait Islander, roatsi;<br />
Asian, roasian; European, roeur; O<strong>the</strong>r, rooth.<br />
Set <strong>of</strong> dummies for English pr<strong>of</strong>iciency with value <strong>of</strong> 1 for efl if<br />
English natural language, else egood if good, epoor if fair, poor or very<br />
poor .<br />
Set <strong>of</strong> dummies with value <strong>of</strong> 1 for capital city, cc14; o<strong>the</strong>r city, oc14;<br />
country town, ct14; rural area, ra14; overseas, os14.<br />
Set <strong>of</strong> dummies with value <strong>of</strong> 1 for capital city, cc84; o<strong>the</strong>r city, oc84;<br />
country town, ct84; rural area, ra84;<br />
mar<br />
Nswact; vic<br />
qld; sant;<br />
watas<br />
Roeur,<br />
roatsi,<br />
roasian,<br />
rooth.<br />
Efl, egood,<br />
epoor.<br />
Cc14, oc14,<br />
ct14, ra14,<br />
os14.<br />
Cc84, oc84,<br />
ct84, ra84.
261<br />
Description Derivation and details <strong>of</strong> construction Variable<br />
name in<br />
data<br />
Lived with parents at The prefix m indicate mo<strong>the</strong>r, <strong>the</strong> prefix f indicates fa<strong>the</strong>r. mo<strong>the</strong>r not Mnpres14,<br />
respondent age 14<br />
Mo<strong>the</strong>r’s and<br />
Fa<strong>the</strong>r’s occupation<br />
at age respondent 14<br />
if lived with <strong>the</strong>m at<br />
14<br />
Mo<strong>the</strong>r’s and<br />
Fa<strong>the</strong>r’s highest<br />
educational<br />
qualification at<br />
respondent age 14 if<br />
lived with <strong>the</strong>m<br />
Attitude to working<br />
women<br />
Number <strong>of</strong> o<strong>the</strong>r<br />
children in family<br />
Religion<br />
Type <strong>of</strong> school last<br />
attended<br />
Highest overall<br />
qualification in 1984<br />
present, mnpres14; fa<strong>the</strong>r not present, fnpres14<br />
The prefix m indicate mo<strong>the</strong>r, <strong>the</strong> prefix f indicates fa<strong>the</strong>r. The 2-digit<br />
ASCO 169 codes, where dummies with value 1 for categories, for<br />
mo<strong>the</strong>r labourer, mlabo; plant operative, mplan; sales, msale; clerical,<br />
mcler; tradespeople, mtrad; not employed, mnemp; mmpp, for mo<strong>the</strong>r<br />
manager, pr<strong>of</strong>essional or para-pr<strong>of</strong>essional. For fa<strong>the</strong>r labourer, flabo;<br />
plant operative, fplan; sales, fsale; clerical, fcler; tradespeople, ftrad;<br />
not employed, fnemp; fmpp, for fa<strong>the</strong>r manager, pr<strong>of</strong>essional or parapr<strong>of</strong>essional.<br />
The prefix m indicate mo<strong>the</strong>r, <strong>the</strong> prefix f indicates fa<strong>the</strong>r. Mpsq and<br />
fpsq dummy where value <strong>of</strong> 1 for any post-school qualification<br />
including degree, trade/apprenticeship, o<strong>the</strong>r post-school qualification.<br />
Those with zero attended secondary school, primary school, no school.<br />
Based on questions that examine attitude to women in work. An<br />
underlying variable is created, where Sexist is an index between 0–7 <strong>of</strong><br />
how many questions respondents agreed or strongly agreed with<br />
reactionary attitudes to women and work. ksink5p is <strong>the</strong>n a dummy<br />
with value 1 for sexist greater than five respectively. An interaction<br />
variable ks5pfem interacts ksink5p with female.<br />
Total number <strong>of</strong> older and younger siblings (nsib).<br />
Religion brought up in as a set <strong>of</strong> dummies, with value 1 where Church<br />
<strong>of</strong> England, coe; Roman Catholic, cath; Presbyterian, pres; Methodist,<br />
meth; o<strong>the</strong>r Christian, othx; o<strong>the</strong>r religion, othrel; no religion, norel).<br />
Type <strong>of</strong> last school if went to school in Australia, dummies for<br />
government, lsgov; Roman Catholic, lsrcs; o<strong>the</strong>r type, including<br />
private, lspriv; lsos if last attended school overseas.<br />
Highest overall qualification in 1984 survey. Set <strong>of</strong> dummies, with<br />
value <strong>of</strong> 1 for degree, diploma/o<strong>the</strong>r cert Hq_dd84;<br />
apprenticeship/trainee/ATS, hq_tra84; o<strong>the</strong>r post-school, hq_oth84;<br />
years <strong>of</strong> school, year 12 hq12_84; years <strong>of</strong> school year 11 hq11_84;<br />
years <strong>of</strong> school year 10 hq10_84; hq9_84=1 if year school 9 or less.<br />
Time varying dummies for overall highest qualification. Takes <strong>the</strong><br />
values as above, for each later survey year 1985, 1986.<br />
fnpres14<br />
Mlabo,<br />
mplan.<br />
Msale,<br />
mcler,<br />
mtrad,<br />
mnemp,<br />
mmpp;<br />
Fabo, fplan.<br />
fsale, fcler,<br />
ftrad,<br />
fnemp,<br />
fmpp;<br />
Mpsq, fpsq<br />
Ksink5p;<br />
ks5pfem<br />
nsib<br />
Coe, cath,<br />
pres, meth,<br />
othx, othrel<br />
Lsgov,<br />
lsrcs,<br />
lspriv, lsos.<br />
Hqdd84,<br />
hqtra84,<br />
hqoth84,<br />
hq12_84,<br />
hq11_84,<br />
hq10_84,<br />
hq9_84<br />
Hq_dd,<br />
hq_tra,<br />
hq_oth,<br />
hq12, hq11,<br />
hq10, hq9<br />
169 <strong>Australian</strong> Standard Classification for Occupation, a skill-based classification <strong>of</strong> occupations as defined<br />
by <strong>the</strong> <strong>Australian</strong> Bureau <strong>of</strong> Statistics ABS Catalogue number: 1221.0 ASCO First Edition which was used<br />
to code occupation data in surveys to May 1996.
262<br />
Description Derivation and details <strong>of</strong> construction Variable<br />
name in<br />
data<br />
Work limited by Work limited by health, dummy with value <strong>of</strong> 1 if work restricted by Health84<br />
health in 1984 health in type or amount.<br />
Time varying ‘Work limited by health’, takes <strong>the</strong> values as above, for health<br />
Partner’s<br />
employment in 1984<br />
Children in 1984<br />
Past work experience<br />
to 1984<br />
each later survey year 1985, 1986.<br />
Spouse employed if living in household, 1984 survey. Dummy with<br />
value 1 if employed, 0 o<strong>the</strong>rwise.<br />
Time varying dummy for Spouse employed if living in household.<br />
Takes <strong>the</strong> values as above, for each later survey year 1985, 1986<br />
Own children living in household, 1984 survey.<br />
Also <strong>the</strong> interaction term ch84fem, where gender interacted with<br />
children female with child84.<br />
Time varying dummy. Takes <strong>the</strong> values as above, for each later survey<br />
year 1985, 1986<br />
Length <strong>of</strong> longest recorded job/business held up to <strong>the</strong> first interview<br />
in 1984. Includes jobs or self-employment spells. A set <strong>of</strong> dummies<br />
with value <strong>of</strong> 1 for work where, longj0, for less than 1 year; longj1 for<br />
1 year to less than 2 years; longj2 for 2 years to less than 3 years;<br />
longj3p for 3 or more years; longjno for never had a job/business.<br />
Spemp84<br />
spemp<br />
Child84,<br />
Ch84fem<br />
Child,<br />
childfem<br />
Longj0,<br />
Longj1,<br />
long j2,<br />
longj3p,<br />
long jno.<br />
Employment<br />
outcome<br />
Ever studying fulltime<br />
Referrals<br />
O<strong>the</strong>r programme<br />
participation<br />
Proportion <strong>of</strong> time<br />
spent unemployed in<br />
1984 survey<br />
reference period,<br />
prior to June 1984<br />
Those not in FTED<br />
ALS List sample<br />
Data syntax editors<br />
Employed in an unsubsidised, non-government programme job, at any<br />
time during <strong>the</strong> 1986 wave. This includes ‘retained’ jobs that were<br />
initially subsidised during 1984/85, excluding <strong>the</strong> first 17 weeks <strong>of</strong><br />
<strong>the</strong>ir duration.<br />
A dummy taking <strong>the</strong> value 1 if studying full-time in any wave up to<br />
that point, and also if ever studying full-time in <strong>the</strong> final wave.<br />
Referred to a Community Employment Programme place by <strong>the</strong> CES<br />
in 1984/1985<br />
Go on o<strong>the</strong>r government programme in 1984 or 1985, after 3 June<br />
1984.<br />
Government programme job in 1986<br />
Constructed from <strong>the</strong> calendar, where number <strong>of</strong> weeks unemployed to<br />
June 1984 is converted to a proportion <strong>of</strong> <strong>the</strong> time period, which is <strong>the</strong><br />
reference period in 1984 survey. The reference period in <strong>the</strong> 1984<br />
survey started 1 January 1984 OR <strong>the</strong> week in which entered <strong>the</strong> labour<br />
force if later.<br />
This variable identifies panel entry and exit, and when <strong>the</strong>y leave fulltime<br />
education, for example those in school or aged 25+ when<br />
interviewed in first wave. A value <strong>of</strong> zero means <strong>the</strong>y are not observed<br />
to leave FTED, ‘84’ <strong>the</strong>y had left FTED when observed in <strong>the</strong> 1984<br />
survey, and similarly for <strong>the</strong> later survey years. So those with value<br />
‘84’ have left FTED, and this is one <strong>of</strong> <strong>the</strong> exclusions used for <strong>the</strong><br />
analytical selection – only those with enteryr=84 are used, ie. Only<br />
those who had left FTED.<br />
Mcrae, I.; Parkinson, G.; Woyzbun, L. (1984-1987) <strong>Australian</strong><br />
Longitudinal Survey Waves 1 to 4, Level 1. Principal Investigators:<br />
Mcrae, I.; Parkinson, G.; Woyzbun, L.; Data collected by Roy Morgan<br />
Research Centre, Canberra. Ian Mcrae, Bureau <strong>of</strong> Labour Market<br />
Research, [producer], Canberra. Social Science Data Archives, The<br />
<strong>Australian</strong> National University, Canberra [distributor].<br />
npjob18<br />
stever<br />
Cepref84<br />
gpoth<br />
Gpjob86<br />
upropjn<br />
enteryr<br />
SSDA<br />
study<br />
numbers<br />
377, 410,<br />
420, 491.<br />
The authors <strong>of</strong> <strong>the</strong> syntax used to <strong>the</strong> derive <strong>the</strong> variables were Lorraine Dearden,<br />
Alex Heath, Henry Overman, James Richardson, and <strong>the</strong> syntax <strong>the</strong>y wrote takes <strong>the</strong><br />
ALS original SPSS files through extensive STATA processing.
Appendix 2 Tables<br />
263
264<br />
Table A2.0a Univariate probit for employment in 1986, as estimated in <strong>the</strong> bivariate<br />
probit replication <strong>of</strong> Richardson (1998)<br />
npjob18<br />
Coefficient<br />
(t stat)<br />
Marginal<br />
effects 170<br />
syetp 0.59 0.13<br />
(3.26)** (3.26)**<br />
Gender=female -0.41 -0.12<br />
(4.15)** (4.15)**<br />
Married 1984 -0.10 -0.03<br />
(0.51) (0.51)<br />
Children 1984 -0.25 -0.08<br />
(1.00) (1.00)<br />
Children*female -1.25 -0.45<br />
(3.84)** (3.84)**<br />
Spouse employed 1984 0.57 0.13<br />
(2.48)* (2.48)*<br />
Aboriginal/Torres Strait Islander -0.34 -0.11<br />
(1.48) (1.48)<br />
O<strong>the</strong>r ethnic minority -0.33 -0.10<br />
State interviewed in 1984 (1.66) (1.66)<br />
Victoria -0.05 -0.02<br />
(0.44) (0.44)<br />
Queensland -0.01 -0.00<br />
(0.05) (0.05)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.35 -0.11<br />
(2.33)* (2.33)*<br />
Western Australia/Tasmania -0.03 -0.01<br />
(0.22) (0.22)<br />
Education school overseas -0.05 -0.01<br />
(0.16) (0.16)<br />
Roman Catholic school -0.06 -0.02<br />
(0.37) (0.37)<br />
Private school 0.53 0.12<br />
Highest qualification in 1984 (1.79) (1.79)<br />
Degree/diploma 0.53 0.13<br />
(3.41)** (3.41)**<br />
Apprenticeship 0.42 0.10<br />
(1.97)* (1.97)*<br />
O<strong>the</strong>r Post-School qualification 0.06 0.02<br />
(0.35) (0.35)<br />
Year 12 <strong>of</strong> school -0.00 -0.00<br />
(0.01) (0.01)<br />
Year 11 <strong>of</strong> school 0.41 0.10<br />
(2.62)** (2.62)**<br />
Year 9 <strong>of</strong> school or less -0.25 -0.08<br />
(1.79) (1.79)<br />
Longest job by 1984 none 0.02 0.01<br />
(0.11) (0.11)<br />
< 1 year 0.23 0.07<br />
(1.79) (1.79)<br />
170 Log likelihood for <strong>the</strong> probit gives Marginal effect is {δΦ (xb) / δx i }| x = µ , where Φ is <strong>the</strong> cumulative<br />
standard normal, µ= mean.
2 years 0.28 0.07<br />
(1.70) (1.70)<br />
3 years + 0.63 0.15<br />
(3.71)** (3.71)**<br />
Enter o<strong>the</strong>r govt programme -0.66 -0.22<br />
(5.55)** (5.55)**<br />
duration <strong>of</strong> Pre-June 1984 unemployment -0.45 -0.13<br />
(3.87)** (3.87)**<br />
Work limited by health -0.36 -0.11<br />
Family background (3.07)** (3.07)**<br />
O<strong>the</strong>r city before aged 14 -0.27 -0.08<br />
(2.15)* (2.15)*<br />
Country town before aged 14 -0.07 -0.02<br />
(0.63) (0.63)<br />
Rural area before aged 14 -0.23 -0.07<br />
(1.25) (1.25)<br />
Overseas before aged 14 0.41 0.10<br />
(1.09) (1.09)<br />
Number <strong>of</strong> siblings -0.03 -0.01<br />
(1.64) (1.64)<br />
English good 0.39 0.10<br />
(1.77) (1.77)<br />
English poor 0.97 0.17<br />
(2.44)* (2.44)*<br />
Sexist -0.42 -0.14<br />
(2.19)* (2.19)*<br />
Sexist*female 0.38 0.09<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.02) (1.02)<br />
Fa<strong>the</strong>r not present when resp 14 -0.22 -0.07<br />
(0.94) (0.94)<br />
Labourer 0.12 0.03<br />
(0.48) (0.48)<br />
Plant operative 0.05 0.01<br />
(0.19) (0.19)<br />
Sales -0.07 -0.02<br />
(0.27) (0.27)<br />
Tradesperson -0.26 -0.08<br />
(1.18) (1.18)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.11 0.03<br />
(0.50) (0.50)<br />
Not employed 0.10 0.03<br />
(0.36) (0.36)<br />
Fa<strong>the</strong>r holds post-school qualification when resp 14 0.10 0.03<br />
Mo<strong>the</strong>rs occupation when resp. 14 (0.87) (0.87)<br />
Mo<strong>the</strong>r not present when resp 14 -0.25 -0.08<br />
(0.99) (0.99)<br />
Labourer -0.13 -0.04<br />
(0.53) (0.53)<br />
Plant operative -0.49 -0.16<br />
(1.92) (1.92)<br />
Labourer -0.38 -0.12<br />
(1.68) (1.68)<br />
Plant operative -0.20 -0.06<br />
(0.59) (0.59)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.21 0.06<br />
(0.91) (0.91)<br />
265
266<br />
Not employed -0.08 -0.02<br />
(0.41) (0.41)<br />
mo<strong>the</strong>r post-school qualification when resp 14 -0.04 -0.01<br />
Religion brought up in (0.27) (0.27)<br />
Catholic 0.36 0.10<br />
(2.85)** (2.85)**<br />
Presbyterian 0.49 0.11<br />
(2.35)* (2.35)*<br />
Methodist 0.15 0.04<br />
(0.84) (0.84)<br />
O<strong>the</strong>r Christian -0.08 -0.02<br />
(0.38) (0.38)<br />
O<strong>the</strong>r religion -0.03 -0.01<br />
(0.16) (0.16)<br />
No religion 0.35 0.09<br />
(2.11)* (2.11)*<br />
constant 1.16<br />
(3.65)**<br />
Observations 1283 1283<br />
Log likelihood -569.76<br />
LR chi 2 (59) 171 329.72<br />
Mcfadden’s Pseudo R 2 172 0.2244<br />
Akaike Information Criterion 0.98<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%<br />
171 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
172<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(Mfull) to a model with just <strong>the</strong> intercept (Mintercept). It is defined as R2 = 1 – (log likelihood Mfull / log<br />
likelihood Mintercept). The value <strong>of</strong> Mcfadden’s Pseudo R2 increases as new variables are added.
267<br />
Table A2.0b Univariate Probit <strong>of</strong> participation in SYETP, as estimated in <strong>the</strong> bivariate<br />
probit replication <strong>of</strong> Richardson (1998)<br />
SYETP<br />
Marginal effects<br />
Coefficient<br />
(t stat)<br />
Gender=female 0.08 0.01<br />
(0.67) (0.67)<br />
Age at 1984 survey -0.11 -0.01<br />
(3.21)** (3.21)**<br />
Married 1984 -0.97 -0.06<br />
(1.62) (1.62)<br />
Children 1984 0.49 0.07<br />
(0.74) (0.74)<br />
Children*female -0.33 -0.03<br />
(0.39) (0.39)<br />
Spouse employed 1984 0.60 0.09<br />
(0.91) (0.91)<br />
Aboriginal/Torres Strait Islander -0.45 -0.03<br />
(0.95) (0.95)<br />
O<strong>the</strong>r ethnic minority 0.06 0.01<br />
State interviewed in 1984 (0.25) (0.25)<br />
Victoria 0.11 0.01<br />
(0.73) (0.73)<br />
Queensland -0.21 -0.02<br />
(0.99) (0.99)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.14 -0.01<br />
(0.66) (0.66)<br />
Western Australia/Tasmania 0.32 0.04<br />
(1.81) (1.81)<br />
Education school overseas 0.14 0.02<br />
(0.38) (0.38)<br />
Roman Catholic school -0.30 -0.02<br />
(1.23) (1.23)<br />
Private school -0.73 -0.04<br />
Highest qualification in 1984 (1.56) (1.56)<br />
Degree/diploma 0.05 0.00<br />
(0.20) (0.20)<br />
Apprenticeship -0.15 -0.01<br />
(0.47) (0.47)<br />
O<strong>the</strong>r Post-School qualification 0.07 0.01<br />
(0.28) (0.28)<br />
Year 12 <strong>of</strong> school 0.40 0.05<br />
(2.25)* (2.25)*<br />
Year 11 <strong>of</strong> school 0.16 0.02<br />
(0.85) (0.85)<br />
Year 9 <strong>of</strong> school or less -0.11 -0.01<br />
(0.53) (0.53)<br />
Longest job by 1984 none -0.42 -0.03<br />
(1.68) (1.68)<br />
< 1 year -0.04 -0.00<br />
(0.24) (0.24)<br />
2 years 0.15 0.02<br />
(0.70) (0.70)<br />
3 years + -0.34 -0.03
(1.32) (1.32)<br />
CEP referrals 1984 0.16 0.02<br />
(2.36)* (2.36)*<br />
duration <strong>of</strong> Pre-June 1984 unemployment 0.46 0.05<br />
(2.79)** (2.79)**<br />
Work limited by health -0.60 -0.04<br />
Family background (2.32)* (2.32)*<br />
O<strong>the</strong>r city before aged 14 -0.25 -0.02<br />
(1.49) (1.49)<br />
Country town before aged 14 -0.43 -0.04<br />
(2.76)** (2.76)**<br />
Rural area before aged 14 -0.46 -0.03<br />
(1.70) (1.70)<br />
Overseas before aged 14 -0.71 -0.04<br />
(1.38) (1.38)<br />
Number <strong>of</strong> siblings -0.01 -0.00<br />
(0.83) (0.83)<br />
English good -0.12 -0.01<br />
(0.48) (0.48)<br />
English poor -0.56 -0.04<br />
(1.03) (1.03)<br />
Sexist 0.25 0.03<br />
(0.95) (0.95)<br />
Sexist*female -0.76 -0.04<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.24) (1.24)<br />
Fa<strong>the</strong>r not present when resp 14 -0.22 -0.02<br />
(0.79) (0.79)<br />
Labourer -0.16 -0.01<br />
(0.52) (0.52)<br />
Plant operative -0.21 -0.02<br />
(0.75) (0.75)<br />
Sales -0.10 -0.01<br />
(0.30) (0.30)<br />
Tradesperson -0.25 -0.02<br />
(0.90) (0.90)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.08 -0.01<br />
(0.33) (0.33)<br />
Not employed -0.39 -0.03<br />
(1.08) (1.08)<br />
Fa<strong>the</strong>r holds post-school qualification when resp 14 -0.30 -0.03<br />
Mo<strong>the</strong>rs occupation when resp. 14 (2.00)* (2.00)*<br />
Mo<strong>the</strong>r not present when resp 14 0.52 0.08<br />
(1.64) (1.64)<br />
Labourer 0.14 0.02<br />
(0.45) (0.45)<br />
Plant operative 0.65 0.10<br />
(2.10)* (2.10)*<br />
Sales 0.19 0.02<br />
(0.65) (0.65)<br />
Tradesperson 0.09 0.01<br />
(0.21) (0.21)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.23 -0.02<br />
(0.76) (0.76)<br />
Not employed 0.05 0.01<br />
(0.21) (0.21)<br />
Mo<strong>the</strong>r post-school qualification when resp 14 0.25 0.03<br />
268
269<br />
Religion brought up in (1.52) (1.52)<br />
Catholic 0.06 0.01<br />
(0.38) (0.38)<br />
Presbyterian 0.32 0.04<br />
(1.34) (1.34)<br />
Methodist 0.12 0.01<br />
(0.46) (0.46)<br />
O<strong>the</strong>r Christian 0.10 0.01<br />
(0.35) (0.35)<br />
O<strong>the</strong>r religion 0.16 0.02<br />
(0.72) (0.72)<br />
No religion 0.19 0.02<br />
(0.95) (0.95)<br />
constant 0.71<br />
(0.93)<br />
Observations 1283 1283<br />
Log likelihood -307.19<br />
LR chi 2 (59) 173 107.57<br />
Mcfadden’s Pseudo R 2 174 0.1490<br />
Akaike Information Criterion 0.57<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%<br />
173 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero. It is defined as LR = 2 (log likelihood M full – 2 log likelihood M intercept ). The degrees <strong>of</strong> freedom <strong>of</strong> this<br />
chi squared distributed statistic are equal to <strong>the</strong> number <strong>of</strong> constrained parameters i.e. <strong>the</strong> number <strong>of</strong><br />
coefficients being tested.<br />
174<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index. It compares <strong>the</strong> full model <strong>of</strong> parameters<br />
(M full ) to a model with just <strong>the</strong> intercept (M intercept ). It is defined as R 2 = 1 – (log likelihood M full / log<br />
likelihood M intercept ) . The value <strong>of</strong> Mcfadden’s Pseudo R 2 increases as new variables are added.
270<br />
Table A2.1 Means and bias after matching, one to one nearest neighbour 0.001 and 0.005 propensity score radius<br />
Matching results shown in<br />
Table 4.6<br />
0.001 propensity score radius 0.005 propensity score radius<br />
Means var Means var Bias Means var Means<br />
in <strong>the</strong><br />
in <strong>the</strong><br />
in <strong>the</strong><br />
in <strong>the</strong><br />
compari<br />
SYETP<br />
compari<br />
SYETP<br />
son<br />
group<br />
son<br />
group<br />
group<br />
matched<br />
group<br />
matched<br />
matched<br />
matched<br />
var<br />
Bias<br />
female Gender=female 0.49 0.25 0.45 0.25 8.00 0.5 0.25 0.45 0.25 10.00<br />
age84 Age at 1984 survey 19.16 5.34 19.11 3.78 2.34 19.03 5.27 19.1 3.83 3.28<br />
mar84 Married 1984 0.04 0.04 0.03 0.03 5.35 0.03 0.03 0.03 0.03 0.00<br />
child84 Children 1984 0 0 0.02 0.02 20.00 0 0 0.02 0.02 20.00<br />
ch84fem Children*female 0 0 0.01 0.01 14.14 0 0 0.01 0.01 14.14<br />
spemp84 Spouse employed 1984 0.04 0.04 0.02 0.02 11.55 0.03 0.03 0.02 0.02 6.32<br />
roatsi<br />
Aboriginal/Torres Strait<br />
Islander 0.01 0.01 0.01 0.01 0.00 0.01 0.01 0.01 0.01 0.00<br />
rones O<strong>the</strong>r ethnic minority 0.06 0.06 0.08 0.07 7.84 0.08 0.07 0.08 0.07 0.00<br />
vic<br />
State interviewed in<br />
1984Victoria 0.33 0.22 0.29 0.21 8.63 0.33 0.22 0.28 0.2 10.91<br />
qld Queensland 0.11 0.1 0.09 0.08 6.67 0.11 0.1 0.08 0.07 10.29<br />
sant<br />
South Australia/Nor<strong>the</strong>rn<br />
watas<br />
Territory 0.13 0.11 0.14 0.12 2.95 0.14 0.12 0.14 0.12 0.00<br />
Western<br />
Australia/Tasmania 0.1 0.09 0.15 0.13 15.08 0.09 0.08 0.2 0.16 31.75<br />
lsos Education school overseas 0.06 0.06 0.02 0.02 20.00 0.06 0.05 0.02 0.02 21.38<br />
lsrcs Roman Catholic school 0.04 0.04 0.07 0.06 13.42 0.05 0.04 0.06 0.06 4.47<br />
lspriv Private school 0.01 0.01 0.01 0.01 0.00 0.01 0.01 0.01 0.01 0.00<br />
hqdd84<br />
Highest qualification in<br />
1984 Degree/diploma 0.1 0.09 0.09 0.08 3.43 0.09 0.08 0.08 0.07 3.65<br />
hqtra84 Apprenticeship 0.03 0.02 0.03 0.03 0.00 0.02 0.02 0.03 0.03 6.32<br />
hqoth84<br />
O<strong>the</strong>r Post-School<br />
qualification 0.11 0.1 0.07 0.06 14.14 0.11 0.1 0.07 0.07 13.72<br />
hq12_84 Year 12 <strong>of</strong> school 0.14 0.12 0.2 0.16 16.04 0.14 0.12 0.23 0.18 23.24<br />
hq11_84 Year 11 <strong>of</strong> school 0.13 0.11 0.2 0.16 19.05 0.15 0.13 0.18 0.15 8.02
271<br />
hq9_84 Year 9 <strong>of</strong> school or less 0.09 0.08 0.1 0.09 3.43 0.09 0.08 0.11 0.1 6.67<br />
longjno Longest job by 1984 none 0.16 0.14 0.14 0.12 5.55 0.15 0.13 0.12 0.11 8.66<br />
longj0 < 1 year 0.48 0.25 0.52 0.25 8.00 0.49 0.25 0.54 0.25 10.00<br />
longj2 2 years 0.19 0.16 0.14 0.12 13.36 0.22 0.17 0.14 0.12 21.01<br />
longj3p 3 years + 0.05 0.05 0.07 0.06 8.53 0.05 0.04 0.06 0.06 4.47<br />
cepref84 CEP referrals 1984 0.49 1.48 0.31 1.03 16.07 0.5 1.59 0.27 0.9 20.61<br />
upropjn duration <strong>of</strong> Pre-June 1984<br />
unemployment 0.66 0.16 0.67 0.14 2.58 0.67 0.16 0.68 0.14 2.58<br />
health84 Work limited by health 0.03 0.02 0.05 0.04 11.55 0.02 0.02 0.04 0.04 11.55<br />
oc14<br />
Family background O<strong>the</strong>r<br />
ct14<br />
city before aged 14 0.28 0.2 0.18 0.15 23.90 0.27 0.2 0.17 0.14 24.25<br />
Country town before aged<br />
14 0.19 0.16 0.21 0.17 4.92 0.18 0.15 0.18 0.15 0.00<br />
ra14 Rural area before aged 14 0.03 0.02 0.05 0.04 11.55 0.02 0.02 0.04 0.04 11.55<br />
os14 Overseas before aged 14 0 0 0.01 0.01 14.14 0 0 0.01 0.01 14.14<br />
nsib2 Number <strong>of</strong> siblings 4.29 120.03 3.14 4.47 14.58 4.09 108.22 2.99 4.17 14.67<br />
egood English good 0.09 0.08 0.08 0.07 3.65 0.09 0.08 0.08 0.07 3.65<br />
epoor English poor 0 0 0.01 0.01 14.14 0 0 0.01 0.01 14.14<br />
ksink5p Sexist 0.08 0.07 0.07 0.06 3.92 0.07 0.06 0.06 0.06 4.08<br />
ks5pfem Sexist*female 0.04 0.04 0.01 0.01 18.97 0.03 0.03 0.01 0.01 14.14<br />
fnpres14<br />
flabo<br />
Fa<strong>the</strong>r not present when<br />
resp 14 0.14 0.12 0.18 0.15 10.89 0.16 0.14 0.19 0.15 7.88<br />
Fa<strong>the</strong>rs occupation<br />
Labourer 0.06 0.06 0.09 0.08 11.34 0.08 0.07 0.09 0.08 3.65<br />
fplan Plant operative 0.16 0.14 0.16 0.14 0.00 0.16 0.14 0.17 0.14 2.67<br />
fsale Sales 0.08 0.07 0.06 0.05 8.16 0.07 0.06 0.05 0.05 8.53<br />
ftrad Tradesperson 0.1 0.09 0.16 0.14 17.69 0.1 0.09 0.15 0.13 15.08<br />
fmpp<br />
Manager/pr<strong>of</strong>essional/para<br />
-pr<strong>of</strong>essional 0.3 0.21 0.22 0.17 18.35 0.3 0.21 0.25 0.19 11.18<br />
fnemp Not employed 0.08 0.07 0.05 0.04 12.79 0.07 0.06 0.04 0.04 13.42<br />
fpsq<br />
mnpres14<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14 0.32 0.22 0.26 0.2 13.09 0.3 0.21 0.26 0.19 8.94<br />
Mo<strong>the</strong>r not present when<br />
resp 14 0.09 0.08 0.07 0.06 7.56 0.11 0.1 0.08 0.07 10.29<br />
mlabo Labourer 0.06 0.06 0.08 0.07 7.84 0.06 0.05 0.07 0.07 4.08<br />
mplan Plant operative 0.09 0.08 0.09 0.08 0.00 0.11 0.1 0.1 0.09 3.24<br />
msale Sales 0.11 0.1 0.09 0.08 6.67 0.1 0.09 0.09 0.08 3.43
272<br />
mtrad Tradesperson 0.03 0.02 0.02 0.02 7.07 0.05 0.04 0.03 0.03 10.69<br />
mmpp<br />
Manager/pr<strong>of</strong>essional/para<br />
-pr<strong>of</strong>essional 0.09 0.08 0.08 0.07 3.65 0.08 0.07 0.07 0.07 3.78<br />
mnemp Not employed 0.49 0.25 0.48 0.25 2.00 0.45 0.25 0.5 0.25 10.00<br />
mpsq<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14 0.15 0.13 0.2 0.16 13.13 0.14 0.12 0.2 0.16 16.04<br />
cath<br />
Religion brought up in<br />
Catholic 0.3 0.21 0.28 0.2 4.42 0.3 0.21 0.27 0.2 6.63<br />
pres Presbyterian 0.04 0.04 0.07 0.06 13.42 0.03 0.03 0.08 0.07 22.36<br />
meth Methodist 0.03 0.02 0.06 0.05 16.04 0.02 0.02 0.06 0.06 20.00<br />
othx O<strong>the</strong>r Christian 0.08 0.07 0.07 0.06 3.92 0.08 0.07 0.07 0.07 3.78<br />
othrel O<strong>the</strong>r religion 0.06 0.06 0.08 0.07 7.84 0.07 0.06 0.08 0.07 3.92<br />
norel No religion 0.19 0.16 0.13 0.11 16.33 0.22 0.17 0.14 0.12 21.01<br />
Average bias 9.66 9.90
273<br />
Table A2.1 Continued Means and bias after matching, nearest neighbour one to one 0.01 and 0.05 propensity score radius<br />
Matching results shown in<br />
Table 4.6<br />
0.01 propensity score radius 0.05 propensity score radius<br />
Means var Means var Bias Means var Means<br />
in <strong>the</strong><br />
in <strong>the</strong><br />
in <strong>the</strong><br />
in <strong>the</strong><br />
compari<br />
SYETP<br />
control<br />
SYETP<br />
son<br />
group<br />
group<br />
group<br />
group<br />
matched<br />
matched<br />
matched<br />
matched<br />
var<br />
Bias<br />
female Gender=female 0.51 0.25 0.44 0.25 14.00 0.51 0.25 0.43 0.25 16.00<br />
age84 Age at 1984 survey 19.01 5.26 19.08 3.84 3.28 19.01 5.26 19.03 3.89 0.94<br />
mar84 Married 1984 0.03 0.03 0.03 0.03 0.00 0.03 0.03 0.03 0.03 0.00<br />
child84 Children 1984 0 0 0.02 0.02 20.00 0 0 0.02 0.02 20.00<br />
ch84fem Children*female 0 0 0.01 0.01 14.14 0 0 0.01 0.01 14.14<br />
spemp84 Spouse employed 1984 0.03 0.03 0.02 0.02 6.32 0.03 0.03 0.02 0.02 6.32<br />
roatsi<br />
Aboriginal/Torres Strait<br />
Islander 0.01 0.01 0.01 0.01 0.00 0.01 0.01 0.01 0.01 0.00<br />
rones O<strong>the</strong>r ethnic minority 0.08 0.07 0.08 0.07 0.00 0.08 0.07 0.08 0.07 0.00<br />
vic<br />
State interviewed in<br />
1984Victoria 0.33 0.22 0.27 0.2 13.09 0.33 0.22 0.27 0.2 13.09<br />
qld Queensland 0.11 0.1 0.09 0.08 6.67 0.11 0.1 0.09 0.08 6.67<br />
sant<br />
South Australia/Nor<strong>the</strong>rn<br />
watas<br />
Territory 0.13 0.12 0.14 0.12 2.89 0.13 0.12 0.13 0.12 0.00<br />
Western<br />
Australia/Tasmania 0.09 0.08 0.2 0.16 31.75 0.09 0.08 0.2 0.16 31.75<br />
lsos Education school overseas 0.06 0.05 0.03 0.03 15.00 0.06 0.05 0.04 0.04 9.43<br />
lsrcs Roman Catholic school 0.04 0.04 0.06 0.06 8.94 0.04 0.04 0.06 0.05 9.43<br />
lspriv Private school 0.01 0.01 0.01 0.01 0.00 0.01 0.01 0.01 0.01 0.00<br />
hqdd84<br />
Highest qualification in<br />
1984 Degree/diploma 0.09 0.08 0.08 0.07 3.65 0.09 0.08 0.08 0.07 3.65<br />
hqtra84 Apprenticeship 0.02 0.02 0.03 0.03 6.32 0.02 0.02 0.03 0.03 6.32<br />
hqoth84<br />
O<strong>the</strong>r Post-School<br />
qualification 0.11 0.1 0.07 0.06 14.14 0.11 0.1 0.07 0.06 14.14<br />
hq12_84 Year 12 <strong>of</strong> school 0.13 0.12 0.23 0.18 25.82 0.13 0.12 0.23 0.18 25.82<br />
hq11_84 Year 11 <strong>of</strong> school 0.16 0.13 0.18 0.15 5.35 0.16 0.13 0.17 0.14 2.72
274<br />
hq9_84 Year 9 <strong>of</strong> school or less 0.09 0.08 0.11 0.1 6.67 0.09 0.08 0.11 0.1 6.67<br />
longjno Longest job by 1984 none 0.15 0.13 0.12 0.1 8.85 0.15 0.13 0.12 0.1 8.85<br />
longj0 < 1 year 0.49 0.25 0.55 0.25 12.00 0.49 0.25 0.56 0.25 14.00<br />
longj2 2 years 0.21 0.17 0.14 0.12 18.38 0.21 0.17 0.13 0.12 21.01<br />
longj3p 3 years + 0.04 0.04 0.06 0.06 8.94 0.04 0.04 0.06 0.05 9.43<br />
cepref84 CEP referrals 1984 0.49 1.57 0.29 0.96 17.78 0.49 1.57 0.3 0.95 16.93<br />
upropjn duration <strong>of</strong> Pre-June 1984<br />
unemployment 0.67 0.16 0.68 0.14 2.58 0.67 0.16 0.68 0.14 2.58<br />
health84 Work limited by health 0.02 0.02 0.04 0.04 11.55 0.02 0.02 0.04 0.04 11.55<br />
oc14<br />
Family background O<strong>the</strong>r<br />
ct14<br />
city before aged 14 0.27 0.2 0.17 0.14 24.25 0.27 0.2 0.16 0.14 26.68<br />
Country town before aged<br />
14 0.19 0.16 0.18 0.15 2.54 0.19 0.16 0.17 0.14 5.16<br />
ra14 Rural area before aged 14 0.02 0.02 0.04 0.04 11.55 0.02 0.02 0.04 0.04 11.55<br />
os14 Overseas before aged 14 0 0 0.01 0.01 14.14 0 0 0.01 0.01 14.14<br />
nsib2 Number <strong>of</strong> siblings 4.07 107.04 2.98 4.14 14.62 4.07 107.04 2.97 4.07 14.76<br />
egood English good 0.09 0.08 0.08 0.07 3.65 0.09 0.08 0.08 0.07 3.65<br />
epoor English poor 0 0 0.01 0.01 14.14 0 0 0.01 0.01 14.14<br />
ksink5p Sexist 0.07 0.06 0.06 0.06 4.08 0.07 0.06 0.07 0.06 0.00<br />
ks5pfem Sexist*female 0.03 0.03 0.01 0.01 14.14 0.03 0.03 0.01 0.01 14.14<br />
fnpres14<br />
flabo<br />
Fa<strong>the</strong>r not present when<br />
resp 14 0.16 0.13 0.2 0.16 10.50 0.16 0.13 0.19 0.16 7.88<br />
Fa<strong>the</strong>rs occupation<br />
Labourer 0.08 0.07 0.09 0.08 3.65 0.08 0.07 0.1 0.09 7.07<br />
fplan Plant operative 0.17 0.14 0.17 0.14 0.00 0.17 0.14 0.16 0.14 2.67<br />
fsale Sales 0.07 0.06 0.05 0.05 8.53 0.07 0.06 0.05 0.05 8.53<br />
ftrad Tradesperson 0.1 0.09 0.15 0.13 15.08 0.1 0.09 0.14 0.12 12.34<br />
fmpp<br />
Manager/pr<strong>of</strong>essional/para<br />
-pr<strong>of</strong>essional 0.29 0.21 0.25 0.19 8.94 0.29 0.21 0.25 0.19 8.94<br />
fnemp Not employed 0.07 0.06 0.04 0.04 13.42 0.07 0.06 0.04 0.04 13.42<br />
fpsq<br />
mnpres14<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14 0.29 0.21 0.25 0.19 8.94 0.29 0.21 0.26 0.19 6.71<br />
Mo<strong>the</strong>r not present when<br />
resp 14 0.11 0.1 0.09 0.08 6.67 0.11 0.1 0.09 0.08 6.67<br />
mlabo Labourer 0.06 0.05 0.07 0.06 4.26 0.06 0.05 0.08 0.07 8.16<br />
mplan Plant operative 0.12 0.11 0.1 0.09 6.32 0.12 0.11 0.1 0.09 6.32<br />
msale Sales 0.1 0.09 0.09 0.08 3.43 0.1 0.09 0.1 0.09 0.00
275<br />
mtrad Tradesperson 0.04 0.04 0.03 0.03 5.35 0.04 0.04 0.03 0.03 5.35<br />
mmpp<br />
Manager/pr<strong>of</strong>essional/para<br />
-pr<strong>of</strong>essional 0.08 0.07 0.07 0.06 3.92 0.08 0.07 0.07 0.06 3.92<br />
mnemp Not employed 0.45 0.25 0.49 0.25 8.00 0.45 0.25 0.48 0.25 6.00<br />
mpsq<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14 0.15 0.13 0.2 0.16 13.13 0.15 0.13 0.2 0.16 13.13<br />
cath<br />
Religion brought up in<br />
Catholic 0.29 0.21 0.26 0.2 6.63 0.29 0.21 0.26 0.19 6.71<br />
pres Presbyterian 0.04 0.04 0.08 0.07 17.06 0.04 0.04 0.08 0.07 17.06<br />
meth Methodist 0.02 0.02 0.06 0.06 20.00 0.02 0.02 0.06 0.05 21.38<br />
othx O<strong>the</strong>r Christian 0.08 0.07 0.07 0.06 3.92 0.08 0.07 0.07 0.06 3.92<br />
othrel O<strong>the</strong>r religion 0.07 0.06 0.09 0.08 7.56 0.07 0.06 0.09 0.08 7.56<br />
norel No religion 0.21 0.17 0.14 0.12 18.38 0.21 0.17 0.15 0.13 15.49<br />
Average bias 9.74 9.57
276<br />
Table A2.2 Applying Different methods for <strong>the</strong> missing cases in <strong>the</strong> SYETP probit before<br />
sample reduction<br />
Mean substitution method<br />
for missing values for some<br />
Regressor variables<br />
Using Missing dummy<br />
Before<br />
sample<br />
reduction,<br />
unweighted<br />
Before<br />
sample<br />
reduction,<br />
with weight<br />
Deletion method for<br />
missing values for some<br />
Regressor variables<br />
No missing dummy,<br />
where missing cases<br />
dropped<br />
Before<br />
sample<br />
reduction,<br />
unweighted<br />
Before<br />
sample<br />
reduction,<br />
with weight<br />
(1) (2) (3) (4)<br />
syetp syetp syetp syetp<br />
Gender=female -0.02 -0.03 -0.02 -0.03<br />
(0.16) (0.33) (0.23) (0.33)<br />
Age at 1984 survey -0.08 -0.06 -0.07 -0.05<br />
(3.21)** (2.41)* (2.84)** (1.94)<br />
Married 1984 -0.59 -0.68 -0.61 -0.72<br />
(1.78) (2.66)** (1.69) (2.31)*<br />
Children 1984 0.48 0.38 -0.01 -0.11<br />
(1.17) (1.14) (0.02) (0.24)<br />
Children*female -0.17 0.16 0.06 0.43<br />
(0.30) (0.35) (0.09) (0.69)<br />
Spouse employed 1984 0.15 0.20 0.25 0.33<br />
(0.34) (0.56) (0.52) (0.81)<br />
Aboriginal/Torres Strait Islander -0.73 -0.70 -0.70 -0.66<br />
(1.78) (1.73) (1.64) (1.61)<br />
O<strong>the</strong>r ethnic minority 0.07 -0.03 0.08 -0.03<br />
State interviewed in 1984 (0.35) (0.16) (0.40) (0.16)<br />
Victoria 0.13 0.15 0.05 0.08<br />
(1.12) (1.17) (0.43) (0.61)<br />
Queensland -0.05 -0.01 -0.08 -0.06<br />
(0.35) (0.07) (0.53) (0.38)<br />
South Australia/Nor<strong>the</strong>rn Territory 0.12 0.13 0.12 0.14<br />
(0.78) (0.82) (0.76) (0.89)<br />
Western Australia/Tasmania 0.31 0.35 0.32 0.37<br />
(2.24)* (2.44)* (2.18)* (2.47)*<br />
Education school overseas 0.07 0.24 -0.05 0.15<br />
(0.28) (0.90) (0.18) (0.50)<br />
Roman Catholic school -0.38 -0.40 -0.37 -0.39<br />
(2.07)* (2.21)* (1.92) (2.01)*<br />
Private school -0.75 -1.01 -0.80 -1.06<br />
Highest qualification in 1984 (1.85) (2.56)* (1.90) (2.71)**<br />
Degree/diploma -0.11 -0.12 -0.15 -0.18<br />
(0.62) (0.65) (0.80) (0.94)<br />
Apprenticeship -0.33 -0.29 -0.29 -0.23<br />
(1.32) (1.22) (1.13) (0.98)<br />
O<strong>the</strong>r Post-School qualification -0.09 -0.00 -0.13 -0.11<br />
(0.44) (0.01) (0.61) (0.55)<br />
Year 12 <strong>of</strong> school 0.15 0.23 0.12 0.18<br />
(1.12) (1.60) (0.83) (1.25)<br />
Year 11 <strong>of</strong> school 0.06 0.10 0.06 0.08<br />
(0.45) (0.73) (0.42) (0.56)
277<br />
Year 9 <strong>of</strong> school or less -0.19 -0.10 -0.09 0.00<br />
(1.30) (0.63) (0.61) (0.01)<br />
Longest job by 1984 none -0.42 -0.34 -0.31 -0.24<br />
(2.20)* (1.83) (1.63) (1.29)<br />
< 1 year -0.04 -0.04 0.03 0.01<br />
(0.30) (0.32) (0.22) (0.08)<br />
2 years -0.03 -0.07 -0.01 -0.08<br />
(0.18) (0.39) (0.04) (0.43)<br />
3 years + -0.35 -0.47 -0.30 -0.45<br />
(1.82) (2.58)** (1.51) (2.43)*<br />
CEP referrals 1984 0.08 0.08 0.09 0.09<br />
(1.66) (1.48) (1.84) (1.63)<br />
Duration <strong>of</strong> Pre-June 1984 unemployment 0.40 0.30 0.36 0.24<br />
(3.24)** (2.64)** (2.85)** (2.03)*<br />
Work limited by health -0.50 -0.60 -0.48 -0.58<br />
Family background (2.69)** (3.33)** (2.53)* (3.14)**<br />
O<strong>the</strong>r city before aged 14 -0.11 -0.21 -0.15 -0.25<br />
(0.89) (1.69) (1.18) (1.97)*<br />
Country town before aged 14 -0.32 -0.36 -0.35 -0.41<br />
(2.62)** (2.83)** (2.77)** (3.16)**<br />
Rural area before aged 14 -0.28 -0.39 -0.24 -0.35<br />
(1.38) (1.86) (1.15) (1.61)<br />
Overseas before aged 14 -0.73 -0.66 -0.71 -0.65<br />
(1.59) (1.58) (1.53) (1.54)<br />
Number <strong>of</strong> siblings 0.01 0.01 -0.00 0.00<br />
(0.58) (0.60) (0.29) (0.10)<br />
English good -0.13 -0.18 -0.24 -0.27<br />
(0.68) (1.03) (1.18) (1.40)<br />
English poor -0.68 -0.74 -0.63 -0.68<br />
(1.50) (1.50) (1.36) (1.37)<br />
Sexist 0.11 0.19 0.04 0.08<br />
(0.59) (0.97) (0.20) (0.36)<br />
Sexist*female -0.66 -0.76 -0.57 -0.62<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.24) (1.67) (1.06) (1.30)<br />
Fa<strong>the</strong>r not present when resp 14 -0.21 -0.24 -0.19 -0.21<br />
(1.02) (1.16) (0.88) (1.00)<br />
Labourer -0.06 0.00 -0.07 0.01<br />
(0.28) (0.02) (0.31) (0.04)<br />
Plant operative -0.13 -0.09 -0.16 -0.13<br />
(0.65) (0.47) (0.75) (0.61)<br />
Sales -0.19 -0.27 -0.16 -0.24<br />
(0.70) (1.00) (0.56) (0.85)<br />
Tradesperson -0.05 -0.05 -0.09 -0.09<br />
(0.24) (0.27) (0.42) (0.46)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.04 -0.08 -0.04 -0.09<br />
(0.21) (0.44) (0.19) (0.42)<br />
Not employed -0.31 -0.23 -0.28 -0.20<br />
(1.13) (0.90) (0.98) (0.75)<br />
Fa<strong>the</strong>r holds post-school qualification when resp -0.29 -0.26 -0.29 -0.25<br />
14<br />
Mo<strong>the</strong>rs occupation when resp. 14 (2.60)** (2.24)* (2.49)* (2.08)*<br />
Mo<strong>the</strong>r not present when resp 14 0.34 0.29 0.31 0.21<br />
(1.39) (1.25) (1.23) (0.87)<br />
Labourer 0.12 0.08 0.08 -0.04<br />
(0.55) (0.38) (0.33) (0.20)<br />
Plant operative 0.38 0.31 0.36 0.25
278<br />
(1.62) (1.30) (1.46) (1.01)<br />
Sales 0.18 0.26 0.05 0.10<br />
(0.79) (1.12) (0.22) (0.44)<br />
Tradesperson 0.02 -0.02 -0.05 -0.13<br />
(0.05) (0.06) (0.13) (0.36)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.14 -0.19 -0.19 -0.27<br />
(0.60) (0.83) (0.79) (1.14)<br />
Not employed 0.08 -0.00 -0.02 -0.13<br />
(0.46) (0.03) (0.13) (0.74)<br />
M<strong>the</strong>r post-school qualification when resp 14 0.21 0.19 0.20 0.19<br />
Religion brought up in (1.64) (1.45) (1.59) (1.39)<br />
Catholic 0.16 0.12 0.10 0.06<br />
(1.27) (0.97) (0.81) (0.46)<br />
Presbyterian 0.28 0.21 0.30 0.24<br />
(1.46) (1.05) (1.57) (1.22)<br />
Methodist 0.12 0.24 0.11 0.22<br />
(0.60) (1.11) (0.51) (1.03)<br />
O<strong>the</strong>r Christian -0.06 -0.11 -0.04 -0.08<br />
(0.26) (0.52) (0.17) (0.36)<br />
O<strong>the</strong>r religion 0.16 0.22 0.17 0.22<br />
(0.94) (1.16) (0.94) (1.14)<br />
No religion 0.14 0.17 0.08 0.11<br />
(0.94) (1.08) (0.55) (0.69)<br />
missing fa<strong>the</strong>rs qualifications -0.49 -0.55<br />
(1.60) (1.69)<br />
missing mo<strong>the</strong>rs qualifications 0.01 0.10<br />
(0.02) (0.42)<br />
missing proportion <strong>of</strong> time spent unemployed -0.07 -0.08<br />
(0.12) (0.16)<br />
missing number <strong>of</strong> siblings 0.20 0.17<br />
(0.79) (0.68)<br />
Constant -0.01 -0.33 0.04 -0.26<br />
(0.03) (0.60) (0.07) (0.45)<br />
Observations 2368 2368 2150 2150<br />
Log likelihood -503.74 -500.54 -473.52 -467.48<br />
LR chi 2 175 (63) 137.31 (63) 150.58 (59) 125.41 (59) 134.91<br />
Mcfadden’s Pseudo R 2 176<br />
0.1199 0.1284 0.1169 0.1253<br />
Akaike Information Criterion 0.48 0.48 0.50 0.49<br />
Coefficient with t statistic in brackets. Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%, **<br />
significant at 1%. Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT,<br />
government school, highest qualification in 1984 year 10 at school, Longest job by 1984 is 1 year, lived<br />
mostly in state capital city until respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when<br />
respondent aged 14, mo<strong>the</strong>r clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
175 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero.<br />
176<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index.
279<br />
Table A2.3 Probit <strong>of</strong> SYETP participation, no missing dummies<br />
Before sample<br />
reduction,<br />
unweighted<br />
Before sample<br />
reduction, with<br />
response weight<br />
After sample<br />
reduction,<br />
unweighted<br />
After sample<br />
reduction, with<br />
response weight<br />
(1) (2) (3) (4)<br />
syetp syetp syetp syetp<br />
Gender=female -0.02 -0.03 0.08 0.06<br />
(0.23) (0.33) (0.66) (0.48)<br />
Age at 1984 survey -0.07 -0.05 -0.11 -0.08<br />
(2.84)** (1.94) (3.21)** (2.86)**<br />
Married 1984 -0.61 -0.72 -0.97 -1.09<br />
(1.69) (2.31)* (1.62) (4.20)**<br />
Children 1984 -0.01 -0.11 0.49 0.33<br />
(0.02) (0.24) (0.73) (0.82)<br />
Children*female 0.06 0.43 -0.30 0.03<br />
(0.09) (0.69) (0.36) (0.06)<br />
Spouse employed 1984 0.25 0.33 0.59 0.70<br />
(0.52) (0.81) (0.91) (1.93)<br />
Aboriginal/Torres Strait Islander -0.70 -0.66 -0.45 -0.37<br />
(1.64) (1.61) (0.96) (0.78)<br />
O<strong>the</strong>r ethnic minority 0.08 -0.03 0.06 0.00<br />
State interviewed in 1984 (0.40) (0.16) (0.24) (0.01)<br />
Victoria 0.05 0.08 0.12 0.11<br />
(0.43) (0.61) (0.75) (0.70)<br />
Queensland -0.08 -0.06 -0.20 -0.13<br />
(0.53) (0.38) (0.97) (0.66)<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory<br />
0.12 0.14 -0.12 -0.11<br />
(0.76) (0.89) (0.61) (0.59)<br />
Western Australia/Tasmania 0.32 0.37 0.32 0.35<br />
(2.18)* (2.47)* (1.79) (2.02)*<br />
Education school overseas -0.05 0.15 0.14 0.38<br />
(0.18) (0.50) (0.40) (1.07)<br />
Roman Catholic school -0.37 -0.39 -0.31 -0.27<br />
(1.92) (2.01)* (1.28) (1.15)<br />
Private school -0.80 -1.06 -0.72 -0.97<br />
Highest qualification in 1984 (1.90) (2.71)** (1.54) (2.32)*<br />
Degree/diploma -0.15 -0.18 0.05 -0.01<br />
(0.80) (0.94) (0.21) (0.03)<br />
Apprenticeship -0.29 -0.23 -0.14 -0.12<br />
(1.13) (0.98) (0.43) (0.41)<br />
O<strong>the</strong>r Post-School qualification -0.13 -0.11 0.08 0.05<br />
(0.61) (0.55) (0.32) (0.19)<br />
Year 12 <strong>of</strong> school 0.12 0.18 0.41 0.41<br />
(0.83) (1.25) (2.27)* (2.29)*<br />
Year 11 <strong>of</strong> school 0.06 0.08 0.16 0.22<br />
(0.42) (0.56) (0.85) (1.14)<br />
Year 9 <strong>of</strong> school or less -0.09 0.00 -0.14 -0.02<br />
(0.61) (0.01) (0.68) (0.09)<br />
Longest job by 1984 none -0.31 -0.24 -0.41 -0.31<br />
(1.63) (1.29) (1.62) (1.33)<br />
< 1 year 0.03 0.01 -0.04 -0.12<br />
(0.22) (0.08) (0.22) (0.67)<br />
2 years -0.01 -0.08 0.15 0.04
280<br />
(0.04) (0.43) (0.66) (0.17)<br />
3 years + -0.30 -0.45 -0.34 -0.51<br />
(1.51) (2.43)* (1.30) (2.12)*<br />
CEP referrals 1984 0.09 0.09 0.15 0.12<br />
(1.84) (1.63) (2.27)* (1.64)<br />
Duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
0.36 0.24 0.47 0.34<br />
(2.85)** (2.03)* (2.85)** (2.46)*<br />
Work limited by health -0.48 -0.58 -0.59 -0.65<br />
Family background (2.53)* (3.14)** (2.30)* (2.84)**<br />
O<strong>the</strong>r city before aged 14 -0.15 -0.25 -0.25 -0.40<br />
(1.18) (1.97)* (1.51) (2.40)*<br />
Country town before aged 14 -0.35 -0.41 -0.43 -0.51<br />
(2.77)** (3.16)** (2.78)** (3.32)**<br />
Rural area before aged 14 -0.24 -0.35 -0.46 -0.39<br />
(1.15) (1.61) (1.70) (1.53)<br />
Overseas before aged 14 -0.71 -0.65 -0.69 -0.62<br />
(1.53) (1.54) (1.36) (1.32)<br />
Number <strong>of</strong> siblings -0.00 0.00 0.01 0.02<br />
(0.29) (0.10) (0.42) (0.56)<br />
English good -0.24 -0.27 -0.11 -0.14<br />
(1.18) (1.40) (0.43) (0.56)<br />
English poor -0.63 -0.68 -0.60 -0.73<br />
(1.36) (1.37) (1.11) (1.18)<br />
Sexist 0.04 0.08 0.25 0.31<br />
(0.20) (0.36) (0.95) (1.12)<br />
Sexist*female -0.57 -0.62 -0.76 -0.88<br />
Fa<strong>the</strong>rs occupation when resp. (1.06) (1.30) (1.25) (1.60)<br />
14<br />
Fa<strong>the</strong>r not present when resp 14 -0.19 -0.21 -0.22 -0.22<br />
(0.88) (1.00) (0.82) (0.81)<br />
Labourer -0.07 0.01 -0.17 -0.07<br />
(0.31) (0.04) (0.58) (0.24)<br />
Plant operative -0.16 -0.13 -0.22 -0.13<br />
(0.75) (0.61) (0.79) (0.51)<br />
Sales -0.16 -0.24 -0.09 -0.25<br />
(0.56) (0.85) (0.27) (0.71)<br />
Tradesperson -0.09 -0.09 -0.25 -0.27<br />
(0.42) (0.46) (0.91) (1.03)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.04 -0.09 -0.09 -0.19<br />
(0.19) (0.42) (0.35) (0.74)<br />
Not employed -0.28 -0.20 -0.41 -0.47<br />
(0.98) (0.75) (1.15) (1.42)<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14<br />
Mo<strong>the</strong>rs occupation when resp.<br />
14<br />
Mo<strong>the</strong>r not present when resp<br />
14<br />
-0.29 -0.25 -0.29 -0.22<br />
(2.49)* (2.08)* (1.97)* (1.61)<br />
0.31 0.21 0.47 0.37<br />
(1.23) (0.87) (1.48) (1.24)<br />
Labourer 0.08 -0.04 0.13 -0.04<br />
(0.33) (0.20) (0.43) (0.14)<br />
Plant operative 0.36 0.25 0.65 0.52<br />
(1.46) (1.01) (2.10)* (1.69)<br />
Sales 0.05 0.10 0.20 0.25
281<br />
(0.22) (0.44) (0.67) (0.89)<br />
Tradesperson -0.05 -0.13 0.11 0.13<br />
(0.13) (0.36) (0.25) (0.31)<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.19 -0.27 -0.23 -0.32<br />
(0.79) (1.14) (0.78) (1.11)<br />
Not employed -0.02 -0.13 0.03 -0.10<br />
(0.13) (0.74) (0.13) (0.45)<br />
Mo<strong>the</strong>r post-school qualification 0.20 0.19 0.25 0.24<br />
when resp 14<br />
Religion brought up in (1.59) (1.39) (1.55) (1.40)<br />
Catholic 0.10 0.06 0.05 -0.02<br />
(0.81) (0.46) (0.32) (0.11)<br />
Presbyterian 0.30 0.24 0.32 0.25<br />
(1.57) (1.22) (1.31) (1.05)<br />
Methodist 0.11 0.22 0.11 0.23<br />
(0.51) (1.03) (0.44) (0.96)<br />
O<strong>the</strong>r Christian -0.04 -0.08 0.09 0.10<br />
(0.17) (0.36) (0.34) (0.36)<br />
O<strong>the</strong>r religion 0.17 0.22 0.14 0.13<br />
(0.94) (1.14) (0.63) (0.54)<br />
No religion 0.08 0.11 0.17 0.16<br />
(0.55) (0.69) (0.84) (0.83)<br />
Constant 0.04 -0.26 0.66 0.39<br />
(0.07) (0.45) (0.86) (0.59)<br />
Observations 2150 2150 1283 1283<br />
Log likelihood -473.52 -467.48 -307.52 -307.63<br />
LR chi 2 177 (59) 125.41 (59) 134.91 (59) 106.91 (59) 126.23<br />
Mcfadden’s Pseudo R 2 178<br />
0.1169 0.1253 0.1481 0.1553<br />
Akaike Information Criterion 0.50 0.49 0.57 0.57<br />
Coefficient with t statistic in brackets. Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; **<br />
significant at 1%.<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
177 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero.<br />
178<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index.
282<br />
Table A2.4 Summary statistics for distribution <strong>of</strong> <strong>the</strong> weights constructed for attrition,<br />
non-response and survey design<br />
wt86<br />
Percentiles Smallest<br />
1% 0.457385 0.350283<br />
5% 0.619839 0.356677<br />
10% 0.769777 0.413743 Obs 1283<br />
25% 0.927171 0.421961 Sum <strong>of</strong> Wgt. 1283<br />
50% 1.158263 Mean 1.286986<br />
Largest Std. Dev. 0.596588<br />
75% 1.50368 3.763253<br />
90% 1.955461 4.14118 Variance 0.355917<br />
95% 2.445457 4.843395 Skewness 3.083631<br />
99% 3.289642 9.10959 Kurtosis 28.41954
283<br />
Table A2.5a Univariate probit for employment in 1986, as estimated in <strong>the</strong> bivariate<br />
probit, weighted for attrition<br />
Dy/Dx 179 , (t statistic)<br />
Employment<br />
(npjob18)<br />
syetp 180 0.13<br />
(3.06)**<br />
Gender=female -0.13<br />
(4.49)**<br />
Married 1984 0.01<br />
(0.08)<br />
Children 1984 -0.13<br />
(1.48)<br />
Children*female -0.46<br />
(3.74)**<br />
Spouse employed 1984 0.13<br />
(2.27)*<br />
Aboriginal/Torres Strait Islander -0.11<br />
(1.47)<br />
O<strong>the</strong>r ethnic minority -0.12<br />
State interviewed in 1984 (2.00)*<br />
Victoria -0.04<br />
(0.98)<br />
Queensland 0.01<br />
(0.31)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.16<br />
(3.15)**<br />
Western Australia/Tasmania -0.03<br />
(0.56)<br />
Education school overseas -0.05<br />
(0.54)<br />
Roman Catholic school -0.01<br />
(0.18)<br />
Private school 0.13<br />
Highest qualification in 1984 (2.00)*<br />
Degree/diploma 0.12<br />
(3.11)**<br />
Apprenticeship 0.09<br />
(1.73)<br />
O<strong>the</strong>r Post-School qualification 0.03<br />
(0.58)<br />
Year 12 <strong>of</strong> school -0.02<br />
(0.34)<br />
Year 11 <strong>of</strong> school 0.10<br />
(2.53)*<br />
Year 9 <strong>of</strong> school or less -0.09<br />
(2.02)*<br />
Longest job by 1984 none -0.01<br />
(0.22)<br />
179 Log likelihood for <strong>the</strong> probit gives Marginal effect is {δΦ (xb) / δx i }| x = µ , where Φ is <strong>the</strong> cumulative<br />
standard normal, µ= mean.<br />
180 Mean <strong>of</strong> SYETP is (.080747); 95 per cent confidence interval for dydx gives (0.066445, 0.190978).
1 year 0.06<br />
(1.70)<br />
2 years 0.06<br />
(1.31)<br />
3 years + 0.13<br />
(3.24)**<br />
O<strong>the</strong>r govt programme -0.25<br />
(5.44)**<br />
Duration <strong>of</strong> Pre-June 1984 unemployment -0.12<br />
(3.55)**<br />
Work limited by health -0.11<br />
Family background (2.77)**<br />
O<strong>the</strong>r city before aged 14 -0.09<br />
(2.27)*<br />
Country town before aged 14 -0.03<br />
(0.95)<br />
Rural area before aged 14 -0.14<br />
(2.31)*<br />
Overseas before aged 14 0.09<br />
(1.01)<br />
Number <strong>of</strong> siblings -0.01<br />
(1.72)<br />
English good 0.10<br />
(2.03)*<br />
English poor 0.17<br />
(2.56)*<br />
Sexist -0.12<br />
(1.84)<br />
Sexist*female 0.12<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.50)<br />
Fa<strong>the</strong>r not present when resp 14 -0.06<br />
(0.93)<br />
Labourer 0.05<br />
(0.74)<br />
Plant operative 0.04<br />
(0.68)<br />
Sales 0.01<br />
(0.14)<br />
Tradesperson -0.05<br />
(0.79)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.06<br />
(1.10)<br />
Not employed 0.02<br />
(0.27)<br />
Fa<strong>the</strong>r holds post-school qualification when resp 14 0.02<br />
Mo<strong>the</strong>rs occupation when resp. 14 (0.69)<br />
Mo<strong>the</strong>r not present when resp 14 -0.04<br />
(0.58)<br />
Labourer -0.05<br />
(0.61)<br />
Plant operative -0.15<br />
(1.73)<br />
Sales -0.12<br />
(1.69)<br />
Tradesperson -0.10<br />
(0.93)<br />
284
285<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional 0.02<br />
(0.24)<br />
Not employed -0.02<br />
(0.39)<br />
Mo<strong>the</strong>r post-school qualification when resp 14 -0.00<br />
Religion brought up in (0.02)<br />
Catholic 0.11<br />
(3.22)**<br />
Presbyterian 0.12<br />
(2.40)*<br />
Methodist 0.07<br />
(1.35)<br />
O<strong>the</strong>r Christian -0.03<br />
(0.49)<br />
O<strong>the</strong>r religion 0.00<br />
(0.08)<br />
No religion 0.10<br />
(2.49)*<br />
Observations 1283<br />
Log likelihood -555.44<br />
LR chi 2 (59) 181 288.06<br />
Mcfadden’s Pseudo R 2 182 0.2438<br />
Akaike Information Criterion 0.96<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%<br />
181 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero.<br />
182<br />
This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index.
286<br />
Table A2.5b Univariate probit <strong>of</strong> participation in SYETP, as estimated in <strong>the</strong> bivariate<br />
probit, weighted for attrition<br />
Dy/Dx 183 , (t statistic)<br />
syetp<br />
Gender=female 0.01<br />
(0.52)<br />
Age at 1984 survey -0.01<br />
(2.87)**<br />
Married 1984 -0.06<br />
(3.91)**<br />
Children 1984 0.04<br />
(0.85)<br />
Children*female -0.01<br />
(0.14)<br />
Spouse employed 1984 0.12<br />
(1.96)*<br />
Aboriginal/Torres Strait Islander -0.03<br />
(0.78)<br />
O<strong>the</strong>r ethnic minority -0.00<br />
State interviewed in 1984 (0.00)<br />
Victoria 0.01<br />
(0.58)<br />
Queensland -0.01<br />
(0.62)<br />
South Australia/Nor<strong>the</strong>rn Territory -0.01<br />
(0.63)<br />
Western Australia/Tasmania 0.05<br />
(2.18)*<br />
Education school overseas 0.05<br />
(1.04)<br />
Roman Catholic school -0.02<br />
(1.09)<br />
Private school -0.05<br />
Highest qualification in 1984 (2.25)*<br />
Degree/diploma -0.00<br />
(0.11)<br />
Apprenticeship -0.01<br />
(0.38)<br />
O<strong>the</strong>r Post-School qualification 0.01<br />
(0.24)<br />
Year 12 <strong>of</strong> school 0.05<br />
(2.19)*<br />
Year 11 <strong>of</strong> school 0.02<br />
(1.05)<br />
Year 9 <strong>of</strong> school or less -0.00<br />
(0.02)<br />
Longest job by 1984 none -0.03<br />
(1.37)<br />
< 1 year -0.01<br />
(0.47)<br />
2 years 0.01<br />
(0.25)<br />
183 Log likelihood for <strong>the</strong> probit gives Marginal effect is {δΦ (xb) / δx i }| x = µ , where Φ is <strong>the</strong> cumulative<br />
standard normal, µ= mean.
3 years + -0.04<br />
(2.13)*<br />
CEP referrals 1984 0.01<br />
(1.75)<br />
Duration <strong>of</strong> Pre-June 1984 unemployment 0.03<br />
(2.40)*<br />
Work limited by health -0.04<br />
Family background (2.91)**<br />
O<strong>the</strong>r city before aged 14 -0.03<br />
(2.39)*<br />
Country town before aged 14 -0.04<br />
(3.34)**<br />
Rural area before aged 14 -0.03<br />
(1.61)<br />
Overseas before aged 14 -0.04<br />
(1.25)<br />
Number <strong>of</strong> siblings -0.00<br />
(0.91)<br />
English good -0.01<br />
(0.56)<br />
English poor -0.04<br />
(1.06)<br />
Sexist 0.04<br />
(1.26)<br />
Sexist*female -0.04<br />
Fa<strong>the</strong>rs occupation when resp. 14 (1.54)<br />
Fa<strong>the</strong>r not present when resp 14 -0.02<br />
(0.79)<br />
Labourer -0.01<br />
(0.19)<br />
Plant operative -0.01<br />
(0.46)<br />
Sales -0.02<br />
(0.66)<br />
Tradesperson -0.02<br />
(1.02)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.02<br />
(0.66)<br />
Not employed -0.03<br />
(1.31)<br />
Fa<strong>the</strong>r holds post-school qualification when resp 14 -0.02<br />
Mo<strong>the</strong>rs occupation when resp. 14 (1.59)<br />
Mo<strong>the</strong>r not present when resp 14 0.05<br />
(1.24)<br />
Labourer -0.00<br />
(0.16)<br />
Plant operative 0.08<br />
(1.80)<br />
Sales 0.02<br />
(0.76)<br />
Tradesperson 0.02<br />
(0.46)<br />
Manager/pr<strong>of</strong>essional/para-pr<strong>of</strong>essional -0.03<br />
(1.10)<br />
Not employed -0.01<br />
(0.57)<br />
287
288<br />
Mo<strong>the</strong>r post-school qualification when resp 14 0.02<br />
(1.26)<br />
Religion brought up in -0.00<br />
Catholic (0.05)<br />
0.04<br />
Presbyterian (1.29)<br />
0.03<br />
Methodist (1.19)<br />
0.01<br />
O<strong>the</strong>r Christian (0.21)<br />
0.01<br />
O<strong>the</strong>r religion (0.56)<br />
0.02<br />
No religion (0.84)<br />
Observations 1283<br />
Log likelihood -303.41<br />
LR chi 2 (59) 184 131.99<br />
Mcfadden’s Pseudo R 2 185 0.1572<br />
Akaike Information Criterion 0.56<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%<br />
184 This is <strong>the</strong> likelihood ratio test <strong>of</strong> <strong>the</strong> hypo<strong>the</strong>sis that all coefficients except <strong>the</strong> intercept are equal to<br />
zero.<br />
185 This measure <strong>of</strong> fit is also known as <strong>the</strong> likelihood ratio index.
289<br />
Table A2.6 Part A Employment equation from bivariate probit where age included in<br />
employment model, attrition weights<br />
ever employed in 1986<br />
survey<br />
coefficient t statistic probability <strong>of</strong> t<br />
SYETP 1.06 0.77 0.440<br />
Age at 1984 survey 0.01 0.30 0.768<br />
Gender=female -0.46** -4.34 0.000<br />
Married 0.03 0.14 0.891<br />
Children -0.40 -1.48 0.140<br />
Children*female -1.27** -3.73 0.000<br />
Spouse employed 0.55** 2.22 0.027<br />
Aboriginal/Torres Strait<br />
Islander<br />
-0.34 -1.36 0.174<br />
O<strong>the</strong>r ethnic minority -0.38* -2.00 0.045<br />
Victoria -0.12 -0.99 0.324<br />
Queensland 0.05 0.34 0.731<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory<br />
-0.48** -2.90 0.004<br />
Western<br />
Australia/Tasmania<br />
-0.11 -0.64 0.523<br />
Education school overseas -0.22 -0.63 0.531<br />
Roman Catholic school -0.02 -0.08 0.935<br />
Private school 0.63* 1.95 0.051<br />
Degree/diploma 0.50** 2.84 0.004<br />
Apprenticeship 0.37 1.73 0.083<br />
O<strong>the</strong>r Post-School<br />
qualification<br />
0.09 0.50 0.616<br />
Year 12 <strong>of</strong> school -0.09 -0.49 0.621<br />
Year 11 <strong>of</strong> school 0.38* 2.17 0.030<br />
Year 9 <strong>of</strong> school or less -0.29* -2.03 0.043<br />
Longest job by 1984 none -0.01 -0.06 0.949<br />
< 1 year 0.24 1.68 0.093<br />
2 years 0.21 1.26 0.208<br />
3 years + 0.56** 3.11 0.002<br />
Enter o<strong>the</strong>r govt prog -0.73** -5.14 0.000<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment<br />
-0.45** -3.46 0.001<br />
Work limited by health -0.34* -2.42 0.016<br />
O<strong>the</strong>r city before aged 14 -0.26 -1.58 0.114<br />
Country town before aged<br />
14<br />
-0.07 -0.42 0.674<br />
Rural area before aged 14 -0.40 -1.73 0.084<br />
Overseas before aged 14 0.42 1.10 0.273<br />
Number <strong>of</strong> siblings -0.04 -1.66 0.097<br />
English good 0.43* 2.08 0.037<br />
English poor 1.03** 2.60 0.009<br />
Sexist -0.40* -1.95 0.051<br />
Sexist*female 0.61 1.56 0.119<br />
Fa<strong>the</strong>r not present when<br />
respondent 14<br />
-0.19 -0.85 0.398<br />
Fa<strong>the</strong>r Labourer 0.18 0.75 0.451<br />
Fa<strong>the</strong>r Plant operative 0.16 0.70 0.484<br />
Fa<strong>the</strong>r Sales 0.05 0.19 0.848<br />
Fa<strong>the</strong>r Tradesperson -0.15 -0.69 0.492
290<br />
Fa<strong>the</strong>r<br />
0.24 1.13 0.259<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
Fa<strong>the</strong>r Not employed 0.10 0.36 0.717<br />
Fa<strong>the</strong>r holds post-school 0.10 0.78 0.437<br />
qualification when resp 14<br />
Mo<strong>the</strong>r not present when -0.18 -0.67 0.501<br />
resp 14<br />
Mo<strong>the</strong>r Labourer -0.15 -0.61 0.539<br />
Mo<strong>the</strong>r Plant operative -0.52 -1.81 0.070<br />
Mo<strong>the</strong>r Sales -0.41 -1.77 0.076<br />
Mo<strong>the</strong>r Tradesperson -0.33 -0.98 0.326<br />
Mo<strong>the</strong>r<br />
0.07 0.30 0.766<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
Mo<strong>the</strong>r Not employed -0.07 -0.37 0.711<br />
Mo<strong>the</strong>r post-school -0.02 -0.14 0.890<br />
qualification when resp 14<br />
Catholic 0.43** 3.12 0.002<br />
Presbyterian 0.49* 2.10 0.036<br />
Methodist 0.25 1.14 0.253<br />
O<strong>the</strong>r Christian -0.10 -0.49 0.623<br />
O<strong>the</strong>r religion 0.01 0.04 0.965<br />
No religion 0.41* 2.24 0.025<br />
rho -0.28<br />
Wald test <strong>of</strong> Rho=0 (chi2 0.11<br />
(1) statistic)<br />
Observations 1283<br />
Log likelihood -858.68<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until respondent aged 14,<br />
English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r clerical worker when respondent<br />
aged 14, religion brought up in is Anglican.<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%.
291<br />
Table A2.6 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit where age<br />
included in employment model, attrition weights<br />
Model <strong>of</strong><br />
SYETP<br />
participation,<br />
1986 survey<br />
data<br />
coefficient t statistic probability <strong>of</strong> t<br />
Gender=female 0.07 0.53 0.596<br />
Age at 1984 survey -0.08** -2.90 0.004<br />
Married 1984 -1.00** -3.30 0.001<br />
Children 1984 0.35 0.86 0.389<br />
Children*female -0.10 -0.17 0.866<br />
Spouse employed 1984 0.68 1.72 0.086<br />
Aboriginal/Torres Strait<br />
Islander<br />
-0.38 -0.79 0.429<br />
O<strong>the</strong>r ethnic minority 0.01 0.05 0.961<br />
Victoria 0.09 0.56 0.574<br />
Queensland -0.14 -0.67 0.504<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory<br />
-0.12 -0.63 0.526<br />
Western<br />
Australia/Tasmania<br />
0.38* 2.20 0.028<br />
Education school overseas 0.35 0.85 0.396<br />
Roman Catholic school -0.26 -1.10 0.271<br />
Private school -0.90* -2.14 0.032<br />
Degree/diploma 0.01 0.04 0.965<br />
Apprenticeship -0.11 -0.38 0.706<br />
O<strong>the</strong>r Post-School<br />
qualification<br />
0.01 0.04 0.967<br />
Year 12 <strong>of</strong> school 0.41* 2.19 0.028<br />
Year 11 <strong>of</strong> school 0.18 0.90 0.370<br />
Year 9 <strong>of</strong> school or less 0.02 0.10 0.919<br />
Longest job by 1984 none -0.29 -1.03 0.304<br />
< 1 year -0.06 -0.33 0.745<br />
2 years 0.07 0.31 0.760<br />
3 years + -0.48* -1.92 0.054<br />
CEP referrals 1984 0.13 1.79 0.074<br />
Proportion <strong>of</strong> time Pre-June 0.34* 2.39 0.017<br />
1984 spent in<br />
unemployment<br />
Work limited by health -0.68** -2.92 0.003<br />
O<strong>the</strong>r city before aged 14 -0.39* -2.33 0.020<br />
Country town before aged<br />
14<br />
-0.53** -3.21 0.001<br />
Rural area before aged 14 -0.41 -1.58 0.115<br />
Overseas before aged 14 -0.61 -1.26 0.209<br />
Number <strong>of</strong> siblings -0.01 -0.89 0.375<br />
English good -0.17 -0.63 0.530<br />
English poor -0.68 -1.10 0.272<br />
Sexist 0.38 1.23 0.220<br />
Sexist*female -0.90 -1.59 0.111<br />
Fa<strong>the</strong>r not present when<br />
respondent aged 14<br />
-0.25 -0.84 0.403<br />
Fa<strong>the</strong>r Labourer -0.09 -0.29 0.772
292<br />
Fa<strong>the</strong>r Plant operative -0.15 -0.53 0.596<br />
Fa<strong>the</strong>r Sales -0.21 -0.60 0.546<br />
Fa<strong>the</strong>r Tradesperson -0.28 -1.07 0.287<br />
Fa<strong>the</strong>r<br />
-0.19 -0.71 0.475<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
Fa<strong>the</strong>r Not employed -0.44 -1.34 0.179<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14<br />
-0.23 -1.61 0.108<br />
Mo<strong>the</strong>r not present when<br />
resp 14<br />
0.37 1.22 0.222<br />
Mo<strong>the</strong>r Labourer -0.03 -0.09 0.926<br />
Mo<strong>the</strong>r Plant operative 0.58 1.81 0.070<br />
Mo<strong>the</strong>r Sales 0.22 0.78 0.435<br />
Mo<strong>the</strong>r Tradesperson 0.20 0.46 0.648<br />
Mo<strong>the</strong>r manager/ pr<strong>of</strong><br />
/para-pr<strong>of</strong>essional<br />
-0.32 -1.12 0.265<br />
Mo<strong>the</strong>r Not employed -0.13 -0.59 0.555<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14<br />
0.22 1.27 0.203<br />
Catholic 0.00 -0.03 0.977<br />
Presbyterian 0.33 1.34 0.179<br />
Methodist 0.25 0.93 0.351<br />
O<strong>the</strong>r Christian 0.06 0.20 0.838<br />
O<strong>the</strong>r religion 0.14 0.58 0.559<br />
No religion 0.14 0.69 0.492<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%.
293<br />
Table A2.7 Part A Employment equation from bivariate probit where CEP referrals<br />
included in employment model, attrition weights<br />
ever employed in 1986<br />
coefficient t statistic probability <strong>of</strong> t<br />
SYETP 1.03 0.83 0.404<br />
CEP referrals -0.03 -0.36 0.716<br />
Gender=female -0.46 -4.45 0.000<br />
Married 0.03 0.14 0.886<br />
Children -0.39 -1.46 0.143<br />
Children*female -1.27 -3.74 0.000<br />
Spouse employed 0.55 2.23 0.026<br />
Aboriginal/Torres Strait<br />
Islander -0.34 -1.36 0.175<br />
O<strong>the</strong>r ethnic minority -0.37 -1.96 0.051<br />
Victoria -0.13 -1.01 0.312<br />
Queensland 0.05 0.34 0.736<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory -0.48 -2.97 0.003<br />
Western<br />
Australia/Tasmania -0.12 -0.67 0.504<br />
Education school overseas -0.22 -0.65 0.518<br />
Roman Catholic school -0.02 -0.08 0.933<br />
Private school 0.64 1.96 0.050<br />
Degree/diploma 0.51 3.13 0.002<br />
Apprenticeship 0.37 1.73 0.084<br />
O<strong>the</strong>r Post-School<br />
qualification 0.10 0.54 0.590<br />
Year 12 <strong>of</strong> school -0.08 -0.48 0.631<br />
Year 11 <strong>of</strong> school 0.39 2.30 0.021<br />
Year 9 <strong>of</strong> school or less -0.29 -2.03 0.043<br />
Longest job by 1984 none -0.03 -0.18 0.860<br />
< 1 year 0.22 1.57 0.116<br />
2 years 0.22 1.32 0.185<br />
3 years + 0.57 3.14 0.002<br />
Enter o<strong>the</strong>r govt prog -0.72 -5.07 0.000<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment -0.44 -3.58 0.000<br />
Work limited by health -0.34 -2.44 0.014<br />
O<strong>the</strong>r city before aged 14 -0.26 -1.71 0.088<br />
Country town before aged<br />
14 -0.08 -0.53 0.595<br />
Rural area before aged 14 -0.40 -1.81 0.071<br />
Overseas before aged 14 0.40 1.07 0.284<br />
Number <strong>of</strong> siblings -0.04 -1.62 0.104<br />
English good 0.43 2.10 0.036<br />
English poor 1.03 2.61 0.009<br />
Sexist -0.40 -1.97 0.049<br />
Sexist*female 0.61 1.58 0.115<br />
Fa<strong>the</strong>r not present when<br />
respondent 14 -0.20 -0.87 0.383<br />
Fa<strong>the</strong>r Labourer 0.18 0.75 0.451<br />
Fa<strong>the</strong>r Plant operative 0.15 0.68 0.498<br />
Fa<strong>the</strong>r Sales 0.05 0.18 0.856
294<br />
Fa<strong>the</strong>r Tradesperson -0.16 -0.71 0.479<br />
Fa<strong>the</strong>r<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
0.24 1.13 0.257<br />
Fa<strong>the</strong>r Not employed 0.09 0.34 0.737<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14 0.09 0.78 0.436<br />
Mo<strong>the</strong>r not present when<br />
resp 14 -0.17 -0.64 0.519<br />
Mo<strong>the</strong>r Labourer -0.14 -0.58 0.564<br />
Mo<strong>the</strong>r Plant operative -0.51 -1.83 0.067<br />
Mo<strong>the</strong>r Sales -0.39 -1.73 0.083<br />
Mo<strong>the</strong>r Tradesperson -0.33 -0.98 0.326<br />
Mo<strong>the</strong>r<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
0.07 0.30 0.761<br />
Mo<strong>the</strong>r Not employed -0.06 -0.34 0.730<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14 -0.02 -0.11 0.910<br />
Catholic 0.43 3.19 0.001<br />
Presbyterian 0.50 2.17 0.030<br />
Methodist 0.25 1.19 0.235<br />
O<strong>the</strong>r Christian -0.11 -0.51 0.611<br />
O<strong>the</strong>r religion 0.01 0.04 0.967<br />
No religion 0.40 2.19 0.028<br />
1.13 2.83 0.005<br />
rho -0.26<br />
Wald test <strong>of</strong> Rho=0 (chi2 0.12<br />
(1) statistic)<br />
Observations 1283<br />
Log likelihood -858.64<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until respondent aged 14,<br />
English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r clerical worker when respondent<br />
aged 14, religion brought up in is Anglican.<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%; ** significant at 1%.
295<br />
Table A2.7 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit where CEP<br />
referrals included in employment model, attrition weights<br />
Participation equation<br />
from bivariate probit<br />
Model <strong>of</strong><br />
SYETP<br />
participation,<br />
1986 survey<br />
data<br />
coefficient t statistic probability <strong>of</strong> t<br />
Gender=female 0.07 0.53 0.595<br />
Age at 1984 survey -0.08 -2.74 0.006<br />
Married 1984 -1.00 -3.43 0.001<br />
Children 1984 0.35 0.86 0.389<br />
Children*female -0.10 -0.17 0.867<br />
Spouse employed 1984 0.68 1.74 0.081<br />
Aboriginal/Torres Strait<br />
Islander -0.38 -0.79 0.430<br />
O<strong>the</strong>r ethnic minority 0.01 0.04 0.965<br />
Victoria 0.09 0.57 0.571<br />
Queensland -0.14 -0.67 0.504<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory -0.12 -0.63 0.527<br />
Western<br />
Australia/Tasmania 0.38 2.20 0.028<br />
Education school overseas 0.35 0.86 0.387<br />
Roman Catholic school -0.26 -1.10 0.271<br />
Private school -0.90 -2.16 0.031<br />
Degree/diploma 0.01 0.03 0.975<br />
Apprenticeship -0.11 -0.38 0.706<br />
O<strong>the</strong>r Post-School<br />
qualification 0.02 0.05 0.959<br />
Year 12 <strong>of</strong> school 0.41 2.23 0.026<br />
Year 11 <strong>of</strong> school 0.18 0.91 0.363<br />
Year 9 <strong>of</strong> school or less 0.02 0.09 0.925<br />
Longest job by 1984 none -0.29 -1.02 0.306<br />
< 1 year -0.06 -0.32 0.752<br />
2 years 0.07 0.30 0.763<br />
3 years + -0.49 -1.97 0.049<br />
CEP referrals 1984 0.13 1.75 0.081<br />
Proportion <strong>of</strong> time Pre-June<br />
1984 spent in<br />
unemployment 0.34 2.41 0.016<br />
Work limited by health -0.68 -2.92 0.004<br />
O<strong>the</strong>r city before aged 14 -0.39 -2.33 0.020<br />
Country town before aged<br />
14 -0.52 -3.29 0.001<br />
Rural area before aged 14 -0.41 -1.58 0.115<br />
Overseas before aged 14 -0.61 -1.26 0.208<br />
Number <strong>of</strong> siblings -0.01 -0.89 0.371<br />
English good -0.17 -0.63 0.526<br />
English poor -0.68 -1.10 0.273<br />
Sexist 0.38 1.24 0.215<br />
Sexist*female -0.90 -1.59 0.111<br />
Fa<strong>the</strong>r not present when<br />
respondent aged 14 -0.25 -0.84 0.402<br />
Fa<strong>the</strong>r Labourer -0.09 -0.29 0.773
296<br />
Fa<strong>the</strong>r Plant operative -0.15 -0.53 0.597<br />
Fa<strong>the</strong>r Sales -0.21 -0.61 0.544<br />
Fa<strong>the</strong>r Tradesperson -0.28 -1.07 0.286<br />
Fa<strong>the</strong>r<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.19 -0.71 0.475<br />
Fa<strong>the</strong>r Not employed -0.44 -1.34 0.180<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14 -0.23 -1.61 0.107<br />
Mo<strong>the</strong>r not present when<br />
resp 14 0.37 1.22 0.223<br />
Mo<strong>the</strong>r Labourer -0.03 -0.10 0.920<br />
Mo<strong>the</strong>r Plant operative 0.58 1.83 0.067<br />
Mo<strong>the</strong>r Sales 0.22 0.78 0.437<br />
Mo<strong>the</strong>r Tradesperson 0.19 0.45 0.650<br />
Mo<strong>the</strong>r manager/ pr<strong>of</strong><br />
/para-pr<strong>of</strong>essional -0.32 -1.11 0.265<br />
Mo<strong>the</strong>r Not employed -0.13 -0.59 0.555<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14 0.22 1.27 0.203<br />
Catholic -0.01 -0.03 0.975<br />
Presbyterian 0.32 1.34 0.179<br />
Methodist 0.26 0.96 0.336<br />
O<strong>the</strong>r Christian 0.06 0.21 0.834<br />
O<strong>the</strong>r religion 0.14 0.58 0.559<br />
No religion 0.14 0.72 0.473<br />
constant 0.41 0.60 0.546<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%, ** significant at 1%.
297<br />
Table A2.8 Part A Employment equation from bivariate probit where CEP referrals and<br />
age included in employment model, attrition weights<br />
ever employed in<br />
1986 survey<br />
coefficient t statistic probability <strong>of</strong> t<br />
SYETP 1.19 0.81 0.416<br />
CEP referrals -0.03 -0.40 0.692<br />
Age 1984 0.01 0.33 0.739<br />
Gender=female -0.46 -4.18 0.000<br />
Married 0.03 0.14 0.891<br />
Children -0.39 -1.47 0.143<br />
Children*female -1.26 -3.70 0.000<br />
Spouse employed 0.54 2.18 0.029<br />
Aboriginal/Torres Strait<br />
Islander -0.34 -1.32 0.186<br />
O<strong>the</strong>r ethnic minority -0.37 -1.95 0.051<br />
Victoria -0.13 -1.00 0.318<br />
Queensland 0.05 0.36 0.716<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory -0.48 -2.75 0.006<br />
Western<br />
Australia/Tasmania -0.13 -0.68 0.497<br />
Education school overseas -0.23 -0.66 0.510<br />
Roman Catholic school -0.01 -0.04 0.966<br />
Private school 0.65 1.93 0.054<br />
Degree/diploma 0.49 2.73 0.006<br />
Apprenticeship 0.36 1.71 0.087<br />
O<strong>the</strong>r Post-School<br />
qualification 0.09 0.50 0.615<br />
Year 12 <strong>of</strong> school -0.10 -0.53 0.595<br />
Year 11 <strong>of</strong> school 0.37 1.99 0.046<br />
Year 9 <strong>of</strong> school or less -0.29 -2.02 0.043<br />
Longest job by 1984 none -0.01 -0.03 0.974<br />
< 1 year 0.23 1.64 0.102<br />
2 years 0.21 1.22 0.221<br />
3 years + 0.57 3.14 0.002<br />
Enter o<strong>the</strong>r govt prog -0.72 -4.64 0.000<br />
duration <strong>of</strong> Pre-June 1984<br />
unemployment -0.46 -3.53 0.000<br />
Work limited by health -0.33 -2.26 0.024<br />
O<strong>the</strong>r city before aged 14 -0.25 -1.38 0.167<br />
Country town before aged<br />
14 -0.06 -0.33 0.742<br />
Rural area before aged 14 -0.39 -1.55 0.122<br />
Overseas before aged 14 0.42 1.11 0.265<br />
Number <strong>of</strong> siblings -0.03 -1.59 0.112<br />
English good 0.43 2.10 0.035<br />
English poor 1.04 2.64 0.008<br />
Sexist -0.41 -2.00 0.045<br />
Sexist*female 0.62 1.58 0.113<br />
Fa<strong>the</strong>r not present when<br />
respondent 14 -0.19 -0.81 0.419<br />
Fa<strong>the</strong>r Labourer 0.18 0.76 0.449
298<br />
Fa<strong>the</strong>r Plant operative 0.16 0.70 0.483<br />
Fa<strong>the</strong>r Sales 0.05 0.20 0.844<br />
Fa<strong>the</strong>r Tradesperson -0.15 -0.66 0.507<br />
Fa<strong>the</strong>r<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
0.24 1.14 0.254<br />
Fa<strong>the</strong>r Not employed 0.10 0.38 0.704<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14 0.10 0.81 0.418<br />
Mo<strong>the</strong>r not present when<br />
resp 14 -0.18 -0.68 0.497<br />
Mo<strong>the</strong>r Labourer -0.15 -0.59 0.556<br />
Mo<strong>the</strong>r Plant operative -0.53 -1.85 0.064<br />
Mo<strong>the</strong>r Sales -0.40 -1.77 0.077<br />
Mo<strong>the</strong>r Tradesperson -0.33 -0.99 0.320<br />
Mo<strong>the</strong>r<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
0.08 0.32 0.745<br />
Mo<strong>the</strong>r Not employed -0.06 -0.35 0.730<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14 -0.03 -0.18 0.856<br />
Catholic 0.43 3.09 0.002<br />
Presbyterian 0.48 1.95 0.051<br />
Methodist 0.24 1.05 0.292<br />
O<strong>the</strong>r Christian -0.10 -0.49 0.622<br />
O<strong>the</strong>r religion 0.00 0.03 0.979<br />
No religion 0.40 2.07 0.038<br />
0.89 0.96 0.338<br />
rho -0.36<br />
Wald test <strong>of</strong> Rho=0 (chi2 0.14<br />
(1) statistic)<br />
Observations 1283<br />
Log likelihood -858.55<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until respondent aged 14,<br />
English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r clerical worker when respondent<br />
aged 14, religion brought up in is Anglican.<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%, ** significant at 1%.
299<br />
Table A2.8 Part B Selection/participation equation <strong>of</strong> <strong>the</strong> bivariate probit where CEP<br />
referrals and age included in employment model, attrition weights<br />
Model <strong>of</strong><br />
SYETP<br />
participation,<br />
1986 survey<br />
data<br />
coefficient t statistic probability <strong>of</strong> t<br />
Gender=female 0.07 0.53 0.598<br />
Age at 1984 survey -0.08 -2.91 0.004<br />
Married 1984 -0.98 -2.99 0.003<br />
Children 1984 0.35 0.87 0.386<br />
Children*female -0.11 -0.18 0.856<br />
Spouse employed 1984 0.66 1.62 0.105<br />
Aboriginal/Torres Strait<br />
Islander -0.38 -0.80 0.426<br />
O<strong>the</strong>r ethnic minority 0.02 0.07 0.946<br />
Victoria 0.09 0.56 0.576<br />
Queensland -0.15 -0.68 0.494<br />
South Australia/Nor<strong>the</strong>rn<br />
Territory -0.12 -0.63 0.526<br />
Western<br />
Australia/Tasmania 0.38 2.19 0.029<br />
Education school overseas 0.34 0.81 0.416<br />
Roman Catholic school -0.26 -1.10 0.270<br />
Private school -0.89 -2.08 0.038<br />
Degree/diploma 0.02 0.09 0.932<br />
Apprenticeship -0.11 -0.37 0.708<br />
O<strong>the</strong>r Post-School<br />
qualification 0.00 0.00 0.999<br />
Year 12 <strong>of</strong> school 0.41 2.21 0.027<br />
Year 11 <strong>of</strong> school 0.18 0.87 0.383<br />
Year 9 <strong>of</strong> school or less 0.03 0.14 0.892<br />
Longest job by 1984 none -0.27 -0.95 0.340<br />
< 1 year -0.06 -0.29 0.770<br />
2 years 0.07 0.32 0.749<br />
3 years + -0.48 -1.86 0.062<br />
CEP referrals 1984 0.13 1.76 0.078<br />
Proportion <strong>of</strong> time Pre-June<br />
1984 spent in<br />
unemployment 0.34 2.38 0.017<br />
Work limited by health -0.68 -2.96 0.003<br />
O<strong>the</strong>r city before aged 14 -0.39 -2.31 0.021<br />
Country town before aged<br />
14 -0.53 -3.20 0.001<br />
Rural area before aged 14 -0.41 -1.56 0.118<br />
Overseas before aged 14 -0.61 -1.26 0.207<br />
Number <strong>of</strong> siblings -0.01 -0.89 0.373<br />
English good -0.18 -0.65 0.515<br />
English poor -0.69 -1.11 0.267<br />
Sexist 0.39 1.25 0.211<br />
Sexist*female -0.92 -1.62 0.106<br />
Fa<strong>the</strong>r not present when<br />
respondent aged 14 -0.26 -0.85 0.394<br />
Fa<strong>the</strong>r Labourer -0.11 -0.32 0.751
300<br />
Fa<strong>the</strong>r Plant operative -0.16 -0.55 0.581<br />
Fa<strong>the</strong>r Sales -0.21 -0.59 0.554<br />
Fa<strong>the</strong>r Tradesperson -0.29 -1.08 0.279<br />
Fa<strong>the</strong>r<br />
Manager/pr<strong>of</strong>essional/parapr<strong>of</strong>essional<br />
-0.19 -0.73 0.464<br />
Fa<strong>the</strong>r Not employed -0.44 -1.36 0.174<br />
Fa<strong>the</strong>r holds post-school<br />
qualification when resp 14 -0.23 -1.62 0.106<br />
Mo<strong>the</strong>r not present when<br />
resp 14 0.37 1.21 0.226<br />
Mo<strong>the</strong>r Labourer -0.02 -0.07 0.942<br />
Mo<strong>the</strong>r Plant operative 0.59 1.83 0.068<br />
Mo<strong>the</strong>r Sales 0.22 0.79 0.431<br />
Mo<strong>the</strong>r Tradesperson 0.19 0.45 0.655<br />
Mo<strong>the</strong>r manager/ pr<strong>of</strong><br />
/para-pr<strong>of</strong>essional -0.32 -1.12 0.261<br />
Mo<strong>the</strong>r Not employed -0.13 -0.60 0.550<br />
Mo<strong>the</strong>r post-school<br />
qualification when resp 14 0.22 1.28 0.201<br />
Catholic 0.00 -0.03 0.979<br />
Presbyterian 0.33 1.37 0.172<br />
Methodist 0.24 0.85 0.393<br />
O<strong>the</strong>r Christian 0.05 0.20 0.843<br />
O<strong>the</strong>r religion 0.14 0.59 0.554<br />
No religion 0.13 0.64 0.521<br />
constant 0.43 0.66 0.512<br />
Base categories: European ethnic origin, state interviewed in 1984 NSW/ACT, government school, highest<br />
qualification in 1984 year 10 at school, Longest job by 1984 is 1 year, lived mostly in state capital city until<br />
respondent aged 14, English is first language, fa<strong>the</strong>r clerical worker when respondent aged 14, mo<strong>the</strong>r<br />
clerical worker when respondent aged 14, religion brought up in is Anglican.<br />
Absolute value <strong>of</strong> z statistics in paren<strong>the</strong>ses * significant at 5%, ** significant at 1%.
301
302<br />
Bibliography<br />
AIMA (1985) "Reducing <strong>the</strong> risk: unemployed migrant youth and labour market programs,<br />
Report to <strong>the</strong> Minister for Immigration and Cultural Affairs", <strong>Australian</strong> Institute <strong>of</strong><br />
Multicultural Affairs, Melbourne Australia.<br />
Alderman, H.; Behrman, J.; Kohler, H.; Maluccio, J.; and Watkins, S. (2000) "Attrition in<br />
longitudinal household survey data: some tests for three developing country samples"<br />
World Bank Policy research working paper no. 2447, Washington.<br />
Ashenfelter, O., Harmon, C. and Oosterbeek, H. (1999) “A review <strong>of</strong> estimates <strong>of</strong> <strong>the</strong><br />
schooling/earnings relationship, with tests for publication bias”, Labour Economics<br />
Vol. 6, pp453-470.<br />
Augurzky, B and Schmidt, C.M. (2001) “The propensity score: a means to an end” Institute for<br />
<strong>the</strong> Study <strong>of</strong> Labor (IZA) Discussion Paper no. 271, Institute for <strong>the</strong> Study <strong>of</strong> Labor<br />
(IZA), Bonn<br />
Aungles, P. and Stewart, B.P. (1986)"The duration <strong>of</strong> unemployment <strong>of</strong> CES registered<br />
jobseekers: an econometric analysis" Bureau <strong>of</strong> Labour Market Research Working<br />
Paper no. 66 Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong><br />
Australia, Canberra.<br />
Baird, C. E.; Gregory, R.H.; and Gruen, F. H. (eds.) (1981) "<strong>Youth</strong> employment, education<br />
and training" Conference Papers Academy <strong>of</strong> <strong>the</strong> Social Sciences in Australia, Centre<br />
for Economic Policy Research, <strong>Australian</strong> National University, Canberra.<br />
Baker, S. (1984) "A comparative evaluation <strong>of</strong> <strong>the</strong> impact on participants <strong>of</strong> selected youth<br />
labour force programs" Bureau <strong>of</strong> Labour Market Research Working Paper no. 50<br />
Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia,<br />
Canberra.<br />
Baker, S.; Nicoll, P. and Yik, W.F. (1985) "Aboriginal participation in <strong>the</strong> Community<br />
Employment Program State/Territory element in <strong>the</strong> Nor<strong>the</strong>rn Territory and Australia"<br />
Bureau <strong>of</strong> Labour Market Research Conference Paper no 56, Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, <strong>Australian</strong>,<br />
Government Publishing Service Canberra.<br />
Becketti, S.; Gould, W.; Lillard, L. and Welch, F. (1988) "The panel study <strong>of</strong> Income<br />
Dynamics after 14 years: an evaluation", Journal <strong>of</strong> Labor Economics Vol. 6 (4) pp<br />
473-492.<br />
BLMR Bureau <strong>of</strong> Labour Market Research (1983) "Employment and training programs for<br />
young people: analysis <strong>of</strong> assistance in 1980-81" Bureau <strong>of</strong> Labour Market Research<br />
Research Report no.2, Department <strong>of</strong> Employment and Industrial Relations,<br />
Commonwealth <strong>of</strong> Australia, Canberra.<br />
BLMR Bureau <strong>of</strong> Labour Market Research (1984) "The National Employment and Training<br />
System (NEAT): an evaluation" Bureau <strong>of</strong> Labour Market Research Monograph Series<br />
no.3, Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong><br />
Australia, <strong>Australian</strong>, Government Publishing Service Canberra.<br />
BLMR Bureau <strong>of</strong> Labour Market Research (June 1984) "Submission to committee <strong>of</strong> enquiry<br />
into labour market programs", Bureau <strong>of</strong> Labour Market Research, Research Report no.<br />
5 <strong>Australian</strong> Government Publishing Service, Canberra.
BLMR Bureau <strong>of</strong> Labour Market Research (1986) "The first wave <strong>of</strong> <strong>the</strong> <strong>Australian</strong><br />
Longitudinal Survey: facts and figures about young CES registrants", Bureau <strong>of</strong> Labour<br />
Market Research Monograph Series no.12, Department <strong>of</strong> Employment and Industrial<br />
Relations, Commonwealth <strong>of</strong> Australia, <strong>Australian</strong> Government Publishing Service<br />
Canberra.<br />
Blundell, R. and Costa Dias, M. (2000) "<strong>Evaluation</strong> methods for non-experimental data" Fiscal<br />
Studies Vol. 21 (4) pp 427-468.<br />
Bonjour, D., Dorsett, R., Knight, G., Lissenburgh, S., Mukherjee, A., Payne, J., Range, M.,<br />
Urwin, P., and White, M. (2001) New Deal for Young People: National Survey <strong>of</strong><br />
Participants: Stage 2, Employment Service Research and Development Report ESR67,<br />
Employment Service, Sheffield, UK.<br />
Bonnal, L., Fougere, D. and Serandon, A. (1994) "L'impact des pispositifs d'emploi sur le<br />
devenir des jeunes chomeurs: une evaluation econometrique sur donnees<br />
longitudinales", Economie et Prevision no.115 pp1-28, Crespe, Universite d'Orleans.<br />
Bonnal, L., Fougere, D. and Serandon, A. (1997) "Evaluating <strong>the</strong> impact <strong>of</strong> French<br />
employment policies on individual labour market histories", Review <strong>of</strong> Economic<br />
Studies 64, 683-713.<br />
Breen, R. (1991) "Education, Employment and Training in <strong>the</strong> <strong>Youth</strong> Labour Market", General<br />
Research Series Paper No. 152, Economic and Social Research Institute, Dublin.<br />
Brooks, C. and Volker, P. (1984) "The probability <strong>of</strong> leaving unemployment: <strong>the</strong> evidence<br />
from <strong>Australian</strong> gross flows data" Bureau <strong>of</strong> Labour Market Research Conference<br />
paper no 47, Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong><br />
Australia, <strong>Australian</strong>, Government Publishing Service Canberra.<br />
Bryant, E.C. (1978) " Survey statistics in social program evaluation" pp41-55 in David,<br />
H.A.(ed) "Contributions to survey sampling and applied statistics: papers in honour <strong>of</strong><br />
H. O. Hartley" Academic Press, New York.<br />
Burtless, G. (1990) "The economists' lament: public assistance in America" Journal <strong>of</strong><br />
Economic Perspectives, Vol.4 Issue 1 pp57-78.<br />
Calmfors, L. (1994) "Active labour market policy and unemployment - a framework for <strong>the</strong><br />
analysis <strong>of</strong> crucial design features" OECD Economic Studies V 22, pp7-47.<br />
Capellari, L. (2001) "Earnings mobility among Italian low paid workers", Institute for Social<br />
and Economic Research Working paper 2001-13, Institute for Social and Economic<br />
Research, University <strong>of</strong> Essex.<br />
Chapman, B. (1985) "Continuity and change: labour market programs and education<br />
expenditure", The <strong>Australian</strong> Economic Review 71, pp98-113.<br />
Cochran, W. and Rubin, D.B. (1973), "Controlling Bias in Observational Studies", Sankyha,<br />
35, 417-446.<br />
Cockx, B., Van der Linden B., and Karaa, A. (1996) "On active labour market policies and job<br />
duration: a microeconometric evaluation for Belgium" 1996 Conference Proceedings,<br />
Vol.4. European Association <strong>of</strong> Labour Economists (EALE).<br />
D'Agostino, R.B. and Rubin, D.B. (2000) "Estimating and using propensity scores with<br />
partially missing Data" Journal <strong>of</strong> <strong>the</strong> American Statistical Association, September<br />
2000, Vol. 95 no. 451 pp749-759.<br />
Daly, A. (1991) "Are education and on-<strong>the</strong>-job training complementary or substitute activities?<br />
Evidence from Australia, Great Britain and <strong>the</strong> US", Centre for Aboriginal Economic<br />
Policy Research Discussion Paper no 260, <strong>Australian</strong> National University, Canberra.<br />
303
Friedlander, D.; Greenberg, D.H. and Robins, P.K. (1997) "Evaluating Government Training<br />
Programs for <strong>the</strong> Economically Disadvantaged", Journal <strong>of</strong> Economic Literature, Vol.<br />
35, No. 4., pp1809-1855.<br />
De La Dehesa, G. (1997) "Preventing long-term unemployment: discussion" p349-356 in<br />
Snower, D.J. and De La Dehesa, G. [eds.](1997) "Unemployment policy: government<br />
options for <strong>the</strong> labour market", Cambridge University Press.<br />
DEET (1994) "Newstart evaluation: interim report on effectiveness" Sakkara, M.; Beyers, S.;<br />
Buchanan, K.; Gatenby, P.; Patston, K; Department <strong>of</strong> Employment, Education and<br />
Training, <strong>Evaluation</strong> and Monitoring Branch, Economic Policy Analysis Division,<br />
<strong>Australian</strong> Government Publishing Service, Canberra.<br />
DEETYA (1996) "Working Nation: evaluation <strong>of</strong> <strong>the</strong> employment, education, and training<br />
elements", Department <strong>of</strong> Employment, Education, Training and <strong>Youth</strong> Affairs,<br />
<strong>Australian</strong> Government Publishing Service, Canberra.<br />
Dehija, R. and Wahba, S. (1998) "Propensity score matching methods for non-experimental<br />
causal studies" NBER Working Paper no. 6829.<br />
Dehija, R. and Wahba, S. (1999) "Causal effects in non-experimental studies: re-evaluating <strong>the</strong><br />
evaluation <strong>of</strong> training programs", Journal <strong>of</strong> <strong>the</strong> American Statistical Association 94<br />
(448), pp 1053-1062.<br />
Dewald, W. G.; Thursby, J. G; and Anderson, R.G.; (1986), “Replication in Empirical<br />
Economics: The Journal <strong>of</strong> Money, Credit and Banking Project", The American<br />
Economic Review, Vol. 76, No. 4. pp 587-603.<br />
DEYA (1980) "Employer and manpower services: a guide issued by <strong>the</strong> Department <strong>of</strong><br />
Employment and <strong>Youth</strong> Affairs", pamphlet March 1980, Department <strong>of</strong> Employment<br />
and <strong>Youth</strong> Affairs, Canberra.<br />
DEYA (1983) "<strong>Youth</strong> Policies, programs and issues an <strong>Australian</strong> background paper",<br />
Department <strong>of</strong> Education and <strong>Youth</strong> Affairs <strong>Australian</strong> Government Publishing<br />
Service, Canberra.<br />
Dillman, D.A.; Eltinge, J.; Groves, R.M. and Little, R.J.A.;(2002) "Survey nonresponse in<br />
design, data collection and analysis" pp3-26 in Groves, R.M.; Dillman, D.A.; Eltinge,<br />
J.; Little, R.J.A.; [eds.] (2002)"Survey non-response", Wiley Series in Survey<br />
Methodology, John Wiley New York.<br />
Dreze, J.H. and Sneessens, H. (1997) "Technological development, competition from low<br />
wage economies and low-skilled unemployment" Chapter 8 pp250-277 in Snower, D.J.<br />
and De La Dehesa, G. [eds.](1997) "Unemployment policy: government options for <strong>the</strong><br />
labour market", Cambridge University Press.<br />
Edwards, S. (1987) "Do manpower programmes enhance employment opportunities? An<br />
evaluation <strong>of</strong> two programmes in Wollongong and Newcastle", Industrial Relations<br />
Research Centre Monograph (May 1987) Industrial Relations Research Centre<br />
University <strong>of</strong> New South Wales.<br />
Eichler, M. and Lechner, M. (1998) "An evaluation <strong>of</strong> public employment programmes in <strong>the</strong><br />
East German state <strong>of</strong> Sachsen-Anhalt" Discussion Paper no.9815, Volswirtschaftliche<br />
Abteilung, University <strong>of</strong> St. Gallen, Switzerland.<br />
EPAC (June 1992) "Unemployment in Australia" Economic Planning Advisory Council,<br />
Council Paper no.51, <strong>Australian</strong> Government Publishing Service, Canberra.<br />
Falaris, E.M. and Peters, H.E. (1998) "Survey attrition and schooling choices", The Journal <strong>of</strong><br />
Human Resources Vol. 33 (2) pp 531-554.<br />
304
Fay, R.G. (1996) "Enhancing <strong>the</strong> effectiveness <strong>of</strong> active labour market policies: evidence from<br />
Programme evaluations in OECD countries", Labour market and social policy<br />
Occasional Papers no.18, OECD Working Papers Vol. 4, no.56, Organisation for<br />
Economic Cooperation and Development, Paris.<br />
Fisher, R. (1951)"The design <strong>of</strong> experiments", 6th edition, Oliver and Boyd, London.<br />
Fitzgerald, J.; Gottshalk, P. and M<strong>of</strong>fitt, R, (1998a) "An analysis <strong>of</strong> sample attrition in panel<br />
data", The Journal <strong>of</strong> Human Resources Vol. 33 (2) pp 251-299.<br />
Fitzgerald, J.; Gottshalk, P. and M<strong>of</strong>fitt, R, (1998b) "An analysis <strong>of</strong> <strong>the</strong> impact <strong>of</strong> sample<br />
attrition on <strong>the</strong> second generation <strong>of</strong> respondents in <strong>the</strong> Michigan Panel Study <strong>of</strong><br />
Income Dynamics", The Journal <strong>of</strong> Human Resources Vol. 33 (2) pp 300-344.<br />
Friedlander, D. and Robins, P.K. (1995) "Evaluating Program <strong>Evaluation</strong>s: New Evidence on<br />
Commonly Used Non-experimental Methods" The American Economic Review, Vol.<br />
85, No. 4. , pp 923-937.<br />
Friedlander, D.; Greenberg, D.H. and Robins, P.K. (1997) "Evaluating government training<br />
programs for <strong>the</strong> economically disadvantaged" Journal <strong>of</strong> economic Literature, Vol.35<br />
(December 1997) pp1809-1855.<br />
Frölich, M. (2001) "Non-parametric covariate adjustment: pair-matching versus local<br />
polynomial matching", Discussion Paper 2000-17, Department <strong>of</strong> Economics<br />
University <strong>of</strong> St. Gallen, Switzerland.<br />
Frölich, M.; Heshmati, A. and Lechner, M.; (2000) "A microeconometric evaluation <strong>of</strong><br />
rehabilitation <strong>of</strong> long-term sickness in Sweden" Discussion Paper 2000-04, Department<br />
<strong>of</strong> Economics University <strong>of</strong> St. Gallen, Switzerland.<br />
Frölich, M.; Heshmati, A. and Lechner, M.; (2001) "A microeconometric evaluation <strong>of</strong><br />
rehabilitation <strong>of</strong> long-term sickness in Sweden, revised" Discussion Paper 2000-04,<br />
Department <strong>of</strong> Economics University <strong>of</strong> St. Gallen, Switzerland.<br />
Gerfin, M. and Lechner, M. (2000) "Microeconometric evaluation <strong>of</strong> <strong>the</strong> active labour market<br />
policy in Switzerland" Swiss Institute for International Economics and Applied<br />
Research (SIAW) mimeo www.siaw.unisg.ch/lechner.<br />
Ginpil, S. and Hoy, M. (1984) "Some factors affecting <strong>the</strong> completion rates for young people<br />
under wage subsidy" Bureau <strong>of</strong> Labour Market Research Working paper no.46,<br />
Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia,<br />
<strong>Australian</strong>, Government Publishing Service Canberra.<br />
Greene, W.H. (1991) "Econometric Analysis", Macmillan, New York.<br />
Griliches, Z. (1986) "Economic Data Issues", in Griliches, S. and Intrilligator, M. (eds.) (1986)<br />
"Handbook <strong>of</strong> econometrics Volume 3", North Holland, Amsterdam.<br />
Gronau, R. (1973) "The effect <strong>of</strong> children on <strong>the</strong> housewife's value <strong>of</strong> time" Journal <strong>of</strong> Political<br />
Economy Vol. 81, no 2, part 2, pp S168-S199.<br />
Gronau, R. (1974) “<strong>Wage</strong> comparisons – a selectivity bias” Journal <strong>of</strong> Political Economy Vol.<br />
82 no. 6, pp 1119-1143.<br />
Groves, R.M.; Dillman, D.A.; Eltinge, J.L.; Little, R.J.A.; (eds.)(2002)"Survey nonresponse"<br />
Wiley series in Survey Methodology, John Wiley, New York.<br />
Gu, X. and Rosenbaum, P. (1993) "Comparison <strong>of</strong> multivariate matching methods: structures,<br />
distance and algorithms" Journal <strong>of</strong> computational and Graphical Statistics 2, pp405-<br />
420.<br />
Gueron, J.M. (1990) "Work and welfare: lessons on employment programs" Journal <strong>of</strong><br />
Economic Perspectives, Vol.4 Issue 1 pp79-98<br />
305
Gujurati, D.N. (1988) "Basic Econometrics" McGraw-Hill, Singapore.<br />
Ham, J.C. and Lalonde, R.J. (1996) “The effect <strong>of</strong> sample selection and initial conditions in<br />
duration models: evidence from experimental data on training”, Econometrica 64<br />
pp175-205.<br />
Hamermesh, D.S. (1978) "Subsidies for jobs in <strong>the</strong> private sector", pp87-122 in Palmer, J. L.<br />
(1978) [ed] "Creating jobs: public employment programs and wage subsidies"<br />
Brookings Institute, Washington D.C.<br />
Harkman, A., and Johansson, A. (2000) "Training or Subsidised Jobs - What Works?<br />
Employment Impact <strong>of</strong> Seven Swedish Labour Market Programmes",<br />
Arbetsmarknadsstyrelsen, mimeo.<br />
Harris, M. (1996) "Modelling <strong>the</strong> probability <strong>of</strong> youth unemployment in Australia" Economic<br />
Record, Vol. 72 no 217 p118-130.<br />
Harris, P. (2001) “From relief to mutual obligation: welfare rationalities and unemployment in<br />
20 th century Australia” Journal <strong>of</strong> Sociology Vol. 37 (1) pp5-24.<br />
Hausman, J.A. and Wise, D. (1979) "Attrition bias in experimental and panel data: <strong>the</strong> GARY<br />
income maintenance experiment", Econometrica 46 (6) pp931- 959.<br />
Haveman, R.H. (1982) "Marginal employment subsidies" Organisation for Economic<br />
Cooperation and Development, Paris.<br />
Heckman, J.J. (1974) "Shadow prices, market wages and labor supply" Econometrica Vol.42<br />
pp 679-694.<br />
Heckman, J.J. (1976) "The common structure <strong>of</strong> statistical models <strong>of</strong> truncation, sample<br />
selection and limited dependent variables and simple estimator for such models"<br />
Annals <strong>of</strong> Economics and Social Measurement Vol. 5 pp475-492.<br />
Heckman, J.J. (1978) “Dummy endogenous variables in a simultaneous equation system”<br />
Econometrica 46, pp931-959.<br />
Heckman, J.J. (1979) "Sample selection bias as specification error", Econometrica 47 pp 153-<br />
161.<br />
Heckman, J.J. and Hotz, V.J. (1989) "Choosing among alternative non-experimental methods<br />
for estimating <strong>the</strong> impact <strong>of</strong> social programs: <strong>the</strong> case <strong>of</strong> manpower training" Journal <strong>of</strong><br />
<strong>the</strong> American Statistical Association Vol. 84 no. 408 pp862-874.<br />
Heckman, J.J.; Ichimura, H.; Smith, J. and Todd, P.E. (1998) “Characterising selection bias<br />
with experimental data” Econometrica 66 1017-1098.<br />
Heckman, J.J., Ichimura, H. and Todd, P.E. (1997) "Matching as an econometric evaluation<br />
estimator: evidence from evaluating a job training programme", Review <strong>of</strong> Economic<br />
Studies 64(4), pp605-654.<br />
Heckman, J.J.; Ichimura, H. and Todd, P.E. (1998) "Matching as an econometric evaluation<br />
estimator" Review <strong>of</strong> Economic Studies 65(1), pp261-294.<br />
Heckman, J.J.; Lalonde, R.J.; Smith, J. A. (1999) "The economics and econometrics <strong>of</strong> active<br />
labour market programs" Chapter 31 pp1865-2097 in Ashenfelter, O. C. and Card, D.<br />
[eds.] (1999) "Handbook <strong>of</strong> labour economics Volume 3A", Elsevier Science B.V.<br />
Amsterdam.<br />
Heckman J.J. and Robb, R. Jr. (1985) "Alternative methods for evaluating <strong>the</strong> impact <strong>of</strong><br />
interventions: an overview" Journal <strong>of</strong> Econometrics Vol. 30, pp239-267.<br />
306
Heckman J.J. and Smith, J. A. (1998) "The sensitivity <strong>of</strong> experimental impact estimates:<br />
evidence from <strong>the</strong> National JTPA study" in Freeman, R. and Katz, L. [eds.] (1998)<br />
"<strong>Youth</strong> employment and unemployment in <strong>the</strong> OECD countries, University <strong>of</strong> Chicago<br />
Press, Chicago Illinois.<br />
Hoem (1985) "Weighting, misclassification and o<strong>the</strong>r issues in <strong>the</strong> analysis <strong>of</strong> survey samples<br />
<strong>of</strong> life histories" Chapter 5 in Heckman, J.J. and Singer, B. (Eds.) (1985) "Longitudinal<br />
analysis <strong>of</strong> labor market data", Cambridge University Press.<br />
Hoy, M. (1983) "Review <strong>of</strong> five years <strong>of</strong> operation <strong>of</strong> <strong>the</strong> <strong>Special</strong> <strong>Youth</strong> Employment Training<br />
Program" Bureau <strong>of</strong> Labour Market Research Conference Paper No. 18, Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Hoy, M.A. and Lampe, G.L. (1982) "Women in national training and employment programs"<br />
Bureau <strong>of</strong> Labour Market Research Conference Paper no.13, Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Hoy, M and Paterson, P. (1983) "Data on national training and employment programs" Bureau<br />
<strong>of</strong> Labour Market Research Technical Paper no.6, Department <strong>of</strong> Employment and<br />
Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Hoy, M.A. and Ryan, C.A. (1984) "Participating employers responses to <strong>the</strong> SYETP wage<br />
subsidy scheme", Bureau <strong>of</strong> Labour Market Research Working Paper no. 40, <strong>Australian</strong><br />
Government Publishing Service, Canberra.<br />
Hsaio, C. (1986) "Analysis <strong>of</strong> panel data" Cambridge University Press.<br />
Hujer, R. and Caliendo, M. (2000) "<strong>Evaluation</strong> <strong>of</strong> active labour market policy: methodological<br />
concepts and empirical estimates" Institute for <strong>the</strong> Study <strong>of</strong> Labor (IZA) Discussion<br />
Paper no. 236, Institute for <strong>the</strong> Study <strong>of</strong> Labor (IZA), Bonn.<br />
Imbens, Guido (2000) “The Role <strong>of</strong> <strong>the</strong> Propensity Score in Estimating Dose-Response<br />
Functions” Biometrika. 87: 706-710.<br />
Jackman, R.A and Layard, P.R.G (1980) "The efficiency case for long run labour market<br />
policies", Economica V 47 pp331-349.<br />
Johannesson, J. (1985) "The Swedish experience with labour market policy" Bureau <strong>of</strong> Labour<br />
Market Research Working Paper no.62, Department <strong>of</strong> Employment and Industrial<br />
Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Johnson, G.E. and Layard, P.R.G. (1986) " The natural rate <strong>of</strong> unemployment: explanations<br />
and policy" Chapter 16 pp921-999 Volume 2 Ashenfelter, O. and Layard, R. [Eds.]<br />
"Handbook <strong>of</strong> Labor economics" North-Holland, Amsterdam.<br />
Johnson, G.E. (1980) "The <strong>the</strong>ory <strong>of</strong> labour market intervention", Economica V 47, pp309-<br />
329.<br />
Junankar, P.N. (1986) "Program <strong>of</strong> research and actions on <strong>the</strong> development <strong>of</strong> <strong>the</strong> labour<br />
market: costs <strong>of</strong> unemployment", Commission <strong>of</strong> <strong>the</strong> European Communities,<br />
Luxembourg.<br />
Kaldor, N. (1936) "<strong>Wage</strong> Subsidies as a Remedy for Unemployment" The Journal <strong>of</strong> Political<br />
Economy, Vol. 44, No. 6. (Dec., 1936), pp. 721-742<br />
Katz, L.F. (1996) "<strong>Wage</strong> subsidies for <strong>the</strong> disadvantaged" National Bureau <strong>of</strong> Economic<br />
Research Working Paper 5679, National Bureau <strong>of</strong> Economic Research, Cambridge<br />
Massachusetts.<br />
307
Kesteven, S. (1987) "Commonwealth employment and training schemes since 1973"<br />
Education and Welfare Group, Legislative Research Service Current Issues Paper 1<br />
1987-88, Department <strong>of</strong> <strong>the</strong> Parliamentary Library, The Parliament <strong>of</strong> <strong>the</strong><br />
Commonwealth <strong>of</strong> Australia, Commonwealth <strong>of</strong> Australia.<br />
King, G. (1995) "replication, replication '' with comments from nineteen authors and a<br />
response, "a revised proposal, proposal,'' PS: Political Science and Politics, Vol.<br />
XXVIII, No. 3 (September, 1995): Pp. 443-499.<br />
King, G. (2002) "The future <strong>of</strong> <strong>the</strong> replication movement", April 2002 meetings <strong>of</strong> <strong>the</strong><br />
International Studies Organisation, mimeo http://GKing.Harvard.Edu.<br />
King, G.; Honaker, J.; Joseph, A.; and Scheve, K. (2001) "Analysing incomplete political<br />
science data: an alternative algorithm for multiple imputation" American Political<br />
Science Review Vol.95 No.1 pp49-69.<br />
Kirby, P.E.F. (1981) "An overview <strong>of</strong> <strong>Australian</strong> experience with manpower programs" pp4.1-<br />
4.45 in Baird, C. E.; Gregory, R.H.; Gruen, F. H. (eds.) "<strong>Youth</strong> employment, education<br />
and training" (1981) Conference Papers, Academy <strong>of</strong> <strong>the</strong> Social Sciences in Australia,<br />
Centre for Economic Policy Research, <strong>Australian</strong> National University, Canberra.<br />
Knapp, L. and Seaks, T. (1998) “A Hausman test for a dummy variable in probit” Applied<br />
Economics Letters Vol.5 pp321-323.<br />
Kronenburg, N.; McRae, I. and Nicoll, P; (1985) "The <strong>Australian</strong> longitudinal survey - sample<br />
design" Bureau <strong>of</strong> Labour Market Research Technical paper no 23, Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, <strong>Australian</strong>,<br />
Government Publishing Service Canberra.<br />
Lahiri, K. and Schmidt, P. (1978) “On <strong>the</strong> estimation <strong>of</strong> triangular structural systems”,<br />
Econometrica 46, pp1217-1221.<br />
Lalive, R.; van Ours, J.C. and Zweimuller, J. (2000) " The impact <strong>of</strong> active labor market<br />
programs and benefit entitlement rules on <strong>the</strong> duration <strong>of</strong> unemployment" Centre for<br />
Economic Research (IEW) Working Paper no.2000-41, University <strong>of</strong> Zurich,<br />
Switzerland.<br />
LaLonde, R. J. (1986) "Evaluating <strong>the</strong> Econometric <strong>Evaluation</strong>s <strong>of</strong> Training Programs with<br />
Experimental Data", The American Economic Review, Vol. 76, No. 4. (Sep 1986) pp.<br />
604-620.<br />
Larsson, L. (2000) "<strong>Evaluation</strong> <strong>of</strong> Swedish youth labour market programs", Uppsala<br />
University mimeo.<br />
Layard, R (1997) "Preventing long-term unemployment" chapter 11 pp333-351 in Snower,<br />
D.J. and De La Dehesa, G. [eds.](1997) "Unemployment policy: government options<br />
for <strong>the</strong> labour market", Cambridge University Press.<br />
Layard, R.; Nickell, S. and Jackman, R; (1991) "Unemployment, macroeconomic performance<br />
and <strong>the</strong> labour market" Oxford University Press.<br />
Lechner, M. (2000) "A note on <strong>the</strong> common support problem in applied evaluation studies"<br />
Swiss Institute for International Economics and Applied Economic Research (SIAW)<br />
Discussion Paper no.2001-01, University <strong>of</strong> St. Gallen, Switzerland.<br />
Lechner, M. (2001) "Some practical issues in <strong>the</strong> evaluation <strong>of</strong> heterogeneous labour market<br />
programmes by matching methods" Swiss Institute for International Economics and<br />
Applied Economic Research (SIAW) Discussion Paper 2000-14, University <strong>of</strong> St.<br />
Gallen, Switzerland. [forthcoming in Journal <strong>of</strong> <strong>the</strong> Royal Statistical Society Series A,<br />
165(1) pp59-82.]<br />
308
Lewis, H.G. (1963) "Unionism and relative wages", University <strong>of</strong> Chicago Press, Chicago<br />
Illinois.<br />
Lewis, H.G. (1974) "Comments on selectivity biases in wage comparisons" Journal <strong>of</strong> Political<br />
Economy Vol. 82. (6) pp1145-1155.<br />
Lewis, P.E.T. (1983) "The role <strong>of</strong> relative wages in <strong>the</strong> substitution between young and adult<br />
workers in Australia" Bureau <strong>of</strong> Labour Market Research Working Paper no.19,<br />
Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia,<br />
Canberra.<br />
Lillard, L. and Panis, C.W.A. (1998) “Panel Attrition from <strong>the</strong> Panel Study <strong>of</strong> Income<br />
Dynamics” The Journal <strong>of</strong> Human Resources 33, no. 2: 437-57.<br />
Lindbeck, A. (1994)"The Welfare State and <strong>the</strong> Employment Problem" The American<br />
Economic Review, Vol. 84, No. 2, Papers and Proceedings <strong>of</strong> <strong>the</strong> Hundred and Sixth<br />
Annual Meeting <strong>of</strong> <strong>the</strong> American Economic Association. (May, 1994), pp. 71-75.<br />
Maddala, G.S. (1983)"Limited dependent and qualitative variables in econometrics"<br />
Econometric Society Monograph no 3, Cambridge University Press.<br />
Martin, J.P. (1998) "What works among active labour market policies: evidence from OECD<br />
countries´ experiences", Labour market and social policy Occasional Papers no.35,<br />
OECD Working Papers Vol. 6, no.82, Organisation for Economic Cooperation and<br />
Development, Paris.<br />
Marx, I. (2001) "Job subsidies and cuts in employers' social security contributions: <strong>the</strong> verdict<br />
<strong>of</strong> empirical evaluation studies" International Labour Review Vol. 140 No. 1,<br />
International Labour Organisation, Geneva.<br />
McGillicudy, R.; Athanasou, J. and Longford, R. (1986) “Unemployment registrations with<br />
<strong>the</strong> CES in NSW: a regional Analysis" September 1986, NSW Department <strong>of</strong> Industrial<br />
Relations, Information and policy unit, Human Resources Division.<br />
Mckay, R. and Hope, C. (1986) "Advances in <strong>the</strong> evaluation <strong>of</strong> labour force programs: issues<br />
and methodological approached" Bureau <strong>of</strong> Labour Market Research Conference Paper<br />
no.61, Department <strong>of</strong> Employment and Industrial Relations, Commonwealth <strong>of</strong><br />
Australia, Canberra.<br />
Mcrae, I. (1984) "The <strong>Australian</strong> National Longitudinal Survey" Bureau <strong>of</strong> Labour Market<br />
Research Conference Paper no.44, Department <strong>of</strong> Employment and Industrial<br />
Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Mcrae, I. (1986) "What happens to <strong>the</strong> young unemployed - some facts from <strong>the</strong> ALS",<br />
<strong>Australian</strong> Longitudinal Survey Working Paper no.9, The Office <strong>of</strong> <strong>the</strong> <strong>Australian</strong><br />
Council for Employment and Training.<br />
Mcrae, I.; Parkinson, G.; Woyzbun, L. (1984-1987) <strong>Australian</strong> Longitudinal Survey Waves 1<br />
to 4, Level 1. Principal Investigators: Mcrae, I.; Parkinson, G.; Woyzbun, L.; Data<br />
collected by Roy Morgan Research Centre, Canberra. Ian Mcrae, Bureau <strong>of</strong> Labour<br />
Market Research, [producer], Canberra. Social Science Data Archives, The <strong>Australian</strong><br />
National University, Canberra [distributor].<br />
Mcrae, I.; Adena, M.; Montesin, H.; Parkinson, G.; Kronenbug, N. (1985) "The <strong>Australian</strong><br />
Longitudinal survey: response rates and ad hoc technical issues related to <strong>the</strong> 1984<br />
survey" Bureau <strong>of</strong> Labour Market Research Technical Paper no.30, Bureau <strong>of</strong> Labour<br />
Market Research, Canberra.<br />
Merrilees, W. (1981)"The effects <strong>of</strong> labour market conditions on school enrolment rates",<br />
<strong>Australian</strong> Economic Review, third quarter 1981, pp56-60.<br />
309
Merrilees, W.J. (1984) "Do <strong>Wage</strong> subsidies stimulate training? An evaluation <strong>of</strong> <strong>the</strong> CRAFT<br />
rebate scheme", <strong>Australian</strong> Economic Papers pp235-248.<br />
Millard, S. and Mortenson, D. (1997)" The unemployment and welfare effects <strong>of</strong> labour<br />
market policy: a comparison <strong>of</strong> USA and <strong>the</strong> UK" chapter 17 pp545-572 in Snower,<br />
D.J. and De La Dehesa, G. [eds.](1997) "Unemployment policy: government options<br />
for <strong>the</strong> labour market", Cambridge University Press.<br />
Miller, P. and Volker, P. (1987) "The youth labour market in <strong>Australian</strong>: a survey <strong>of</strong> <strong>the</strong> issues<br />
and evidence", Discussion paper no. 171 Department <strong>of</strong> Economics, Research School<br />
<strong>of</strong> Social Sciences, <strong>Australian</strong> National University.<br />
O'Connell, P.J. and McGinnity, F. (1997) "What works, who works? The employment and<br />
earnings effects <strong>of</strong> active labour market programmes among young people in Ireland",<br />
Work, Employment and Society, Vol. 11, No. 4, pp. 639-661.<br />
OECD (1988) "Measure to assist <strong>the</strong> long-term unemployed: recent experience in some OECD<br />
countries" Organisation for Economic Cooperation and Development, Paris.<br />
OECD [Grubb, D.; Lippoldt, D. and Tergeist, P.;] (2001) "Innovations in labour market<br />
policies: The <strong>Australian</strong> way" Organisation for Economic Cooperation and<br />
Development, Paris.<br />
Pagan, A. and Ullah, A. (1999) "Nonparametric Econometrics", Cambridge University Press,<br />
Cambridge.<br />
Parkinson, G. and Mcrae, I. (1985) "The <strong>Australian</strong> longitudinal survey - questionnaire design<br />
aspects" Bureau <strong>of</strong> Labour Market Research Technical paper no 24, Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, <strong>Australian</strong><br />
Government Publishing Service Canberra.<br />
Paterson, P. (1982) "Manpower program evaluation activities <strong>of</strong> <strong>the</strong> Bureau <strong>of</strong> Labour Market<br />
Research", Bureau <strong>of</strong> Labour Market Research Conference Paper no.12, Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Paterson, P.R. and Mackay, K.R. (1982a) "<strong>Youth</strong> employment in Australia", Bureau <strong>of</strong> Labour<br />
Market Research Working Paper no.8, Department <strong>of</strong> Employment and Industrial<br />
Relations, Commonwealth <strong>of</strong> Australia, <strong>Australian</strong>, Government Publishing Service<br />
Canberra.<br />
Paterson, P.R. and Mackay, K.R. (1982b) "Changes in <strong>the</strong> youth labour market: 1971 to 1981"<br />
Bureau <strong>of</strong> Labour Market Research Working Paper no.11, Department <strong>of</strong> Employment<br />
and Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Pfefferman, D. (1993) "The role <strong>of</strong> sampling weights when modelling survey data"<br />
International Statistical Review Vol. 61 (2), pp317-337.<br />
Phelps, E.S. (1994) "Low-<strong>Wage</strong> Employment Subsidies versus <strong>the</strong> Welfare State" The<br />
American Economic Review, Vol. 84, No. 2, Papers and Proceedings <strong>of</strong> <strong>the</strong> Hundred<br />
and Sixth Annual Meeting <strong>of</strong> <strong>the</strong> American Economic Association. (May, 1994), pp.<br />
54-58.<br />
Phelps, E.S. (1997) "<strong>Wage</strong> subsidy programmes: alternative designs" chapter 7 pp206-244 in<br />
Snower, D.J. and De La Dehesa, G. [eds.](1997) "Unemployment policy: government<br />
options for <strong>the</strong> labour market", Cambridge University Press.<br />
Piggot, J and Chapman, B. (1995) “Costing <strong>the</strong> Job Compact” Economic Record V.71 (215)<br />
pp313-328.<br />
Puhani, P. (2000) "The Heckman correction for sample selection and it's critique a short<br />
survey" Journal <strong>of</strong> Economic Surveys V.14, pp53-68.<br />
310
Quandt, R. (1972) "A new approach to switching regressions", Journal <strong>of</strong> <strong>the</strong> American<br />
Statistical Association Vol. 67, pp306-310.<br />
Rao, G.L. and Jones, F.L. (1986) "Effectiveness <strong>of</strong> youth manpower programs” Bureau <strong>of</strong><br />
Labour Market Research Working paper no.61, Department <strong>of</strong> Employment and<br />
Industrial Relations, Commonwealth <strong>of</strong> Australia, <strong>Australian</strong>, Government Publishing<br />
Service Canberra.<br />
Richardson, J. (1998) "Do wage subsidies enhance employability? Evidence from <strong>Australian</strong><br />
youth" Discussion Paper no.387, April 1998, Centre for Economic Performance,<br />
London School <strong>of</strong> Economics.<br />
Ridder, G. (1990) "Attrition in multi-wave panel data" pp45-67 in Hartog, J., Ridder, G. and<br />
Theeuwes, J. (eds.)(1990) "Panel data and labour market studies", North-Holland,<br />
Amsterdam.<br />
Rosenbaum, P.R. (1984) “From association to causation in observational studies: <strong>the</strong> role <strong>of</strong><br />
tests <strong>of</strong> strongly ignorable treatment assignment”, Journal <strong>of</strong> <strong>the</strong> American Statistical<br />
Association Vol.79 No.38, pp41-48.<br />
Rosenbaum, P.R. and Rubin, D.B. (1983), "The Central Role <strong>of</strong> <strong>the</strong> Propensity Score in<br />
Observational Studies for Causal Effects", Biometrika, 70, 1, 41-55.<br />
Rosenbaum, P.R. and Rubin, D.B. (1985a) "Constructing a control group using multivariate<br />
matched sampling methods that incorporate <strong>the</strong> propensity score" The American<br />
Statistician Vol. 39, pp33-38.<br />
Rosenbaum, P.R. and Rubin, D.B. (1985b) "The bias due to incomplete matching", Biometrics<br />
41 (March 1985) pp103-116.<br />
Rosholm, M. (1998) "Transitions in <strong>the</strong> labour market", University <strong>of</strong> Aarhus Department <strong>of</strong><br />
Economics PhD Thesis, University <strong>of</strong> Aarhus, Denmark.<br />
Ross, R.T. (1988) "Teenagers and <strong>the</strong> labour market 1983-1988", Social Welfare Research<br />
Centre Discussion paper no.8 (December 1988) University <strong>of</strong> NSW.<br />
Ross, R.T. (1989) "Labour market programs and income support: The <strong>Australian</strong> Experience"<br />
in Saunders, P. and Jamrozik, A. (eds.)(September 1989) "Social Policy and Inequity in<br />
Australia and New Zealand", Social Welfare Research Centre, University <strong>of</strong> New<br />
South Wales, Reports and Proceedings no 78.<br />
Routley, V.C. (1984) "Register <strong>of</strong> government employment and training programs", Bureau <strong>of</strong><br />
Labour Market Research Technical Paper no.19 (March 1984), Department <strong>of</strong><br />
Employment and Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
Roy, A.D (1951) "Some thoughts on <strong>the</strong> distribution <strong>of</strong> earnings" Oxford Economics Papers<br />
Vol.3 pp 135-146.<br />
Rubin, D.B. (1974) "Estimating causal effects to treatments in randomized and nonrandomized<br />
studies", Journal <strong>of</strong> Educational Psychology 66, pp688-701.<br />
Rubin, D.B. (1980), "Bias Reduction Using Mahalanobis-Metric Matching", Biometrics, 36,<br />
293-298.<br />
Rubin, D.B. (1986) "Statistics and causal inference, Comment: Which ifs have causal<br />
answers?" Journal <strong>of</strong> <strong>the</strong> American Statistical Association Vol. 81 Issue 396 pp 961-<br />
962.<br />
Scherer, P. (1985) "The Kirby report: implications for training, education and work" Bureau <strong>of</strong><br />
Labour Market Research Conference Paper no.54, Department <strong>of</strong> Employment and<br />
Industrial Relations, Commonwealth <strong>of</strong> Australia, Canberra.<br />
311
Schmertmann, C. P. (1994) "Selectivity bias correction methods in polychotomous sample<br />
selection models", Journal <strong>of</strong> Econometrics 60 pp101-132.<br />
Stricker, P. and Sheehan, P. (1981) "Hidden unemployment: <strong>the</strong> <strong>Australian</strong> experience"<br />
Institute <strong>of</strong> Applied Economic and Social Research, Melbourne.<br />
Sheen, V. and Tre<strong>the</strong>wey, J. (1991) "Unemployment in <strong>the</strong> recession: policies for reform",<br />
Bro<strong>the</strong>rhood <strong>of</strong> St. Lawrence, Melbourne.<br />
Sianesi, B. (2001)"An evaluation <strong>of</strong> <strong>the</strong> Swedish system <strong>of</strong> active labour market programmes<br />
in <strong>the</strong> 1990's" Institute <strong>of</strong> Fiscal Studies Working paper no.02/01, University College<br />
London.<br />
Sianesi, B. (2002)" Swedish system <strong>of</strong> active labour market programmes in <strong>the</strong> 1990's: overall<br />
effectiveness and differential performance" Institute <strong>of</strong> Fiscal Studies Working paper<br />
no.02/03, University College London.<br />
Sloan, J. (1985) "Comment on Continuity and change: labour market programs and education<br />
expenditure", The <strong>Australian</strong> Economic Review 71, pp114-115.<br />
Smith, R.E. (1983) "Employment subsidies in <strong>the</strong>ory and practice: <strong>the</strong> <strong>Special</strong> youth<br />
Employment Training Program?" Centre for Economic Policy Research Discussion<br />
Paper no.69, <strong>Australian</strong> National University, Canberra.<br />
Smith, R.E. (1984a) "Estimating <strong>the</strong> impacts <strong>of</strong> job subsidies on <strong>the</strong> distribution <strong>of</strong><br />
unemployment: reshuffling <strong>the</strong> queue?" Centre for Economic Policy Research<br />
Discussion Paper no.95, (may 1984) <strong>Australian</strong> National University, Canberra.<br />
Smith, R.E. (1984b)"How effective has <strong>the</strong> SYETP job subsidy really been" Discussion paper<br />
no 104, (August 1984) Department <strong>of</strong> economics, Research School <strong>of</strong> Social Sciences,<br />
<strong>Australian</strong> National University.<br />
Smith, J. (2000) "A critical survey <strong>of</strong> empirical methods for evaluating active labour market<br />
policies", Schweiz Zeitschrift für Volkswirtschaft und Statistik Vol. 136(3) pp1-22.<br />
Smith, J. and Todd, P. (2000) "Does matching overcome Lalonde's critique <strong>of</strong> nonexperimental<br />
estimators", University <strong>of</strong> Western Ontario Mimeo.<br />
Smith, J. and Todd, P. (2003) "Does matching overcome Lalonde's critique <strong>of</strong> nonexperimental<br />
estimators", Working paper no. 2003/5 CIBC Working Paper Series<br />
University <strong>of</strong> Western Ontario Department <strong>of</strong> Economics Social Science Centre,<br />
University <strong>of</strong> Western Ontario, Canada.<br />
http://www.ssc.uwo.ca/economics/centres/cibc/wp2003/Smith05.pdf<br />
Snower, D.J. and De La Dehesa, G. [eds.] (1997) "Unemployment policy: government options<br />
for <strong>the</strong> labour market", Cambridge University Press.<br />
Snower, D.J. (1994) "Converting Unemployment Benefits into Employment Subsidies" The<br />
American Economic Review, Vol. 84, No. 2, Papers and Proceedings <strong>of</strong> <strong>the</strong> Hundred<br />
and Sixth Annual Meeting <strong>of</strong> <strong>the</strong> American Economic Association. (May, 1994), pp.<br />
65-70.<br />
Stanley, T.D. and Jarell, S.B. (1998) “Gender <strong>Wage</strong> Discrimination Bias? A Meta-Regression<br />
Analysis”, The Journal <strong>of</strong> Human Resources, Vol. 33, pp947-973.<br />
Stretton, A. (1984) "The short term impact on participants <strong>of</strong> selected labour force programs",<br />
The Journal <strong>of</strong> Industrial Relations Vol. 26(1), March 1984 pp 76-90.<br />
Stretton, A. and Chapman, B.J. (1990) "An analysis <strong>of</strong> <strong>Australian</strong> labour market programs"<br />
Centre for Economic Policy Research Discussion Paper no.247, <strong>Australian</strong> National<br />
University, Canberra.<br />
312
Stromback, M. and Dockery, A. (2000) "Labour market programs, unemployment and<br />
employment hazards: an application using <strong>the</strong> 1994-1997 Survey <strong>of</strong> Employment and<br />
Unemployment Patterns" <strong>Australian</strong> Bureau <strong>of</strong> Statistics {ABS} Occasional Paper Cat.<br />
No. 6293.0.00.002, ABS, Canberra.<br />
Tukey, J.W. (1949) "One degree <strong>of</strong> freedom for non-additivity", Biometrics 5, 232-242.<br />
van de Ven, W.P.M.M. and van Praag, M.S. (1981)"The demand for deductibles in private<br />
health insurance: a probit model with sample selection" Journal <strong>of</strong> Econometrics Vol.<br />
17 pp229-252.<br />
Vella, F. (1998) "Estimating models with sample selection bias: a survey", The Journal <strong>of</strong><br />
Human Resources Vol. 33 (1) pp127-169.<br />
Victorian State Office (1985) "Leaving School <strong>of</strong> out <strong>of</strong> work: a guide for young people"<br />
pamphlet, Victorian State Office, <strong>Australian</strong>, Government Publishing Service<br />
Canberra.<br />
Wainer, H. (1987) "Eelworms, bullet holes and Geraldine Ferraro: some problems in<br />
statistically adjusting for survey non-response", Program Statistics Research, Technical<br />
Report no 87-76 (research report no 87-12), Educational Testing Service, Princeton,<br />
New Jersey.<br />
Webster, E. (1997a) "Labour market programs: a review <strong>of</strong> <strong>the</strong> literature" Working paper no<br />
28/97, Melbourne Institute <strong>of</strong> Applied economic and Social Research, University <strong>of</strong><br />
Melbourne.<br />
Webster, E. (1997b) "Labour Market Programs and <strong>the</strong> <strong>Australian</strong> Beveridge curve: 1978<br />
to1996" Working paper no 25/97, Melbourne Institute <strong>of</strong> Applied economic and Social<br />
Research, University <strong>of</strong> Melbourne.<br />
Webster, E. (1998) "Microeconomic evaluations <strong>of</strong> <strong>Australian</strong> labour market programs", The<br />
<strong>Australian</strong> Economic Review, June 1998, vol. 31, no. 2, pp. 189-201.<br />
Webster, E. and Summers, P. (1999)"The effect <strong>of</strong> labour market programs on wage inflation"<br />
Working paper no 16/99, Melbourne Institute <strong>of</strong> Applied economic and Social<br />
Research, University <strong>of</strong> Melbourne.<br />
White, H. (1980) "A heteroskedasticity consistent covariance matrix estimator and a direct test<br />
for heteroskedasticity" Econometrica 48 (4) pp817-830.<br />
White, H. (1982) "Maximum likelihood estimation <strong>of</strong> misspecified models" Econometrica, 50<br />
(1) pp1-25.<br />
Wielgosz, J.B. (1984)"The referral and placement activity <strong>of</strong> <strong>the</strong> Commonwealth Employment<br />
Service: a case study <strong>of</strong> <strong>the</strong> Brisbane Metropolitan Area" Bureau <strong>of</strong> Labour Market<br />
Research Working paper no.45, Department <strong>of</strong> Employment and Industrial Relations,<br />
Commonwealth <strong>of</strong> Australia, <strong>Australian</strong> Government Publishing Service Canberra.<br />
Wilde, J. (2000) “Identification <strong>of</strong> multiple equation probit models with endogenous dummy<br />
regressors” Economics Letters 69 (2000) pp 309-312.<br />
Windshuttle, K. (1985)"<strong>Youth</strong> wages, employment and unemployment: critique <strong>of</strong> a study by<br />
<strong>the</strong> Bureau <strong>of</strong> Labour Market Research", Department <strong>of</strong> Industrial Relations Working<br />
paper no.60, University <strong>of</strong> New South Wales.<br />
Yatchew, A. and Griliches, Z. (1985) “Specification error in probit models”, The Review <strong>of</strong><br />
Economics and Statistics, Vol. 67 No.1, pp134-139.<br />
Zabel, J. E. (1998) "An analysis <strong>of</strong> attrition in <strong>the</strong> Panel Study <strong>of</strong> Income Dynamics and <strong>the</strong><br />
survey <strong>of</strong> Income and Program Participation with an application to a model <strong>of</strong> labor<br />
market behaviour", The Journal <strong>of</strong> Human Resources Vol. 33 (1) pp479-506.<br />
313
314