Evaluation of the Australian Wage Subsidy Special Youth ...

18
and the possible impact of repeated interviewing on behaviour. Additionally, there is the
issue of overlapping social programs, which can affect even experimental program
evaluations, and make it difficult to quantify the unique contribution of each program.
In the absence of controlled experimentation, a comparison group is selected comprised
of persons who are not participants in the program, but who have similar characteristics.
The gain in employment they make serves as the benchmark against which the progress
of the participants is compared. This is often termed the matched comparison group
design. It is argued that because the comparisons are subjected to similar economic
conditions at the time of the treatment, then any greater gain in employment due to
participants than the comparisons is due to the program. The problem subsequently, is
how to select the comparison group, and adjust the treatment and comparison groups in
order to make them more comparable and so enable attribution of the effect to the
program.
1.3.2 Evaluation problem
Heckman, Lalonde and Smith (1999) p1877 use the phrase ´the evaluation problem´ and
succinctly describe the underlying difficulty in evaluating programs as the issue of reconstructing
counterfactuals. To find the effect of the program on the individual, a
comparison must be made between the observed employment for that treated individual
and the outcome that would have occurred if that person had not participated. Yet what is
observed is the outcome for participants and the outcome for non-participants. Inference
about the impact of the program involves speculation about outcomes had they not
received treatment. The counterfactual of what would have occurred if that person had
not participated cannot be observed. Instead, all methods used for evaluation attempt to
infer an estimate of the counterfactual from the observed data, and then use this to find
the program effect. Hujer and Caliendo (2000) p9 describe the theoretical framework of
this approach as the potential outcome approach, most often attributed to Rubin (1974).
Early authors associated with this model are Fisher (1951), Roy (1951), and Quandt
(1972) (Heckman, Ichimura and Todd (1997): 608). Identifying assumptions, which can
differ for each method, are introduced to help identify the causal effect of the program.