Evaluation of the Australian Wage Subsidy Special Youth ...

139
matching is the ability to weed out the incomparable cases, and find the treatment results
for the most comparable treatment and comparison cases. In addition, the empirical
power of propensity score matching in reducing the problem of selection bias is improved
with a propensity score very similar but preferably equal to that of the treated persons.
Also, the bivariate probit employed here is based on the assumption of a common effect,
which makes the choice among propensity score models simpler because the loss of
treated observations due to lack of a good match does not affect the parameter being
estimated by the propensity score matching (as opposed the scenario when heterogeneous
effects are assumed). The 0.001 caliper result would then be chosen as the result for
comparison upon the basis that it had no great deterioration in performance but compared
the most comparable cases. In light of this, the 0.001 caliper results are compared to the
bivariate probit in the discussion.
4.9 Sensitivity analysis: all-in-Radius matching
As a sensitivity analysis of the choice of matching algorithm, results for ‘all-in-radius’
matching are shown in Table 4.7. ‘All in radius’ matching might achieve some gains in
balance, as pair-matching forces the selection from within the caliper of only one
comparison case. This sensitivity analysis is an exercise to examine the importance of the
choice of matching algorithm.
Table 4.7 repeats the presentation of various calipers as used for the pair patching. The
first column shows the smallest caliper width applied to the difference in the propensity
used to find the matched treated and comparison cases, the caliper width 0.001. Each
column to the right of column 1 uses a wider caliper: Column 2 shows the caliper width
0.005, column 3 shows 0.01 caliper width, column 4 shows 0.02 caliper width and finally
column 5 shows 0.05 caliper width. To assess the matching, for each set of matching
results is shown the matched mean difference, and the F statistic of the statistical
significance of the difference, the number of treat and comparison cases matched, the
number of comparison cases unmatched and the mean standardized bias from the match.