Is headspace making a difference to young people’s lives?
Evaluation-of-headspace-program Evaluation-of-headspace-program
7. Conclusion headspace has been successful in attracting other young people from marginalised and at-risk groups. The service is being accessed by a high proportion of young people who identifies as LGBTI (approximately one in 5 clients), as well as those who are homeless or living in conditions of insecure housing such as couch surfing (approximately one in 10 clients). Access, however, does not guarantee sustained engagement and improvement. The outcomes analysis shows that LGBTI young people are less likely to show clinically or reliably significant improvement following treatment than young people who identified as heterosexual. Furthermore, homeless young people are less likely to return to headspace for a second occasion of service than the general client population. headspace has been less successful at engaging young people who were born overseas and who speak a language other than English at home. These groups are significantly under-represented at headspace. This finding is consistent with the literature that indicates that CALD young people are more reluctant to seek help than young people from mainstream cultures (Rickwood et al, 2007). Evaluation data shows that the centre-based program implements a variety of strategies to make services accessible for a diverse range of young people. Strategies most valued by young people include the provision of a wide range of services available at a single location, free or low cost services, the welcoming space filled with friendly and non-judgemental staff and the innovative and youth-friendly methods used to foster engagement including the use of iPads and social media. Despite efforts to increase service accessibility, barriers remain. Evaluation data indicates that most headspace clients live within 10 kilometres of a centre and the service is much less accessible to those living at greater distances. Perhaps in recognition of this, a significant number of stakeholders argued that headspace needs to expand its outreach services. Other barriers to service usage include centres that do not provide any extended opening hours, have long waiting times for services, and do not ensure that practices are culturally appropriate. In addition, young people identified the stigma of mental illness as a reason why they did not seek help earlier. This barrier is a challenge for all mental health services and evaluation data suggests that headspace may be having a positive impact in this area. Young people and their parents often spoke about accessing headspace after having unsatisfactory experiences with other services and generally headspace appears to be doing better than many mainstream services to engage young people. Data collected for the evaluation suggests that headspace could implement additional strategies to specifically target vulnerable groups of young people identified as under-represented. As part of this, centre staff should engage in genuine and ongoing liaison with services that target these groups of young people, including refugee and CALD support services. This active liaison is required to build links into these communities, which will assist centres to provide services that are more culturally appropriate. The evaluation found that awareness of headspace was relatively low among parents (63% of respondents to the Parents and Carers Survey reported that they either had not heard of headspace before their young person attended a centre or had heard of headspace, but did not know what they did). Interview data indicates that most young people accessed headspace through a multistep referral process (for example through recommendation from a school counsellor or a friend; or a referral from a GP) and parents often assisted them along the way (for example by driving them to a GP). Indeed, survey data indicates that parents provide important practical assistance to encourage their young person to attend appointments (such as driving them to a centre) as well as providing emotional support during the treatment process. Data suggests that parents play a more active role in supporting their young person’s engagement with headspace rather than encouraging access for the first time. Outcomes for young people The qualitative data is overwhelmingly positive about young people’s outcomes. Most young people and their parents attribute improvements across a number of outcome areas to headspace. Findings from the statistical analysis show a small positive improvement in outcomes of young people who sought headspace services relative to similar young people and a functional population. Specifically, the headspace treatment group records a greater reduction in psychological distress (K10 score) when compared with both matched groups over time (‘other treatment’ and ‘no treatment’), with both results statistically significant. The effect size for this outcome indicator may be considered relatively small (-0.11 for diff-in-diff in no treatment and -0.16 for diff-in-diff for other Social Policy Research Centre 2015 headspace Evaluation Final Report 112
7. Conclusion treatment); however, classifying the magnitude of the strength of an effect size is often contentious, especially for a diverse group of people accessing a wide range of different services. Other outcome indicators, including social inclusion and drug and alcohol use, show weaker results. While the ‘headspace treatment’ group shows an overall improvement in social inclusion over time, this improvement is not as strong as that observed for the matched ‘other’ and ‘no treatment’ groups. Further, while no significant change was observed in binge drinking in the ‘headspace treatment’ group, a reduction in binge drinking was observed in the matched ‘no treatment’ group from 1.5 day to just under one day on average each month. The difference-in-differences observed over time are significant at the 1% level, with the matched ‘no treatment’ group reporting a reduction in binge drinking by 0.7 days more than the ‘headspace treatment’ group, and the ‘other treatment’ group reporting a reduction in binge drinking by 0.75 days more than the ‘headspace treatment’ group. No statistically significant differences were observed for cannabis use. These results must be interpreted with caution as these outcome indicators were not included as benchmarks in the matching technique and clear differences are seen at baseline between the groups, particularly for the social inclusion outcome. Results that seek to extend this analysis by testing the prevalence of a clinically significant change show that overall, substantially more young people using headspace services get significantly better (22.7%) than get worse (9.4%) when measured against ‘functional’ benchmarks of psychological distress derived from the general youth population. Further, particularly strong effects arising from the improvements in mental health delivered through headspace include a significantly reduced prevalence of suicidal ideation and self-harm. One of the more important findings in this evaluation is the improvement that can be seen in other valuable outcomes (most notably a reduction in suicidal ideation and self-harm) for those receiving headspace treatments, even among those for whom the K10 measure of psychological distress shows little change. The outcomes analysis also highlighted gains for clients related to enhanced social inclusion and economic participation. Economic and social benefits from improved mental health functioning are delivered through a number of positive outcomes, and to the extent that these can be attributed to headspace treatment, add value to the headspace investment. The strongest economic benefits arise from a significant reduction in the number of days lost due to illness, the number of days cut down, and the reduction in suicide ideation and self-harm. It should be recognised that employment may be a longer term outcome than possible to fully judge from this evaluation. Nevertheless, these findings provide some indication of the economic and social value to society of the improvements in mental health functioning being delivered through headspace. Caution should be taken when interpreting the outcome findings. Young people interviewed attributed improvements across a number of outcome areas to headspace while the statistical data shows a small program effect. Given the timing of the fieldwork, it was not possible to explore statistical analysis results with young people and headspace staff. The headspace service delivery model headspace is a holistic program that operates on a national, community and individual level. The value of national branding and mental health promotion work as well as local community awareness initiatives in educating young people about the services available to them, encouraging help seeking, and reducing the stigma of mental health problems are highlighted by the Centre Managers Survey, the Professional Stakeholders Survey and interview data. The headspace service delivery model is designed to enable young people to receive multiple services from different practitioner disciplines within one location. This minimises the need for referral to other services, which may explain the very low rate of formal referrals to other services for headspace clients. Evaluation data suggests that the majority of referrals that connect young people to other services within the system are informal or verbal. These are not recorded in the administrative data, but staff at all fieldwork sites spoke of referring frequently. It was clear that some centres are working effectively with other local service providers while tensions and challenges are evident in other centres. Centres that have developed good links with local GPs and CAMHS in particular should work with other sites to lead positive change and share successful methods of communication and cooperation. Overall, this evaluation confirms, however, that the service context Social Policy Research Centre 2015 headspace Evaluation Final Report 113
- Page 71 and 72: 4. Outcomes of headspace Clients Ta
- Page 73 and 74: y three percentage points, and 2.1
- Page 75 and 76: 4. Outcomes of headspace Clients So
- Page 77 and 78: 4. Outcomes of headspace Clients I
- Page 79 and 80: 4. Outcomes of headspace Clients in
- Page 81 and 82: 4. Outcomes of headspace Clients Wh
- Page 83 and 84: 5. Service Delivery Model For the a
- Page 85 and 86: 5. Service Delivery Model the provi
- Page 87 and 88: 5. Service Delivery Model headspace
- Page 89 and 90: 5. Service Delivery Model services,
- Page 91 and 92: 5. Service Delivery Model services
- Page 93 and 94: 5. Service Delivery Model Further,
- Page 95 and 96: 5. Service Delivery Model that we
- Page 97 and 98: 5. Service Delivery Model is a cons
- Page 99 and 100: 5. Service Delivery Model connectio
- Page 101 and 102: 5. Service Delivery Model Table 5.2
- Page 103 and 104: 5. Service Delivery Model Table 5.3
- Page 105 and 106: 5. Service Delivery Model I don’t
- Page 107 and 108: 5. Service Delivery Model group is
- Page 109 and 110: 5. Service Delivery Model Figure 5.
- Page 111 and 112: 6. The Costs of headspace One of th
- Page 113 and 114: 6. The Costs of headspace Table 6.1
- Page 115 and 116: 6. The Costs of headspace In-scope
- Page 117 and 118: 6. The Costs of headspace Revenue/
- Page 119 and 120: 6. The Costs of headspace Other gov
- Page 121: 7. Conclusion The evaluation of hea
- Page 125 and 126: 7. Conclusion Overall the evaluatio
- Page 127 and 128: Appendix A Evaluation scope areas h
- Page 129 and 130: Appendix B The current model of hea
- Page 131 and 132: Appendix B and each SA3 in non-capi
- Page 133 and 134: Appendix B chapter 39 . To determin
- Page 135 and 136: Appendix B While there is an overal
- Page 137 and 138: Appendix B Figure B3 describes the
- Page 139 and 140: Appendix B Figure B5 Cost per young
- Page 141 and 142: Appendix B The concept of access as
- Page 143 and 144: Appendix B Figure B7 Heard of heads
- Page 145 and 146: Figure B10 Proportion of headspace
- Page 147 and 148: Appendix B Table B5 Females 18-25 y
- Page 149 and 150: Appendix B following section. Figur
- Page 151 and 152: Appendix B and less access to mains
- Page 153 and 154: Appendix B Figure B17 Estimated pre
- Page 155 and 156: Appendix B Figure B18 SA1s within 1
- Page 157 and 158: Figure B20 Hypothetical allocation
- Page 159 and 160: Appendix B The current model of cen
- Page 161 and 162: Appendix B Current centre funding m
- Page 163 and 164: Appendix B Evaluation of the curren
- Page 165 and 166: Appendix B Level of funding availab
- Page 167 and 168: Appendix B provide a method for ide
- Page 169 and 170: Appendix B centres and Localities i
- Page 171 and 172: Appendix B for greater flexibility
7. Conclusion<br />
treatment); however, classifying the magnitude of the strength of an effect size is often contentious,<br />
especially for a diverse group of people accessing a wide range of different services.<br />
Other outcome indica<strong>to</strong>rs, including social inclusion and drug and alcohol use, show weaker results.<br />
While the ‘<strong>headspace</strong> treatment’ group shows an overall improvement in social inclusion over time,<br />
this improvement is not as strong as that observed for the matched ‘other’ and ‘no treatment’ groups.<br />
Further, while no significant change was observed in binge drinking in the ‘<strong>headspace</strong> treatment’<br />
group, a reduction in binge drinking was observed in the matched ‘no treatment’ group from 1.5 day<br />
<strong>to</strong> just under one day on average each month. The <strong>difference</strong>-in-<strong>difference</strong>s observed over time<br />
are significant at the 1% level, with the matched ‘no treatment’ group reporting a reduction in binge<br />
drinking by 0.7 days more than the ‘<strong>headspace</strong> treatment’ group, and the ‘other treatment’ group<br />
reporting a reduction in binge drinking by 0.75 days more than the ‘<strong>headspace</strong> treatment’ group. No<br />
statistically significant <strong>difference</strong>s were observed for cannabis use. These results must be interpreted<br />
with caution as these outcome indica<strong>to</strong>rs were not included as benchmarks in the matching<br />
technique and clear <strong>difference</strong>s are seen at baseline between the groups, particularly for the social<br />
inclusion outcome.<br />
Results that seek <strong>to</strong> extend this analysis by testing the prevalence of a clinically significant change<br />
show that overall, substantially more <strong>young</strong> people using <strong>headspace</strong> services get significantly better<br />
(22.7%) than get worse (9.4%) when measured against ‘functional’ benchmarks of psychological<br />
distress derived from the general youth population. Further, particularly strong effects arising from<br />
the improvements in mental health delivered through <strong>headspace</strong> include a significantly reduced<br />
prevalence of suicidal ideation and self-harm.<br />
One of the more important findings in this evaluation is the improvement that can be seen in other<br />
valuable outcomes (most notably a reduction in suicidal ideation and self-harm) for those receiving<br />
<strong>headspace</strong> treatments, even among those for whom the K10 measure of psychological distress<br />
shows little change. The outcomes analysis also highlighted gains for clients related <strong>to</strong> enhanced<br />
social inclusion and economic participation. Economic and social benefits from improved mental<br />
health functioning are delivered through a number of positive outcomes, and <strong>to</strong> the extent that these<br />
can be attributed <strong>to</strong> <strong>headspace</strong> treatment, add value <strong>to</strong> the <strong>headspace</strong> investment. The strongest<br />
economic benefits arise from a significant reduction in the number of days lost due <strong>to</strong> illness,<br />
the number of days cut down, and the reduction in suicide ideation and self-harm. It should be<br />
recognised that employment may be a longer term outcome than possible <strong>to</strong> fully judge from this<br />
evaluation. Nevertheless, these findings provide some indication of the economic and social value <strong>to</strong><br />
society of the improvements in mental health functioning being delivered through <strong>headspace</strong>.<br />
Caution should be taken when interpreting the outcome findings. Young people interviewed attributed<br />
improvements across a number of outcome areas <strong>to</strong> <strong>headspace</strong> while the statistical data shows<br />
a small program effect. Given the timing of the fieldwork, it was not possible <strong>to</strong> explore statistical<br />
analysis results with <strong>young</strong> people and <strong>headspace</strong> staff.<br />
The <strong>headspace</strong> service delivery model<br />
<strong>headspace</strong> is a holistic program that operates on a national, community and individual level. The<br />
value of national branding and mental health promotion work as well as local community awareness<br />
initiatives in educating <strong>young</strong> people about the services available <strong>to</strong> them, encouraging help seeking,<br />
and reducing the stigma of mental health problems are highlighted by the Centre Managers Survey,<br />
the Professional Stakeholders Survey and interview data.<br />
The <strong>headspace</strong> service delivery model is designed <strong>to</strong> enable <strong>young</strong> people <strong>to</strong> receive multiple<br />
services from different practitioner disciplines within one location. This minimises the need for<br />
referral <strong>to</strong> other services, which may explain the very low rate of formal referrals <strong>to</strong> other services<br />
for <strong>headspace</strong> clients. Evaluation data suggests that the majority of referrals that connect <strong>young</strong><br />
people <strong>to</strong> other services within the system are informal or verbal. These are not recorded in the<br />
administrative data, but staff at all fieldwork sites spoke of referring frequently. It was clear that some<br />
centres are working effectively with other local service providers while tensions and challenges<br />
are evident in other centres. Centres that have developed good links with local GPs and CAMHS<br />
in particular should work with other sites <strong>to</strong> lead positive change and share successful methods of<br />
communication and cooperation. Overall, this evaluation confirms, however, that the service context<br />
Social Policy Research Centre 2015<br />
<strong>headspace</strong> Evaluation Final Report<br />
113