Is headspace making a difference to young people’s lives?

Recommendations

Info

2. Evaluation Methodology Name Source type/description Dates Survey of young people The evaluation drew upon data collected as part of the National Child and Adolescent Survey of Mental Health and Wellbeing. A sample of this data comprises the W1 12-17 years comparison group data. Wave 2 12-17 years comparison group data; all 18-25 years comparison group data; and headspace treatment group data collected for evaluation Wave 1 YMM data collection period: Jun 2013 to Mar 2014. Wave 2 data collection period: May 2014 to Nov 2014 Wave 1 18-25 years comparison group data collection period: Dec 2013. Wave 2 data collection period: Sep 2014 to Oct 2014 Wave 1 intervention group data collection period: Dec 2013 to Jun 2014. Wave 2 data collection period: Aug 2014 to Jan 2015 Interviews with headspace staff Primary data collected for evaluation Fieldwork conducted Apr to Jun 2013 Interviews with headspace clients Primary data collected for evaluation Fieldwork conducted Apr to Jun 2013 Interviews with parents and carers Primary data collected for evaluation Fieldwork conducted Aug to Oct 2014 Survey of parents and carers Primary data collected for evaluation Online survey open Jun to Sep 2014 Survey of centre managers Primary data collected for evaluation Survey open Jun to Aug 2013 Professional stakeholders survey Primary data collected for evaluation Online survey open Sep to Nov 2014 2.3 Evaluation methods The multiple evaluation methods enable a comprehensive assessment of headspace effectiveness and efficiency. Where possible, the different methods were triangulated to strengthen the findings and to provide more depth to the analysis. The evaluation methods are summarised in Table 2.2 below. Further information on each method including sampling, recruitment and analysis techniques is provided in Appendix C. Table 2.2 Summary of evaluation methods Evaluation method Description of method Analysis of program data: headspace Centres Services Application (hCSA) Analysis of secondary data: (Census) Surveys of young people Analysis of the data contained in the headspace Centres Services Application (hCSA) for the 2013/14 financial year. This dataset contains information collected from both headspace clients and service providers. The hCSA data has been operationalised to collect information about young people’s access and engagement with headspace, aspects of the headspace service delivery model, patterns of client outcomes, government investment in headspace, and the extent of clinically significant change in headspace clients. Descriptive analysis was undertaken to profile headspace clients in comparison to the overall youth population. The hCSA data was also used to provide further information about the headspace survey intervention group. The evaluation was informed by the analysis of secondary data: the 2011 Census of Population and Housing. This secondary dataset was used to inform comparative analysis of the demographic characteristics of headspace clients and young people across the general population. Three surveys were conducted to assess young people’s outcomes for the headspace evaluation. The surveys were conducted in two waves, approximately 9 months apart. However, this timeframe varied considerably. The three surveys undertaken were: the ‘headspace treatment’ group (n = 1,364 at wave 2), a comparison group of 12-17 year olds (via the Young Minds Matter survey; n=1,686 at wave 2), and a comparison group of 18-25 year olds (sourced through a national online panel; n=936 at wave 2). The survey questions were largely drawn from the Young Minds Matter survey. Issues with survey timing and comparability are addressed in the analysis where possible and further information is provided in Appendix C. The three surveys were used to compare outcomes in young people that had sought services from headspace to those that had not sought any treatment and those that had sought other treatment from another mental health provider. A difference-in-difference approach was used to assess the impact of the program using survey cohorts. Propensity score matching was conducted to align the intervention and comparison groups. Social Policy Research Centre 2015 headspace Evaluation Final Report 12
2. Evaluation Methodology Evaluation method Fieldwork at sites Description of method Fieldwork was undertaken at 5 headspace sites around Australia between April and June 2013. Across these sites, a sample of 50 young people attending centres, 5 centre managers and 20 staff members were interviewed. 2. Sexual practices and risk in gay men Survey of centre managers Survey of professional stakeholders Parents and carers study Cost effectiveness analysis Additional economic analysis Interviews were semi-structured and conducted face-to-face. All interviews were recorded with the permission of participants. Interviews were then transcribed and imported into QSR NVivo10, a qualitative data analysis software package to assist coding and thematic analysis. Transcripts were coded using pre-determined coding frameworks (one for headspace staff and another for headspace clients). Following coding, analytic memos were written to summarise each node (or code) used and queries were run to identify frequencies and relationships across nodes. An online survey was distributed to centre managers at 40 operational centres (that is, centres established during rounds 1-3). The survey collected data in June 2013 and valid responses were provided by a total of 29 managers. The survey collected data on the headspace service delivery model and program implementation from a sample of respondents. This survey was conducted to examine specific features of the headspace service model as well as aspects of access and engagement that other data sources do not provide sufficient information on. In particular, the survey examined the perceptions of professionals affiliated with youth mental health care, or providers connected to the broader service system with regard to how headspace is collaborating with local services, and the impact of any collaboration. Representatives from headspace consortium organisations were invited to participate in the survey. In addition, a survey link and advertisement was posted on a number of forums including AIFS’ Child Family Community Australia news section, and the RACGP Friday Facts newsletter. The survey was distributed online through direct email invitation and advertisements with an embedded URL. The survey was open for approximately 4 weeks from early October to early November 2014. A total of 207 professionals participated in the survey. The KeySurvey system that hosted the survey has an automatic report function that provides basic analysis (descriptive results). Further analysis was conducted by exporting the data file into excel. Text provided in open-ended questions was thematically coded. This study explored the views of parents and carers of young people with mental health problems about their own mental health needs, their experiences with headspace services, and how headspace compares to other services. The study comprised two components: an online survey of parents and carers; and 6 focus groups and 3 individual interviews with parents/carers. Study participants were recruited through posters that were located in headspace centre reception areas, advertisements posted on the Facebook pages of beyondblue and the Butterfly Foundation, and through an advertisement and survey link that was posted on the parents and carers section of the headspace website. A total of 226 valid survey responses were received between 1 June and 30 September 2014. An additional 38 parents/carers participated in interviews that were held between August and October 2014. All interviews were transcribed and uploaded to NVivo, a qualitative data analysis software. Transcripts were coded using a pre-determined framework. The survey data was analysed using KeySurvey, a software system that enables analysis through an automatic report function. The analysis report provides basic descriptions of data collected. Further analysis of the survey was conducted following export of survey data into excel. Text provided in open-ended questions was thematically coded. The cost effectiveness analysis of the headspace program examines the government’s investment in headspace and compares costs with the outcomes of young people in comparison to a functional population and to other similar young people that did not receive treatment at headspace. The analysis was undertaken by economists at the Bankwest Curtin Economics Centre at Curtin University and is based upon both the difference-indifference methodology and the clinically significant change method. This component of the evaluation builds on the cost effectiveness analysis to consider different methodologies for increasing the efficiency and economy for the potential expansion of headspace. This analysis was undertaken by the Telethon Kids Institute at the University of Western Australia. It is presented in Appendix B. Social Policy Research Centre 2015 headspace Evaluation Final Report 13
Page 1 and 2: Is headspace making a difference to
Page 3 and 4: Authors This report was written by
Page 5 and 6: 5. Service Delivery Model 73 5.1 Wh
Page 7 and 8: Table 6.1 Table 6.2 Table 6.3 Main
Page 9 and 10: Figure 4.9 Figure 4.10 Figure 4.11
Page 11 and 12: Executive Summary headspace 1 aims
Page 13 and 14: Executive Summary or very high leve
Page 15 and 16: Executive Summary through mental he
Page 17 and 18: value to society of the improvement
Page 19 and 20: 1. Introduction people in Australia
Page 21: 2. Evaluation Methodology 2.1 Aims
Page 25 and 26: 2. Evaluation Methodology Table 2.3
Page 27 and 28: 2. Evaluation Methodology Represent
Page 29 and 30: 3. Access and Engagement with Centr
Page 51 and 52: 4. Outcomes of headspace Clients Th
Page 53 and 54: 4. Outcomes of headspace Clients 35
Page 55 and 56: 4. Outcomes of headspace Clients Ch
Page 57 and 58: 4. Outcomes of headspace Clients No
Page 59 and 60: 4. Outcomes of headspace Clients Th
Page 61 and 62: 4. Outcomes of headspace Clients Fi
Page 63 and 64: 4. Outcomes of headspace Clients A
Page 65 and 66: 4. Outcomes of headspace Clients Ta
Page 67 and 68: 4. Outcomes of headspace Clients nu
Page 69 and 70: 4. Outcomes of headspace Clients No
Page 71 and 72: 4. Outcomes of headspace Clients Ta
Page 73 and 74:
y three percentage points, and 2.1
Page 75 and 76:
4. Outcomes of headspace Clients So
Page 77 and 78:
4. Outcomes of headspace Clients I
Page 79 and 80:
4. Outcomes of headspace Clients in
Page 81 and 82:
4. Outcomes of headspace Clients Wh
Page 83 and 84:
5. Service Delivery Model For the a
Page 85 and 86:
5. Service Delivery Model the provi
Page 87 and 88:
5. Service Delivery Model headspace
Page 89 and 90:
5. Service Delivery Model services,
Page 91 and 92:
5. Service Delivery Model services
Page 93 and 94:
5. Service Delivery Model Further,
Page 95 and 96:
5. Service Delivery Model that we
Page 97 and 98:
5. Service Delivery Model is a cons
Page 99 and 100:
5. Service Delivery Model connectio
Page 101 and 102:
5. Service Delivery Model Table 5.2
Page 103 and 104:
5. Service Delivery Model Table 5.3
Page 105 and 106:
5. Service Delivery Model I don’t
Page 107 and 108:
5. Service Delivery Model group is
Page 109 and 110:
5. Service Delivery Model Figure 5.
Page 111 and 112:
6. The Costs of headspace One of th
Page 113 and 114:
6. The Costs of headspace Table 6.1
Page 115 and 116:
6. The Costs of headspace In-scope
Page 117 and 118:
6. The Costs of headspace Revenue/
Page 119 and 120:
6. The Costs of headspace Other gov
Page 121 and 122:
7. Conclusion The evaluation of hea
Page 123 and 124:
7. Conclusion treatment); however,
Page 125 and 126:
7. Conclusion Overall the evaluatio
Page 127 and 128:
Appendix A Evaluation scope areas h
Page 129 and 130:
Appendix B The current model of hea
Page 131 and 132:
Appendix B and each SA3 in non-capi
Page 133 and 134:
Appendix B chapter 39 . To determin
Page 135 and 136:
Appendix B While there is an overal
Page 137 and 138:
Appendix B Figure B3 describes the
Page 139 and 140:
Appendix B Figure B5 Cost per young
Page 141 and 142:
Appendix B The concept of access as
Page 143 and 144:
Appendix B Figure B7 Heard of heads
Page 145 and 146:
Figure B10 Proportion of headspace
Page 147 and 148:
Appendix B Table B5 Females 18-25 y
Page 149 and 150:
Appendix B following section. Figur
Page 151 and 152:
Appendix B and less access to mains
Page 153 and 154:
Appendix B Figure B17 Estimated pre
Page 155 and 156:
Appendix B Figure B18 SA1s within 1
Page 157 and 158:
Figure B20 Hypothetical allocation
Page 159 and 160:
Appendix B The current model of cen
Page 161 and 162:
Appendix B Current centre funding m
Page 163 and 164:
Appendix B Evaluation of the curren
Page 165 and 166:
Appendix B Level of funding availab
Page 167 and 168:
Appendix B provide a method for ide
Page 169 and 170:
Appendix B centres and Localities i
Page 171 and 172:
Appendix B for greater flexibility
Page 173 and 174:
Appendix B Additional consideration
Page 175 and 176:
Appendix B of achieving more effici
Page 177 and 178:
Appendix B • Used the simulation
Page 179 and 180:
Appendix C information relating to
Page 181 and 182:
Appendix C Regarding gender, in bot
Page 183 and 184:
Appendix C Figure C5 Total number o
Page 185 and 186:
Appendix C Table C1 Information Sum
Page 187 and 188:
Appendix C Data Cleaning and Analys
Page 189 and 190:
Appendix C groups. The age and sex
Page 191 and 192:
Appendix C Figure C8 Demographic ch
Page 193 and 194:
Appendix C The KeySurvey software s
Page 195 and 196:
Appendix C different client groups.
Page 197 and 198:
Appendix D Round State Centre Name
Page 199 and 200:
Appendix E Characteristics HS clien
Page 201 and 202:
Table E4 Cell Sizes for Figure 3.4
Page 203 and 204:
Appendix F Figure F1 Changes in K10
Page 205 and 206:
Appendix F three SEIFA quintiles. A
Page 207 and 208:
Appendix F Occasion of service inte
Page 209 and 210:
Appendix F Table F1 Cell sizes for
Page 211 and 212:
Page 213 and 214:
Page 215 and 216:
Appendix G Supplementary material f
Page 217 and 218:
Appendix G Table G3 Round 3 (10 cen
Page 219 and 220:
Appendix G Table G7 Round 7 (10 cen
Page 221 and 222:
State (youth population) a headspac
Page 223 and 224:
Page 225 and 226:
Appendix G Table G12 Round 11(16 ce
Page 227 and 228:
Table G16 Additional youth populati
Page 229 and 230:
Round Region State Additional youth
Page 231 and 232:
Page 233 and 234:
Page 235 and 236:
Appendix G Figure G1 Percentage of
Page 237 and 238:
Page 239 and 240:
Page 241 and 242:
Page 243 and 244:
Appendix G Table G22 Australians 18
Page 245 and 246:
Appendix G Synthetic small area est
Page 247 and 248:
Appendix G Figure G16 Estimated pre
Page 249 and 250:
Page 251 and 252:
Page 253 and 254:
Appendix G Figure G22 SA1s within 1
Page 255 and 256:
Page 257 and 258:
Page 259 and 260:
Appendix G Table G25 Centres alloca
Page 261 and 262:
References Atkins, D.C., Bedics, J.
Page 263:
References Smith, E., Jones, T., Wa
show all

Is headspace making a difference to young people’s lives?

Create successful ePaper yourself

Delete template?

Save as template?