Is headspace making a difference to young people’s lives?

2. Evaluation Methodology
Representativeness of participants in stakeholder surveys
Although strenuous attempts were made to distribute the Survey of Professional Stakeholders,
to a wide range of service providers, the response rate was lower than anticipated. In particular
the number of GPs who completed the survey was low (n=45). It is not possible to assess the
representativeness of stakeholders as there is no definition of the population. Nevertheless, the
analysis provides some important insights into the strengths and weaknesses of the headspace
model and how it works within the broader health and mental health service context.
2. Sexual practices and
risk in gay men
Similarly, the surveys and focus groups with parents and carers were only conducted with a relatively
small number of parents who were self-selected. Again, this component of the evaluation provided
important insights into the strengths and weaknesses of headspace’s engagement with parents and
carers.
Reliance on the K10
The Kessler 10 Psychological Distress Scale (K10) was selected to assess the effectiveness of
headspace in relation to the mental health outcomes of clients. This instrument is designed to
measure psychological distress based on 10 questions about negative emotional states experienced
in the past 30 days. Young people are asked to complete the K10 at various times throughout their
treatment (first, 3rd, 6th, 10th, and 15th occasion of service), and in addition, the evaluators collected
K10 data at two different time points from a sample of headspace clients and a comparison group
via the young people survey. While the K10 has demonstrated efficacy in identifying mental health
outcomes, it does not provide an unequivocal means of gauging the social, emotional and physical
wellbeing of young people accessing headspace. This is important to consider as the program effect
reported using the Clinically Significant Change method only measures changes in K10 scores.
Because headspace is a holistic program that aims to improve young people’s wellbeing across
multiple domains, the evaluators also examined change across other outcome variables. The
difference-in-difference approach draws upon survey data to assess changes in young people’s
mental health, physical health, drug and alcohol use, and social inclusion.
These limitations, and the methods chosen to address them, are outlined to assist in the
interpretation of evaluation findings. A number of the analyses in this evaluation indicate that the
headspace model is highly variable and there are very significant variations between headspace
centres in the level of service provided, their collaboration with other mental health services, and
their engagement with the communities in which they are situated. This evaluation does not report on
individual centres and, therefore, the findings represent the whole of the headspace program rather
than the performance of particular centres.
2.6 Evaluation ethics
Researchers sought and received approval to conduct evaluation activities from UNSW’s Human
Research Ethics Committee (HREC) (approval reference number HREC 13024). Approval to conduct
this project was sought in two separate stages. This was done to align with the staged evaluation
activities specified by the Department, and to ensure that approval processes did not delay milestone
reporting.
The application for ethics approval for stage 1 of the evaluation was submitted in February 2013.
Following a request for additional information, the HREC granted approval for stage 1 on the 21st of
March 2013. The application for ethics approval for stage 2 of the evaluation was submitted to the
HREC in April 2013. Again, following a request for additional information and some modifications,
approval was given in May 2013.
Social Policy Research Centre 2015
headspace Evaluation Final Report
17