Improving Public Confidence in the Criminal Justice System: An ...

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500Promoting the performance and progress of the CJS carries the uniquechallenge of requiring all three strategies. Whilst for many, knowledgeabout the system is second-hand, not a day passes without the broadcastand print media carrying stories about crime and justice. Accordingly, itwas agreed that the booklet needed to contain facts and figures that werenot only informative but were presented in a persuasive way. Moreover, anoverarching theme had to be created and reinforced so that the reminderfunction could challenge and overcome second-hand stereotypes aboutperformance and progress received from the media.A 20-page A5 booklet printed in colour and on heavyweight, mattfinishedpaper was produced with the intention of conveying to therecipient that this was a quality product worth looking at. On the first twopages the aim was to arouse attention and establish the reminder themethat ‘Things aren’t always what they seem’. 1 Eye-catching graphics wereemployed to both inform and persuade the reader that in their LCJB areacrime was actually going down and prison sentence lengths were gettinglonger. The data used were all based on official statistics and are sourcedfor further reference.Having designed the booklet, the research team needed to test the mosteffective way of delivering it. From a marketing perspective this relates todetermining the appropriate ‘promotional mix’ (Chartered Institute ofMarketing 2004). This refers to the set of tools that can be used tocommunicate with consumers. Broadly speaking there are five tools:advertising; public relations; sales promotion; direct marketing; andpersonal selling. The Chartered Institute of Marketing (CIM) reports thatresearch shows that people need to see an advertisement at least seventimes before it starts to mean anything to them (Chartered Institute ofMarketing 2004). For this reason it can be very costly and was notconsidered appropriate to use in combination with a mail drop. For thesame reason the public relations and sales promotions approaches using,for example, a news or a press release prior to a mail drop were alsodiscounted. This left the direct marketing and personal selling approachesas potential tools to adapt to the particular needs of this project. As CIMpoints out, direct marketing is an effective tool because each letterdistributed can be personalised which improves response rates greatly.However, most effective of all is the face-to-face contact of personal selling.As Chartered Institute of Marketing (2004) states:This is the most effective form of promotion because it allows your approach to betailored to the needs of an individual customer . . . Face-to-face, a sales person canbuild a relationship with the customer – understanding their needs and feedingback this knowledge to the business. (p.4)Applying this knowledge to the delivery of the booklets, three methods ofdelivery were determined. The first method, direct marketing approach,involved posting the booklet in a personalised envelope to the individual.The second and third methods involved personal contact but entailedeither limited interaction through handing the booklet to the individual ormore extensive interaction by explaining it to the person. The purpose of488r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500interviewed the same individuals from all groups on two occasions. Thesecond interview was conducted, on average, four weeks after the first. Theinterview schedule, informed from previous panel surveys (Chapman,Mirrlees-Black and Brawn 2002), contained questions about knowledge andawareness of crime rates and sentencing, and confidence in different aspectsof the CJS, as asked in the BCS. The second survey largely asked the samequestions as the first, but there were a few differences, for example, askingpeople who had received the booklet if they had remembered receiving it, ifthey had read it, and if so what they thought about it.The analyses focused upon changes in people’s knowledge and confidence,by comparing changes in the intervention groups (pre- to post-test) to see ifthese were significantly greater than changes in the control group.Booklet DistributionWhilst adhering to data-sharing protocols, the names and addresses of peopleinterviewed for the first telephone survey were passed to a communicationsdelivery company. People in group 2 were then sent the information bookletin the post. For people in groups 3 and 4, trained staff of the deliverycompany handed over the booklet on the doorstep. Upon ringing thedoorbell, the deliverers introduced themselves, showed identification andfollowed a script: a short script for group 3 and a full script, providing a verbalsummary of the booklet, for group 4. The deliverers presented the bookletonly to the named person participating in the research.Sample SizesOverall, 3,667 people were interviewed in the first survey and 2,846 (78%of those initially interviewed) in the second. Table 2 shows the sample sizesand drop-out through the different phases of the research.In order to examine any potential bias in the demographic characteristicsof people involved in the research, the profile of people interviewedwas compared with the profile of those within the LCJB area as drawn fromTABLE 2Sample Sizes and AttritionGroup Panel 1interviews nAttempted bookletdeliverySuccessful bookletdelivery w Panel 2interviews nn1 845 n/a n/a 7012 826 609 609 5433 1000 920 715 8024 996 923 702 800Total 3667 2452 2026 2846(Notes: n People were called back seven times during the two-week interview period.w A minimum of three delivery attempts were made within the three-week delivery period.nnIn addition to the control group, people who had an attempted booklet delivery were contacted for asecond interview. Here, people were called back seven times during the four-week interview period.)490r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500the 2001 area Census data. This showed that the profiles did not varyconsiderably across the four groups, indicating that any biases affected eachgroup to a similar extent. Furthermore, when investigating if there werecharacteristics of Phase 1 respondents who were most likely to participate atPhase 2, no differences were found, both overall or across the four groups,indicating that non-response bias did not differ across the groups. Taken as awhole, this means that the analysis is able to make comparisons across thefour groups. When conducting the quantitative analysis, the researcherscompared the four sub-groups based upon their initial treatment intent viathe original random group assignment, irrespective of whether or not theperson received and read the information booklet. In other words, anIntention to Treat Analysis was mounted rather than a Treatment ReceivedAnalysis (Lee et al. 1991), to avoid the effects of crossover and drop-outwhich can introduce biases.(ii) Qualitative Data Collection: Focus Group InterviewsAt the end of the second survey, respondents were asked if they would beprepared to participate in a follow-up focus group. Subsequently, aselection of people was contacted and invited to participate in one of eightfocus groups which took place over the summer of 2007. Overall, 49people participated in the focus group interviews.It is recognised that the views of people attending the focus groups arenot necessarily representative of all interviewees. The main aim of the focusgroups was to further explore the use of the information booklet as asource of information about the CJS and to discuss any improvements thatcould be made.ResultsThe statistical analysis simultaneously tests whether the change in theproportion of people providing positive responses from pre- to postinterventiondiffered significantly between the experimental and controlgroups. This parametric test, The Dobby Test, was previously used in theNational Reassurance Policing Programme evaluation (Tuffin, Morris andPoole 2006). 5Crime LevelsRespondents from each of the four groups were asked to assess whether, overthe previous two years they thought there had been more crime, less crime orabout the same amount in the country as a whole. Because there has actuallybeen a reduction in crime, more and same responses were treated as negativeand a less response as positive. The results are given below in Table 3.As can be seen in Table 3, at the post-intervention stage, the responsesfrom all those in the experimental groups performed five percentagepoints better than those in the control group. This statistically significantresult indicates that, overall, the booklet was successful in improving491r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500TABLE 3Proportion Correctly Estimating Crime has Fallen% positiveresponse% pointchange% point gainover controlsP(Z) one tailedExp. vs ControlPre-Post-CONTROL GROUP 3.2 4.4 1.2 n/a n/aTOTALEXPERIMENTAL4.9 11.1 6.2 5.0 0.0000nnnPosted Exp. Group 4.0 8.9 4.9 3.7 0.0046nnHanded Exp. Group 3.6 10.9 7.3 6.1 0.0000nnnExplained Exp. Group 6.8 12.7 5.9 4.7 0.0006nnn(Notes: The term positive response is used in the text where there is a significant positive differencewhen comparing the results for experimental and control groups.npo0.05; nn po0.01; nnn po0.001.)people’s understanding that nationally crime has been falling over the lasttwo years. Furthermore, compared with the control group, each of thedelivery mechanisms employed was effective in improving people’sknowledge; although the ‘handed’ and ‘explained’ approaches were moreeffective than posting the booklet this was not a statistically significantdifference between these delivery approaches.During the focus group interviews the problem with delivering thebooklet by post compared with the ‘personal touch’ of handing or explainingit to the recipient was often mentioned. One postal recipient observed:I [prefer] the personal touch . . . I mean a lot of mail, most mail now is junk. Mostmail is junk mail, isn’t it? And I’ve dropped stuff in the bin probably without evenlooking at it in the past. This just caught my eye, I suppose, because it’s slightlybigger and bulkier. But I think if they actually handed it to you and explained whatit’s about, you’re more likely to look at it.Prison Sentence LengthsAlong with overestimating the level of crime, the public also tends tounderestimate the severity of sentences made by the courts. To measurethe impact of the intervention on this aspect of the perception gap,members of the sample were invited to estimate the average length of aprison sentence for males aged 21 years or over convicted of rape. Basedon the latest sentencing data at the time of the research, covering Januaryto December 2004, the correct answer is seven years (84 months). Answerswere invited in months and grouped into twelve-month clusters. Thoseestimating between 84 and 95 months were considered to have provided apositive response. The findings are reported in Table 4 below.Once again, those in all three of the experimental groups did significantlybetter than those in the control group. As one focus group interviewee stated:492r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500TABLE 4Proportion Correctly Estimating Sentence Length for Rape% positiveresponse% pointchange% point gainover controlsP(Z) one tailedExp. vs ControlPre-Post-CONTROL GROUP 5.1 5.8 0.7 n/a n/aTOTAL EXPERIMENTAL 5.7 10.2 4.5 3.8 0.0010nnnPosted Exp. Group 6.0 10.3 4.3 3.6 0.0186nHanded Exp. Group 6.4 9.7 3.3 2.6 0.0487nExplained Exp. Group 4.9 10.8 5.9 5.2 0.0004nnn(Notes: The term positive response is used in the text where there is a significant positive differencewhen comparing the results for experimental and control groups.npo0.05; nn po0.01; nnn po0.001.)It surprised me. I said five because I knew it was quite low. I was even thinking itcould be two years.Although there were no statistically significant differences between thedelivery mechanisms, the greatest improvement was noted for the peoplewho had the booklet delivered and explained to them. Here a difference ofover five percentage points was found in comparison with the controls.Sentencing SeverityTo gauge whether the booklet had changed attitudes toward sentencingseverity, answers were invited to the question whether the sentenceshanded down by the courts are too tough, about right or too lenient. Theanalysis focused on a shift from tough or lenient to about right. Accordingly, arise in the proportion of those thinking sentences were about right wasregarded as a positive response. Details are shown in Table 5 below.Whilst there was no change registered by the control group, all threeexperimental groups registered improvements in attitudes towardssentencing severity by approximately five percentage points.Confidence in the Criminal Justice System (CJS)The responses made to the headline confidence measure are summarisedin Table 6. Without exception, those receiving the booklet recorded agreater positive response than those not receiving it. As one of therespondents later remarked during the follow-up interviews when asked iftheir views had changed:Well, mine did, I guess, because I would have underestimated the amount of peoplethat they would have brought to justice before and then, when I read the booklet,then I was quite surprised of what I read. So, yes, I guess my opinion probablyimproved after I read the booklet.493r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500TABLE 5Proportion Stating Sentences Given by the Courts Were About Right% positiveresponse% pointchange% point gainover controlsP(Z) one tailedExp. vs ControlPre-Post-Post-CONTROL GROUP 20.3 20.3 0.0 n/a n/aTOTALEXPERIMENTAL20.1 24.9 4.8 4.8 0.0030nnPosted Exp. Group 17.9 22.8 4.9 4.9 0.0141nHanded Exp. Group 19.5 23.8 4.3 4.3 0.0197nExplained Exp. Group 22.1 27.4 5.3 5.3 0.0074nn(Notes: The term positive response is used in the text where there is a significant positive differencewhen comparing the results for experimental and control groups.npo0.05; nn po0.01; nnn po0.001.)TABLE 6Proportion Saying They Were Confident that the Criminal Justice System Was Effective in BringingPeople Who Commit Crimes to Justice% positiveresponse% pointchange% point gainover controlsP(Z) one tailedExp. vs ControlPre-CONTROL GROUP 36.0 42.6 6.6 n/a n/aTOTALEXPERIMENTAL34.0 45.4 11.4 4.8 0.0123nPosted Exp. Group 30.4 40.0. 9.6 3.0 0.1427 nsHanded Exp. Group 33.4 46.4 13.0 6.4 0.0059nnExplained Exp. Group 37.0 48.0 11.0 4.4 0.0434n(Notes: The term positive response is used in the text where there is a significant positive differencewhen comparing the results for experimental and control groups.npo0.05; nn po0.01; nnn po0.001.)Changes were noted for all groups suggesting a degree of volatility inconfidence in the CJS’s effectiveness. However, when their scores wereaggregated all those who got the booklet registered in aggregate an overall11.4 percentage point positive change, which was 4.8 percentage pointshigher than the positive change registered by the control group. Thedifference between the ‘posted’ group and the control group was notstatistically significant, but the differences between both the ‘handed’ and‘explained’ groups and the control group were.494r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500Table 6 therefore provides evidence that, so far as improving publicperceptions about the headline target is concerned, the booklet is aneffective medium but the mechanism of delivery is also very important.Whilst some factual information may be effectively conveyed by post, forexample, on crime levels and sentence lengths, as well as influencingcertain attitudes toward sentencing, for example, on sentencing beingabout right, this is not true for general confidence in the CJS. Accordingly,careful consideration needs to be given to the type of delivery mechanismemployed to maximise the potential of the booklet.Lessons Learned About the Booklet and Delivery MethodsThe results described in Tables 3 to 6 provide evidence that both knowledgeabout, and attitudes toward, the CJS can be improved using a bookletdelivered in different ways. However, a number of challenges had to be facedover and above competing with negative messages from the media (Hough2008) or general public scepticism about official statistics (Goddard 2005).The first hurdle to be overcome was to get people to read literature of thissort. During the post-intervention stage of the survey, those people who hadbeen given a booklet were asked how much of it they had read. Of the 1,545people who recalled receiving a booklet, 43% (n 5 669) read all or most of it.However, the delivery mechanism appears to have been influential, with thelikelihood of reading all or most of the booklet linked to the type of deliverymechanism. Thus those getting it through the post had the lowest proportionof 36.1% saying they read all or most of the booklet, compared with thosehanded it (44%) and those delivered it with an explanation (48%). Those whohad been handed the booklet were significantly more likely to read it thanthose who had received it in the post. 6Amongst the 1,219 people who stated they had read the booklet, 7 62%found it very easy to read with a further 36% finding it fairly easy to read.Nearly nine out of ten of the readers said they had learned something fromthe booklet and just under half (49%) said that at least one item in thebooklet had surprised them. These findings are consistent with thebooklet’s content needing to both inform and persuade.The equal importance of the reminder function was, however, suggestedby another focus group interviewee with regard not only to the design of thebooklet and the need to contain repeated message reinforcement but alsoreinforcement through other communication channels:I think it gives you confidence when you read it but then, to be fair, if you listen tothe local news and you see the free papers that come around and you read aboutcrimes, this really fades into insignificance. So, at first glance you think this is reallyimpressive and then through personal experience perhaps and, like I say, hearingthings locally, you do question what you see.In spite of this important rider, the booklet does appear to have proven tobe a successful medium that was able to overcome the challenges presentedby the broadcast and print media:495r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500Well, I read it, I suppose, so it must be reasonably useful, because I did actually readit, whereas if it was on the news, I might not have paid attention to it quite so much.It would have been in one ear and out the other. So I think this is quite good,actually, this.In addition to lessons relating to the design of the booklet, the groupinterviews were also useful in learning about the perceived strengths andweaknesses of the different delivery mechanisms. Ironically, because thepostal method is a popular and relatively inexpensive way for organisationsto communicate with large numbers of people, there is a serious risk ofmaterial being consigned to the oblivion of junk mail:Somebody putting something through my letterbox, to me it would probably be like apizza pamphlet, it would just be coming straight in and straight into the recycling bin.Two complementary lessons emerged from the group discussions relatingto the value of using personalised envelopes – as distinct from addressing‘The Occupier’ – and the importance of investing in high-quality paper andpresentation:It was a good thing it was in an envelope because I’m afraid it would have gone straightinto the bin. So it was a good thing it was in that envelope and not just shoved throughmy letterbox because I doubt we would have picked it up and looked at it.There is no question however that personal contact, whether handing outor explaining the booklet, carried the added value of creating a sense ofinterest or obligation:Can I just say that I was impressed with it being delivered and that made me read it.I get so much junk mail that I don’t even look at that comes through the letterboxand that goes in the bin. Because somebody actually knocked on the door andhanded it to me, I thought, well, I better read it.This view was further supported by the responses to the survey. A fifth(20%) of those receiving the booklet said that their confidence hadincreased after receiving it. However, the proportion of positive responseswas found to relate to the delivery mechanism with only 15% of the ‘posted’group saying their confidence had increased in contrast to 21% for the‘handed’ and 22% for the ‘explained’ groups. When these responses weredivided into their respective delivery groups and whether after receivingthe booklet confidence had increased, remained the same or decreased,the association between personal contact – as distinct from postal contact –and an improved positive response was found to be statistically significant. 8Thus whereas the number of instances where confidence was reported tohave risen was less than expected for the ‘posted’ group, the reverse wastrue for the other two groups.Suggested Improvements to the Booklet and Delivery MethodsBased on the survey results and feedback elicited from the follow-upinterviews with groups of respondents, the booklet and various deliverymethods were very effective in improving understanding of, and attitudes496r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500toward, the CJS. This is not to say that further enhancements could not orshould not be made. Having employed marketing theory in the design ofthe booklet, a ‘Test Marketing’ approach was also used to check perceivedstrengths and weaknesses of the material and methods used (Evans andBerman 1990).The BookletDuring the post-intervention stage of the survey, people receiving thebooklet were invited to suggest improvements to it. Of the 1,545 whorecalled receiving it, just under two-thirds thought it was fine/good/verygood (22%) or could not think of anything/said they had not read it yet(43%). Of the remaining third of respondents who did suggest improvements(n 5 535), 38% of them said that it needed more detail orinformation and 27%, in line with mistrust of official statistics, said it couldbe improved by addressing the perception of it being a distorted view,through being more open and honest.In the focus groups one of the main areas of improvement frequentlymentioned echoed the findings from the survey and the scepticism aboutofficial statistics and the problem of ‘spin doctoring’:They are certainly true figures. This is the beauty about it. They’re totally truefigures but it’s how they present it. You can do that with figures. That’s the beautyabout it. They all say statistics, statistics and lies. Statistics you can put any way youlike. You can say 95% this and everything else. Well, okay, there’s 5% still left. Youcan highlight that 5% and make it sound bad. We’ve reduced burglaries by 59%.41% of burglaries were domestic. You can put a different spin on it.To address this problem it was suggested that telling the CJS story innumbers as well as, or instead of, percentages would prevent figures frombeing used to create a spurious impression.Similarly, and in line with the suggestion from a number of surveyrespondents, providing a more detailed picture of the CJS process – asdistinct from discrete events within it – was seen as a useful way of makingthe story behind the statistics more transparent and credible:I’d like to know the full range as well, I’d like to know just how many arrests andcharges the police have made, how many of those have gone forward forprosecution to the CPS, how many of those prosecutions were successful. Let’sstart getting some real statistics on the whole of the process.With regard to addressing scepticism, it was suggested that instead ofpresenting information that only contained positive statistics, it would bemore plausible to include some negative figures so as to make it ‘More abalanced view’.Lastly, another type of information suggested which was supported by themajority of interviewees related to providing cameos that, so to speak, putsome flesh on the bones of the statistics and met the human-interest angle.Alternative MediaIn looking for a way to provide the general public with informative andpersuasive information about the CJS, the use of a booklet is an obvious497r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd

The Howard Journal Vol 48 No 5. December 2009ISSN 0265-5527, pp. 485–500ReferencesAllen, J., Edmonds, S., Patterson, A. and Smith, D. (2006) Policing and the CriminalJustice System – Public Confidence and Perceptions: Findings from the 2004/05 BritishCrime Survey (Home Office Online Report 07/06), London: Home Office. Availableat: http://www.homeoffice.gov.uk/rds/pdfs06/rdsolr0706.pdf (accessed 25 June 2009).Chapman, B., Mirrlees-Black, C. and Brawn, C. (2002) Improving Public Attitudes to theCriminal Justice System: The Impact of Information (Home Office Research Study No.245), London: Home Office.Chartered Institute of Marketing (2004) How to Achieve an Effective Promotional Mix (10Minute Guide), Maidenhead: Chartered Institute of Marketing. Available at: http://www.cim.co.uk/resources/commschannels/home.aspx (accessed 25 June 2009).Dalgleish, D. and Myhill, A. (2004) Reassuring the Public: A Review of InternationalPolicing Interventions (Home Office Research Findings 241), London: Home Office.Available at: http://www.homeoffice.gov.uk/rds/pdfs04/r241.pdf (accessed 25 June 2009).Evans, J. and Berman, B. (1990) Marketing, New York: Macmillan.Goddard, E. (2005) Public Confidence in Official Statistics, London: Office for NationalStatistics.Hough, M. (2008) ‘Foreword’, in: B. Duffy, R. Wake, T. Burrows and P. Bremner,Closing the Gap: Crime and Public Perceptions, London: Ipsos MORI.Kershaw, C., Nicholas, S. and Walker, A. (2008) ‘Crime in England and Wales 2007/08’, Home Office Statistical Bulletin 07/08, London: Home Office.Lee, Y.J., Ellenberg, J.H., Hirtz, D.G. and Nelson, K.B. (1991) ‘Analysis of clinical trials bytreatment actually received: is it really an option?’, Statistics in Medicine, 10, 1595–605.Mattinson, J. and Mirrlees-Black, C. (2000) Attitudes to Crime and Criminal Justice: Findingsfrom the 1998 BCS (Home Office Research Study No. 200), London: Home Office.Ministry of Justice (2008) ‘Criminal statistics: England and Wales 2007’, StatisticalBulletin, November 2008, London: Ministry of Justice.Nicholas, S., Povey, D., Walker, A. and Kershaw, C. (2005) ‘Crime in England andWales 2004/2005’, Home Office Statistical Bulletin 11/05, London: Home Office.Page, B., Wake, R. and Ames, A. (2004) Public Confidence in the Criminal Justice System(Research Findings 221), London: Home Office.Parrott, R. (1995) ‘Motivation to attend to health messages’, in: E. Mailbach andR. Parrott (Eds.), Designing Health Messages, London: Sage.Ringham, L. and Salisbury, H. (2004) Support for Victims of Crime: Findings from the 2002/3 British Crime Survey, London: TSO.Salisbury, H. (2004) Public Attitudes to the Criminal Justice System: The Impact of ProvidingInformation to British Crime Survey Respondents (Home Office Online Report 64/04),London: Home Office. Available at: http://www.homeoffice.gov.uk/rds/pdfs04/rdsolr6404.pdf (accessed 25 June 2009).Singer, L. and Cooper, S. (2008) Inform, Persuade and Remind: An Evaluation of a Projectto Improve Public Confidence in the Criminal Justice System (Ministry of Justice ResearchSeries 15/08), London: Ministry of Justice. Available at: http://www.justice.gov.uk/publications/docs/inform-persuade-remind.pdf (accessed 25 June 2009).Tuffin, R., Morris, J. and Poole, A. (2006) An Evaluation of the Impact of the NationalReassurance Policing Programme (Home Office Research Study No. 296), London:Home Office.500r 2009 Crown copyrightJournal compilation r 2009 The Howard League and Blackwell Publishing Ltd