IELTS Research Reports

The cognitive processes of taking IELTS Academic Writing Task 1SCRIPT ID RATER 1 RATER 2 RATER 31 x x2 x x3 x x4 x x5 x x6 x x7 x xTable 3. Double blind marking schemeAt the moderation meetings, we discussed the scripts in question and marked them together. Themoderation exercise was an essential step to improve rater reliability. The average inter-rater reliabilityCronbach’s Alpha was improved (above 0.83).Marking the AWT2 scripts was much more straightforward and achieved high inter-rater consistencybefore moderation: Cronbach’s Alpha = 0.843 between Rater 1 and Rater 2, 0.881 between Rater 1 andRater 3, 0.793 between Rater 2 and Rater 3. Only two scripts had difference larger than 1 and weremarked by a third rater. The participants’ performances in these two writing tasks are reported.N Minimum Maximum Mean Std. DeviationAWT1 no think-aloud 24 4.50 7.00 5.69 0.673AWT2 no think-aloud 24 4.50 7.50 6.32 0.764Table 4. Performance in AWT1 and AWT2 without think-aloudAs shown in Table 4, the mean of the participants’ performance in AWT1 and AWT2 undernormal examination conditions without think-aloud was 5.69 (std. deviation=0.67) and 6.32 (std.deviation=0.76) respectively. Below we report the participants’ performances in the eight tasks underthink-aloud conditions.4.1.3 Participants’ writing performances under think-aloud conditionsThe writings of the eight tasks under think-aloud conditions were randomly assigned to and doubleblind marked by the research team, using the same procedure as for marking the first AWT1 task undernormal examination conditions (see 4.1.1). Overall, the inter-rater reliability Cronbah’s Alphas weresatisfactory: 0.69 for A-instruction scripts, 0.86 for A-layout scripts, 0.85 for A-oil scripts, 0.90 forA-UKCO2 scripts, 0.78 for B-broadcast, 0.74 for B-EUfund, 0.71 for B-map, and 0.79 for B-viewing.The participants’ performances in the eight tasks are reported in Table 5 below.IELTS Research Reports Volume 11387