IELTS Research Reports

Anthony Green and Roger HawkeyThe points made by the three participants in the focus group discussion certainly served astriangulation for the views they had expressed in the preceding IELTS text search and treatment anditem development: flowcharts and discussions already reported. Once again we see strong evidenceof time-consuming searching for suitable texts but uncertainty of the target level(s) of such texts and,to some extent, the topic range; major problems with the design of tasks, in particular multiple choice(MCQ) items and, as might be expected of this non-experienced item writer group, frustration causedby lack of item writing guidance.The research team pursued with the participants certain emerging issues immediately after the endof the participant-led semantic differential discussion, in particular the issue of `the level of Englishlanguage proficiency associated with IELTS’ about which the three participants admitted to beinguncertain. Mathilda had learnt from her own experience as an IELTS test-taker but still felt thatthe IELTS website and other guidance on proficiency levels was ‘vague’. Victoria felt that she hadhad to develop her own proficiency level criteria while selecting her text and making items. Shenoted how the text ‘comprehensibility factor’ seemed to dominate her decisions on text and itemdifficulty. Mathilda felt that her text would not be ‘that easy’ for candidates whose English ‘was notso developed’ as her own. Participants were aware that an IELTS Band of 6 or 6.5 was conventionallyseen as a cut-off point for students entering BA courses. Mary and Victoria were also informed bythe levels of their own IELTS students (IELTS bands 5.0 - 7.5, and 8.0 respectively), which, for Marymeant that her test might not discriminate effectively at the higher end as she felt that she might nothave enough experience of the highest scoring candidates to be able to target items at this group.The discussion was now focusing on the actual reading construct espoused by IELTS. Victoria andMary had heard that EL1 users had difficulty with the IELTS academic reading module, and that testperformance on this module tended anyway to be weaker than on the other IELTS modules, evenfor stronger candidates. This is a common perception of IELTS (see Hawkey 2006), although testresults published on the IELTS website show that overall mean scores for reading are higher than forthe writing and speaking papers. Mathilda wondered whether the IELTS academic reading modulewas perhaps testing concentration rather than ‘reading proficiency’. Victoria recalled that IELTS wasdescribed as testing skimming and scanning, but thought that skimming and scanning would alsoinvolve careful reading once the information necessary for the response had been located. But Marywas sure that reading and trying to understand every word in an IELTS text would mean not finishingthe test. Mary felt that a candidate could not go into an IELTS exam ‘not having been taught how totake an IELTS exam’ and that a test-taker might not do well on the test just as a ‘good reader’. Maryalso claimed that she had never, even as a university student, read anything else as she reads an IELTSreading text. When reading a chapter in a book at university, one generally wants one thing, which oneskims to locate, then ‘goes off’ to do the required reading-related task (although, conversely, Mathildaclaimed often to ‘read the whole thing’).The participants were then asked what other activities the IELTS text selection, editing and itemwriting processes reminded them of. Victoria recalled her experience working for a publisher andediting other people’s reading comprehension passages for the Certificate of Proficiency in English(CPE) examination, which included literary texts (see Appendix B).Mary had worked on online language courses, where editing other people’s work had helped herthinking about the question-setting process (as well as surprising her with how inadequate somepeople’s item-writing could be). The experience had reminded Mary how much easier it was to writegrammatical rather than skills-based items. Victoria agreed, based on her own (admittedly rather286 www.ielts.org