IELTS Research Reports

Anthony Green and Roger HawkeyFollowing editing, material either passes into the IELTS test bank for inclusion in pre-tests to betrialled with groups of test takers, or is returned to the item writer for further revision and anotherround of editing. Pretests are administered to groups of students at selected IELTS centres and datais obtained indicating the measurement characteristics of the test items. A further meeting - thepre-test review meeting - is held to consider the item statistics and feedback from candidates andtheir teachers. Texts are submitted for pretesting with more questions than will appear in the finalversion and those items that fall outside target difficulty ranges or that have weak discrimination areeliminated. Again at this point any unsatisfactory material may be rejected.All IELTS item writers are said to receive extensive training. Ingham (2008) describes the standardprocesses of recruitment and training offered to item writers. This takes place within ‘a framework forthe training and development of the externals with whom [Cambridge ESOL] works in partnership.The framework has the acronym RITCME: Recruitment; Induction; Training; Co-ordination;Monitoring and Evaluation’. To be recruited as item writers, individuals must have a university degree,a suitable qualification in English language teaching and five years’ teaching experience together withsome familiarity with materials production and involvement in preparing students for CambridgeESOL examinations (Ingham 2008). After completing a screening exercise and preparatory tasks(induction), successful applicants are invited to complete a ‘training weekend’ (Ingham, 2008; 5) withCambridge staff and external consultants. The Cambridge item writer trainers work with betweentwelve and sixteen trainees, introducing them, inter alia, to item writing techniques, issues specific tothe testing of different skills and the technical vocabulary used in the Cambridge ESOL context.After joining the item writing team for a specific paper such as the IELTS academic reading paper,writers ‘receive team-specific training before they start to write’ (Ingham 2008, 6). They are invitedto further training sessions with their team, led by the chair, on an annual basis. In time, successfulitem writers gain work on additional products to those for which they were originally recruited andmay progress in the hierarchy to become chairs themselves. Less successful writers who fail togenerate sufficient acceptable material are offered support, but according to Salisbury (2005, 75) may‘gradually lose commissions and eventually drop from the commissioning register’.Salisbury (2005) points out that the role of the item writer appears, superficially, to be limited todelivering material in line with predetermined requirements. However, it is also widely recognised thatformal written specifications can never be fully comprehensive and are always open to interpretation(Clapham 1996a, Fulcher and Davidson 2007). Perhaps inevitably, what Salisbury (2005) describesas ‘non-formalised specifications’, representing the values and experience of the item writing teamand subject officers, emerge to complement the formal set provided by the test developers. Thesenon-formal specifications are less explicit, but more dynamic and open to change than the item writerguidelines. We have already noted that in the Cambridge ESOL model, elements of these non-formalspecifications can become formalised as regular feedback from item writers informs revisions to theguidelines. Item writers are therefore central to the IELTS reading construct.Khalifa and Weir (2009) point to the critical importance of professional cultures or communities ofpractice (Lave and Wenger, 1991) within a testing body such as Cambridge ESOL. They suggest thatquestion paper production perhaps depends as much on the shared expertise and values of the itemproduction team as on the procedures set out in item writer guidelines. All members of this team,whether they be internal Cambridge ESOL staff or external consultants, bring their own expertise andexperience to the process and shape its outcomes at the same time as their own practices are shaped bythe norms of the established community that they are joining.276 www.ielts.org