IELTS Research Reports

Anthony Green and Roger HawkeyTwo groups of item writers were involved in these sessions. One group consisted of four trainedIELTS item writers. This required the cooperation of Cambridge ESOL in facilitating contact withitem writers able to participate in the research, permitting their involvement and in providing theresearchers with access to the item writer guidelines for the academic reading paper. As the guidelinesare confidential we were asked not to discuss them in detail or to quote from them in this report.The second group included three teachers of English for academic purposes with a range of experienceof the IELTS test and of IELTS preparation but no previous experience of writing reading test itemsfor an examinations board. These teachers were familiar with the appearance of the test, but not withits underlying design.Data collection took place over two sessions. On the basis of Salisbury’s (2005) division of theitem writing process into exploratory, concerted and refining phases, the first session concentratedretrospectively on the exploratory phase and prospectively and concurrently on the concerted phase(see above). In the second session the item writers worked as a group to further refine their texts anditems to make them more suitable for the test (as the trained item writers would normally do in anactual test editing meeting). In Salisbury’s terms, this session may be said to have been concernedretrospectively with the concerted phase and prospectively and concurrently with the refining phase.In preparation for Phase 2, each participating item writer was sent a commissioning letter (AppendixA), based on a model provided by Cambridge ESOL, inviting them to choose a text that wouldbe suitable for use in IELTS, to edit this text as appropriate and to write 16 or 17 test questions toaccompany the text.In the first session of Phase 2, we sought insights into the strategies that item writers use in selectingand preparing texts and the role that the test specifications, experience and other sources of knowledgemight play in this process for experienced and inexperienced writers. Writers were interviewed abouttheir selection of texts for item writing purposes. Key questions for this session included how itemwriters select texts, how they adapt the texts to shape them for the purposes of the test and how theygenerate items. The focus was on the specific text selected by the item writer for this exercise, thefeatures that made it attractive for the purpose of writing IELTS items and the edits that might havebeen required to shape the text to meet the requirements of the test.The second session of Phase 2 was similar to an IELTS editing meeting (see above). Item writersbrought their texts and items to the focus group to discuss whether these did, as intended, meet therequirements of the test. Again, observation of differences between the experienced and inexperiencedwriters was intended to provide insights into the practices of those item writers working within theIELTS system for test production. Here the researchers sought to understand the kinds of issuesthat item writers attend to in texts prepared by others, the changes that they suggest and features oftexts and test questions that are given approval or attract criticism. Once again, the analyses of thedeliberations linked themes and categories emerging from the recordings and transcripts to the insightsprovided by the socio-cognitive framework Weir 2005, Khalifa and Weir 2009, Weir et al 2009a). Itwas expected that differences between the experienced and non-experienced groups would highlightthe practices of item writers working within the IELTS system for test production and the nature oftheir expertise. As will be seen below, the study provides insights into how item writers prepare textsand items, and their focus of attention in texts prepared by others; also into the features of texts andtest questions that attract approval or criticism in editing.278 www.ielts.org