IELTS Research Reports

An empirical investigation of the process of writing Academic Readingtest items for the International English Language Testing SystemFirstly consideration should be given to better informing teachers about the test by increasing theamount of guidance offered concerning the reading skills being targeted and the intentions behindusing the variety of item types on the test. The information currently offered on the IELTS websitedoes not seem to be sufficient to inform teachers about the construct of the test. The non-experiencedwriters felt that, based on the examples they had seen, they had a clear sense of the kinds of texts beingused in IELTS, but were less clear on why such texts were being used. This impression was largelyborne out by the texts they produced, which resembled the texts produced by the experienced groupand those analysed by Weir et al. (2009a). However, the untrained writers did not feel well equippedto produce items that would test the same skills as the IELTS academic reading test. Although allwere familiar with the academic reading test, they did not feel well informed about the functionof the different items types or the implications of these for the types of reading being tested. Moreinformation on these aspects of the test could be communicated through the IELTS handbook, websiteand other communication channels.From the testimony of both groups, there seems little doubt that observing the processes of editingand refinement that we report in this study is of great help to item writers in developing their ownskills. Indeed, we would suggest that this research could be of use in training new item writers byproviding insights into how texts and items are reshaped for the test and might also serve to inform awider audience about the extent of the work that goes into producing each IELTS item. However, therewould seem to be a need for an additional strand of training and guidance that pays greater attentionto the construct of academic reading intended to be operationalised through the IELTS academicreading module.In an earlier study (Weir et al 2009a) we recommended that objective text analysis tools could playa valuable role in assisting the evaluation of texts (and perhaps items) as part of the review process.We repeat this recommendation here because, as exemplified in our analyses, such tools can help inthe identification of infrequent or technical vocabulary, help to highlight inconsistencies between thetexts used across versions of the test and assist in identifying differences in textual genre that might behelpful in better defining the requirements for texts suited to the purposes of the test.The participant interview and focus groups raised a number of questions that should be addressed inthe item writer guidelines or related training packages:■■■■■■■■What are the reading skills that the test as a whole is intended to address? And in whatproportion?Why these reading skills? And how do they relate to the available task types?Within each task, what kinds of linguistic relationships should T/F/NG (and other types ofitems) have to the text and in what proportion? What are the implications of these for thereading skills being targeted?What range of skills should be addressed in each section of the test and what item typesshould be used to target them?The experienced item writers speculated on a number of issues including the kinds of informationthat could legitimately be targeted by MCQ and the extent to which inference could legitimately betargeted: there appeared to be room for individual interpretation in these areas. It is perhaps inevitablethat individual writers (and separate teams of writers) will interpret specifications differently (and thatthere will be some misinterpretation), but questions of this kind should be answerable through the itemwriter guidelines. To the extent that they are, there should be greater attention to the guidelines duringediting. To the extent they are not, the guidelines should be updated to address them.IELTS Research Reports Volume 11325