IELTS Research Reports

Anthony Green and Roger HawkeyThe test providers should keep item writers informed about relevant assessment issues includingcurrent theoretical perspectives on the reading process, the nature of the reading demands made onbeginning university students and the implications of these for IELTS. Such meetings, by raisingissues of concern to writers, could also serve to direct further research into these questions that willinform the design of the test.Elizabeth made reference to the discontinued practice of asking item writers to identify the skills beingtested by each of their items. Elizabeth had found this difficult, but useful and consideration might begiven to re-introducing such a practice as a training exercise if not as a routine requirement. It mightalso be advisable to introduce clearer controls on the range of task types and the range of skills to betargeted for each text.Item writers reported that from their perspective some decisions made about test content couldappear inconsistent. The fairness review element of the pre-editing and editing process was one areaof concern. Items based on factual details in a text might inevitably advantage candidates who arefamiliar with the subject matter, but the question of which facts should be considered to be widelyknown and which not was a grey area for our participants. Similarly, these writers, who all workedon other Cambridge ESOL papers as well as IELTS, felt that there might be inconsistencies in thedefinition of potentially ‘offensive’ or ‘sensitive’ material across examinations. It may be that thereis a rationale for such differences based in the nature of the candidatures for these different tests, butthe implications for test content were not sufficiently clear to the item writing team. If this view isshared more generally by item writers, mechanisms should be found to create greater consistency inthe interpretation of the rules, or to better articulate to item writers justified differences across testingprogrammes within Cambridge ESOL.Finally, we believe that this study points towards a number of interesting avenues for future research.A comparison between the item writer practices investigated here and test taker strategies of the kindinvestigated by Weir et al (2009b) would provide insights into the extent to which candidate readingbehaviours conform to item writer expectations. Similarly, it would be interesting to obtain candidateviews on the kinds of editing changes made by item writers or to compare candidate judgements ofwhat constitutes ‘key’ information in a text with item writer judgements. It would be useful, as a formof evaluation, to carry out a follow-up study after changes to item writer training and guidance havebeen implemented.326 www.ielts.org