IELTS Research Reports

An empirical investigation of the process of writing Academic Readingtest items for the International English Language Testing System6.1.2 Change of view caused by the editing process?Victoria still liked her Dreams text but was now less confident about her tasks. She considered itnecessary to do far more analysis of potential texts and tasks. The three in the group still did not knowthe optimum processes but were, rather, acting on the basis of common sense and their experience asteachers. Mathilda felt the need for a whole range of IELTS tests and tasks to analyse to increase herawareness of suitable texts, tasks and what they are supposed be testing. Mary agreed, not having beentrained as an IELTS item writer, it was difficult to know which words you can use in a text, how muchyou can test inferences.Victoria would like to know about technical testing matters such as evenness of distractor and responselengths, Mathilda wanted more official information on IELTS to know more about IELTS level ofdifficulty, mark allocation and analysis. All three participants felt that the ‘rules’ of IELTS are ‘prettywell hidden’. Their own help to their IELTS students in how to deal with IELTS reading test items wascommon sense rather than officially informed.Victoria, who was aware of IELTS writing paper pre-testing and other validation procedures,wondered whether the reading paper was subject to similar procedures. As will be apparent fromour review above, IELTS does publish information on these issues on its website and through othersources such as Cambridge Research Notes. The response of this group therefore may indicate lack ofawareness rather than lack of availability.In response to the final question, what the three participants felt they had learnt from their day:■■■■■■■■■■■■■■■■Victoria, assuming that reading for an IELTS reading test was different from other readingand in spite of having looked at information on task types on the IELTS website, stillwished to know how test-takers should read in the test.Mary, on the issue of what we are meant to be testing and how do we test it, wonderedwhen is it appropriate to be testing vocabulary and when is it not?Mathilda, wished to know, relatedly, how questions towards the broader and narrowerunderstanding of a text should be balanced.Learning from the activities of the day, the participants noted the following things thatthey would do differently in future:Mary would, in advance, chart her intended items in terms of their intended difficulty andscope, to ensure evenness of coverage.Mathilda would like to challenge herself using a fictional, more literary text (for examplea short story) for her item writing; she still wondered why IELTS reading seemed not touse such texts; her impression was that the range of topics covered by IELTS academicreading modules was somewhat narrow.Victoria would do a close analysis of more IELTS reading papers before she began settingher own; she was seeking a match between text type and task type; this could mean achange of direction from choosing text and topic first; as an item writer, she might preferto feel more responsible for the kinds of task she was going to set.Victoria did not feel that the experience of the day had clarified why test-takers oftenfound the IELTS reading module more difficult than the other modules (although, asnoted above, reading scores are generally higher than for writing and speaking). Perhapsit was less clear with the reading module than with the others what test-takers weresupposed to be doing.IELTS Research Reports Volume 11311