IELTS Research Reports

Anthony Green and Roger Hawkeyapproach to the editing meeting and worked intensively on improving the texts and items. Each writercontributed numerous suggestions and the chair sought consensus on the proposed changes.The experienced group were pleased with the guidance they had received from the item writerguidelines and from the experience of training and editing meetings and felt that this had contributedto their expertise. Nonetheless there were clear inconsistencies in the interpretation of taskrequirements between the experienced writers. The group seemed to share a conception that IELTStasks should target key, salient facts or opinions expressed in a text and appeared less concerned withthe reading skills involved.The group had discussed at some length the nature of the information that could be targeted using Type1 MCQ items and the extent to which inferences might be tested using Type 8 T/F/NG items. Thesediscussions left open the possibility that different writers might be targeting different reading skillswhen using the same item type - as observed in Section 8, each set of T/F/NG items bore a somewhatdifferent relationship to its partner text. This has implications for the comparability of different formsof the test as it makes it more challenging to ensure that every form reflects the required range ofreading skills. These issues had not been resolved by the end of the session.When reviewing and revising items, the writers identified ambiguities and suggested clarifications, butdid not generally discuss the implications of changes of wording on the nature of the reading skills thatmight be required in arriving at a correct response or to the balance of skills being tested in a passage.The three task types in Anne’s submission, for example, all appear to involve careful local reading. Theitems include eight Type 8 T/F/NG items, which involve paraphrase of information in the text, and twoType 7 Locating Information items which are also based on recognising paraphrases of information inthe text - in this case distinguishing between the two sentences that paraphrase the information in thetext (similar to True items) and the three that do not (similar to False and Not Given items). The itembelow illustrates how similar this is to a T/F/NG item. There are similar lexical relationships involvingrepetition (speech), synonymy (develop: evolve) and co-reference (early man: our ancestors).In the item:In the text:Human speech began to develop when early man ceased walking on four legs.When our ancestors stood up on two feet, the chest was freed of these mechanicaldemands making it possible for speech to evolve.The third item set - Type 4 Summary Completion - involves selecting two-word expressions fromthe text to complete a summary of paragraphs 3, 4 and 5 - also seems to require understanding at alocal level.8 CONCLUSIONS AND RECOMMENDATIONSThe researchers were favourably impressed by the conscientiousness and professionalism of theIELTS item writers that we interviewed and observed and the quality of the texts and items that theyproduced. Nonetheless, we would suggest that there are a number of recommendations that could bemade on the basis of our study to refine the IELTS academic reading item production process. Theinter- and intra- group differences revealed by our research have implications for test preparation thatcould be addressed through information provided to teachers of IELTS candidates and implications forthe consistency of test material that could be addressed through the guidelines and training given toitem writers and the process of text and test review.324 www.ielts.org