Assessing and reporting performances on pre-sessional EAP courses

More documents

Recommendations

Info

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 51 for institutional purposes (e.g., the Institutional TOEFL). Some institutions, in addition to dem<strong>and</strong>ing a minimum level of performance on an external test, require students to take an inhouse test. The results of this test are used to determine whether the students need further language support before they begin their academic programme or, in some institutions, while they follow their programme. In other institutions (particularly those in the UK), students who meet all the other requirements for admission but whose performance on the external test falls short of that required for unconditional admission can be admitted provided that they successfully complete a pre-sessional EAP course. There is no common approach to assessing <strong>and</strong> <strong>reporting</strong> performance on pre-sessional EAP courses. In the case of institutions that will only accept the results of external tests, students are required to re-take the tests at the end of their pre-sessional EAP courses. One recently reported example is that of Australian universities where pre-sessional EAP courses serve not only to prepare students for academic study but also for the IELTS test (Moore & Morton, 2005). At least one institution uses the results of an external test that has been piloted on its students; others design their own tests, which may or may not be modelled on tests created elsewhere for other purposes. Some institutions prefer to judge students on their in-course performance, combining internal test scores, performance on written assignments (projects <strong>and</strong> course assignments, sometimes collected together in portfolios), formal presentations, <strong>and</strong> classroom participation. The weighting given to these elements varies, but it is common to give more weight to tasks that are completed near the end of the course. Most institutions are required to send individual reports to the central admissions office or the receiving department, either with a test result (from one or more of the tests described above), a grade determined by a combination of test results <strong>and</strong> other assessments, or a pass-fail judgement which is based either on tutors’ impressions or on a more straightforward criterion such as attendance. An estimated IELTS score is required in some institutions. A recent development is the use of ‘can-do’ scales, which consist of lists of performance objectives <strong>and</strong> columns where EAP tutors indicate whether students are able or not to achieve each objective. The history of assessment on Lancaster University’s pre-sessional EAP courses reveals that we have adopted a number of these approaches at different times. In the early 1990s, we were asked to provide estimated IELTS scores. The admissions officers found these estimates useful as they could interpret them in the same way they had processed genuine test results when making their initial decisions. We, however, became increasingly concerned about the lack of validity <strong>and</strong> reliability in our judgements because: (i) We did not have access to the official rating scales for speaking <strong>and</strong> writing. (ii) Estimating reading <strong>and</strong> listening abilities was problematic, since this depended on inferences about students’ ‘inner processes’ rather than analyses of products such as essays or oral presentations, <strong>and</strong> (iii) None of the teaching team had been trained as IELTS examiners. We ab<strong>and</strong>oned this practice in favour of writing a profile report for each student, commenting on their general ability in listening, oral presentations, group discussions, reading <strong>and</strong> writing. The report was written by the student’s main tutor, who had at least 8 h of contact with the student every week <strong>and</strong> also had access to feedback sheets <strong>and</strong> comments from the student’s other tutors. The main tutor would also give an overall judgement of the student’s ability. The report was discussed with the student before being passed on to the course co-ordinators. The course co-ordinators reviewed <strong>and</strong> edited it to st<strong>and</strong>ardise the style <strong>and</strong> to ensure that it was
52 J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 a summary of the student’s progress <strong>and</strong> ability rather than a collection of anecdotes. Copies were sent to the University’s admissions officers, the departments <strong>and</strong> to the students themselves. An example of a completed profile report can be found in Appendix A. 1.1. Problems with the <strong>reporting</strong> system This system continued until 2001 without too many objections or queries, but there were a number of problems: (i) The reports went to two different audiences: the administrative audience, consisting of the central university admissions officers <strong>and</strong> the students’ departmental admissions officers, <strong>and</strong> the students themselves. It was difficult to write in a way which satisfied the needs of two such distinct audiences—in particular, to say what needed to be said without discouraging the students. (ii) Tutors had varying interpretations of what constituted readiness for entry into a university department <strong>and</strong> there were few opportunities in either pre-course or in-course staff meetings to discuss definitions of the key criteria. Table 1 contains a compilation of the types of features that tutors mentioned in the “Writing” section of a sample of 50 final reports produced in 2001. The left-h<strong>and</strong> column contains comments which relate to some notion of writing proficiency. It is interesting to note that features which often figure in textbooks on writing, such as cohesiveness, coherence, <strong>and</strong> checking drafts, occur rarely. The comments about linguistic performance (under “Language”) are general rather than detailed. It is a matter of debate whether the comments in the right-h<strong>and</strong> column (which relate to other facets of the students’ performance) reveal anything substantial about the students’ ability to express them in formal writing <strong>and</strong> to conform to st<strong>and</strong>ard academic conventions. (iii) The brevity of the courses, the need to give fair coverage to all skill areas, <strong>and</strong> a course approach which emphasised out-of-class preparation <strong>and</strong> in-class group discussions meant that there were many aspects of performance that were either not visible in the classroom or that manifested themselves when students were talking with one another rather than with the tutor. This affected the depth of knowledge any tutor had of a student’s ability. (iv) We have already explained that all the course reports were edited by the course coordinators in order to st<strong>and</strong>ardise the style <strong>and</strong> to lift the level of commentary from the more specific to the more general. It was inevitable that those doing the editing changed the meaning of some of the reports, either on purpose (toning them down) or inadvertently, through a lack of underst<strong>and</strong>ing of the original text. As a result of changes that made the language more diplomatic, admissions officers sometimes had problems with interpretation, which made their job harder rather than easier, thus undermining one of the purposes of the exercise. It is clear from the survey of other institutions that Lancaster University’s practices before 2001 were not unusual. However, we remained concerned about the validity <strong>and</strong> reliability of our exit assessments. We wanted to ensure that our practice was well grounded in theory, that it was transparent to users (our students <strong>and</strong> admissions personnel), <strong>and</strong> that it was practical in terms of tutors’ skills <strong>and</strong> time.
Page 1: Assessing
Page 5 and 6: 54 J. Banerjee, D. Wall / Journal o
Page 17 and 18: 66 Can take satisfactory notes Vari
Page 19 and 20: 68 Instrument When used Function De

Assessing and reporting performances on pre-sessional EAP courses

Create successful ePaper yourself

Delete template?

Save as template?