Assessing and reporting performances on pre-sessional EAP courses

<strong>Assessing</strong> <strong>and</strong> <strong>reporting</strong> <strong>performances</strong> on pre-sessional
EAP courses: Developing a final assessment checklist
<strong>and</strong> investigating its validity
Abstract
Journal of English for Academic Purposes
5 (2006) 50–69
Jayanti Banerjee *, Dianne Wall
Department of Linguistics <strong>and</strong> English Language, Lancaster University, Lancaster LA1 4YT, UK
As the number of non-native speakers of English studying at universities in the United Kingdom (UK)
has grown, so has the need to provide pre-sessional English for Academic Purposes (EAP) courses. One of
the challenges facing the organisers of such courses is deciding how to determine whether students have
completed them successfully <strong>and</strong> whether they are ready to enter their academic departments. This paper
reviews research into assessment on pre-sessional <strong>and</strong> other EAP support courses, <strong>and</strong> summarises the
results of a survey of practice in a number of universities in the UK <strong>and</strong> other countries. It describes how
assessment on pre-sessional courses has evolved at one British institution (Lancaster University), <strong>and</strong>
outlines some of the problems that have led to changes in recent years. Explanations will be given of the
methods used to pilot <strong>and</strong> refine a new assessment procedure <strong>and</strong> of the attempts that have been made to
investigate its validity.
q 2005 Elsevier Ltd. All rights reserved.
Keywords: English for academic purposes; Assessment; Validation; Pre-sessional EAP courses
1. <strong>Assessing</strong> <strong>and</strong> <strong>reporting</strong> performance on pre-sessional EAP courses
www.elsevier.com/locate/jeap
All English-medium universities stipulate a minimum level of English language proficiency
for admission to their courses, but they set down different routes by non-native speakers of
English can satisfy their requirements. We surveyed institutions in the UK <strong>and</strong> other countries in
order to identify how different universities decided whether students were proficient enough to
begin studying in their academic departments. The survey revealed that some institutions will
only accept the results of external tests (e.g., IELTS, TOEFL, the test of English for International
Communication—TOEIC) or tests which have been designed by external agencies <strong>and</strong> released
* Corresponding author. Tel.: C44 1524 593039/C44 1524 840352; fax: C44 1524 843085.
E-mail address: j.banerjee@lancaster.ac.uk (J. Banerjee).
1475-1585/$ - see front matter q 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.jeap.2005.11.003

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 51
for institutional purposes (e.g., the Institutional TOEFL). Some institutions, in addition to
dem<strong>and</strong>ing a minimum level of performance on an external test, require students to take an inhouse
test. The results of this test are used to determine whether the students need further
language support before they begin their academic programme or, in some institutions, while
they follow their programme. In other institutions (particularly those in the UK), students who
meet all the other requirements for admission but whose performance on the external test falls
short of that required for unconditional admission can be admitted provided that they
successfully complete a pre-sessional EAP course.
There is no common approach to assessing <strong>and</strong> <strong>reporting</strong> performance on pre-sessional EAP
courses. In the case of institutions that will only accept the results of external tests, students are
required to re-take the tests at the end of their pre-sessional EAP courses. One recently reported
example is that of Australian universities where pre-sessional EAP courses serve not only to
prepare students for academic study but also for the IELTS test (Moore & Morton, 2005). At
least one institution uses the results of an external test that has been piloted on its students; others
design their own tests, which may or may not be modelled on tests created elsewhere for other
purposes. Some institutions prefer to judge students on their in-course performance, combining
internal test scores, performance on written assignments (projects <strong>and</strong> course assignments,
sometimes collected together in portfolios), formal presentations, <strong>and</strong> classroom participation.
The weighting given to these elements varies, but it is common to give more weight to tasks that
are completed near the end of the course.
Most institutions are required to send individual reports to the central admissions office or the
receiving department, either with a test result (from one or more of the tests described above), a
grade determined by a combination of test results <strong>and</strong> other assessments, or a pass-fail
judgement which is based either on tutors’ impressions or on a more straightforward criterion
such as attendance. An estimated IELTS score is required in some institutions. A recent
development is the use of ‘can-do’ scales, which consist of lists of performance objectives <strong>and</strong>
columns where EAP tutors indicate whether students are able or not to achieve each objective.
The history of assessment on Lancaster University’s pre-sessional EAP courses reveals that
we have adopted a number of these approaches at different times. In the early 1990s, we were
asked to provide estimated IELTS scores. The admissions officers found these estimates useful
as they could interpret them in the same way they had processed genuine test results when
making their initial decisions. We, however, became increasingly concerned about the lack of
validity <strong>and</strong> reliability in our judgements because:
(i) We did not have access to the official rating scales for speaking <strong>and</strong> writing.
(ii) Estimating reading <strong>and</strong> listening abilities was problematic, since this depended on
inferences about students’ ‘inner processes’ rather than analyses of products such as
essays or oral presentations, <strong>and</strong>
(iii) None of the teaching team had been trained as IELTS examiners.
We ab<strong>and</strong>oned this practice in favour of writing a profile report for each student, commenting
on their general ability in listening, oral presentations, group discussions, reading <strong>and</strong> writing.
The report was written by the student’s main tutor, who had at least 8 h of contact with the
student every week <strong>and</strong> also had access to feedback sheets <strong>and</strong> comments from the student’s
other tutors. The main tutor would also give an overall judgement of the student’s ability. The
report was discussed with the student before being passed on to the course co-ordinators. The
course co-ordinators reviewed <strong>and</strong> edited it to st<strong>and</strong>ardise the style <strong>and</strong> to ensure that it was

52
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
a summary of the student’s progress <strong>and</strong> ability rather than a collection of anecdotes. Copies
were sent to the University’s admissions officers, the departments <strong>and</strong> to the students
themselves.
An example of a completed profile report can be found in Appendix A.
1.1. Problems with the <strong>reporting</strong> system
This system continued until 2001 without too many objections or queries, but there were a
number of problems:
(i) The reports went to two different audiences: the administrative audience, consisting of
the central university admissions officers <strong>and</strong> the students’ departmental admissions
officers, <strong>and</strong> the students themselves. It was difficult to write in a way which satisfied the
needs of two such distinct audiences—in particular, to say what needed to be said
without discouraging the students.
(ii) Tutors had varying interpretations of what constituted readiness for entry into a
university department <strong>and</strong> there were few opportunities in either pre-course or in-course
staff meetings to discuss definitions of the key criteria. Table 1 contains a compilation of
the types of features that tutors mentioned in the “Writing” section of a sample of 50 final
reports produced in 2001. The left-h<strong>and</strong> column contains comments which relate to some
notion of writing proficiency. It is interesting to note that features which often figure in
textbooks on writing, such as cohesiveness, coherence, <strong>and</strong> checking drafts, occur rarely.
The comments about linguistic performance (under “Language”) are general rather than
detailed. It is a matter of debate whether the comments in the right-h<strong>and</strong> column (which
relate to other facets of the students’ performance) reveal anything substantial about the
students’ ability to express them in formal writing <strong>and</strong> to conform to st<strong>and</strong>ard academic
conventions.
(iii) The brevity of the courses, the need to give fair coverage to all skill areas, <strong>and</strong> a course
approach which emphasised out-of-class preparation <strong>and</strong> in-class group discussions
meant that there were many aspects of performance that were either not visible in the
classroom or that manifested themselves when students were talking with one another
rather than with the tutor. This affected the depth of knowledge any tutor had of a
student’s ability.
(iv) We have already explained that all the course reports were edited by the course coordinators
in order to st<strong>and</strong>ardise the style <strong>and</strong> to lift the level of commentary from the
more specific to the more general. It was inevitable that those doing the editing changed
the meaning of some of the reports, either on purpose (toning them down) or
inadvertently, through a lack of underst<strong>and</strong>ing of the original text. As a result of changes
that made the language more diplomatic, admissions officers sometimes had problems
with interpretation, which made their job harder rather than easier, thus undermining one
of the purposes of the exercise.
It is clear from the survey of other institutions that Lancaster University’s practices before
2001 were not unusual. However, we remained concerned about the validity <strong>and</strong> reliability of
our exit assessments. We wanted to ensure that our practice was well grounded in theory, that it
was transparent to users (our students <strong>and</strong> admissions personnel), <strong>and</strong> that it was practical in
terms of tutors’ skills <strong>and</strong> time.

Table 1
Final report 2001—the types of comments recorded in the section for writing
Related to writing Interesting, but are they related to writing?
GENERAL Confident
Excellent writing Has improved
Thoughtful writing (3) Feels has improved
Transmits ideas clearly Will improve
Cohesive (4) No improvement in first 3 weeks, then sudden
improvement
Coherent (6) Raised self-awareness
Fluent Has become aware of the need to write well
CONTENT
Related to reading: Aware of difficulties <strong>and</strong> trying to overcome (2)
Reveals she’s understood reading Has taken note of principles (3)
Can analyse data Puts in effort (2)
Can respond critically to data (10) Has applied self
Argumentation:
Good argumentation (11) Responds to feedback (9)
Does not hesitate to give own views Found module helpful (2)
Needs to read widely <strong>and</strong> critically Has enjoyed this module
Style:
Reader-friendly (9) Settled down quickly
Academic I have no worries
Concise expression Should be able to cope with dem<strong>and</strong>s (2)
Verbose I hope she will continue to improve
Conventions:
Has explored conventions of academic writing Weaknesses likely to disappear soon
Can convey message in own words Weak in xxx but once that improves I hope
Editing:
Developing editing skills Will benefit from continued academic support
Less than rigorous when checking drafts (2)
ORGANISATION (31)
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 53
LANGUAGE
General comment about language (13)
Grammatical problems (5)
Good word choice
2. Research into EAP assessment
One obvious option when restructuring the EAP exit assessment was to require the students to
re-take an external EAP test such as IELTS or TOEFL. However, there are a number of
drawbacks in this approach. The first is practical in that there is a time delay between the taking
of the test <strong>and</strong> the release of the official results. The pre-sessional EAP courses finish just before
the academic year begins so this means that students would need to take the external test before
the pre-sessional course ended. Since, the shortest (<strong>and</strong> largest) of Lancaster University’s pre-

54
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
sessional EAP courses is only four weeks, the students would probably have to take the external
test mid-way through the programme, possibly before they were able to make measurable
advances in their language proficiency.
The second, <strong>and</strong> stronger, argument is made by research that has analysed the dem<strong>and</strong>s of
university writing tasks, particularly research that has compared these dem<strong>and</strong>s to those on
external EAP tests. Horowitz (1986) <strong>and</strong> Canseco <strong>and</strong> Byrd (1989), the latter looking
specifically at the writing dem<strong>and</strong>s in postgraduate degree programmes, report that university
students need to be able to select relevant data from sources, reorganise the data to
answer the question set, <strong>and</strong> also need to write their text using academic register. Other
important writing skills include the ability to present information in tables <strong>and</strong> graphs, <strong>and</strong>
to revise <strong>and</strong> edit drafts. Students are rarely required to refer to personal experience. This, as
Moore <strong>and</strong> Morton (2005) point out, marks an important difference between the dem<strong>and</strong>s of the
target language situation <strong>and</strong> the writing dem<strong>and</strong>ed by at least one major external test—the
IELTS. IELTS Task 2 typically requires students to draw upon their personal experience
for examples to support claims that they make rather than to draw on reading sources or
primary data.
Other differences documented by Moore & Morton (2005) include:
(i) IELTS Task 2 falls within a restricted genre of a written argument or case whereas
university writing tasks cover reviews, case study reports, research reports, research
proposals, summaries, exercises <strong>and</strong> short answer tasks.
(ii) The predominant rhetorical functions in IELTS Task 2 are evaluation <strong>and</strong> hortation.
Though evaluation is the predominant rhetorical function of university writing tasks, other
rhetorical functions such as description, summarisation <strong>and</strong> comparison are also common
<strong>and</strong> hortation is relatively rare.
Consequently, Moore & Morton argue that it is not possible to combine preparation for the
IELTS test with preparation for university study.
Another possible reason for lack of fit between EAP tests <strong>and</strong> the language dem<strong>and</strong>s students
face at university may lie in the criteria used for assessment in EAP contexts. If the assessment
criteria used in EAP tests do not reflect the criteria against which the students’ performance will
be judged in academic contexts, then the scores achieved are less easily interpretable with
reference to the students’ ability to perform tasks in those contexts. For instance, Thorp <strong>and</strong>
Kennedy (2003) have reported that ‘idiomaticity’ rather than the ability to write within the
academic genre is rewarded on the IELTS writing test. Jacoby <strong>and</strong> McNamara (1999) comment
that the assessment criteria in specific purposes tests usually employ only language-focused
categories whereas <strong>performances</strong> in a specific purposes context are not wholly language based.
This last point is problematic because the main purpose of a pre-sessional EAP course is to boost
students’ ability to perform in English in an academic environment so it can be argued that the
focus should therefore be upon language. However, this would be to ignore the complexity of
language for specific purposes contexts. It appears, therefore, that external EAP tests such as
IELTS <strong>and</strong> TOEFL (in their current forms) [2005], while suitable as a pre-entry measure, are
insufficiently representative of the construct of EAP (particularly EAP writing) to be used as an
exit measure for pre-sessional EAP courses.
With some notable exceptions (e.g., Cho, 2003), there is little published on EAP-course finalassessment
procedures. However, the construct of EAP assessment has been addressed from a
number of other perspectives (see Blue, Milton, & Saville, 2000 for a compendium of recent

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 55
Table 2
Key factors in successful academic performance, taken from Ginther <strong>and</strong> Grant (1996)
Reading Writing
Underst<strong>and</strong>ing the main idea of their reading Organisation
Reaching valid conclusions Summarisation
Making critical evaluations of content Well-formed sentences
Comprehending significant detail Vocabulary
Underst<strong>and</strong>ing explicitly stated information Usage
Detecting inferences between the lines Research skills
Economy
Clarity
Providing sufficient evidence
Grammatical
Correctly punctuated
Ability to use ‘st<strong>and</strong>ard’ academic discourse
Knowing what your tutor–examiner values
(<strong>and</strong> giving that to him/her)
work). Studies like Horowitz (1986), Canseco <strong>and</strong> Byrd (1989) <strong>and</strong> Moore <strong>and</strong> Morton (2005)
have analysed the writing dem<strong>and</strong>s placed upon university students. Others have looked
more broadly at the language <strong>and</strong> academic needs of university students. Weir’s (1983)
groundbreaking work focused upon the needs of overseas students. More recently, as part of
the development programme for the TOEFL internet-based test (TOEFL iBT), the
Educational Testing Service has published a series of commissioned papers covering
the reading, writing, speaking <strong>and</strong> listening dem<strong>and</strong>s on university students (see Bejar,
Douglas, Jamieson, Nissan, & Turner, 2000; Butler, Eignor, Jones, McNamara, & Suomi, 2000;
Cumming, Kantor, Powers, Santos, & Taylor, 2000; Enright, Grabe, Koda, Mosenthal,
Mulcahy-Ernt, & Schedl, 2000; Hamp-Lyons & Kroll, 1997; Jamieson, Jones, Kirsch,
Mosenthal, & Taylor, 2000; Waters, 1996). Though the primary purpose of these papers was
to define the initial construct of the revised TOEFL test, they also form an excellent resource for
the EAP construct in all four-language skills.
Another useful resource is Ginther <strong>and</strong> Grant’s (1996) meta-analysis of research into the
academic needs of native English-speaking university students in the US. This provides
information about the language dem<strong>and</strong>s placed on all students (not only international students).
They note that speaking <strong>and</strong> listening needs have not been investigated (possibly because these
skills are taken for granted among native speakers or because they are rarely formally assessed in
an academic context) <strong>and</strong> that reading is often commented on in relation to writing. Their metaanalysis
suggests a number of reading <strong>and</strong> writing features that are key factors in good academic
performance. These are listed in Table 2.
Yet other researchers have explored the criteria applied by subject tutors to university students’
<strong>performances</strong>. For instance, as part of a study to design suitable writing tasks for post-graduate
students Wall (1981) analysed the criteria that Economics tutors applied to the students’ written
work. She found that the tutors applied some or all of the following criteria when assessing students’
finished work: knowledge of subject matter, critical ability, structuring of the essay, answering the
question, <strong>and</strong> the use of sources. Tutors responded negatively to verbosity/irrelevance <strong>and</strong>
carelessness/lack of rewriting. They made relatively few comments about specific language
features. Wall, Nickson, Jordan, Allwright <strong>and</strong> Houghton (1988) arrived at a similar finding when
they compared the reactions of an academic tutor to a student essay with those of three EAP tutors.

56
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
Table 3
Factors influencing lecturers’ judgements of student writing, from Errey (2000, p. 156)
Content Layout
Use of sources Punctuation
Coherence Underst<strong>and</strong>ing of reading
Organisation Appropriate register
Addresses topic Appropriate expression
Pre-writing strategies Accurate vocabulary
Development of ideas Essential vocabulary
Completion of task Length
General linguistic accuracy Paragraphing
Grammatical accuracy L1 interference
Fluency/style Legibility
Spelling Cohesion
Although the EAP tutors differed in their approaches to helping the student they all seemed to share
the opinion of the academic tutor that “a higher return to language teacher input can be achieved by a
shift in teaching emphasis towards the wider questions of essay structure in general <strong>and</strong> the art of
logical presentation in particular” (1988, p. 127). More recently, Hartill (2000) <strong>and</strong> Errey (2000)
have investigated the assessment dem<strong>and</strong>s faced by students at two UK universities. Hartill reports
Table 4
‘Cost’ factors listed in Banerjee (2003, pp. 335 <strong>and</strong> 343)
Language related source of “cost” Course related source of “cost”
Inability to underst<strong>and</strong> the lecturers’ accents The structure of the course
Inability to underst<strong>and</strong> native-speaking colleagues Course content
Inability to underst<strong>and</strong> local residents –Theoretical orientation
Inability to underst<strong>and</strong> other non-native speakers –Inadequate coverage
Difficulty in following radio broadcasts –Rigid (insufficient allowance for student contributions)
Need to translate into L1 in order to underst<strong>and</strong> concepts –British centred examples
Need to translate from L1 when speaking or writing –No focus on current affairs
Difficulty in making themselves understood –Mathematics
Restricted/inadequate lexical resources Course delivery
Inability to retrieve lexical resources quickly
(particularly problematic under examination conditions)
–Poor lecturing
Negative perception of their ability to communicate Conflict between preferred study method <strong>and</strong> the
dem<strong>and</strong>s of the course
Slow reading speed Conflict between study aims <strong>and</strong> the dem<strong>and</strong>s of the
course
Difficulty in/inability to identify key points in reading Volume of assessment
British academic writing conventions Lack of variety in assessment
Avoidance of resources in their L1 because of a
perceived incompatibility between the two languages
Lack of background in the subject
Preference for resources in their L1 Relationship with supervisor
Inability to memorise easily (particularly problematic Behaviour of course colleagues
under examination conditions)
Poor examination performance

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 57
that departments tend to value “communicative effectiveness,” but that this does not necessarily
encompass linguistic accuracy. Errey’s introspective verbal report study with lecturers from five
different departments at Oxford Brookes University has revealed 24 factors that influence the
lecturers’ judgements of student writing. These factors are presented in Table 3.
A few researchers (e.g., Geoghegan, 1983; Banerjee, 2003) have explored study needs from
the students’ perspective by drawing on their on-going study experiences. Banerjee (2003)
conducted a longitudinal study of 25 postgraduate students, interviewing them at a number of
different points during their Masters degrees. She found that they experienced difficulties (she
termed this “cost”) stemming from factors related to both language <strong>and</strong> the courses they were
following (Table 4). She also found that students experienced difficulties in adjusting to
the British system of education, particularly the lecture-seminar structure of their courses, the
emphasis on learner independence, the dem<strong>and</strong> for critical <strong>and</strong> independent thought, academic
conventions, the criteria used for marking (particularly the way they were applied), the role of
the teacher, <strong>and</strong> the emphasis in many courses on group work.
To summarise, our previous experience of <strong>reporting</strong> student performance on the pre-sessional
EAP course <strong>and</strong> the studies reported above narrowed our options with respect to the <strong>reporting</strong>
mechanisms that we could consider. It was clear that the use of an external EAP test would risk
construct under-representativeness <strong>and</strong> would undermine the usefulness of the report for the
admitting departments <strong>and</strong> the students. Our experience of using profile reports based on tutors’
personal criteria was similarly unsatisfactory. We wanted to devise a report form which would
incorporate key features from the research we have reviewed <strong>and</strong>, because much of this focused
on writing, would also incorporate features from scales such as those found in the English
Speaking Union (ESU) framework (Carroll & West, 1989). We would complement this with our
own experience, not only as pre-sessional <strong>and</strong> in-sessional tutors but also as members of an
academic department.
In the sections that follow, we describe both the design of an exit assessment checklist <strong>and</strong> the
initial stages in its validation.
3. Developing a final assessment checklist
We took a number of decisions in order to ensure that the new exit assessment procedure
provided more useful guidance to tutors, students <strong>and</strong> the admissions officers; was more
practical; <strong>and</strong> explicitly reflected theories of the EAP construct. These decisions were:
(i) Function <strong>and</strong> audience—The main function of the report would be to give the admissions
officers an accurate picture of the student’s abilities. Students would receive a copy of the
report, <strong>and</strong> it was hoped they would benefit from receiving a frank account of their
strengths <strong>and</strong> weaknesses, but we would not edit the reports to soften the blow for
students who were not performing adequately.
(ii) Coverage—The report form would explicitly reflect current EAP theory. It would specify
the skills <strong>and</strong> strategies that had emerged from our analysis of previous research, the
report forms prepared in 2001, <strong>and</strong> our previous experience as EAP teachers <strong>and</strong>
academic tutors. We decided that it would be inappropriate to comment on attitude,
aptitude, motivation, awareness, or any other quality which was a feature of personality
rather than of linguistic or EAP ability.
(iii) Evidence—We wanted to make it clear that we were only commenting on features
that we had evidence for. We also wanted to make clear the limits of our judgements.

58
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
For instance, we stated explicitly that the tasks that students performed during the presessional
course approximated (but could not fully replicate) the dem<strong>and</strong>s students
were likely to encounter in their study contexts. We would indicate how students had
performed on the tasks set during the course rather than predicting whether they
would do well in their future settings. Our intention was to provide our primary
users—the admissions personnel, with an evidential basis for deciding whether or not
a student was ready to begin studying on a particular programme. This allowed
admissions tutors for different degree programmes to interpret the evidence
differently, depending on the aspects of EAP proficiency they considered more
important for their fields of study.
(iv) Format—It was clear that prose reports were difficult for the tutors to write <strong>and</strong> timeconsuming
to edit. They were also difficult for the end-users (primarily university
admissions personnel) to interpret. One way of eliminating the problems of
composing, editing <strong>and</strong> interpreting was to devise a checklist where tutors would
only have to place ticks in columns to indicate whether the student had or had not
demonstrated certain abilities.
The procedure for devising the checklist was as follows:
1. We drew up an exhaustive list of the features of academic reading, writing, listening <strong>and</strong>
speaking abilities.
2. We went through an iterative process of combining some items that were similar <strong>and</strong>, in
contrast, breaking down some broad descriptors into sub-categories. For instance, we split an
item taken from Errey (2000)—“content’”—into two sub-categories, “can analyse the topic
of the assignment” <strong>and</strong> “can produce relevant content”.
3. We grouped items according to the language skill they represented. At times these decisions
were complex. For example, “providing sufficient evidence” (Ginther & Grant 1996) isa
writing skill but is also dependent on the student’s ability to underst<strong>and</strong> <strong>and</strong> make use of
reading. We resolved this particular problem by breaking the skill down into two components
“can analyse argumentation in academic texts” (classified in the checklist as a reading skill)
<strong>and</strong> “can reproduce others’ ideas using his/her own words” (classified as a writing skill).
4. The checklist was formatted to include two columns for each can-do descriptor: “yes” (the
student can do this) or “must pay attention to” (this skill or strategy).
5. The draft checklist was then presented to the University’s postgraduate admissions officer
<strong>and</strong> to the pre-sessional course tutors. Changes were made on the basis of their feedback.
6. The revised checklist was piloted by the pre-sessional course tutors on three of their
students (a strong student, a weak student, <strong>and</strong> one whose ability fell between these two
extremes)—approximately 27 students. On the basis of this pilot procedure, further changes
were made.
7. Taking account of recommendations by Weigle (1994) for rater familiarisation with the
adopted rating scale, there was extensive opportunity, at each point during the consultation
<strong>and</strong> piloting phase, for discussion of what the can-do descriptors meant <strong>and</strong> how they might
be applied. Indeed, the raters (the pre-sessional course tutors) contributed to the content,
wording <strong>and</strong> final format of the exit assessment checklist. The details of this consultation
are presented in Section 3.1 (below).
8. We also met with the students in order to introduce the checklist to them as well as to elicit
their feedback on the items included. Changes were made on the basis of this feedback.

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 59
9. The final form of the exit assessment checklist (Appendix B) was used during the last week of the
pre-sessional course. Each student received their report <strong>and</strong> a copy was sent to the University’s
postgraduate admissions officer <strong>and</strong> the admissions personnel in admitting departments.
3.1. Consulting the stakeholders <strong>and</strong> piloting the checklist
As has already been stated, the aims of the consultation <strong>and</strong> piloting phase were threefold:
† To provide some of our stakeholders (the University’s admissions personnel <strong>and</strong> the presessional
course tutors) with an opportunity to comment on the initial draft <strong>and</strong> suggest
changes or additions before the piloting stage (we consulted students during the piloting
stage).
† To pilot the revised instrument in order to ascertain its usability <strong>and</strong> coverage.
† To maximise the course tutors’ opportunities to familiarise themselves with the checklist <strong>and</strong>
to arrive at shared interpretations of each of the ‘can-do’ statements.
During the consultation <strong>and</strong> piloting phase, we received valuable feedback in a number of
areas:
(i) Usability of the report form—The tutors found the report form quick to complete
<strong>and</strong>, despite some suggestions for changes, they believed that they understood the
can-do statements. The University’s Postgraduate Admissions Officer noted that the
form provided judgements in four skill areas, commenting that this paralleled the
report formats of language proficiency test scores like the IELTS. She thought this
was a step forward, arguing that this form was more transparent than its predecessor
<strong>and</strong> that the information was easy to retrieve because it was not presented in prose.
She believed that course directors in the academic departments would be able to
return to this form at a later date in order to compare the students’ on-going
performance in their departments with what was reported at the end of the EAP
course.
(ii) Format of the overall report—The tutors initially argued that if a student seemed to
be satisfactory in all respects it would be sufficient to present a summary judgement
of his/her performance on the course. They suggested that the form for such students
should consist of a single page with a box to tick for each of the four skills
(listening, speaking, reading <strong>and</strong> writing). We resisted this suggestion for two
reasons. The first was that we needed to be sure (inasmuch as this is possible) that
all the tutors shared the same underst<strong>and</strong>ing of the construct for each skill area. We
felt that if they had to place a tick opposite each aspect listed under each skill they
would be obliged to consider all the traits that we considered to be important. The
second reason had to do with accountability <strong>and</strong>, ultimately, face validity: we felt it
unlikely that the University would accept our judgements on the adequacy or
otherwise of students’ <strong>performances</strong> unless we demonstrated how we had arrived at
our decisions.
Nevertheless, the course tutors prevailed upon us to include a summary judgement
page (this became the final page of the exit assessment form). They argued that they
would feel more comfortable being “honest” in the checklist if they could also

60
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
indicate how seriously they viewed a student’s particular difficulties. They would not
worry about ticking individual boxes if they could indicate in the summary section
that they were “generally satisfied with the student’s performance”. This would
allow them to give their students specific feedback <strong>and</strong> guidance without suggesting
that they had not “successfully completed” the course.
(iii) Wording of the judgement—The tutors argued that some of the diplomacy of the
original <strong>reporting</strong> format had been lost <strong>and</strong> that, in order to correct the apparent
baldness of the claims being made, the descriptors needed to be presented within the
context in which the judgements took place. The outcome of this discussion was that
all the judgements are prefaced with the phrase “Our evidence suggests that.”
The tutors also argued that it should be possible to indicate when there was no
evidence for a particular judgement. For instance, a skill may have been taught but
the student may not have taken advantage of opportunities to demonstrate it. This
resulted in the inclusion of the third column, “We have no evidence.” It should be
noted at this point that the tutors’ request for this column proved somewhat
problematic when the report was used in its final form (see below).
Thirdly, in line with the spirit of the new exit assessment, the tutors rejected the
formulations “will have difficulty” <strong>and</strong> even “has difficulty” as they implied a
prediction of future performance. They preferred instead to locate their judgements
within the period of the course, hence the formulation “has had difficulty”.
(iv) Evidence available on which to base a judgement—Though the tutors considered the
report form easy to use <strong>and</strong> transparent, they were concerned about the extent to which
they could make detailed judgements about the students’ listening <strong>and</strong> reading abilities.
They argued that since the processes involved in listening <strong>and</strong> reading are internal, they
would be reliant on student self-report (which might not be reliable) or on their
interpretation of students’ listening <strong>and</strong> reading abilities based on what they produced
when speaking <strong>and</strong> writing. This problem has not been fully or satisfactorily resolved.
Indeed, it remains one of the difficulties in any assessment of receptive skills.
The tutors also argued that students with good language skills might fail to give evidence
of certain can-do statements such as strategies like “asking for clarification,” since they
might never be in a position where they have misunderstood <strong>and</strong> have to ask for
clarification. The solution we have adopted has been to orientate the students to the criteria
by which they are to be judged. When the students register they receive a self-assessment
checklist which requires them to make judgements about their language abilities at the start
of the course. This checklist replicates the can-do statements used in the exit assessment
checklist <strong>and</strong> has proved a useful consciousness-raising tool particularly since students
discuss their self-assessments during their first consultation with their tutor. Nevertheless,
we are still concerned that students might fail to fully demonstrate their abilities according
to the “can-do” statements <strong>and</strong> we plan to explore this area in a later study.
4. Investigating the validity of the checklist
The final version of the checklist incorporated as much of the feedback as was feasible from
the course tutors <strong>and</strong> the University’s Postgraduate Admissions Officer. It was then used to
assess 86 students, all of whom completed the 4-week pre-sessional EAP course. In a further

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 61
stage of developing <strong>and</strong> reviewing this new assessment format, we investigated the validity of
the exit assessment in two phases:
† Content relevance <strong>and</strong> coverage
† Interpretation <strong>and</strong> use of the new instrument
4.1. Content relevance <strong>and</strong> coverage
The aims of this validation phase were to determine where evidence for items on the checklist
might be available from assessment opportunities during the programme <strong>and</strong> what opportunities
for collecting evidence were being missed. The first step in this process was to list all the
assessment instruments that were used during the course, when they were used, what function
they performed, what they consisted of, <strong>and</strong> where the results of the assessment were kept for
reference. A table containing the details of 12 instruments can be found in Appendix C.
We distributed this list <strong>and</strong> copies of all the instruments to a seminar of colleagues working in
language testing, <strong>and</strong> asked them to analyse the instruments to determine whether there were
features on the final checklist for which no evidence was being collected during the course <strong>and</strong>
whether there was evidence being collected on the course for which there were no features
mentioned in the checklist.
The main conclusion emerging from the discussion was that there was a great deal of
assessment on the pre-sessional course, both formal <strong>and</strong> informal, but the overall system was not
integrated. Each instrument had its own function <strong>and</strong> its own rationale but the instruments did
not fit together in a coherent whole. The following types of problems were identified:
† Entry on checklist but no evidence available through instruments—for example, the checklist
asks for a judgement concerning the student’s ability to participate in small group
discussions, but there is no structured way of gathering this evidence.
† Evidence available but no entry on checklist—for example, the tutor feedback form for oral
presentations includes several features of oral performance which do not appear on the
checklist.
† Terminology: some of the terminology in the checklist <strong>and</strong> in the instruments was not defined
so it was not clear whether they were referring to the same thing—for example, the TEEP
(Test of English for Educational Purposes) attribute writing scale, which is used to assess the
students’ first piece of writing, contains a criterion called “cohesion”. There is no equivalent
term on the checklist. The nearest notion seems to be “Can make appropriate use of heading
<strong>and</strong> subheadings,” but this may have been quite different from what the TEEP designers
originally had in mind.
† What do these things look like in the classroom? The checklist contained several notions such
as “Can cope with heavy reading load” <strong>and</strong> “Can underst<strong>and</strong> gist” but it was not clear what
the teachers should look for to confirm whether students were able to do these things.
It was to be expected that there would be inconsistencies within the overall system, given that
different instruments <strong>and</strong> practices were designed at different times in the history of the
development of the programme. We have begun to harmonise all of the components so that
evidence is available for all the features specifically mentioned in the checklist.

62
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
4.2. Interpretation <strong>and</strong> use of the checklist
In the second phase of our validity investigation, we explored how the pre-sessional course
tutors used the checklist to make judgements about individual students. Since, the course tutors
had been so closely involved in the development of the instrument, we were particularly
interested in the extent to which the co-operative development process had benefited them in the
way they used it.
Three tutors were selected for individual interviews (out of a teaching team of nine) based on
their general teaching experience <strong>and</strong> their experience of teaching EAP (particularly, the
Lancaster University pre-sessional course). One was a teacher with many years of experience in
both teaching <strong>and</strong> course management. He had worked on the teaching team the previous year
<strong>and</strong> had used the previous <strong>reporting</strong> format. The second was a tutor who had also worked on the
teaching team the previous year but who was less experienced. The third tutor selected had no
previous EAP teaching experience <strong>and</strong> was teaching on this course for the first time. She had
therefore not used the old <strong>reporting</strong> format.
Each interview followed a three-part structure. In the first stage, the tutors were asked for
their general impressions of the form <strong>and</strong> the process of using it. They were then shown a blank
copy of the form <strong>and</strong> asked to work through it, pointing out any difficulties they faced in
completing it. In the final stage, they were given the report forms they had previously completed
for their class. They were asked to select three students—the top student, the bottom student, <strong>and</strong>
one in the middle—<strong>and</strong> then asked to recount the process they had gone through when making
judgements about them.
The interviews revealed three perceived strengths of the report form. First, the tutors were
impressed by the amount of time saved in using this report form. The tutor who was teaching
on the course for the first time commented that she could not imagine having to write prose
reports in addition to the other pressures on tutors’ time. The tutors also appreciated the
transparency of the report form. One tutor commented that he did not “have to be creative.”
He also argued that it was clearer for departments who no longer had “to wade through .
diplomatically couched language to find out what the tutor was really getting at.” Finally, the
tutors were in favour of the disclaimer on the front page which delimited the judgements
being made to the context of the course.
The interviews also revealed that the tutors were able identify the behaviours on which
they had based their judgements <strong>and</strong> that they had used both the more formal opportunities
for monitoring progress (such as essays <strong>and</strong> oral presentations) <strong>and</strong> the learning
opportunities provided in class (such as group discussions <strong>and</strong> poster presentations) to
gather evidence of students’ language ability. Clearly, they found it easiest to comment on
language areas for which they had formal evidence. This was particularly true of writing
because, as one tutor commented, “it is the skill we gather the most evidence for”.
Listening <strong>and</strong> reading definitely presented problems, not least because the tutors were
unable to observe their students in all the contexts for which they were making judgements.
For instance, though the students attended lectures each week (delivered by invited
members of academic departments), the tutors responsible for judging their progress were
not necessarily present <strong>and</strong> so did not have an opportunity to observe them.
Where tutors did not have direct access to a particular ability such as “Can underst<strong>and</strong> a
variety of non-native speaker accents,” they found that they were able to consult with the
students. This is a particularly interesting outcome because, during the consultation <strong>and</strong>
piloting process the tutors had expressed some concern about how much they should rely on

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 63
student self-assessment. Since, the exit assessment report represented high stakes for the
students they might well have described themselves in a way that they thought appropriate in
order to successfully complete the course. However, the tutors we interviewed felt that their
students had largely been forthright in their self-assessments <strong>and</strong> individual tutorials. In fact,
one tutor commented that the students “were more honest than perhaps we give them credit
for.” Indeed many students encouraged their tutors to be “truthful” so that they could set
learning goals for the rest of the year.
Nevertheless, a number of issues also emerged from the interviews. Most of them are clearly
training issues which need to be addressed as soon as possible:
1. The checklist is criterion-referenced <strong>and</strong> each student is to be judged against the can-do
statements. However, at least one tutor not only considered whether the student had the stated
ability or had had difficulty but also tried to rank his students. For example, in the case of the
writing item “Can produce grammatically correct text” he ticked that a student had this
ability not because she performed well in this area but because he wanted to distinguish her
from another good student who did not.
2. During the consultation <strong>and</strong> piloting phase (see Section 3.1, above), the tutors specifically
requested a column entitled “we have no evidence” to cover cases where students did not
demonstrate a particular ability. The purpose of this column was discussed extensively at the
time yet, during the implementation of the checklist, at least one tutor agonised over using
the “no evidence” column, worrying that it would imply that the student did not have that
language ability.
3. The exit assessment checklist includes a summary judgement on the final page. This was
included at the request of the course tutors (see Section 3.1, above). However, the interviews
revealed that the route by which the tutors arrived at their summary judgements differed. One
tutor said that she had a global sense of whether a student had made satisfactory progress or
not <strong>and</strong> based her judgement on that rather than on the distribution of ticks across the report.
Another tutor based her judgement on the number of ticks the student received in the “has
had some difficulties” column as well as her judgement of the gravity of these ticks. This last
practice is particularly interesting for it introduces a level of detail not explicit in the report
form. This has implications for the relationship between the detailed checklist <strong>and</strong> the
summary section of the report form <strong>and</strong> is again an issue to be dealt with in training.
Though there is still much to be done to help tutors adjust to the new <strong>reporting</strong> system <strong>and</strong> to
orientate new users of the checklist, the remaining concern relates to how the tutors applied the
“can-do” statements. It is important to remember that the tutors had discussed their
interpretations of items during the consultation <strong>and</strong> piloting stage (described earlier) <strong>and</strong> were
therefore familiar with the checklist. Indeed, they had contributed to its wording. Yet, as Lumley
(2002) points out, when making their judgements all raters have to reconcile the rating scale (in
this case the can-do statements) <strong>and</strong> their observation <strong>and</strong> interpretation of student
<strong>performances</strong>. This proved to be an issue for the tutors we interviewed. For instance, one
tutor commented on the word “satisfactory” in the statement “can take satisfactory notes.” Since,
notes are generally for personal use, this tutor wondered whether she was the best judge of
whether a student’s notes were “satisfactory” for the student’s own purposes. Much can be done
to discuss with the course tutors the behaviours that constitute evidence of a particular ability (as
well as the behaviours that suggest difficulty). However, the tutor is still the ultimate arbitrator
<strong>and</strong>, as Lumley (2002) argues, it is not possible to cover all eventualities during rater training.

64
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
5. Planning future directions
In this paper, we have presented the rationale for a new assessment instrument <strong>and</strong> the process
we went through to tailor the instrument to our needs <strong>and</strong> to investigate its validity. Our work so
far has identified two areas for immediate action:
1. Integrating continuous assessment with final assessment—Our review of the instruments we use
for continuous assessment (Appendix C) clearly shows that there is a need to review <strong>and</strong> rationalise
our system, so that each formative judgement contributes to the summative checklist. This should
also provide our students with sufficient opportunities to demonstrate whether they have mastered
the abilities in the final checklist. This will, of course, have an impact upon syllabus design.
2. Training the tutors—Despite our involvement of course tutors in the development of the checklist,
it is clear that they still diverged in their interpretation <strong>and</strong> implementation of the instrument. This
problem could be exacerbated in cases where tutors are teaching on the course for the first time. We
must prepare materials <strong>and</strong> procedures that we can use with these tutors, both in our orientation
work before teaching begins <strong>and</strong> during the rest of the course, to make sure that they underst<strong>and</strong> the
different meanings behind terms such as ‘coherence’ <strong>and</strong> to bring all tutor judgements into line.
Our investigations so far have identified key elements to be included in the training material.
This study has also revealed a number of longer-term objectives. We are keenly aware that we
are still in the early stages of design <strong>and</strong> that there is work to be done before we can be satisfied
that this form of assessment is entirely appropriate for our situation. There are two further
challenges which must be dealt with in the near future:
1. Interpretation <strong>and</strong> use of the instrument by admissions personnel—One of our aims for this
checklist was to disambiguate the information that the pre-sessional exit assessment procedure
provides to admitting departments. The preliminary feedback that we received (see Section
3.1, above) indicated that we had succeeded in this aim. However, a further phase of validating
the checklist should explore the ways in which admissions personnel interpret the results of the
checklist <strong>and</strong> arrive at a decision on whether or not to admit a conditional applicant.
2. Refining the construct—The checklist that we have designed was based on our underst<strong>and</strong>ing
of the issues present in the literature on EAP assessment, our own experience both as EAP
tutors <strong>and</strong> as academic lecturers, <strong>and</strong> the experience of colleagues in other universities in
Britain <strong>and</strong> other parts of the world. Despite the rich resources available, there is still much to
learn about the dem<strong>and</strong>s placed on students at different levels in the university system
(undergraduates <strong>and</strong> postgraduates) <strong>and</strong> in all the different departments that our students will
be entering. In particular, we need to underst<strong>and</strong> better what aspects of language ability are
crucial in determining students’ success in meeting these dem<strong>and</strong>s. Investigating this will
involve in-depth interviews with academic staff <strong>and</strong> both native <strong>and</strong> non-native English
speaking students, bearing in mind that students have been found to construct ‘success’ rather
differently from their tutors (Basturkmen & Lewis, 2002).
The issues involved in carrying out end-of-course assessment on pre-sessional courses have
not been well covered in the literature, yet the assessment that takes place on these courses is
high-stakes for the students involved. Failure to meet the st<strong>and</strong>ards set could result in the
students being refused entry to the university of their choice. Our survey of assessment practices
on pre-sessional courses has revealed that EAP professionals are aware of these pressures <strong>and</strong>

the importance of getting it right. We hope that this paper will encourage this work to move into
the public domain so that we can all benefit from the insights gained.
Appendix A. End-of-course profile, English for Academic Purposes course 2001
NAME: XX XXX
Skill Comments
LISTENING She did not, to begin with, underst<strong>and</strong> very much of what was said to her, <strong>and</strong>
this probably accounted for the fact that she did not make as much progress as
she should have during the first two weeks. However, in week 4 she suddenly
appeared to underst<strong>and</strong> more of what was said, <strong>and</strong> showed more progress
ORAL PRESENTATION Her oral presentation was fine, <strong>and</strong> no doubt her presentations will get even
better with practice. She needs to think carefully though about how much
information she can usefully fit on overhead transparencies
GROUP DISCUSSION She tended to be on the quiet side for the first half of the course but her
participation improved in the final week. She knows that she should talk
more <strong>and</strong> says she will do her best to speak more in her undergraduate classes
READING She initially had trouble underst<strong>and</strong>ing the materials she was supposed to
read, but she is using better reading strategies <strong>and</strong> her reading is now getting
quicker <strong>and</strong> better focussed
WRITING For the first two weeks I did not see any improvement, but in the third week,
when she word-processed her essay, there seemed to be a great improvement.
She did not hesitate to give her own views, <strong>and</strong> she succeeded in making it
clear that these were indeed her own views
Additional comments:
XXX’s problem at the moment is mainly linguistic. She was a very conscientious student, but
she had such difficulties with English syntax <strong>and</strong> vocabulary that she was not able, until week 4
of the course, to demonstrate her improvement in study skills. However, in this final week her
comprehension suddenly improved; she spoke more <strong>and</strong> appeared to be more confident.
Academic tutor course.
Course Co-ordinator’s signature.
Date: 1st October 2001
Appendix B. Final assessment checklist (the full version is available at http://www.ling.
lancs.ac.uk/groups/ltrg/docs/ssreport.pdf)
Oral skills
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 65
Listening Speed
Can respond quickly to what is said to him/her
Note-taking
Can underst<strong>and</strong> the gist of lectures
Can underst<strong>and</strong> the gist of seminar discussions
Can identify the main points in a lecture
Can identify the main points in a seminar discussion
Our evidence suggests
that the student
Has this
ability
Has had
some
difficulty
We have no
evidence
(continued on next page)

66
Can take satisfactory notes
Varieties of English
Can underst<strong>and</strong> classmates who speak quickly
Can underst<strong>and</strong> a variety of native speaker accents
Can underst<strong>and</strong> a variety of non-native speaker
accents
Speaking Speed
Can process the input <strong>and</strong> express his/her thoughts
quickly
Participation
Can participate in large group discussions (e.g.
plenary sessions)
Can participate in small group discussions
Can participate in informal group discussions (e.g. a
social event)
Can participate in one-to-one discussions (pairs or
with a tutor)
Can make major contributions to group discussions
Can express own point of view effectively
Can debate the views of other group members
Presentations
Can select appropriate content
Can establish rapport with the audience (e.g. eye
contact)
Can h<strong>and</strong>le questions from the audience
Strategies
Can ask for clarification
Can use turn-taking strategies to participate in
debates/discussions
Quality of language
Can be easily understood (pronunciation)
Can use appropriate vocabulary
Can speak with few/no grammar mistakes
Written skills
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
Reading Speed
Can skim texts for gist
Can scan texts to locate specific information
Quantity
Can cope with heavy reading load
Note-taking
Can identify the main points in a text
Can take satisfactory notes
Analysis of data
Our evidence suggests
that the student
Has this
ability
Has had
some
difficulty
Our evidence suggests that the
student
Has this ability Has had some
difficulty
We have no
evidence
We have no
evidence

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 67
Can underst<strong>and</strong> visual displays of data <strong>and</strong>/or
results of data analyses (tables, charts,
diagrams)
Critical reading
Can analyse argumentation in academic texts
Can evaluate argumentation in academic texts
Can support or refute argumentation in
academic texts
Writing Content
Can analyse the topic of the assignment
Can produce relevant content
Can reproduce others’ ideas using my own
words
Can write concisely
Can respond critically to input material
(lectures, texts, discussions)
Can produce well-supported argumentation
Can produce effective conclusions
Organisation
Can write informative introductions
Can organise texts on the macro-level
Can make appropriate use of headings <strong>and</strong>
sub-headings
Can use paragraphs
Acknowledging sources
Can use proper conventions to acknowledge
sources
When paraphrasing
When quoting verbatim
Can construct bibliography correctly
Language
Can produce grammatically correct text
Can select appropriate vocabulary
Can use the appropriate academic style
Can use spell-checker
Can use appropriate punctuation
Appendix C. Inventory of course assessment instruments
Our evidence suggests that the
student
Has this ability Has had some
difficulty
Instrument When used Function Description Where kept
Initial grammar test Day 1 Grouping,
diagnosis
Initial listening test Day 1 Grouping,
diagnosis
Grammar in context—mcq,
short answer, gap-filling,
re-ordering
TEEP-style ‘communicative
dictation’—underst<strong>and</strong>ing of
key point
Tutor’s file
Tutor’s file
We have no
evidence
(continued on next page)

68
Instrument When used Function Description Where kept
Initial writing test Day 1 Grouping,
diagnosis
Criteria for marking
writing
Self-evaluation
questionnaire
Day 1 Grouping,
diagnosis
Week 1 Awarenessraising
<strong>and</strong>
goal-setting
Tutorial form Weeks 1–4 Recordkeeping
Reading-review of
critical reading <strong>and</strong>
discussion
Writing-Guidance to
tutors on marking
written assignments
Writing-feedback
form
Oral Presentationstutor
feedback form
Oral presentationspeer
feedback from
Seminar discussionself-evaluation
References
J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69
Week 4 Selfevaluation
Weeks 2–4 Focusing
feedback
Weeks 2–4 Feedback to
student on
each writing
assignment
Weeks 1–4 Feedback to
student on
each
presentation
Weeks 1–4 Feedback from
peers on each
presentation
Weeks 2–4 Reflection,
goal-setting
TEEP writing task-read
12-page article <strong>and</strong> write a
summary
TEEP attribute writing scale
(adapted)-content, organisation,
cohesion, vocabulary,
grammar, punctuation, spelling,
lack of plagiarism
Five pages-questions on
abilities in all skill areas,
short composition
Three pages-personal details,
summary of self-evaluation,
test results, student’s reflections/concerns
re classes,
tutor’s evaluation/feedback re
all skills areas
Student’s record of preparation
for class, notes on
important points, thoughts re
future action
Features to comment on when
marking each weekly
assignment
Hamp-lyons formative feedback
profile-communicative
quality, ideas <strong>and</strong> organisation,
grammar <strong>and</strong> vocabulary,
surface features
Short comments on content,
organisation, vocabulary,
grammar, pronunciation <strong>and</strong>
other features related to public
speaking
Adapted from Lynch <strong>and</strong>
Anderson (1992). Tick boxes
to evaluate 15 aspects of
speaking
Nine questions re student’s
participation in seminar
Tutor’s file
Tutor’s file
Tutor’s file
Tutor’s file
Student keeps only
copy. Tutor never sees
Tutor’s file
Student keeps one
copy, <strong>and</strong> one kept in
tutor’s file
Student keeps one
copy, <strong>and</strong> one kept in
tutor’s file
Student keeps all
copies. Tutor never
sees
Student keeps copy.
Tutor never sees
Banerjee, J. V. (2003). Interpreting <strong>and</strong> using proficiency test scores. Unpublished doctoral dissertation. Lancaster, UK:
Lancaster University.
Basturkmen, H., & Lewis, M. (2002). Learner perspectives of success in an EAP writing course. <strong>Assessing</strong> Writing, 8(1),
31–46.
Bejar, I., Douglas, D., Jamieson, J., Nissan, S., & Turner, J. (2000). TOEFL 2000 speaking framework: A working paper.
TOEFL Research Monograph Series MS-19. Princeton, NJ: Educational Testing Service.
Blue, G. M., Milton, J., & Saville, J. (2000). <strong>Assessing</strong> English for academic purposes. Bern: Peter Lang, AG.

J. Banerjee, D. Wall / Journal of English for Academic Purposes 5 (2006) 50–69 69
Butler, F. A., Eignor, D., Jones, S., McNamara, T., & Suomi, B. K. (2000). TOEFL 2000 speaking framework: A working
paper. TOEFL Research Monograph Series MS-20. Princeton, NJ: Educational Testing Service.
Canesco, G., & Byrd, P. (1989). Writing required in graduate courses in business administration. TESOL Quarterly,
23(2), 305–316.
Carroll, B. J., & West, R. (1989). ESU framework: Performance scales for English language examinations. Harlow,
Essex: Longman.
Cho, Y. (2003). <strong>Assessing</strong> writing: Are we bound by only one method? <strong>Assessing</strong> Writing, 8(3), 165–191.
Cumming, A., Kantor, R., Powers, D., Santos, T., & Taylor, C. (2000). TOEFL 2000 writing framework: A working
paper. TOEFL research monograph series MS-18. Princeton, NJ: Educational Testing Service.
Enright, M., Grabe, W., Koda, K., Mosenthal, P., Mulcahy-Ernt, P., & Schedl, M. (2000). TOEFL 2000 reading
framework: A working paper. TOEFL research monograph series MS-17. Princeton, NJ: Educational Testing
Service.
Errey, L. (2000). Stacking the decks: What does it take to satisfy academic readers’ requirements?. In G. M. Blue, J.
Milton, & J. Saville (Eds.), <strong>Assessing</strong> English for academic purposes (pp. 147–168). Bern: Peter Lang, AG.
Geoghegan, G. (1983). Non-native speakers of English at Cambridge University. Cambridge: The Bell Educational Trust
in association with Wolfson College.
Ginther, A., & Grant, L. (1996). A review of the academic needs of native English-speaking college students in the United
States Research monograph series MS-1. Princeton, NJ: Educational Testing Service.
Hamp-Lyons, L., & Kroll, B. (1997). TOEFL 2000: Writing: Composition, community <strong>and</strong> assessment. Princeton, NJ:
Educational Testing Service.
Hartill, J. (2000). <strong>Assessing</strong> postgraduates in the real world. In G. M. Blue, J. Milton, & J. Saville (Eds.), <strong>Assessing</strong>
English for academic purposes (pp. 117–130). Bern: Peter Lang, AG.
Horowitz, D. M. (1986). What professors actually require: Academic tasks for the ESL classroom. TESOL Quarterly,
20(3), 445–462.
Jacoby, S., & McNamara, T. (1999). Locating competence. English for Specific Purposes, 18(3), 213–241.
Jamieson, J., Jones, S., Kirsch, I., Mosenthal, P., & Taylor, C. (2000). TOEFL 2000 framework: A working paper. TOEFL
research monograph series RM-00-3. Princeton, NJ: Educational Testing Service.
Lumley, T. (2002). Assessment criteria in a large-scale writing test: What do they really mean to the raters? Language
Testing, 19(3), 246–276.
Moore, T., & Morton, J. (2005). Dimensions of difference: A comparison of university writing <strong>and</strong> IELTS writing.
Journal of English for Academic Purposes, 4(1), 43–66.
Thorp, D., & Kennedy, C. (2003, April). What makes a ‘good’ IELTS answer in academic writing?. Paper presented at
the British Association of Lecturers in English for Academic Purposes (BALEAP) Conference, Southampton, UK.
Wall, D. (1981). A pre-sessional academic writing course for postgraduate students in Economics. In G. Sealy (Vol. Ed.),
Practical papers in English language education: Vol. 4 (pp. 31–105). Lancaster: Institute for English Language
Education, Lancaster University.
Wall, D., Nickson, A., Jordan, R. R., Allwright, J., & Houghton, D. (1988). Developing student writing: A subject tutor
<strong>and</strong> writing tutors compare points of view. In P. Robinson (Ed.), Academic writing: Process <strong>and</strong> product (pp. 117–
129). London: Modern English Publications in association with The British Council.
Waters, A. (1996). A review of research into needs in English for academic purposes of relevance to the North American
higher education context. TOEFL research monograph series MS-6. Princeton, NJ: Educational Testing Service.
Weigle, S. C. (1994). Effects of training on raters of ESL compositions. Language Testing, 11(2), 197–223.
Weir, C. (1983). Identifying the language needs of overseas students in tertiary education in the United Kingdom.
Unpublished doctoral dissertation, University of London, London, UK.
Dr Jayanti Banerjee is a Lecturer at Lancaster University. She has published in Language Teaching <strong>and</strong> has also
contributed chapters to edited collections such as C. Elder et al. (eds.), Experimenting with uncertainty: Essays in honour
of Alan Davies, Cambridge: CUP. Her main interests are Language Testing <strong>and</strong> Assessment <strong>and</strong> English for Academic
Purposes.
Dr Dianne Wall is a Lecturer at Lancaster University. She is co-author (with Charles Alderson <strong>and</strong> Caroline Clapham) of
Language Test Construction <strong>and</strong> Evaluation <strong>and</strong> author of The Impact of High-Stakes Testing on Classroom Teaching
(both published by Cambridge University Press). She has published articles in Language Testing, Applied Linguistics <strong>and</strong>
System.