learning - Academic Conferences Limited

Tim Cappelli
In order to determine this process, the team carried out a series of interviews with each of the relevant
stakeholders and in doing so also identified issues inherent in the process that stakeholders wanted
to improve or eliminate. From this a set of key objectives were articulated, namely:
Provision of faster results and feedback to students.
Reduction of the opportunities for errors in data handling and erroneous results.
Provision of timely textual feedback to all students.
Reduction of time-pressured manual handling of the marking forms.
Less labour-intensive and faster preparation of forms for the different OSCE exams with less
manual form preparation.
More efficient and faster preparation and transmission of data for analysis by the different roles
involved in administration, management and quality assurance.
3. The solution
According to Love and Cooper (2004), assessment systems should consider the interface,
accessibility, security, reliability and the information to be captured. In considering a potential
technology solution, it was therefore important to consider these issues alongside the system’s ability
to achieve the pilot’s objectives. In particular, the following features where considered crucial:
Data security: transmission of data must be secure and storage of exam scenarios, forms and
results is also secure.
Performance: OSCE exams run on a tight time schedule. The system must not introduce risk of
time delays due to slow response times.
Connectivity: the system must offer solutions for the communications infrastructure that is in
place in the hospitals. Wireless internet availability or provision for wired networks cannot be
guaranteed.
Data integrity: it must be ensured that no data is lost in transmission and that there can be no
mix-up between fields in a form or between forms. It would be an advantage if mandatory fields
could be checked in real-time so that only complete forms are submitted to the system.
Scalability: the system must be able to support the volume of forms needed in the annual cycle
of OSCE exams.
Human Computer Interaction: the examiners’ interaction with the system must be intuitive and
not alter the marking process for the examiner. For example, the examiner must be able to amend
his marks on a form prior to submission.
Training: the device chosen must also be intuitive enough to avoid the need for lengthy or
expensive training for examiners.
Robustness: the chosen technology must be robust and reliable, with a failsafe backup system,
as any delay in the OSCE exam is untenable.
When the team looked at potential technology, there emerged two principal approaches to supporting
the OSCE process. These were:
Hand held devices together with an electronic forms software application. The types of devices
considered were:
Mobile phone
iPad
Tablet PC
Netbooks
Digital Pens together with form management software.
Digitised forms and digital pens
Digital pen and sensor with non-digitised forms
Preliminary research into the technologies and their suitability to support the above process indicated
that this list could be narrowed down still further. The consensus was that the screens available on
mobile phones are too small for marking sheets, allowing examiners to see only a fraction of the sheet
93