Guenter Hilmer - Phd Thesis - Glamorgan Dspace - University of ...

6.2 Analysis of the manual markup task 6 INTERVIEW EVALUATION
6.2 Analysis of the manual markup task
It was not only the interviews of the test candidates that provided insight into the
3-Layer-Model and the created framework, but also the candidates’ observations
during their work on the markup tasks (cf. section 5.2).
The first markup task, the manual markup of the provided material, produced the
following results.
6.2.1 Provided material
It was observed that the provided material was not immediately comprehensible
to the test candidates. This problem was mainly based on the fact that the material
was provided by another person and from an unfamiliar knowledge domain. The
problems were more of a content based kind than of an educational type.
In general it can be said that it was difficult for the test candidates to markup
material that existed on paper and which had not been created by themselves in
the first place.
6.2.2 EFTECS and 3-Layer-Model
It was necessary for most of the test candidates to get a second explanation of
the 3-Layer-Model during their first steps through the markup tasks. The educa-
tional concept of the 3-Layer-Model was not immediately comprehensible to the
candidates. However, after a second explanation of the 3-Layer-Model and the
framework every candidate of the evaluation was able to carry out the given tasks
as designed in section 5.2.
6.2.3 Educational experts
The educational experts seemed to have no major problems with the given markup
tasks. After a short while they felt very confident about the given tasks and worked
right through them. However, it was observed that every educational expert put
151