learning - Academic Conferences Limited

Anne Jelfs and Chetz Colwell
that when working together with all types of participants we realise there is a different level of
engagement by them.
4. Putting the study into practice - the framework
Our way of working together is informed in part by the needs of the students and the conformance to
standards such as WAI (2010); BS 8878 (2010); and QAA (2010). A substantial part of this work is
conducted through our ‘expert’ walkthroughs and heuristic evaluations. The third part of our
framework is the user input to the new developing websites. Within the framework we work together,
but separately at the same points within the evaluation:
1. At the wireframe stage we will work independently reviewing the proposed sites and then come
together to report back to the developers. At this stage the Accessibility standards will also be
considered. We will review the wireframes from different student viewpoints.
2. At the ‘click through’ stage we will again work independently and report back individually to our
developer colleagues.
3. At the next stage which may not be a fully functioning site we will bring in students for our
observational studies. This will usually be 5 disabled students and 5 non-disabled students.
It could be argued that observational studies, particularly under laboratory conditions, are not the
same conditions as those the student works in at home. However, to gauge some understanding of
student use and relevant feedback on the web resources, we have found these types of studies
invaluable in our resource development. We acknowledge their limitations, but value the insight they
provide. In this section we discuss how we have attempted to alleviate as many of the problems as
possible.
We have found that student recruitment can often be problematic, especially when trying to identify
people with specific accessibility needs for our studies. Usually students are recruited via notices on
module or university based websites or through existing contacts. In conducting the studies we also
have to make additional allowances, such as how far the disabled toilet facilities are and specific
specialist requirements such as software e.g. JAWS and Dragon Naturally Speaking. There is also a
need to recruit interpreters for deaf students who do not lip read and for one evaluator to be in the
same room facing the student for those students who do lip-read in order to communicate.
The way we work is that students are invited to attend one of a number of sessions for an
observational study which is carried out in the Jennie Lee Research Labs in the Institute of
Educational Technology (IET) at the Open University. This facility has one specifically allocated
Accessibility Lab which contains additional support materials such as an adjustable desk, various
keyboards, and devices such as trackballs. There are also two other Usability Labs, a Games
Research lab, an observation room and reception room. The cameras capture human–computer
interaction, through simultaneous ‘views’ of: the user, the computer they are working at and
interactions with the application on the screen. The resulting video enables the evaluator to
concentrate on the interaction as it occurs and to have a recorded observation for future analysis. A
number of scenarios are set up for the students to complete during the observational study to
simulate some of the activities that students would be completing in their interaction with the virtual
learning environment, such as wikis, forums and assessment. The aim is to work for about 45 minutes
before a break and all participants are free to leave at any point in the evaluation.
At the start of the evaluation session, participants are informed of the purpose of the evaluation, about
the cameras and what would happen during the session. It is explained to participants that they will be
asked to think aloud when they use the website during the task completion. We also ask students to
sign a consent form allowing us to record their activities and permission to use their findings in
publications. We usually make a small payment plus travel expenses in acknowledgment of their
participation. The use of the laboratory facilities can be daunting to participants at first so we try and
create a relaxed environment and offer tea, coffee and biscuits at the start of the sessions, during any
breaks and if necessary we also offer lunch.
329