DEliverable 2.3 - the School of Engineering and Design - Brunel ...

ICT Project 3D VIVANT– Deliverable 2.3
Contract no.:
248420
User Acceptance Validation Plan
The most comprehensive source on how to actually conduct usability tests, which describes in all
details the way the partners have organized usability tests in recent years, seems to be Rubin and
Chisnell’s “Handbook of Usability Testing” (2008). The envisaged validation tests will try and
investigate differences in perceived usefulness and usability with respect to gender, age and prior
experience with IT, in order to cater for varying expectations in a large (potential) audience as that of
the broadcasting partners in the project.
2.4.1 User Acceptance and Usability
The key aim of the user acceptance or usability testing is to evaluate the efficiency and effectiveness
of the system. As the user interacts with a system via some form of a user interface (typically
graphical), usability testing is closely associated with the study of interaction between users and
computers.
The user acceptance of the proposed use cases (the Broadcast TV as well as the Online Hyperlinking
use cases) will be tested according to the following four criteria:
• Usability – Whether the system can be operated with ease and efficiency.
• Usefulness – Whether the system allows the user to complete the relevant tasks successfully.
• Intuitiveness – Whether the system and its interface components (navigation, etc.) can be easily
recognised.
• Learnability – Whether the system can be mastered quickly, minimizing the user’s learning
curve.
User acceptance testing will involve a mixed qualitative methodology.
The user acceptance testing refers to the testing and evaluation of the developed use cases, namely the
Broadcast TV and Online Hyperlinking use cases as well as the 3D graphical user interface (3D GUI)
that will be developed as part of these use cases. The user acceptance testing will involve both
professional as well as end-users.
2.4.2 Data Collection Methods
The following data collection methods will be utilised to gather user feedback. The individual
sections on the technologies (Section 3 and Section 4) to be tested will name applicable methods
more precisely.
• Questionnaires – Comprising of both closed and open ended questions.
• Interviews – Semi-structured interviews comprising of a selected user sample and one/two
interviewees.
• Task-centred testing – Users fulfil prepared tasks without help from experts. Their methods and
success of fulfilling these are being watched and documented.
• Thinking aloud method –Users tell the tester what they are thinking, while fulfilling tasks to
give designers and developers an idea of what users would have expected.
The user acceptance methodology will be employed for the testing of both components of the
interactive experience testing as discussed and detailed in Section 4.2.1 for the hyperlinking
(particularly questionnaires, task-centred testing, the thinking aloud method and observations) and
Section 4.2.2 for the search and retrieval framework (particularly questionnaires, task-centred testing,
the thinking aloud method and observations).
Focus groups and interviews will be employed primarily for the production-side testing as described
01.09.11 14