Anais do IHC'2001 - Departamento de Informática e Estatística - UFSC

Anais do IHC’2001 - IV Workshop sobre Fatores Humanos em Sistemas Computacionais 73
Comparing usability evaluation methods applied to
corporate web portals
Cláudia Dias
Depto de Ciência da Informação e Documentação - Universidade de Brasília
AOS 04 Bloco A apt 522 - Brasília - 70660-041, Brazil
claudiaad@yahoo.com
Abstract. The variety of usability evaluation methods, the need to adapt them to
the system's context of use, and the technological evolution of information
management - the corporate web portal - suggested the research problem of
comparing usability evaluation methods, applied to corporate portals’ context of
use. The goal of the present study was first to compare four usability evaluation
methods, when employed by non-expert people, and second to verify the influence
of the evaluators' experience and abilities on the results of usability evaluations
of web portals.
Keywords: usability evaluation, corporate portal, web interface, humancomputer
interaction
Introduction
In order to compare usability evaluation methods applied to corporate web portals, the
context of use of the sample portal, selected for this study, was first analyzed, identifying
its typical users, their characteristics, the tasks they seek to achieve, and their
organizational and computing environments. After that, during the literature review about
usability and human-computer interaction, 17 usability evaluation methods were selected:
eight inspection methods and nine user testing methods.
To choose the methods that would be compared in this study, 11 criteria were
established: procedure and typical location of data collection, data type and source,
development phase and status, usability factors measured by the method, cost of
evaluation, reliability of measurements, technical support needed, and additional costs, like
users' involvement, materials, training. Among the evaluation methods pre-selected, four
inspection methods were chosen, according to these criteria:
• a standard for the ergonomy and usability of information technology products - the ISO
9241-10 Dialogue principles (ISO, 1996);
• an ergonomic design guideline - the Ergonomic Criteria (Bastien & Scapin, 1993);
• a checklist - the Usability Index Checklist for Websites (Keevil, 1998); and
• the heuristic evaluation, whose support tool - the Corporate Portal Heuristics (Dias,
2001) - was elaborated for this research, based on the context of use of the sample
corporate portal and on several principles or heuristics published by usability authors.
To compare not only the quality of the chosen methods, but also the effect of the
evaluators' experience and abilities, the usability inspection of the sample corporate portal