Anais do IHC'2001 - Departamento de Informática e Estatística - UFSC

76
Anais do IHC’2001 - IV Workshop sobre Fatores Humanos em Sistemas Computacionais
were not able to complete all three tasks). Some web pages displayed poorly on low
resolution screens, requiring horizontal scrolling, and there was lack of format consistency
between some of them.
To compare the selected evaluation methods, the criteria previously mentioned
were analyzed in detail. In order to easily identify the evaluation groups, the following
acronyms will be used throughout this paper:
Table 1 - Evaluation groups.
Non-experts
ISO 9241-10 group Non_E_ISO
Control group Non_E_Cntl
Ergonomic Criteria group Non_E_EC
Heuristic Evaluation group Non_E_Heu
Usability Index group
Software engineers
Non_E_Index
Heuristic Evaluation group SE_Heu
Usability Index group
Usability experts
SE_Index
Heuristic Evaluation group UE_Heu
Usability Index group UE_Index
Average evaluation time
Comparing the time spent by all evaluation groups, it was noticed that non-experts took
more time to complete this activity than software engineers or usability experts. The largest
average time (4:27h) was spent by the EC non-expert group, while the shortest one (1:11h)
was that of the usability expert group using the Heuristics method. Among the Index
groups, software engineers were the fastest group (avg time = 1:54h).
The time spent by the researcher was 46% smaller with software engineers and
usability experts, in comparison to non-expert groups, because it was unnecessary to be
present during evaluation sessions, and to correct evaluation form fields during data
analysis of expert groups results.
No usability expert or software engineer asked the researcher to solve any doubts
about the evaluation process, and no filling errors were identified on their evaluation forms.
Allied to their previous computer and/or usability knowledge, the evaluation guide, sent by
e-mail to these groups, was considered clear enough for their understanding of the steps to
be followed.
Number and type of identified usability problems
Table 2 shows the distribution of usability problems found by all groups. The predominant
types of usability problems were: 'Error prevention', 'Visibility, recognition and
conduction', and 'Flexibility and efficiency of use'. Software engineers (SE) and usability
experts (UE) using the Index method were the groups that identified more of these types of