Heuristic Evaluation (lab 8)

Heuristic Evaluation (lab 8)
EEE459 fall 2012
Contents
1 Due date and submission
2 Goal
3 Requirement
3.1 Heuristic evaluation
3.2 Consolidated heuristic list
3.3 Written report
3.3.1 Questions regarding the inspection process
3.3.2 Question regarding your follow-up
4 Example heuristic violation report
1 Due date and submission
It's a little complicated this week:
• as you complete your heuristic violation reviews in the lab today, you must immediate email
them to the members of the group whose product you're reviewing, cc: me (mailto:greg.phillips@rmc.ca).
• your written lab report (in PDF format) and heuristic violation spreadsheet (in MS Excel format)
must be submitted by to me (mailto:greg.phillips@rmc.ca) not later than 0800 hrs Monday 26 November
2012.
• your revised prototype must be ready to demonstrate to the course not later than the lab
period at 14h40, Monday 26 November 2012.
2 Goal
The goal of this lab is to allow you to practice the heuristic evaluation usability inspection technique.
In the process of doing this, you will also be providing feedback to other lab groups and
improving your own prototype based on the feedback from the other lab groups.
3 Requirement
In this lab, you will perform a heuristic evaluation on each of the other two lab groups' designs.
This will be done individually.
You will receive the heuristic evaluation reports from the members of the other groups who evaluated
your product. You will be required to consolidate the reports, assign severity ratings, and
modify your prototype to account for the identified defects.
3.1 Heuristic evaluation
This section is to be done individually.
Carefully follow the procedure described in the heuristic evaluation overview, using Jakob
Nielsen's ten heuristics. Record all the heuristic violations that you identify using the provided MS
Excel template.
Each violation report must include a sequential problem ID, a list of the heuristic(s) violated (identified
by heuristic number), and a description of where the problem occurs and what it is. See the
Example heuristic violation report section below.
In your initial submissions, do not include severity ratings.
1 of 3

As soon as you have completed a review of another groups' design, email your violation spreadsheet,
to all the members of that group, cc: me (mailto:greg.phillips@rmc.ca).
3.2 Consolidated heuristic list
This section is to be done in your lab groups.
You should receive four or five heuristic evaluation reports on your groups' design. Merge these
into a single spreadsheet, consolidating and combining any reported violations that are duplicates.
Be intelligent about this! You may need to rewrite the violation descriptions to some extent.
Assign severity ratings to each reported problem.
3.3 Written report
Complete a written lab report in your lab groups. The report must represent a consolidated consensus
in your group and must include an introduction, answers to the questions shown below,
and a conclusion.
3.3.1 Questions regarding the inspection process
1. During the inspections, did the written heuristics allow you to identify usability problems that
you feel you otherwise might have missed If yes, give one example, explaining how the
heuristics helped. If not, explain why you don't believe that the heuristics helped you.
2. Did you find any usability problems in the interface that did not appear to correspond to any
of the heuristics If yes, give an example. If not, explain (speculate) as to why this was so.
3. How confident are you that all the heuristic violations you identified are actually usability
problems Explain your degree of confidence.
4. How confident are you that you identified all the usability problems in the interface Explain
your degree of confidence.
3.3.2 Question regarding your follow-up
5. Explain which heuristic violations you chose to address, and which ones you did not. Justify
your decision in each case.
4 Example heuristic violation report
An example heuristic violation report (for Apple's iCal program) is shown in the table below.
Problem
ID
Heuristic(s)
violated
Problem location and description
Severity
(0-4)
42 7 When entering information into the event editing pane
using the keyboard, on some fields text focus does not
move to the next logical element after the user makes an
entry. For example, to make an event repeat for four
weeks, the user presses [tab] until the "repeat" field is
highlighted, presses [space] to activate the selection box,
presses [down] twice to highlight "every week", and
presses [enter] to select it. This causes a new field "end"
to appear immediately below "repeat" and logically the
"end" selection box should now be active. However, text
focus stays on the "repeat" entry field, requiring the user
2 of 3

Problem
ID
Heuristic(s)
violated
Problem location and description
Severity
(0-4)
to press [tab] to mode to the "end" field. This behaviour is
common to the following event detail fields: "all day", the
minutes fields in "from" and "to", "repeat", repeat "end",
"calendar", "alarm", and alarm "how much before".
Things to note about this report and to emulate in your own:
• Here only one heuristic was violated: number 7, "flexibility and efficiency of use." You may
find that some problems are simultaneous violations of more than one heuristic, in which
case you should list them like, e.g., "2, 5, 7".
• The problem description clearly indicates where on the interface the problem occurs and in
what circumstance, and what the problem is.
• The problem identified here actually occurs at multiple points in the interface; rather than
raise a separate violation for each, the problem is described once and the places it occurs
are listed.
• The problem has a real impact on usability and is therefore more than “cosmetic”; however,
the interface is still fairly usable even with the defect, so it probably doesn't rate a “major”
severity level. We have not yet assigned it a severity rating; this happens later in the process.
— Greg Phillips (http://last3.in/greg.html)
From http://last3.in © Greg Phillips. Unlimited use is permitted within the Government of Canada. All other use
is governed by the Creative Commons by-nc 3.0 license; see http://last3.in/colophon.html for details.
3 of 3