Jan Stage, Aalborg Universitet: Interaktionsdesign - IT-Vest

Interaktionsdesign
Jan Stage
Research leader, Professor
Aalborg University
Department of Computer Science
HCI-Lab
jans@cs.aau.dk

Agenda
• Baggrund
• Usability og usability-evaluering
• Undervisning i usability
• Aktiviteter og teknikker
2

Agenda
• Baggrund
• Min baggrund
• Forskningsgruppe
• Usability og usability-evaluering
• Undervisning i usability
• Aktiviteter og teknikker
3

Min baggrund
• Mat-fys student, Esbjerg Statsskole, 1978
• Datalog, Aalborg Universitet, 1984
Speciale om Metoder til analyse, samarbejde med Mentor informatik
• PhD i Informatik, Universitetet i Oslo, 1989
Afhandling om Arbejdsformer for analyse og design i systemudvikling
• Adjunkt-Lektor-Professor, Institut for Datalogi, Aalborg Universitet, 1988-
• Gæst ved SUNY Binghamton (1991), University of Auckland (2002-2003)
• Forskning inden for:
• Usability-evaluering
• Interaktionsdesign
• Objekt-orienterede metoder – OOA&D
• Systemudvikling og systemudviklingsmetoder
• Prototyping
Institut for Datalogi
4
4

Human-Computer Interaction Research Group
Generally:
• Design and evaluation of the interaction between
a user and a computerised system
Two focus areas:
• Design of human-computer interaction
• Usability evaluation
Applications:
• Desktop systems
• Web applications
• Mobile systems
Colleagues (HCI): Mikael B. Skov, Jesper Kjeldskov, Jeni Paay, Anders Bruun,
6 PhD-studerende, 10-20 specialestuderende
5

Agenda
• Baggrund
• Usability og usability-evaluering
• Definition
• Betydning
• Undervisning i usability
• Aktiviteter og teknikker
6

Usability: Hvad er det og hvad er det ikke?
Klassisk definition (ISO 1998):
The effectiveness, efficiency and satisfaction with which a specified set of users
can achieve a specified set of tasks in a particular environment
Nye typer systemer (spil, underholdning
osv.) og nye brugssituationer (anytime,
anywhere osv.) giver anledning til
nye definitioner
Moderne definitioner skelner
mellem tre begreber
Konstruktivt mål: usability-problemer
Eksempel:
Booking/Ledige tider: kan ikke tvinge en akut operation ind.
Utility
User experience
Usability
7

Usability: Hvorfor er det vigtigt?
Et stigende antal software-organisationer har fokus på usability
Kravene til systemerne ændrer sig:
• Risiko: flere systemer, hvor fejl påvirker menneskers helbred
• Nødvendighed: flere systemer uden mulighed for hjælp i brugssituationen
• Konkurrence: flere systemer, som er i direkte konkurrence med andre tilsvarende
systemer
• Kundekrav: flere systemer, hvor kunden eksplicit stiller krav til usability
Systemerne udvikles:
• Antallet og typen af brugere
• Brugssituationen og -formålet
• Kompleksiteten af software (og hardware)
Mange systemer lever slet ikke op til dette
8

Usability: Hvorfor er det vigtigt?
Dårlig usability vedbliver med at give problemer
Tiden heler ikke dårligt design
• Longitudinal study of usability with EPR
system: IBM IPJ 2.3 (05-2002 og 08-2003)
• Evaluation with nurses
2002
2003
2002
2003
2002
2003
9

Hvorfor: Dårlig usability koster
• Evaluering af IKEA’s hjemmeside
”Det er vist lettere at køre derned” (til Århus)
• Evaluering af Budget Cars hjemmeside
Lej en bil på 60 sekunder
10 personer: 4-17 min.; snit: 7,5 min.
79 usability problemer (5 / 36 / 38)
• Dokumenteret, at det koster kunder
10

Agenda
• Baggrund
• Usability og usability-evaluering
• Undervisning i usability
• Materiale
• Erfaringer
• Aktiviteter og teknikker
11

Undervisning i usability
Der kan undervises i det selv på
introducerende niveau
Giver en god forståelse for systemers
brug – gerne før de lærer at
konstruere
Note til gymnasiet
12

Training Course
Teach software developers and
designers to conduct usability
evaluations
Provide participants with skills in
formative usability evaluation
No prerequisites
It was done in a week
Result: a usability report
Could be reduced to the core
# Lecture Exercises
1 Introduction to the course and
basic website technology
2 Basic introduction to usability
issues and guidelines for
interaction design
3 The think-aloud protocol and how
to set up a test scenario. User
groups and their different needs
4 Application of questionnaires for
collecting data and how to use
different kinds of questions
5 Computer architecture and
website technology
6 Describing the usability testing
method and how to collect and
analyze empirical data
7 Other usability evaluation
methods and how to conduct a
full-scale usability test session
8 Website structures, information
search and web surfing
9 Guidelines for website design and
principles for orientation and
navigation
10 Principles for visual design and
different interaction styles
Pilot test:
Each team conducts simple
pilot usability tests of websites
to train their practical skills in
usability evaluation.
The teams choose the website
themselves. Experience with
conducting tests and the results
achieved are discussed
afterwards.
Usability evaluation:
The teams conduct a usability
evaluation of the Hotmail
website according to a
specification provided by the
course instructors.
The usability evaluations are
conducted at the university in
assigned rooms for each team.
After the usability test
sessions, the teams analyze the
empirical data and make a
usability report that describes
the identified usability
problems.
13

Method
Website: hotmail.com
Participants:
• First-year university students
• Four educations:
• Informatics
• Architecture and design
• Planning and environment
• Chartered surveyor
Setting: their group offices
Assessment based on their usability report
Total
number of
students
Total
number of
teams
Team size
Average
234 36 6.5 4 / 8
Number of
test subjects
Average
Number of
test subjects
Min / Max
Age of test
subjects
Average
Team size
Min / Max
Age of test
subjects
Min / Max
3.6 2 / 5 21,2 19 / 30
14

Results (1)
Teams
Student
(N=36)
Professional
(N=8)
Conducting the
evaluation
Evaluation
Task quality and
relevance
The students did quite well in conducting the evaluation
The professionals did significantly better
Questionnaire/
Interviews
3.42 (0.73) 3.22 (1.05) 2.72 (1.00)
4.38 (0.74) 3.13 (1.64) 3.50 (1.69)
On task quality and relevance the students seem to do better than the
professionals (but not significant)
15

Results (2)
Teams
Student
(N=36)
Professional
(N=8)
Test
description
Data quality
Clarity of
problem list
Report
Executive
summary
Clarity of
report
The students did well on describing the test and providing underlying data in
appendices
Layout of
report
3.03 (0.94) 3.19 (1.33) 2.53 (1.00) 2.39 (0.80) 2.97 (0.84) 2.94 (0.89)
4.00 (1.31) 2.13 (0.83) 3.50 (0.93) 3.38 (1.06) 4.25 (0.71) 3.25 (0.71)
The worst student performance was in clarity of the problem list and executive
summary
16

Team
Results (3)
Student
(N=36)
Professional
(N=8)
Number of
problems
Problem
categorization
Practical
relevance
Qualitative
results
overview
Results
Quantitative
results
overview
Use of
literature
The students did poorly on problem identification and description
Conclusion
Evaluation
of test
2.56 (0.84) 2.06 (1.22) 3.03 (1.00) 3.03 (1.00) 2.28 (1.14) 3.08 (0.81) 2.64 (0.90) 2.44 (1.08)
4.13 (1.13) 3.25 (1.75) 4.25 (1.49) 3.75 (1.16) 2.00 (1.51) 3.13 (0.35) 3.88 (0.64) 2.88 (1.13)
All did poorly in describing the quantitative results (efficiency and effectiveness)
17

Agenda
• Baggrund
• Usability og usability-evaluering
• Undervisning i usability
• Aktiviteter og teknikker
• Interaktionsdesign
• Aktiviteter i usability-evaluering
• Teknik til usability-evaluering
18

Interaktionsdesign
Aktiviteter og samspil i interaktionsdesign (HCI-design)
Udgangspunkt
Generer
design
Identificer behov
Etabler krav
Byg interaktiv
version
Evaluer
design
Resultat:
Endeligt
produkt
Alternativ start:
• Byg noget og prøv
det af
• Lav flere design af
et system
• Evaluér noget
eksisterende
19

Aktiviteter i usability-evaluering
Evaluation plan:
• Purpose
• Key questions
• User profile
• Participants and roles
• Test methods
• Task list
• Context and
equipment
• Data to be collected
• Report structure
Each user:
• Introduction
• Task solving
• Data collection
• Logging
• Debriefing
Determine
basics
Plan
evaluation
Prepare
tests
Conduct
tests
Interpret
data
• Form: formative/summative
• When in development process:
• Exploratory
• Assessment
• Validation
• Comparison
• Participants and roles
• Location and equipment
• Recreation of context
• Selection of test subjects
• Design of tasks
• Transcription of log files
• Data summary
• Data analysis
• Documentation (report)
20

Udfordring: Tidsforbrug til usability-evaluering
• Rolf Molich: CUE-2 projektet (http://www.dialogdesign.dk/cue.html)
• 8 grupper, men ikke alle opsamlede systematisk data om tidsforbrug
A B C D E F G H J Gennemsnit
Forberedelse 37 40 57 10 34 70 41
Testing 43 36 9 10 18 18 72 29
Analyse og rapportering 56 47 18 30 55 27 76 44
Samlet tidsforbrug 136 123 84 130 50 107 46 218 112
• Andre tal: 100-150 timer for en total evaluering med 6-8 brugere og
systematisk video-baseret dataanalyse
• Varighed: 4-6 uger
21

Teknik: Instant Data Analysis (IDA)
Conduct 4-6 standard think-aloud sessions. A
test monitor and a data logger are present
Conduct a 1-1½ hour data analysis session:
based on brainstorming and systematic
discussion. Organized by a facilitator
•Brainstorm
• Screen printouts
• Notes
Let the facilitator spend 1-1½ hours on writing up
the content of the whiteboard into a ranked list of
usability problems with clear references to the
system
Review the problem list for final consensus
22

IDA: Experiment
We studied the use of Instant Data Analysis
through an exploratory experiment
Purpose
• Gaining practical experience with the use of the technique
• Comparing results produced “instantly” with results from traditional video
data analysis
• Identifying opportunities and challenges
for improving IDA
The system: resource booking at a large hospital
Participants
• 5 test subjects
• 1 test monitor + 1 data logger
• 1 IDA facilitator
Observer
Observation
Room
Subject Room
Test subject
Test monitor
Data logger
and video
equipment
operator
Control
Room
23

IDA: Results
Instant Data Analysis can …
• Assist usability evaluators in quickly identifying the most critical and
serious usability problems experienced by users in a think-aloud
evaluation
• Was conducted in 10% of the time it takes to do a traditional video data
analysis
• Reduce the noise of unique (false?) usability problems
24

Afslutning
• Baggrund
• Min baggrund
• Forskningsgruppe
• Usability og usability-evaluering
• Definition
• Betydning
• Systemudvikling
• Undervisning i usability
• Materiale
• Erfaringer
• Aktiviteter og teknikker
• Interaktionsdesign
• Aktiviteter i usability-evaluering
• Teknik til usability-evaluering
25

Litteratur
Kjeldskov, J., Skov, M. B. and Stage, J. (2012) Interaktionsdesign.
http://iftek.dk/interaktionsdesign
Kjeldskov, J., Skov, M. B. and Stage, J. (2004) Instant Data Analysis: Evaluating Usability
in a Day. Proceedings of NordiCHI 2004, pp. 233-240. ACM.
Kjeldskov, J., Skov, M. B. and Stage, J. (2010) A Longitudinal Study of Usability in Health
Care: Does Time Heal? International Journal of Medical Informatics, 79(6):135-143.
Skov, M. B. and Stage, J. (2009) A Conceptual Tool for Usability Problem Identification in
Website Development. International Journal of Information Technology and Web
Engineering, 4(4):22-35.
Skov, M. B. and Stage, J. (2011) Training Software Developers and Designers to Conduct
Usability Evaluations. Behaviour and Information Technology (in press, available
online, DOI: 10.1080/01449290903398208).
26

IT-Vest: Master i IT for IT-lærere
Fagpakke i Interaktionsdesign og Usability-evaluering
• Interaktionsdesign
• Metoder til usability-evaluering
• Evaluering og design i praksis
Link:
http://www.master-it-vest.dk/fagpakker/interaktionsdesign-og-multimedier/detaljer/fagpakke/interaktionsdesign-og-usability-evaluering/2.html#.UGPv91HQys0
27