Jan Stage, Aalborg Universitet: Interaktionsdesign - IT-Vest
Jan Stage, Aalborg Universitet: Interaktionsdesign - IT-Vest Jan Stage, Aalborg Universitet: Interaktionsdesign - IT-Vest
Interaktionsdesign Jan Stage Research leader, Professor Aalborg University Department of Computer Science HCI-Lab jans@cs.aau.dk
- Page 2 and 3: Agenda • Baggrund • Usability o
- Page 4 and 5: Min baggrund • Mat-fys student, E
- Page 6 and 7: Agenda • Baggrund • Usability o
- Page 8 and 9: Usability: Hvorfor er det vigtigt?
- Page 10 and 11: Hvorfor: Dårlig usability koster
- Page 12 and 13: Undervisning i usability Der kan un
- Page 14 and 15: Method Website: hotmail.com Partici
- Page 16 and 17: Results (2) Teams Student (N=36) Pr
- Page 18 and 19: Agenda • Baggrund • Usability o
- Page 20 and 21: Aktiviteter i usability-evaluering
- Page 22 and 23: Teknik: Instant Data Analysis (IDA)
- Page 24 and 25: IDA: Results Instant Data Analysis
- Page 26 and 27: Litteratur Kjeldskov, J., Skov, M.
<strong>Interaktionsdesign</strong><br />
<strong>Jan</strong> <strong>Stage</strong><br />
Research leader, Professor<br />
<strong>Aalborg</strong> University<br />
Department of Computer Science<br />
HCI-Lab<br />
jans@cs.aau.dk
Agenda<br />
• Baggrund<br />
• Usability og usability-evaluering<br />
• Undervisning i usability<br />
• Aktiviteter og teknikker<br />
2
Agenda<br />
• Baggrund<br />
• Min baggrund<br />
• Forskningsgruppe<br />
• Usability og usability-evaluering<br />
• Undervisning i usability<br />
• Aktiviteter og teknikker<br />
3
Min baggrund<br />
• Mat-fys student, Esbjerg Statsskole, 1978<br />
• Datalog, <strong>Aalborg</strong> <strong>Universitet</strong>, 1984<br />
Speciale om Metoder til analyse, samarbejde med Mentor informatik<br />
• PhD i Informatik, <strong>Universitet</strong>et i Oslo, 1989<br />
Afhandling om Arbejdsformer for analyse og design i systemudvikling<br />
• Adjunkt-Lektor-Professor, Institut for Datalogi, <strong>Aalborg</strong> <strong>Universitet</strong>, 1988-<br />
• Gæst ved SUNY Binghamton (1991), University of Auckland (2002-2003)<br />
• Forskning inden for:<br />
• Usability-evaluering<br />
• <strong>Interaktionsdesign</strong><br />
• Objekt-orienterede metoder – OOA&D<br />
• Systemudvikling og systemudviklingsmetoder<br />
• Prototyping<br />
Institut for Datalogi<br />
4<br />
4
Human-Computer Interaction Research Group<br />
Generally:<br />
• Design and evaluation of the interaction between<br />
a user and a computerised system<br />
Two focus areas:<br />
• Design of human-computer interaction<br />
• Usability evaluation<br />
Applications:<br />
• Desktop systems<br />
• Web applications<br />
• Mobile systems<br />
Colleagues (HCI): Mikael B. Skov, Jesper Kjeldskov, Jeni Paay, Anders Bruun,<br />
6 PhD-studerende, 10-20 specialestuderende<br />
5
Agenda<br />
• Baggrund<br />
• Usability og usability-evaluering<br />
• Definition<br />
• Betydning<br />
• Undervisning i usability<br />
• Aktiviteter og teknikker<br />
6
Usability: Hvad er det og hvad er det ikke?<br />
Klassisk definition (ISO 1998):<br />
The effectiveness, efficiency and satisfaction with which a specified set of users<br />
can achieve a specified set of tasks in a particular environment<br />
Nye typer systemer (spil, underholdning<br />
osv.) og nye brugssituationer (anytime,<br />
anywhere osv.) giver anledning til<br />
nye definitioner<br />
Moderne definitioner skelner<br />
mellem tre begreber<br />
Konstruktivt mål: usability-problemer<br />
Eksempel:<br />
Booking/Ledige tider: kan ikke tvinge en akut operation ind.<br />
Utility<br />
User experience<br />
Usability<br />
7
Usability: Hvorfor er det vigtigt?<br />
Et stigende antal software-organisationer har fokus på usability<br />
Kravene til systemerne ændrer sig:<br />
• Risiko: flere systemer, hvor fejl påvirker menneskers helbred<br />
• Nødvendighed: flere systemer uden mulighed for hjælp i brugssituationen<br />
• Konkurrence: flere systemer, som er i direkte konkurrence med andre tilsvarende<br />
systemer<br />
• Kundekrav: flere systemer, hvor kunden eksplicit stiller krav til usability<br />
Systemerne udvikles:<br />
• Antallet og typen af brugere<br />
• Brugssituationen og -formålet<br />
• Kompleksiteten af software (og hardware)<br />
Mange systemer lever slet ikke op til dette<br />
8
Usability: Hvorfor er det vigtigt?<br />
Dårlig usability vedbliver med at give problemer<br />
Tiden heler ikke dårligt design<br />
• Longitudinal study of usability with EPR<br />
system: IBM IPJ 2.3 (05-2002 og 08-2003)<br />
• Evaluation with nurses<br />
2002<br />
2003<br />
2002<br />
2003<br />
2002<br />
2003<br />
9
Hvorfor: Dårlig usability koster<br />
• Evaluering af IKEA’s hjemmeside<br />
”Det er vist lettere at køre derned” (til Århus)<br />
• Evaluering af Budget Cars hjemmeside<br />
Lej en bil på 60 sekunder<br />
10 personer: 4-17 min.; snit: 7,5 min.<br />
79 usability problemer (5 / 36 / 38)<br />
• Dokumenteret, at det koster kunder<br />
10
Agenda<br />
• Baggrund<br />
• Usability og usability-evaluering<br />
• Undervisning i usability<br />
• Materiale<br />
• Erfaringer<br />
• Aktiviteter og teknikker<br />
11
Undervisning i usability<br />
Der kan undervises i det selv på<br />
introducerende niveau<br />
Giver en god forståelse for systemers<br />
brug – gerne før de lærer at<br />
konstruere<br />
Note til gymnasiet<br />
12
Training Course<br />
Teach software developers and<br />
designers to conduct usability<br />
evaluations<br />
Provide participants with skills in<br />
formative usability evaluation<br />
No prerequisites<br />
It was done in a week<br />
Result: a usability report<br />
Could be reduced to the core<br />
# Lecture Exercises<br />
1 Introduction to the course and<br />
basic website technology<br />
2 Basic introduction to usability<br />
issues and guidelines for<br />
interaction design<br />
3 The think-aloud protocol and how<br />
to set up a test scenario. User<br />
groups and their different needs<br />
4 Application of questionnaires for<br />
collecting data and how to use<br />
different kinds of questions<br />
5 Computer architecture and<br />
website technology<br />
6 Describing the usability testing<br />
method and how to collect and<br />
analyze empirical data<br />
7 Other usability evaluation<br />
methods and how to conduct a<br />
full-scale usability test session<br />
8 Website structures, information<br />
search and web surfing<br />
9 Guidelines for website design and<br />
principles for orientation and<br />
navigation<br />
10 Principles for visual design and<br />
different interaction styles<br />
Pilot test:<br />
Each team conducts simple<br />
pilot usability tests of websites<br />
to train their practical skills in<br />
usability evaluation.<br />
The teams choose the website<br />
themselves. Experience with<br />
conducting tests and the results<br />
achieved are discussed<br />
afterwards.<br />
Usability evaluation:<br />
The teams conduct a usability<br />
evaluation of the Hotmail<br />
website according to a<br />
specification provided by the<br />
course instructors.<br />
The usability evaluations are<br />
conducted at the university in<br />
assigned rooms for each team.<br />
After the usability test<br />
sessions, the teams analyze the<br />
empirical data and make a<br />
usability report that describes<br />
the identified usability<br />
problems.<br />
13
Method<br />
Website: hotmail.com<br />
Participants:<br />
• First-year university students<br />
• Four educations:<br />
• Informatics<br />
• Architecture and design<br />
• Planning and environment<br />
• Chartered surveyor<br />
Setting: their group offices<br />
Assessment based on their usability report<br />
Total<br />
number of<br />
students<br />
Total<br />
number of<br />
teams<br />
Team size<br />
Average<br />
234 36 6.5 4 / 8<br />
Number of<br />
test subjects<br />
Average<br />
Number of<br />
test subjects<br />
Min / Max<br />
Age of test<br />
subjects<br />
Average<br />
Team size<br />
Min / Max<br />
Age of test<br />
subjects<br />
Min / Max<br />
3.6 2 / 5 21,2 19 / 30<br />
14
Results (1)<br />
Teams<br />
Student<br />
(N=36)<br />
Professional<br />
(N=8)<br />
Conducting the<br />
evaluation<br />
Evaluation<br />
Task quality and<br />
relevance<br />
The students did quite well in conducting the evaluation<br />
The professionals did significantly better<br />
Questionnaire/<br />
Interviews<br />
3.42 (0.73) 3.22 (1.05) 2.72 (1.00)<br />
4.38 (0.74) 3.13 (1.64) 3.50 (1.69)<br />
On task quality and relevance the students seem to do better than the<br />
professionals (but not significant)<br />
15
Results (2)<br />
Teams<br />
Student<br />
(N=36)<br />
Professional<br />
(N=8)<br />
Test<br />
description<br />
Data quality<br />
Clarity of<br />
problem list<br />
Report<br />
Executive<br />
summary<br />
Clarity of<br />
report<br />
The students did well on describing the test and providing underlying data in<br />
appendices<br />
Layout of<br />
report<br />
3.03 (0.94) 3.19 (1.33) 2.53 (1.00) 2.39 (0.80) 2.97 (0.84) 2.94 (0.89)<br />
4.00 (1.31) 2.13 (0.83) 3.50 (0.93) 3.38 (1.06) 4.25 (0.71) 3.25 (0.71)<br />
The worst student performance was in clarity of the problem list and executive<br />
summary<br />
16
Team<br />
Results (3)<br />
Student<br />
(N=36)<br />
Professional<br />
(N=8)<br />
Number of<br />
problems<br />
Problem<br />
categorization<br />
Practical<br />
relevance<br />
Qualitative<br />
results<br />
overview<br />
Results<br />
Quantitative<br />
results<br />
overview<br />
Use of<br />
literature<br />
The students did poorly on problem identification and description<br />
Conclusion<br />
Evaluation<br />
of test<br />
2.56 (0.84) 2.06 (1.22) 3.03 (1.00) 3.03 (1.00) 2.28 (1.14) 3.08 (0.81) 2.64 (0.90) 2.44 (1.08)<br />
4.13 (1.13) 3.25 (1.75) 4.25 (1.49) 3.75 (1.16) 2.00 (1.51) 3.13 (0.35) 3.88 (0.64) 2.88 (1.13)<br />
All did poorly in describing the quantitative results (efficiency and effectiveness)<br />
17
Agenda<br />
• Baggrund<br />
• Usability og usability-evaluering<br />
• Undervisning i usability<br />
• Aktiviteter og teknikker<br />
• <strong>Interaktionsdesign</strong><br />
• Aktiviteter i usability-evaluering<br />
• Teknik til usability-evaluering<br />
18
<strong>Interaktionsdesign</strong><br />
Aktiviteter og samspil i interaktionsdesign (HCI-design)<br />
Udgangspunkt<br />
Generer<br />
design<br />
Identificer behov<br />
Etabler krav<br />
Byg interaktiv<br />
version<br />
Evaluer<br />
design<br />
Resultat:<br />
Endeligt<br />
produkt<br />
Alternativ start:<br />
• Byg noget og prøv<br />
det af<br />
• Lav flere design af<br />
et system<br />
• Evaluér noget<br />
eksisterende<br />
19
Aktiviteter i usability-evaluering<br />
Evaluation plan:<br />
• Purpose<br />
• Key questions<br />
• User profile<br />
• Participants and roles<br />
• Test methods<br />
• Task list<br />
• Context and<br />
equipment<br />
• Data to be collected<br />
• Report structure<br />
Each user:<br />
• Introduction<br />
• Task solving<br />
• Data collection<br />
• Logging<br />
• Debriefing<br />
Determine<br />
basics<br />
Plan<br />
evaluation<br />
Prepare<br />
tests<br />
Conduct<br />
tests<br />
Interpret<br />
data<br />
• Form: formative/summative<br />
• When in development process:<br />
• Exploratory<br />
• Assessment<br />
• Validation<br />
• Comparison<br />
• Participants and roles<br />
• Location and equipment<br />
• Recreation of context<br />
• Selection of test subjects<br />
• Design of tasks<br />
• Transcription of log files<br />
• Data summary<br />
• Data analysis<br />
• Documentation (report)<br />
20
Udfordring: Tidsforbrug til usability-evaluering<br />
• Rolf Molich: CUE-2 projektet (http://www.dialogdesign.dk/cue.html)<br />
• 8 grupper, men ikke alle opsamlede systematisk data om tidsforbrug<br />
A B C D E F G H J Gennemsnit<br />
Forberedelse 37 40 57 10 34 70 41<br />
Testing 43 36 9 10 18 18 72 29<br />
Analyse og rapportering 56 47 18 30 55 27 76 44<br />
Samlet tidsforbrug 136 123 84 130 50 107 46 218 112<br />
• Andre tal: 100-150 timer for en total evaluering med 6-8 brugere og<br />
systematisk video-baseret dataanalyse<br />
• Varighed: 4-6 uger<br />
21
Teknik: Instant Data Analysis (IDA)<br />
Conduct 4-6 standard think-aloud sessions. A<br />
test monitor and a data logger are present<br />
Conduct a 1-1½ hour data analysis session:<br />
based on brainstorming and systematic<br />
discussion. Organized by a facilitator<br />
•Brainstorm<br />
• Screen printouts<br />
• Notes<br />
Let the facilitator spend 1-1½ hours on writing up<br />
the content of the whiteboard into a ranked list of<br />
usability problems with clear references to the<br />
system<br />
Review the problem list for final consensus<br />
22
IDA: Experiment<br />
We studied the use of Instant Data Analysis<br />
through an exploratory experiment<br />
Purpose<br />
• Gaining practical experience with the use of the technique<br />
• Comparing results produced “instantly” with results from traditional video<br />
data analysis<br />
• Identifying opportunities and challenges<br />
for improving IDA<br />
The system: resource booking at a large hospital<br />
Participants<br />
• 5 test subjects<br />
• 1 test monitor + 1 data logger<br />
• 1 IDA facilitator<br />
Observer<br />
Observation<br />
Room<br />
Subject Room<br />
Test subject<br />
Test monitor<br />
Data logger<br />
and video<br />
equipment<br />
operator<br />
Control<br />
Room<br />
23
IDA: Results<br />
Instant Data Analysis can …<br />
• Assist usability evaluators in quickly identifying the most critical and<br />
serious usability problems experienced by users in a think-aloud<br />
evaluation<br />
• Was conducted in 10% of the time it takes to do a traditional video data<br />
analysis<br />
• Reduce the noise of unique (false?) usability problems<br />
24
Afslutning<br />
• Baggrund<br />
• Min baggrund<br />
• Forskningsgruppe<br />
• Usability og usability-evaluering<br />
• Definition<br />
• Betydning<br />
• Systemudvikling<br />
• Undervisning i usability<br />
• Materiale<br />
• Erfaringer<br />
• Aktiviteter og teknikker<br />
• <strong>Interaktionsdesign</strong><br />
• Aktiviteter i usability-evaluering<br />
• Teknik til usability-evaluering<br />
25
Litteratur<br />
Kjeldskov, J., Skov, M. B. and <strong>Stage</strong>, J. (2012) <strong>Interaktionsdesign</strong>.<br />
http://iftek.dk/interaktionsdesign<br />
Kjeldskov, J., Skov, M. B. and <strong>Stage</strong>, J. (2004) Instant Data Analysis: Evaluating Usability<br />
in a Day. Proceedings of NordiCHI 2004, pp. 233-240. ACM.<br />
Kjeldskov, J., Skov, M. B. and <strong>Stage</strong>, J. (2010) A Longitudinal Study of Usability in Health<br />
Care: Does Time Heal? International Journal of Medical Informatics, 79(6):135-143.<br />
Skov, M. B. and <strong>Stage</strong>, J. (2009) A Conceptual Tool for Usability Problem Identification in<br />
Website Development. International Journal of Information Technology and Web<br />
Engineering, 4(4):22-35.<br />
Skov, M. B. and <strong>Stage</strong>, J. (2011) Training Software Developers and Designers to Conduct<br />
Usability Evaluations. Behaviour and Information Technology (in press, available<br />
online, DOI: 10.1080/01449290903398208).<br />
26
<strong>IT</strong>-<strong>Vest</strong>: Master i <strong>IT</strong> for <strong>IT</strong>-lærere<br />
Fagpakke i <strong>Interaktionsdesign</strong> og Usability-evaluering<br />
• <strong>Interaktionsdesign</strong><br />
• Metoder til usability-evaluering<br />
• Evaluering og design i praksis<br />
Link:<br />
http://www.master-it-vest.dk/fagpakker/interaktionsdesign-og-multimedier/detaljer/fagpakke/interaktionsdesign-og-usability-evaluering/2.html#.UGPv91HQys0<br />
27