HumanâRobot Interaction - LASA - EPFL
HumanâRobot Interaction - LASA - EPFL
HumanâRobot Interaction - LASA - EPFL
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
© CORBIS & JOHN FOXX<br />
An Exclusive Course for<br />
Computer Scientists and Engineers<br />
The field of human–robot interaction (HRI)<br />
addresses the design, understanding, and evaluation<br />
of robotic systems, which involve humans and<br />
robots interacting through communication [1]. As<br />
the field matures, education of students<br />
becomes increasingly important.<br />
Courses in HRI provide the canonical<br />
set of knowledge and core skills<br />
that represent the current state of the<br />
field and permit the evolution of<br />
knowledge and methods to be transferred<br />
from research to a broad set of<br />
students. In addition, coursework in HRI<br />
creates a workforce capable of transferring<br />
Digital Object Identifier 10.1109/MRA.2010.936953<br />
BY ROBIN R. MURPHY,<br />
TATSUYA NOMURA,<br />
AUDE BILLARD,<br />
AND JENNIFER L. BURKE<br />
HRI theory to practice. However, as would be expected with an<br />
emerging field, HRI courses are largely ad hoc.<br />
Teaching HRI is challenging because the subject is multidisciplinary,<br />
and there is lack of educational materials, such as textbooks<br />
and resources such as robots and interfaces.<br />
This article summarizes the discussion and<br />
findings from the “Teaching Humans<br />
About Human–Robot <strong>Interaction</strong>”<br />
workshop on the development of an<br />
HRIcourseforcomputerscientistsand<br />
engineers. This half-day workshop was<br />
held at the IEEE/Robotics Society of<br />
Japan International Conference on Intelligent<br />
Robots and Systems (IROS), 22 September<br />
2008, in Nice, France. The motivation for the workshop was a<br />
direct response to a key finding from the National Science<br />
JUNE 2010 1070-9932/10/$26.00ª2010 IEEE IEEE Robotics & Automation Magazine 85
The IEEE Robotics and Automation<br />
Society sponsors a Technical<br />
Committee on Human–<br />
Robot <strong>Interaction</strong>.<br />
Foundation (NSF)-sponsored HRI Young Pioneers Workshops<br />
[2], held in conjunction with the annual Association of Computing<br />
Machinery (ACM)/IEEE Conference on Human–Robot<br />
<strong>Interaction</strong>. The findings consistently emphasized the need for<br />
an interdisciplinary course or curriculum in HRI to be taught at<br />
the university level. However, until the IROS workshop, there<br />
has been no reported venue for faculty to gather and discuss such<br />
a curriculum or teaching methods. Although this workshop was<br />
limited in both time and the number of participants, it offers a<br />
starting point and some insight into HRI education.<br />
The objectives of the workshop were to identify what is<br />
essential in an HRI course by leveraging the experiences to<br />
date in teaching HRI and then use this list of fundamentals to<br />
define course content. The workshop was also expected to<br />
create a community of educators within the emerging HRI<br />
research community, foster the exchange of best practices and<br />
pedagogical methods, and provide reference materials, if any,<br />
for instructors teaching HRI.<br />
The rest of this article is organized as follows. It first<br />
describes the workshop in terms of participants and activities.<br />
The challenges for a course in HRI identified by the participants<br />
follow next. Suggested course content, both in terms of<br />
the set of candidate topics for a course, and a sequence of lectures<br />
for advanced students in artificial intelligence (AI) and<br />
robotics follows. Next, possible course projects and assignments<br />
are discussed. The article then concludes with a distillation of<br />
the workshop into a set of six major findings.<br />
Workshop Description<br />
The workshop was attended by 18 participants, representing<br />
France, Germany, India, Japan, Korea, Switzerland, and the<br />
United States, and was organized around group discussions.<br />
Graduate students slightly exceeded the number of professors<br />
and industry researchers. The workshop consisted of four<br />
parts; beginning with each participant positing what they<br />
believed should be included in a HRI course and what is currently<br />
missing from HRI education.<br />
Second, a discussion on available resources for HRI education<br />
was initiated with an invited talk by Dr. Kojiro Matsushita<br />
from the University of Tokyo and demonstration of his two<br />
low-cost robot kits [3]. One kit was made from servomotors<br />
and plastic water bottles and can be constructed by beginners<br />
in less than 6 h after 2 h of instruction, making it suitable for<br />
nonrobotics students. The resulting robot can take many configurations,<br />
including legs and snake structures, and more<br />
emotive shapes similar to puppets. The robot can be controlled<br />
directly, learn motions from the user guiding the robot, or<br />
controlled by noninvasive contact sensors measuring muscle<br />
strain. Another kit is based on a toy hand. Dr. Matsushita is<br />
working on an English translation of his book on how to build<br />
and use these robots.<br />
The next discussion, led by Dr. Aude Billard from the<br />
Ecole Polytechnique Federale de Lausanne (<strong>EPFL</strong>), concentrated<br />
on lessons learned from both instructor and student<br />
experiences with HRI. Three professors described HRI classes<br />
at the Indian Institute of Information Technology Allahabad<br />
(Prof. G.C. Nandi), <strong>EPFL</strong> (Prof. Aude Billard), and at the<br />
University of South and Texas A&M (Prof. Robin Murphy).<br />
Rod Gutierrez, a graduate student at the University of South<br />
Florida, presented feedback from the 2008 Young Pioneers<br />
Workshop at the 2008 ACM/IEEE International Conference<br />
on Human–Robot <strong>Interaction</strong> with amplifying comments<br />
from that workshops attendees.<br />
The fourth discussion took the form of breakout groups.<br />
Participants were split into two groups: one to discuss the perfect<br />
syllabus, or sequence of lectures, for an HRI course, and<br />
the other to determine the perfect set of assignments and projects<br />
(Figure 1). The groups then gave reports summarizing<br />
their thoughts. The fifth component was a short recap and<br />
discussion of future activities, primarily increased involvement<br />
in the annual HRI conference.<br />
Figure 1. Participants in one of the two breakout groups.<br />
(Photo courtesy of Robin Murphy.)<br />
Challenges<br />
Creating a new course is always challenging, but the field of<br />
HRI provides three additional challenges for education.<br />
First, HRI is multidisciplinary, incorporating contributions<br />
from communications, computer science, engineering, psychology,<br />
and theater, creating challenges in creating course<br />
content that covers the field in sufficient depth without requiring<br />
a large number of prerequisites. Balancing coverage depth<br />
while minimizing prerequisites is particularly hard because the<br />
background between the engineering sciences and the human<br />
sciences was felt to be large. In response, the workshop participants<br />
quickly restricted discussion to teaching students in the<br />
engineering sciences; even within that restriction, the differences<br />
between individual engineering disciplines and computer<br />
science were significant.<br />
86 IEEE Robotics & Automation Magazine<br />
JUNE 2010
Second, the diversity of the HRI field also extends to<br />
resources, and as a result, there are no dedicated HRI resources,<br />
although possible materials can be extracted from mature<br />
fields. For example, HRI does not have a journal or textbook.<br />
There is a dedicated conference, the annual ACM/<br />
IEEE International Conference on HRI now in its fourth<br />
year, but the majority of participants were not aware of the<br />
conference. A related conference, the IEEE International<br />
Symposium on Robot and Human Interactive Communication<br />
(Ro-Man), also publishes HRI research. The IEEE<br />
Robotics and Automation Society sponsors a technical committee<br />
on HRI.<br />
Third, there is a lack of cost-effective, pedagogically<br />
appropriate robots and rich interfaces. As detailed below,<br />
hands-on projects in HRI are highly desirable. Robots such<br />
as Lego Mindstorms are inexpensive and do not require<br />
extensive programming expertise but, as noted by the students,<br />
may not provide sufficient capability to support key<br />
HRI topics. Humanoid robots vary in price but often have<br />
significant limitations for general HRI topics. For example,<br />
the design of the HOAP-3 robot prevents the camera in the<br />
head from seeing the hands, curtailing physical interaction<br />
and learning manipulation tasks. Few robots in any price<br />
range support human–computer interfaces such as haptics,<br />
touch screens, or gestures. Speech recognition remains unreliable,<br />
obviating the easy application of natural language to a<br />
survey course. One promising robot resource that addresses<br />
the third challenge is the robot kits presented by Dr. Matsushita<br />
and shown in Figure 2.<br />
Course Content<br />
Starting from the discussion and<br />
throughout the workshop, topics for<br />
inclusion in course emerged. The objectives<br />
of an HRI course were proposed,<br />
and a subset of these topics was arranged<br />
into one possible sequence of lectures<br />
aimed at advanced robotics or AI students.<br />
The individual topics were not<br />
rated as to relative importance because<br />
of time constraints.<br />
The topics not only largely borrowed<br />
from robotics, AI, and psychology themes<br />
but also included more unique HRI subjects<br />
and applications. Robot control and<br />
humanoid robot design and control were<br />
two robotcentric topics suggested for<br />
inclusion, along with user interfaces. Skill<br />
acquisition, often associated with traditional<br />
robot learning, has been experiencing<br />
a renaissance with the new emphasis<br />
offered by HRI. In particular, it was<br />
noted that students often do not understand<br />
limits on the range of motion or<br />
degrees of freedom in humanoid robots<br />
and thus become confused when trying<br />
to generate naturalistic motions. Natural<br />
(a)<br />
Robot control and humanoid robot<br />
design and control were two<br />
robotcentric topics suggested for<br />
inclusion.<br />
language processing and machine learning, staples of AI, were<br />
also deemed important. Psychology and cognitive engineering<br />
topics were tools and methods to measure HRI, joint attention<br />
theory, teams, and user-centered design. The participants noted<br />
that there was no concise list of qualitative and quantitative evaluation<br />
methods or tools, nor was there a clear mapping of particular<br />
techniques to desired outcomes, e.g., what technique<br />
would be best to measure X Social behaviors, emotion or affective<br />
expressions, interaction modalities, social learning, user<br />
expectations, safety, the Uncanny Valley, and ethics emerged as<br />
unique HRI topics. The topic of social behaviors actually is<br />
composed of two topics: one is “what are social behaviors” and<br />
the other is “how can robots be programmed to generate social<br />
behaviors” Rehabilitation and therapy was singled out as major<br />
HRI application areas.<br />
Course objectives should include at a minimum:<br />
u definition of HRI<br />
u the basic modalities for interacting with a robot<br />
u the key issues in HRI<br />
u the current applications<br />
u the process of making robots into social platforms<br />
u the importance of social skills in robots (role of learning,<br />
a theory of mind).<br />
Figure 2. Demonstration of walking robot and robot hand kits by Dr. Matsushita. (a)<br />
Legged robot from water bottles and (b) robot hand. (Photo courtesy of Rodrigo<br />
Gutierrez.)<br />
(b)<br />
JUNE 2010 IEEE Robotics & Automation Magazine 87
Psychology and cognitive<br />
engineering topics were tools and<br />
methods to measure HRI, joint<br />
attention theory, teams, and usercentered<br />
design.<br />
The prerequisites for an HRI course depend on the target<br />
audience and scope of material, although probability and statistics<br />
was considered a universal prerequisite. In addition to<br />
probability and statistics, related concepts such as regression<br />
analysis and experimental design would be helpful for a<br />
course focused on methodology. Robotics and AI (capturing<br />
control and automation), sensors, and machine vision<br />
are starting points for robotics students to study HRI. In<br />
addition, having signal processing and machine learning<br />
might be very helpful, although participants noted that<br />
machine learning was a topic that should also be covered in<br />
the course.<br />
Assuming an advanced robotics student with a background<br />
in AI, a set of possible lectures spans robot inputs to ethics.<br />
These are:<br />
u modalities and types of knowledge acquired through<br />
interactions, including vision, speech, and haptics<br />
u representing the world and the intentions of others<br />
u case studies of social learning and interaction<br />
u evaluation methodologies, both qualitative and quantitative<br />
u ethics.<br />
Course Projects and Assignments<br />
Having a hands-on component to an HRI class was strongly<br />
recommended by those who have taught HRI, who wish to<br />
teach HRI, and students. The recommended pattern was to<br />
have a series of small assignments either directly related to<br />
the current course material or scaffolded in complexity, then<br />
a final project chosen by the students. Assignments and projects<br />
directly involving robots and users were seen as the most<br />
desirable. However, working with users and robots raises<br />
many issues. Availability of platforms and of users is a concern.<br />
User-studies often requires a great deal of planning and<br />
preparation, including getting any institutional human–subject<br />
protocol approvals. Working with robots is costly, and<br />
there are concerns that sufficient robots will not be operational<br />
when needed. Robot simulations may prove to be<br />
a viable alternative to directly using a robot. Simulations<br />
such as Microsoft Robotics Studio can be programmed at a<br />
high level of abstraction, allowing the students to move and<br />
direct the robot without having to focus on the details of the<br />
robot or robot programming. Regardless of whether real or<br />
simulated robots are used, two applications are particularly<br />
attractive for a course. Search and rescue robotics has a<br />
strong societal benefit, whereas social robots are engaging<br />
and entertaining.<br />
Summary of Findings<br />
The workshop focused on teaching roboticists (computer science<br />
and engineers) at graduate level, generally discussing<br />
issues from an instructor’s viewpoint (e.g., pedagogy and<br />
resources) with a presentation and feedback from students.<br />
The six findings from the workshop are summarized below.<br />
u Finding 1: Students prefer HRI courses with a high<br />
degree of interaction between students and between<br />
students and robots over courses that are primarily lecture<br />
based. <strong>Interaction</strong>, both through discussion and<br />
hands-on projects, appears to be the desired style for<br />
teaching HRI.<br />
u Finding 2: Candidate topics for coverage in an HRI<br />
course include emotion, ethics, humanoid robot design<br />
and control, interaction modalities, joint attention theory,<br />
machine learning, natural language processing, robot<br />
control, safety, skill acquisition, social behaviors, social<br />
learning, teams, tools and methods to measure HRI, the<br />
Uncanny Valley, user interfaces, user-centered design,<br />
and user expectations. The choice of topics to include<br />
depends on the course prerequisites. On one hand, course<br />
prerequisites permit content to go deeper or free up time<br />
in the course schedule to include more of these topics.<br />
On the other hand, prerequisites may exclude students<br />
from the human sciences or even from a particular engineering<br />
science discipline. This could undermine the<br />
benefits of interdisciplinary courses and the discussionoriented<br />
teaching style desired by the students.<br />
u Finding 3: The most prominent deficits for creating<br />
course content in HRI are the lack of: 1) a set of key<br />
principles of HRI, 2) a survey of mechanisms on how to<br />
generate social behaviors, and 3) a succinct synopsis of<br />
user evaluation methods. We note that the fist deficit in<br />
the list reflects the lack of consensus in the HRI<br />
community over HRI. However, the second and third<br />
deficits highlight gaps in robotics that must be filled by<br />
multidisciplinary work; the second deficit shows the<br />
need to connect control theory with the behavioral sciences,<br />
whereas the third deficit necessitates a transfer of<br />
quantitative and qualitative methods pioneered outside<br />
of robotics.<br />
u Finding 4: The major missing pedagogical tools for<br />
instructors are cost-effective robots and a corpus of case<br />
studies, illustrating key principles of HRI. Cost is<br />
viewed as a major driver of a robot that can be adopted<br />
by a large number of universities for teaching HRI.<br />
u Finding 5: Course development should consider industry<br />
needs as well as instructor constraints and student learning<br />
preferences, as not all students will become HRI<br />
researchers. This includes understanding anthropomorphic<br />
robots as well as nonanthropomorphic forms.<br />
u Finding 6: Regardless of the target audience, an<br />
HRI course will most likely require students to have<br />
a background in statistics and will, at a minimum,<br />
cover interaction modalities, issues, social interactions,<br />
and applications.<br />
88 IEEE Robotics & Automation Magazine<br />
JUNE 2010
The workshop briefly touched on the way ahead. In terms<br />
of facilitating general progress in HRI education, there was a<br />
hope that the HRI conference would become a clearing house<br />
for HRI-specific resources. In terms of continuing the discussion<br />
on HRI education, it would be interesting to elicit the<br />
viewpoints of other disciplines, especially psychology, on<br />
what they believe are fundamental topics and how HRI<br />
should be taught.<br />
Acknowledgments<br />
The authors thank Dr. Matsushita for his demonstration of<br />
low-cost robots, Rod Gutierrez for his presentation and general<br />
assistance during the workshop, Dr. Ephriam Glinert for<br />
his support of the HRI Young Pioneers Workshop (NSF<br />
Grant IS-0813909), and the IROS 2008 tutorial chairs, Dr.<br />
Rachid Alami and Dr. Roland Siegwart.<br />
Keywords<br />
Human–robot interaction, robotics education.<br />
References<br />
[1] M. A. Goodrich and A. C. Schultz, “Human-robot interaction: A survey,”<br />
Found. Trends Hum.-Comput. Interact., vol. 1, no. 3, pp. 203–275, 2007.<br />
[2] J. Burke, R. Murphy, and C. Kidd, “Young researchers in HRI workshop<br />
2006,” Interact. Stud., vol. 8, no. 2, pp. 343–358, 2007.<br />
[3] K. Matsushita, H. Yokoi, and T. Arai, “Plastic-bottle-based robots in educational<br />
robotics courses—Understanding embodied artificial intelligence,”<br />
J. Robot. Mechatron., vol. 19, no. 2, pp. 212–222, 2007.<br />
Robin R. Murphy received a B.M.E. degree in mechanical<br />
engineering, and M.S. and Ph.D. degrees in computer science in<br />
1980, 1989, and 1992, respectively, from Georgia Tech, where<br />
she was a Rockwell International Doctoral Fellow. She is the<br />
Raytheon Professor of Computer Science and Engineering at<br />
Texas A&M. In 2008, she was awarded the Al Aube Outstanding<br />
Contributor Award by the Association for Unmanned Vehicle<br />
Systems International Foundation for her insertion of ground,<br />
air, and sea robots for urban search and rescue at the 9/11 World<br />
Trade Center disaster, Hurricanes Katrina and Charley, and the<br />
Crandall Canyon Utah mine collapse. She is a distinguished<br />
speaker for the IEEE Robotics and Automation Society and has<br />
served on numerous boards, including the Defense Science<br />
Board, U.S. Air Force Scientific Advisory Board, NSF Computer<br />
and Information Science and Engineering Advisory Council,<br />
and the Defense Advanced Research Projects Agency Information<br />
Science and Technology Study Group. She is a Senior<br />
Member of the IEEE. Her research interests include AI, HRI,<br />
and heterogeneous teams of robots.<br />
Tatsuya Nomura received the M.S. degree in mathematics<br />
from Osaka University, Japan, in 1989, and the D.E. degree in<br />
engineering from Kyoto University, Japan, in 1998. From 1989 to<br />
2000, he was with the Corporate Research and Development<br />
Group at Sharp Corporation. From 2000 to 2004, he was with<br />
Hannan University, Osaka, Japan. He is currently an associate<br />
professor in the Department of Media Informatics, Ryukoku<br />
University, Otsu, Japan, and a researcher in the Advanced<br />
The prerequisites for an HRI<br />
course depend on the<br />
target audience and scope<br />
of material, although probability<br />
and statistics was considered a<br />
universal prerequisite.<br />
Technology and Research Intelligent Robotics and Communication<br />
Laboratories, Japan. He is a member of the Japanese<br />
Psychological Association, the Japanese Cognitive Science<br />
Society, and the Mathematical Society of Japan. He is a Member<br />
of the IEEE. His research interests include intelligent<br />
robots and human—robot interaction.<br />
Aude Billard received a B.Sc. degree in physics from <strong>EPFL</strong>,<br />
with specialization in particle physics at the European Center for<br />
Nuclear Research (CERN) in 1994. She received her M.Sc.<br />
degrees in physics from the same university, with specialization in<br />
particle physics at the CERN and in knowledge-based systems in<br />
1996 and a Ph.D. degree in AI from the Department of Artificial<br />
Intelligence at the University of Edinburgh in 1998. She is an<br />
associate professor and head of the Learning Algorithms and Systems<br />
Laboratory at the School of Engineering, <strong>EPFL</strong>. Before this,<br />
she was a research assistant professor at the Department of<br />
Computer Sciences at the University of Southern California,<br />
where she retained an adjunct faculty position to this day. She is a<br />
Member of the IEEE. Her research interests focus on machine<br />
learning tools to support robot learning through human guidance.<br />
This extends also to research on complementary topics,<br />
including machine vision and its use in human–machine interaction<br />
and computational neuroscience to develop models of<br />
learning in humans.<br />
Jennifer L. Burke received the B.A. degree in business from<br />
Florida State University, the M.S. degree in counseling from<br />
the University of North Florida, and the M.S. and Ph.D.<br />
degrees in industrial-organizational psychology (minor: man–<br />
machine interaction) from the University of South Florida, in<br />
1980, 1990, and 2006, respectively. She is a practicing human<br />
factors engineer at SA Technologies, specializing in robotic<br />
interface design. She is active in the robotics and psychology/<br />
human factors communities and is the author of more than 30<br />
publications in fields of robotics, human performance, and<br />
workplace studies. She is a member of the ACM, the American<br />
Psychological Society, and the Human Factors and Ergonomics<br />
Society. Her research interests include team processes<br />
and human–robot interaction.<br />
Address for Correspondence: Robin R. Murphy, Computer<br />
Science and Engineering, Texas A&M University, College<br />
Station, TX, USA. E-mail: murphy@cs.tamu.edu.<br />
JUNE 2010 IEEE Robotics & Automation Magazine 89