The work-reflection-learning cycle - Department of Computer and ...

Birgit Rognebakke Krogstie
The work-reflection-learning
cycle in software engineering
student projects: Use of
collaboration tools
Thesis for the degree philosophiae doctor
Trondheim, June 2010
Norwegian University of Science and Technology
Faculty of Information Technology, Mathematics and
Electrical Engineering
Department of Computer and Information Science

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
NTNU
Norwegian University of Science and Technology
Thesis for the degree doctor philosophiae
Faculty of Information Technology, Mathematics and Electrical Engineering
Department of Computer and Information Science
© 2010 Birgit Rognebakke Krogstie
ISBN 978-82-471-2025-5 (printed version)
ISBN 978-82-471-2026-2 (electronic version)
ISSN 1503-8181
Doctoral thesis at NTNU, 2010:35
Printed in Norway by NTNU-trykk, Trondheim
ii

Figure 1: Most used terms in the thesis (the research papers excluded); diagram
generated in Wordle (http://www.wordle.net)
iii

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
iv

Abstract
In project based learning, students learn from hands-on experience with the challenges
and complexities of real work. To achieve this learning, reflection is essential. In project
work in industry, retrospective reflection on the project process is established practice,
e.g. in project debriefings. Student projects often include retrospective reflection as a
mandatory exercise seen as important to learning but not as a part of the „real work‟.
The research in this thesis is guided by the idea that making retrospective reflection
integral to the work practice in project based learning can help students learn more from
their project experience. The thesis explores how retrospective reflection in Software
Engineering student projects can be supported, particularly taking into account the use
of collaboration tools – typically lightweight tools – by teams in their daily project
work.
To this end, a set of interpretive case studies have been conducted. The studies examine
the current use of state-of-the-art lightweight collaboration tools in Software
Engineering student projects, focusing on the role of the tools in work and learning
within the teams and in collaboration with other project stakeholders. The thesis
research also includes studies in which interventions have been made in the projects by
introducing facilitated retrospective reflection activities with and without the aid of
collaboration tools. These interventions can be considered as initial steps of a design
research effort with the aim of improving the pedagogical design of the course as well
as developing new theory on reflection in project based learning.
The resulting contributions include new knowledge about the use of various types of
lightweight collaboration tools to support day-to-day work in the projects; within the
teams and in collaboration between the teams and other project stakeholders. These
contributions can be used to aid the organization of future SE student projects including
the choice and use of collaboration tools in the projects. The thesis also provides
insights on how retrospective reflection, seen as a part of a collaborative work practice,
can be supported in SE student projects and project work more generally. This research
contribution includes a design for retrospective reflection workshops in educational
settings (adapted from industry practice and empirically tested in a project course), a
prototype tool extending wiki functionality to support navigation of relevant historical
data, and empirical results demonstrating how a collaboration tool used in daily project
work can be used as an aid to memory in retrospective reflection. Finally, a main
contribution of this PhD work is a model of reflection, conceptualized in terms of
distributed cognition. Drawing on learning theory as well as current practice for
Software Engineering retrospectives, the model represents a novel and practical view of
v

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
the work-reflection-learning cycle in collaborative work. The model incorporates
individual and collaborative steps of reflection on a collaborative work process, making
explicit the role of internal and external representations of the process as an aid to
reflection. The model also outlines the potential role of collaboration tools in supporting
day-to-day work and retrospective reflection on that work, thereby integrating these
aspects of the work practice. In this way shedding light on complexities and
opportunities related to reflection on collaborative work the model can aid design for
retrospective reflection in project based learning in educational and professional
settings.
vi

Preface
This thesis is submitted to the Norwegian University of Science and Technology
(NTNU) for partial fulfilment of the requirements for the degree of philosophiae doctor.
The doctoral work has been performed at the Department of Computer and Information
Science at NTNU. Prof. Monica Divitini has been the main supervisor and Ove
Haugaløkken has been the co-supervisor.
The doctoral work is funded by MOTUS2, a research project within the NTNU research
programme for Learning and ICT.
vii

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
viii

Acknowledgements
I would like to thank all the people who in different ways have contributed to this thesis.
My colleagues at IDI have been an invaluable resource for discussing and exchanging
ideas and have provided good company throughout my PhD years. Sobah Abbas
Petersen and Kirsti Berntsen have been great office mates and friends. Glenn
Munkvold, being a couple of years ahead of me on the PhD schedule, provided valuable
support and advice on many aspects of PhD work. Torstein Hjelle is always helpful and
faithfully joins in on the morning visit to the coffee shop. Thomas Østerlie, Sven
Ziemer, Tor Stålhane, Torgeir Dingsøyr, Eric Monteiro, Dag Svanæs, Ilaria Canova
Calori, Reidar Conradi, Gro Alice Hamre, Øyvind Hauge, Hallvard Trætteberg,
Gasparas Jarulaitis, Letizia Jaccheri, Ole Andreas Alsos, Guttorm Sindre, Gry Seland,
Inger Dybdahl Sørby and Chiara Rossitto have all been generous in taking the time to
discuss with me when I have been in need of opinions and insights. More names could
have been added to this list, and I colloquially thank the people who make my daily
work at IDI interesting and enjoyable.
Thanks to Bendik Bygstad and Tor-Morten Grønli, my former colleagues at NITH, for
good collaboration on research papers.
Glenys Hamilton did efficient editing work, and I recommend her as an editor.
Ove Haugaløkken, my co-supervisor, and Leif Martin Hokstad, in practice my second
co-supervisor, have provided constructive and useful input from the side of educational
theory and computer-supported collaborative learning throughout the PhD period.
Special thanks go to my main supervisor Monica Divitini. She is supportive and
generous and has been doing a great job as co-author of several of my research papers.
Applying her strong analytic skills to my ideas and writings she has frequently provided
the essential keys to bringing the PhD work another step forward.
Thanks to my parents Ellen and Jan Rognebakke and to my friends Idunn Løvseth
Kavlie and Ragnhild Torsvik Bamrud for listening when I have been in need of talking
about the challenges of life as a PhD student.
Finally I thank my beloved children Øystein, Håvard and Idunn for putting up with a
mother who has often been stressed, impatient and absorbed in her PhD work. I thank
my dear husband John for love and support. I am very fortunate to be married to a man
who not only knows the requirements of academic work but who is also highly
respectful of our differences in terms of coping with work and life.
ix

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
x

Contents
1 Introduction ........................................................................................................ 1
1.1 Problem statement 1
1.2 Problem refinement and research questions 3
1.3 Research design 4
1.4 Results and contributions 5
1.5 Structure of the thesis 7
2 Work, learning and reflection: theoretical background ................................. 9
2.1 Understanding work and learning 9
2.1.1 A constructionist perspective on work and learning ............................ 9
2.1.2 Communities of practice and learning ................................................ 10
2.1.3 Distributed cognition .......................................................................... 12
2.2 Reflection as a bridge between work and learning 13
2.3 Trajectory as representation of a process 16
3 Software Engineering student projects: state-of-the-art .............................. 19
3.1 The characteristics of SE student projects 19
3.2 Use of lightweight collaboration tools in SE student projects 22
3.3 Supporting reflection on SE work 23
3.4 Using work tools as tools for learning 26
4 Research approach and research process ...................................................... 29
5 Results ................................................................................................................ 37
5.1 P1: Cross-Community Collaboration and Learning in Customer-Driven
Software Engineering Student Projects 37
5.2 P2: Power through brokering. OSS participation in SE projects 38
5.3 P3: Do‟s and Don‟ts of Instant Messaging in Students‟ Project Work 40
5.4 P4: The wiki as an integrative tool in project work 40
xi

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
5.5 P5: Using Project Wiki History to Reflect on the Project Process 42
5.6 P6: Shared timeline and individual experience: Supporting retrospective
reflection in student software engineering teams 44
5.7 P7: A model of retrospective reflection in project based learning utilizing
historical data in collaborative tools 46
5.8 P8: Supporting Reflection in Software Development with Everyday
Working Tools 48
6 Contributions and implications ....................................................................... 53
6.1 Project work as cross-community collaboration 53
6.2 Lightweight collaboration tools in project work 55
6.3 Supporting retrospective reflection 57
6.4 A model of work and reflection 60
7 Evaluation ......................................................................................................... 63
7.1 Evaluation of the contributions with respect to the research questions 63
7.2 Evaluation of the contributions with respect to state-of-the-art of the SE
Education, TEL and CSCW research fields 64
7.3 Evaluation of the research process 67
7.3.1 On the work in the thesis as interpretive research .............................. 67
7.3.2 On the work in the thesis as design research ...................................... 72
7.4 On the choice of theory 73
8 Conclusion and further work .......................................................................... 77
9 References.......................................................................................................... 81
Glossary .................................................................................................................... 91
Appendix A: Research papers ................................................................................ 95
Research paper P1 97
Research paper P2 107
Research paper P3 119
xii

Research paper P4 135
Research paper P5 149
Research paper P6 161
Research paper P7 171
Research paper P8 189
Appendix B: Other research publications ........................................................... 211
Practice-Based Learning as Mobile Learning: The Role of Boundary Objects 211
Learning from achievement: scaffolding student projects in software engineering
212
xiii

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
List of Figures
Figure 1: Most used terms in the thesis (the research papers excluded); diagram
generated in Wordle (http://www.wordle.net) ......................................................... iii
Figure 2: The problem domain of the PhD thesis ............................................................. 3
Figure 3: Research papers and the main topics of the research contributions .................. 7
Figure 4: The model of experiential learning (Kolb et al. 1971) .................................... 14
Figure 5: A model of the reflective process. From (Boud et al. 1985a) ......................... 15
Figure 6: Timeline indicating when the data collection for the empirical studies took
place ........................................................................................................................ 30
Figure 7: Team A having a meeting at the customer's site. The customer (second from
the right) has ordered pizza. ................................................................................... 33
Figure 8: The whiteboard after a reflection workshop with a student team. (The photo
has been manipulated to make the experience curves more visible in print). ........ 34
Figure 9: Field notes. ...................................................................................................... 35
Figure 10: Screen shot from a project wiki main page as the project approached final
deadline. Note the strikethroughs of completed tasks in the to-do-list. ................ 41
Figure 11: Wiki Walkthrough Tool: adding a tag to the current version of a page........ 43
Figure 12: Snapshot of the whiteboard after the reflection workshop of one project
team, showing the shared project timeline with the individual satisfaction curves.
The numbers along Max’s curve indicate where he explained this experience to the
rest of the team. From P6. ...................................................................................... 46
Figure 13: Retrospective reflection as distributed cognition. From P7. ......................... 47
Figure 14: Examination of historical data in Trac. From P8. ........................................ 49
Figure 15: A model outlining how Trac supports project work and retrospective
reflection on the project process in the case study of P8. From P8. ....................... 51
Figure 16: Diagrams showing the researchers' interpretation of how Team F changed
their story of their project in their reflection workshop.......................................... 71
xiv

Figure 17: Support for learning in Software Engineering student projects as approached
in this thesis ............................................................................................................ 77
xv

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
List of Tables
Table 1: Research questions for the PhD thesis ............................................................... 4
Table 2: Empirical research activity vs. research questions and research papers .......... 31
Table 3: Timetable of the retrospective reflection workshop conducted with each of the
teams in a SE project course. From P6. .................................................................. 45
Table 4: Theory informing the work presented in the research papers .......................... 68
xvi

Abbreviations
CSCL
CSCW
IS
NITH
NTNU
OSS
TEL
Computer-Supported Collaborative Learning
Computer-Supported Cooperative Work
Information Systems
Norwegian School of Information Technology
Norwegian University of Science and Technology
Open Source Software
Technology Enhanced Learning
xvii

1 Introduction
For we live not in a settled and finished world, but in one which is going
on, and where our main task is prospective, and where retrospect – and
all knowledge as distinct from thought is retrospect – is of value in the
solidity, security, and fertility it affords our dealings with the future.
1.1 Problem statement
(Dewey 2008 (1909))
This thesis is about project based learning (PBL) and how it can be supported in
Software Engineering (SE) student projects. Project based learning is an educational
approach at the intersection of work practice and education, and students‟ experience of
being practitioners is essential to their motivation and learning (Dohn 2009; Schön
1987). The students do real work – independent, challenging, complex, aided by tools
and in accordance with the norms and rules of the work practice, the latter mainly
known to them through formal SE education. In project based learning, by participating
in the community of practice (Wenger 1998) for which the project is preparing them,
the students develop their insights and skills within their team as well as through
interaction with other stakeholders such as the supervisor.
Project activities perceived to be „school assignments‟ are not approached with the same
enthusiasm as the challenges associated with doing „the real thing‟ in the project (i.e.
solving the project task). In a SE student project, this task would typically be to develop
a working software product, maybe for an external customer.
As professional work practices involve collaboration (e.g. in project teams), project
based learning typically takes place through work in teams. While collaboration adds
opportunities to achieve what is not possible for individuals alone, it introduces
complexity and the need for collaborative skills. In this thesis, focus will be on project
based learning in teams.
To support collaborative work, tools are used. In PBL, getting the desired work
experience includes getting experience with the relevant tools. Knowledge about
adequate tool use can be drawn from the work domain in question (e.g. in the case of SE
1

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
student projects, using tools similar to those used in SE industry). However, adding to
the complexity of PBL, the adequate use of tools for supporting work may not be so
clearly determined: as new technologies and patterns of using them rapidly develop
across different domains, student teams may find themselves at the frontier of
technology use. For instance, current project work practices frequently involve the use
of lightweight collaborative tools, many of which are being used for social as well as
professional purposes, and many of which are especially widespread among young
people. The use of such tools in workplace settings is currently being explored (Decker
et al. 2007; Fuchs-Kittowski and Köhler 2005; Isaacs et al. 2002b; Stafford 2008). In
the research community of Computer Supported Cooperative Work (CSCW), it is
acknowledged that further research is needed in this area (Pipek and Wulf 2007).
Learning from experience entails reflection (Boud et al. 1985b; Dewey 1933; Kolb and
Fry 1975). In context of a work practice, work and reflection are intertwined. Some
reflection is inherent to the meaning making (Bruner 1990) of day-to-day work, and
some requires more distance to the experience reflected upon (Schön 1983). The aspects
of work that can be elaborated through reflection range from those that are easily made
explicit to those that can be considered tacit (Polanyi 1966) but may be codified and
shared within a common frame of reference (Hildrum 2009), and include the affective
elements of personal experience (Boud et al. 1985b; Eraut 2000).
The role of reflection in learning has long been recognized in the educational field and
incorporated into current practices (e.g. of project based learning (Duim et al. 2007;
Helle et al. 2006)). In industry, the need to learn from experience is manifest in current
industry practices of facilitated reflection, such as in project post-mortem evaluations
(Dingsøyr 2005; Kerth 2001). Still, in educational as well as workplace settings, there
are obstacles to making reflection a useful part of the work practice that the reflection is
intended to support (Armarego 2007; Kasi et al. 2008).
Due to the central role of tools in mediating work and learning (Kirschner and Erkens
2006; Kuutti and Kaptelinin 1997; Stahl 2002), research on the support for reflection in
project based learning has a place in the research fields of Technology Enhanced
Learning (TEL) and Computer-Supported Collaborative Learning (CSCL).
As an empirical case of project based learning for the PhD work presented in this thesis,
Software Engineering (SE) student projects were selected. The projects are
collaborative, complex and based on active use of state-of-the-art collaboration
technology. The extensive use of lightweight collaboration tools in the projects makes
them a good case for exploring the role of this type of technology in work and learning.
Regarding support for reflection, SE project course designs typically include activities
2

Introduction
to help the students reflect on their project process. However, there is a tendency that
such activities are conducted only half-heartedly, being perceived as school assignments
rather than part of the „real work‟. This indicates a learning potential left partially
unexploited.
How work and reflection are, and could be, supported and connected in SE student
projects, taking into account the role of tools, is the problem domain (Figure 2) to be
explored in this thesis. The domain is approached from different angles, framed in the
research contexts of CSCW, TEL/CSCL and SE education.
Figure 2: The problem domain of the PhD thesis
1.2 Problem refinement and research questions
The main research question for the PhD work is:
How can work and reflection be supported and bridged in software
engineering student projects, taking into account the use of
collaboration tools?
The refinement of the main research question into a set of research questions guiding
the investigation of the problem domain was based on the following research strategy:
• Start by examining current practices of work in student SE teams (particularly
stressing the role of collaboration technology) and thereby identify challenges to
work and learning
• Provide solutions to the identified challenges, and evaluate these solutions.
3

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
The following four research questions were formulated (Table 1):
Table 1: Research questions for the PhD thesis
RQ1: What characterizes software engineering student projects?
RQ2: What is the current usage of lightweight collaboration tools to support work in
software engineering student projects?
RQ3: How can retrospective reflection be supported in software engineering student
projects?
RQ4: How can collaboration tools that are used in daily project work be utilized as
tools for project based learning?
1.3 Research design
Case studies of SE projects with a focus on the as-is situation were appropriate for
unveiling current challenges in the projects in line with RQ1 and RQ2 and guide the
selection of issues to be explored more deeply in the thesis. Particularly in the case of
two longitudinal studies, the research was guided by the principles of interpretive field
studies (Klein and Myers 1999).
To explore the selected issue of retrospective reflection in accordance with RQ3 and
RQ4, the research was designed to include interventions to current practice, exploring
and evaluating approaches to retrospective reflection. This can be considered part of a
design research (Kelly et al. 2008b) process with the goal of improving the SE project
course in question as well as contributing with theoretical insights and practical
approaches in the area of project based learning more generally.
A SE project course at NTNU was a good candidate for the empirical research needed
to explore RQ1-4. The course is a capstone project course of an undergraduate IT study
program, and while the students in the projects are relative novices in the field of
Software Engineering, they have relatively advanced SE knowledge and skills. Further
adding to the realism of the projects as compared to professional SE work, the projects
have external customers for whom working software products are to be developed.
Being a researcher and some of the time course staff in various roles in the project
course, I had opportunities to get insights about the course organization and about
specific projects of interest. I had access to various project documentation and
opportunities for data collection across the teams in the course as well as the
4

Introduction
opportunity to do longitudinal studies of single teams. This permitted the collection of a
large amount of data over several years to gain an in-depth understanding of the case.
Also, I had the opportunity to make interventions with the course itself. The dual role of
researcher and staff and its possible impacts on the research were systematically
addressed in the research. While most of the case material for the thesis was drawn from
the SE project course at NTNU, some of the empirical data for research paper P1 was
collected in a similar project course at NITH, the university college that was my former
workplace.
In my PhD research I take a constructionist perspective on work and learning,
deliberately seeking to focus on their social and cognitive aspects. I have chosen to be
relatively restrictive in applying theoretical concepts in the initial phases of the research,
including theory as seen useful for the different research activities to support data
collection and analysis in a way coherent with a constructionist view.
1.4 Results and contributions
The research questions RQ1-RQ4 are answered in the following research papers:
P1
P2
P3
Krogstie, B. and B. Bygstad. Cross-Community Collaboration and Learning in
Customer-Driven Software Engineering Student Projects. Twentieth Conference
on Software Engineering Education and Training (CSEE&T) 2007, Dublin.
IEEE Computer Society. (Krogstie and Bygstad 2007)
Krogstie, B. Power through brokering. OSS participation in SE projects. ICSE
2008, Leipzig. IEEE Computer Society. (Krogstie 2008a)
Krogstie, B.R. Do‟s and Don‟ts of Instant Messaging in Students‟ Project Work.
NOKOBIT 2009, Trondheim. Tapir. (Krogstie 2009a)
P4 Krogstie, B.R. The wiki as an integrative tool in project work. in COOP 2008.
Carry-le-Rouet, Provence, France. Institut d‟Etudes Politiques d‟Aix-en-
Provence. (Krogstie 2008b)
P5
P6
Krogstie, B.R. Using Project Wiki History to Reflect on the Project Process.
42nd Hawaii International Conference on System Sciences (HICSS‟42) 2009.
Big Island, Hawaii: IEEE Computer Society. (Krogstie 2009c)
Krogstie, B.R. and M. Divitini. Shared timeline and individual experience:
Supporting retrospective reflection in student software engineering teams.
CSEE&T 2009, Hyderabad. IEEE Computer Society. (Krogstie and Divitini
2009)
5

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
P7
P8
Krogstie, B.R. A model of retrospective reflection in project based learning
utilizing historical data in collaborative tools. EC-TEL 2009, Nice. Springer.
(Krogstie 2009b)
Krogstie, B.R. and M. Divitini. Supporting Reflection in Software Development
with Everyday Working Tools. COOP 2010, Aix-en-Provence, France. Springer.
(Krogstie and Divitini 2010)
In Figure 3 the four transparent rectangles indicate the general research topics addressed
in the research contributions of the thesis and which research papers address each topic.
The topics are:
• Project work as cross-community collaboration (Section 6.1)
• The use of lightweight collaboration tools (Section 6.2)
• Retrospective reflection, with and without the aid of collaboration tools (Section
6.3)
• The integration of work and reflection in a work-reflection-learning cycle,
taking a distributed cognition perspective (Section 6.4)
The „swim lanes‟ in the diagram show the research communities (SE Education, TEL
and CSCW) in which the research papers have been peer reviewed and accepted for
publication.
6

Introduction
Figure 3: Research papers and the main topics of the research contributions
1.5 Structure of the thesis
The thesis is structured as follows:
Chapter 2 outlines relevant background theory on work, learning and reflection, with a
focus on the choice of an adequate theoretical framework to address the main research
question of the thesis.
Chapter 3 gives an overview of state-of-the art research in the areas of project based
learning and lightweight tool support for work in Software Engineering and SE student
projects.
Chapter 4 presents the research approach of the PhD work and discusses its strengths
and limitations.
7

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Chapter 5 summarizes the results from the research papers.
Chapter 6 outlines the contributions of the thesis, addressing their implications.
Chapter 7 contains an evaluation of the PhD work.
Chapter 8 concludes the thesis and suggests future research.
Appendix A contains the PhD papers P1-P8.
Appendix B contains the abstract of two research papers that were written during the
PhD scholarship period but not included in the PhD paper collection
8

2 Work, learning and reflection: theoretical
background
[Theory] can be a useful signpost for those who need directions, or for
that matter for those who are busily writing PhD theses and feel they need
a crutch.
(Randall et al. 2007)
In this chapter, I briefly account for the theory guiding the research in the thesis.
Concepts from various theoretical frameworks have been combined in order to shed
light on issues of interest within a coherent conceptual frame. I start by summarizing
theory on how to understand work and learning. I next consider theory illuminating how
reflection can bridge work and learning. Finally, I turn to the concept of trajectory as a
way of conceiving and making sense of a process.
2.1 Understanding work and learning
Outlining the main theory on work and learning underlying the thesis I address the
constructionist perspective, theory of Communities of Practice (CoP) and learning, and
the theoretical framework of Distributed cognition.
2.1.1 A constructionist perspective on work and learning
Constructionism and constructivism are the core concepts of a large and non-unified
body of research (Phillips 1995) building on the assumption that human knowledge is
constructed in an interplay between individuals‟ thought processes and social processes
(Berger and Luckmann 1966; Brown et al. 1989; Palincsar 1998; Vygotsky 1978). This
view “discards the notion that knowledge could or should be a representation of an
observer-independent world-in-itself and replaces it with the demand that the conceptual
constructs we call knowledge be viable in the experiential world of the knowing
subject” (von Glasersfeld 1989). Such a perspective has profound implications for how
learning is regarded, and sets learners‟ experience at the centre of attention. The
9

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
constructionist perspective is common in CSCL research (Stahl 2002; Strijbos et al.
2004; Suthers 2006) and also within CSCW (e.g. with activity theory (Kaptelinin and
Nardi 2006)).
The constructionist perspective has implications for the thesis in the following ways:
• Ontologically/epistemologically: The reality to be understood and designed for
(in student SE project teams) should be considered a social construct.
Participants (including the researcher) constantly engage in the construction of
this reality.
• Research focus: If we want to understand work and learning, knowledge
construction among participants is an important object of study.
• Research approach: It must always be kept in mind that participants have
different interpretations and that these are made in certain social and historical
contexts (Klein and Myers 1999).
Viewing human activity mainly from a socio-cultural perspective or mainly from a
cognitive one can be seen as two different strands of constructionism. In this thesis, I
take an intermediate position (Cobb 1994; Lin et al. 1999), stressing the social as well
as the cognitive aspects of work and learning in collaborative settings. This provides a
starting point for shedding light on the interplay between thought processes and social
processes.
2.1.2 Communities of practice and learning
Communities of practice (CoP) (Lave and Wenger 1991; Wenger 1998) are an
important arena for the construction of knowledge. CoPs are characterized by members
having mutual engagement, a joint enterprise, and a shared repertoire. They have a
shared history of learning and boundaries with other communities. Learning the practice
of a CoP is something that happens through participation in the community. A new
participant entering a community starts out as a novice engaged in “legitimate
peripheral participation” and gradually increases expertise through situated learning
(Lave and Wenger 1991). Along the same line, Brown and colleagues describe learning
as situated cognition (Brown et al. 1989), arguing that activity and situations are
integral to cognition and learning.
A student team engaged in a project throughout a semester can be seen as a CoP, albeit
a small and relatively short-lived one. The team develops a shared history of learning
and has boundaries with other communities (e.g. those of project stakeholders). The
10

Work, learning and reflection: theoretical background
members of the teams are simultaneously members of other communities such as the
profession for which the education is preparing the students, and the students‟ personal
circles of friends. There are frequently different levels of expertise among the students
with respect to the practice of the project. For instance, in the case of SE projects,
programming skills often vary greatly within a team. From a CoP perspective, team
members‟ work and learning thus takes place within and across communities with
varying goals, norms and practices, and with expertise in these practices differing
among community members and over time. In the thesis, team members‟ multiple
memberships in different communities are a core issue in the research papers P1 and P2.
A potential arena for learning in the context of CoPs is the boundaries between
communities. Individuals who are simultaneously members of two communities can
serve as brokers, contributing to the transfer and transformation of knowledge from one
community into the other. This typically involves artifacts (Gal et al. 2005; Star 1990)
integral to the practice of each of the communities and serving as boundary objects.
Changes to the boundary objects (e.g. through collaboration), may have implications for
the communities involved and be a source of learning and change. In the thesis, the
issue of brokering is important in P2.
In research papers P3 and P4, which focus on current use of lightweight technologies in
SE student teams, the CoP concept helps shed light on the differences between activity
within the team and that which involves collaboration with other stakeholders (e.g.
customer, supervisor) and can be considered cross-community collaboration.
A student SE team can also be considered as a particular type of CoP, namely a learning
community (Riel and Polin 2004). Riel and Polin provide a categorization of online
learning communities which is also appropriate for characterizing communities of
learners that are fully or partially collocated, such as student SE team. The categories
are intended by the authors to be analytic categories, and a specific learning community
may have the characteristics of one or more of them:
Task-based learning communities are “groups of people organized around a task
who work intently together for a specified period of time to produce a product.
While the specific group may not, in the strictest sense, share all of the
properties of a community, the people who participate in them often experience
a strong sense of identification with their partners, the task, and the organization
that supports them.” (Riel and Polin 2004, p.20)
Practice-based learning communities are larger groups with shared goals that
offer their members richly contextualized and supported arenas for learning.
11

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Knowledge-based learning communities “often share many of the features of a
community of practice but focus on the deliberate and formal production of
external knowledge about the practice”. This includes being committed to
recording and sharing knowledge “outside its immediate use or active context”
(Riel and Polin 2004, p.21).
In SE student projects and project based learning more generally, while knowledge
building beyond the project itself may be important in some projects, the teams are
typical task-based learning communities. This theoretical point is central to the
perspective on the use of lightweight tools in research paper P4. At the same time in
project based learning, participation in authentic work (e.g the CoP of the professional
community or possibly a particular organization) is a motivating force. Issues arising
from SE project students simultaneously being members of different types of learning
communities are discussed in P1.
2.1.3 Distributed cognition
From a constructionist perspective, learning in a SE student team can be viewed in
terms of knowledge construction. To understand how knowledge is constructed (e.g. by
the aid of technology) it is useful to go into more detail on the role of tools and how
knowledge is distributed.
The impact on activity (e.g. project work) of tools is captured in the idea of mediation:
the use of tools forming the activity and the activity over time forming the design of the
tools. This is a key point in various theoretical frameworks explaining the interplay of
the individual and the social (e.g. activity theory (Engeström 1987; Leont'ev 1981;
Vygotsky 1978) and actor network theory (Latour 2005)). Tools can be understood in a
broad sense, comprising physical artifacts, theories and concepts as well as
computerized tools. Within the framework of distributed cognition (DCog) (Hutchins
1995), the central unit of analysis is the functional system, “a collection of individuals
and artifacts and their relations to each other in a particular work practice” (Rogers and
Ellis 1994). As in actor network theory, the artifacts (e.g. tools) are considered entities
analytically on a par with human actors, which helps shed light on the dynamics of tool
mediation. Analyzing a case of collaborative work from the perspective of DCog, the
main goal is to account for how the distributed structures are coordinated. This implies
examining the contributions of the environment in which the work activity takes place,
individuals‟ interactions and the use of artifacts in interaction, and the representational
media used. DCog has a focus on the way knowledge is propagated across different
representational states along various communicative pathways (Rogers and Ellis 1994).
12

Work, learning and reflection: theoretical background
In approaching a case of collaborative work with the aim of understanding the role of
tools in knowledge construction, the focus on representational media and states makes
DCog a particularly useful analytical framework. DCog has been used as a framework
for understanding collaborative work in general (Rogers and Ellis 1994), organizational
memory (Ackerman and Halverson 2004), SE project work (Sharp and Robinson 2006)
as well as educational practice (Blandford and Furniss 2006; Daradoumis and Marques
2002). In this thesis, DCog has been applied in the analysis of the case study in P8 and
in the development of the conceptual model of reflection presented in P7.
A further link between tools and knowledge construction can be made by considering
computerized tools that are used to aid thinking as mindtools (Pea 1985) or cognitive
tools (Kim and Reeves 2007; Kirschner and Erkens 2006; Kuutti and Kaptelinin 1997).
Cognitive tools can be seen as “cognitive reflection and amplification tools that help
learners to construct their own realities by designing their own knowledge bases”
(Jonassen 1995, p.43). In the thesis, the concept of cognitive tools is central to P7.
The DCog perspective can shed light on the role of individual participants in the
functional system of collaborative work by distinguishing between internal and external
representations. The need to focus on the individual learner within a DCog perspective
on collaborative learning has been stressed (Salomon 1993). From a related theoretical
viewpoint, personal knowledge is “the cognitive resources which a person brings to a
situation which enables them to think and perform” when (typically) combined with
codified knowledge (Eraut 2000). The latter is by definition explicit, whereas personal
knowledge can be either explicit or tacit (Polanyi 1966). In this thesis, the DCog
perspective is used in P7 and P8 as the role of external and internal representations of a
work process in work and reflection is explored.
2.2 Reflection as a bridge between work and learning
Reflection is a core concept in the thesis and a main topic of the research papers P5-P8.
Situated in the philosophical school of pragmatism, John Dewey developed a theory of
learning from experience:
“To „learn from experience‟ is to make a backward and forward
connection between what we do to things and what we enjoy or
suffer from things in consequence. Under such conditions, doing
becomes a trying; an experiment with the world to find out what it
is like; the undergoing becomes instruction – discovery of the
connection of things.” (Dewey 2008 (1909), p.140)
13

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
To understand the details of this connection, reflective thinking is needed. Dewey
defined this as “active, persistent, and careful consideration of any belief or supposed
form of knowledge in the light of the grounds that support it and the further conclusions
in the direction of which it tends” (Dewey 1933, p.133).
Kolb, Rubin and McIntyre (1971) presented a model of the learning process termed an
experiential learning model, including a step of reflection (“Observations and
reflections”) preceding the formation of abstract concepts and generalizations.
Figure 4: The model of experiential learning (Kolb et al. 1971)
Stahl (2002) presents a model of collaborative knowledge building which integrates
cycles of „building personal knowing‟ with a cycle of „building collaborative knowing‟.
The model is fairly complex, including steps of externalizing collaborative knowing
into cultural artifacts which again mediate activity and thereby individuals‟ personal
knowing and contribution to the collective discourse and interaction. While Kolb places
stronger focus on experience and Stahl on knowledge building, the models can both be
seen as varieties of the learning cycle: the collective version of Kolb‟s „formation of
abstract concepts and generalizations‟ of which implications are to be tested in new
situations (arguably) corresponds to Stahl‟s externalizing of collaborative knowing into
cultural artifacts which again mediate activity.
Experience is not purely rational, and not all experience can easily be expressed. This
point is made by McCarthy and Wright (2004), addressing experience with technology:
“Developing an account of felt experience with technology is difficult partially because
the word „experience‟ is simultaneously rich and elusive. It is also difficult because we
can never step out of experience and look at it in a detached way” (p.15).
The complexity of experience and the necessity to consider more than its rational and
easily observable aspects are captured in the model of the reflective process (Figure 5)
14

Work, learning and reflection: theoretical background
presented by Boud, Keogh and Walker (1985a; 1985b), who extend Kolb‟s learning
cycle and go further into details on the constituent elements of reflection.
Figure 5: A model of the reflective process. From (Boud et al. 1985a)
Experiences(s) is what is reflected upon, including behaviour, ideas and feelings. (One
may for instance think of the experience of collaborating with others in a project over a
period of time.) The reflective process consists of returning to experience, attending to
feelings, and re-evaluating experience (e.g. considering what was good and bad about a
project process). Theis model indicates a possible iteration between experiencing and
reflecting. Outcomes include new perspectives on the experience reflected upon, change
in behaviour, readiness for application, and commitment to action (e.g. to change an
ongoing project in another direction or to make sure to do a certain task in a certain
phase of the next project). The authors stress that an instance of reflection need not
contain all the elements outlined in the model. Also, there is no fixed sequence of steps
in the process. Still, with its checklist of elements of a reflective process and its
grounding in learning theory, the model can be used both to understand reflective
processes and to aid the design for them. In the thesis, the model has been used in the
research presented in P6 and P7.
From a DCog perspective, reflection can be understood as an intrinsic aspect of how
information is used (i.e. through simultaneous consideration of multiple representations,
internal and external). Information is seen as interactive and “used cognitively and
socially. By interactive, it is meant that external representations used in situ are
interpreted in coordination with the implicitly shared and individual knowledge of
current events and work practices” (Rogers and Ellis 1994, p.131). Bruner, in
accounting for human meaning making (Bruner 1990), touches upon similar issues. He
argues that meaning making is about negotiating and renegotiating by the mediation of
narrative interpretation: “ [] human beings, in interacting with another, form a sense of
the canonical and ordinary as a background against which to interpret and give narrative
meaning to breaches in and deviations from „normal‟ states of the human condition”
15

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
(Bruner 1990, p.67). The comparison of different representations can be seen as a key
point in reflection. For instance, a project team‟s reflection on their project process may
entail the team members‟ consideration and interpretation of an external representation
of the process created by one team member, or the team‟s collaboratively constructed
timeline of project events considered in light of the „ideal‟ process from the point of
view of a certain methodology for conducting the project work.
Many aspects of work experience worth learning from, and thus worth returning to in a
reflective process, are tacit (Polanyi 1966) and not immediately accessible to thought
and discussion. Hildrum argues that sharing of some tacit knowledge through
codification is possible between participants who have a shared frame of reference
(Hildrum 2009). From the perspective of social learning theory (Bandura 1996) learning
from tacit knowledge can happen through observational learning, which includes
observation and elaboration of symbolic models of the experience. These are examples
of research on learning that can be used to underpin the idea that reflection, by aid of
appropriate external representations (e.g. as seen within DCog), can help participants
learn from aspects of work that are in themselves less explicit. In the thesis, project
team members‟ externalization of knowledge about their project process as an important
part of reflection, is a main concern in P6, P7 and P8.
Finally, reflection as part of a work practice is to a large extent integral to daily work,
taking the form of reflection-in-action. Sometimes reflection requires more distance
from the activity, which can be seen as reflection-on-action. (Schön 1983; Schön 1987).
this thesis, while not diminishing the significance of reflection-in-action, has a focus on
reflection with some distance to the process reflected upon, looking particularly at
facilitated retrospective reflection (P5-P8).
2.3 Trajectory as representation of a process
Strauss (1993) argues that when people make sense of a process (e.g. one of cooperative
work), they think of it as a trajectory. A trajectory is “(1) the course of any experienced
phenomenon as it evolves over time” and “(2) the actions and interactions contributing
to its evolution” (pp.53-54). Trajectories can be individual or shared, and they can be
past ones or projected future ones. The role of trajectories as seen by Strauss bears
resemblance to the role of narratives in meaning making (Bruner 1990).
Strauss‟ trajectory concept is part of his theory of action, a framework for understanding
human acting and social phenomena, influenced by pragmatism (e.g. (Dewey 1978
(1910))) and interactionism (e.g. (Blumer 1986; Mead 1934)). The theory of action was
used by Fitzpatrick in the Locales framework (Fitzpatrick 2003), illustrating the
16

Work, learning and reflection: theoretical background
usefulness of, for example, the trajectory concept in the context of understanding
cooperative work and support for that work.
The basic assumptions about social interaction underlying Strauss‟ theory of action are
compatible with the theoretical underpinnings of the work in this thesis. The trajectory
concept has been applied in the research papers P5-P8 as a core concept shedding light
on how team members conceptualize a project process and what may be adequate
external representations of a project process when people (e.g. project team members)
are to reconstruct, discuss and reflect on their process.
17

3 Software Engineering student projects: state-of-theart
In this section, I provide a brief overview of state-of-the-art research identifying
challenges that are addressed through the research contributions of the thesis. The
chapter is organized in line with the research questions RQ1-RQ4
3.1 The characteristics of SE student projects
A first main characteristic of SE student projects is that they are instances of software
engineering. The IEEE Computer Society defines software engineering as: "(1) The
application of a systematic, disciplined, quantifiable approach to the development,
operation, and maintenance of software; that is, the application of engineering to
software. (2) The study of approaches as in (1)." (IEEE 1990). The knowledge areas
associated with software engineering (Abran et al. 2004) comprise technical knowledge
as well as knowledge of the human and social aspects of cooperation. Software
development constitutes “an exercise in complex interrelationships” with frequent adhoc
collaboration (Cherry and Robillard 2008). There may be many project stakeholders
(e.g. customers, users, technology providers), each with their own perspectives and
interests (Poole 2003). Collaboration with the stakeholders is essential and frequently
challenging (Brady et al. 2006). The complexity of design work (Carstensen and
Schmidt 2002 (1999)) makes SE work unpredictable and difficult to coordinate. The
many project failures in SE industry (Keil et al. 2000) point to a need to learn from
experience (Lyytinen and Robey 1999). The SE student projects of the case studies in
this thesis have external customers, which contributes to a realistic complexity of work.
Based on recent increased understanding of the need for flexibility and adaptability to
change in many SE projects, many popular software development methodologies are
based on iterations and incremental development (Boehm 1988; Kruchten 2003) and –
in case of agile methodologies – an increased focus on user/customer interaction rather
than extensive documentation (Beck 2000; Cockburn 2006; Dybå and Dingsøyr 2008)
(Germain and Robillard 2004). In a given software development project, the choice of
development methodology should be made with the specific characteristics of that
project in mind (Cockburn 2000). Project management is to some extent linked to the
19

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
development methodology but is not necessarily strongly supported within the
development methodology itself. In this thesis, the SE student projects of the case
studies generally use iterative methodologies to structure their work.
At this point there is a need for clarification about the use of the terms software
development (SD) and software engineering (SE), for which definitions vary in the
associated professions and research fields. This thesis concentrates on the SD
components of software engineering work. In the research papers I use the term
software engineering when I refer to and address the research fields and professions of
software engineering and SE education. In the publications for the CSCW and TEL
communities, the focus is on software development as a particular domain of work, and
I refer to software development. In the thesis introduction, to be consistent I use only the
term software engineering.
A second, major characteristic of SE student projects is that they are based on the
pedagogical approach of project based learning (PBL). To be considered an instance of
PBL, a project must have the following characteristics (Thomas 2000):
• be central, not peripheral to the curriculum
• be focused on questions or problems that “drive” students to encounter (and
struggle with) the central concepts and principles of a discipline
• involve students in a constructive investigation
• be student-driven to some significant degree
• be realistic, not like school work
PBL bears similarity to problem based learning, and the distinction between the two
approaches is not necessarily clean in practice (Prince and Felder 2007). One main
difference is that PBL, as opposed to problem based learning, involves the construction
of concrete artifacts (Helle et al. 2006). Another difference is that PBL is based on
students mainly applying knowledge that is previously acquired, with the final product
as the primary focus. Problem based learning generally has a stronger focus on the
solution process and no formal instruction to provide the necessary background for
solving the problem (Prince and Felder 2007).
As argued in Section 1.1, PBL resides at the intersection between education and work.
This creates a tension between the associated educational and practice logics (Dohn
2009). Educational practices are generally based on an acquisition metaphor of
learning, in which the goals are external to participation: the learner, by participating in
20

Software Engineering student projects: state-of-the-art
learning practice, should change in a way enabling more competent participation in
working life practices. In other words, the educational practice should mainly prepare
the learner for future participation in work life. At the same time, PBL is meant to
provide work practice, the goals of which are intrinsic to participation (e.g. being a
practitioner, proving oneself as a competent practitioner), solving the task being the
main proof of successful participation, and knowledge and competence being viewed as
situated doing. Dohn (2009) argues that PBL is one of the pedagogical approaches that
succeed in bridging the educational and practice logics by challenging the problem from
within, “bringing authentic working life problems into schools settings as authentic
problems (not just as illustrative examples) and/or by organizing educational tasks
within working life settings.” (p.354).
To provide support for PBL, it is useful to know how the students perceive the tension
between education and work, and, in the same vein, what are their objectives for project
participation. For instance, Berglund and Eckerdal (2006) found in a case study that
project students strive from three different motives: academic achievement, project and
team working capacity (including becoming a better professional), and social
competence. Students‟ priorities among various objectives, and how they see certain
tasks and aspects of project work as related to these objectives, have an impact on their
work and learning, and ultimately on what are the most useful ways of supporting work
and learning in the projects.
Also, students‟ understanding of the practices and objectives of the various CoPs
involved (e.g. the university, the project team, the customer organization, the work
profession) impacts on their collaboration with project stakeholders. Due to the
importance and challenges of stakeholder collaboration in Software Engineering, it is
acknowledged as vital to have SE project students learn to communicate with project
stakeholders (McMillan 1999; Poole 2003).
In this thesis, SE project students‟ objectives for project participation and understanding
of the objectives of other stakeholders (e.g. the university, the customer organization)
are explored in research paper P1. Collaboration between project teams and
stakeholders is addressed in the papers P2 and P3: P2 elaborating on a successful case
of project-critical and tool-mediated collaboration, and P3 discussing an unsuccessful
one.
21

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
3.2 Use of lightweight collaboration tools in SE student
projects
In project based learning, tools are used to support learning and work. Technology
introduced to support learning, what may be called learning technology, primarily has a
role within the education logics of PBL. Learning technology includes tools for
structuring learning activities (Xu 2007), having course staff communicate with,
monitor and evaluate students (Coates et al. 2005; Trentin 2009) or in other ways
support students‟ collaborative learning activities (e.g. building knowledge (Cress and
Kimmerle 2008), engaging in discussions (Zuhrieh 2009) and comparison with expert
thinking (Lin et al. 1999)). The use of software tools to support learners‟ construction of
knowledge representations to aid collaborative learning was addressed in Suthers and
Hundhausen (2001). Among the lightweight tools wikis are cited as particularly good
for collaborative knowledge construction and open, low-threshold participation
(Hickerson and Giglio 2009; Kim and Lee 2002; Lund and Smørdal 2006).
While acknowledging the relevance of learning technology for PBL, this thesis has a
focus on tools primarily used to support work, e.g. participation in SE practice. It is the
role and usage of a tool in a project that makes it a work tool: for instance a wiki may be
introduced by a SE team as a tool for project management.
An important part of effective project communication is to use appropriate technology
to support it (Burnett 2001). This points to the need for students in PBL not only to gain
experience with using collaboration technologies, for example in stakeholder
communication, but to the need to learn to judge the appropriateness of the technology
for the purpose.
The focus in the thesis is on the use of lightweight collaboration tools. These are tools
that can be acquired and used at low cost (e.g. money and time to learn) for individuals
and their organization. A lightweight collaboration tool typically provides a limited set
of features to support one aspect of collaborative work and may thus be relatively easily
integrated into existing work processes rather than imposing a certain process on the
user. Many lightweight collaboration tools are associated with Web 2.0 (Dohn 2009),
for example wikis, discussion forums, and instant messaging.
Lightweight collaboration tools are actively used in work life. Tools used to support
work practices include blogs, instant messaging, discussion forums and wikis (e.g.
Isaacs et al. 2002a; Lovejoy and Grudin 2003; Majchrzak et al. 2006; Muller et al.
2003; Nardi et al. 2000; Niinimaki 2008; Quan-Haase et al. 2005). In the area of
Software Engineering, textual chat and discussion forums have been found to be
22

Software Engineering student projects: state-of-the-art
powerful tools in distributed software development (Gutwin et al. 2004a; Gutwin et al.
2004b; Herbsleb et al. 2001). Wikis have been found useful for supporting various
aspects of SE work (Louridas 2006; Radziwill and Shelton 2004) including stakeholder
participation in requirements engineering (Decker et al. 2007).
Lightweight collaboration tools that are used in professional work are also used in other
areas of life and/or among young people in particular (Baron 2004; Grinter and Palen
2002; Huang and Yen 2003).
In the thesis P3 provides a brief overview of the research literature on the use of instant
messaging in project work, and P4 and P5 outlines relevant research on the use of wikis
in SE project work as well as in student projects in particular.
As lightweight collaboration technology continuously develops along with their patterns
of use, there is a need for research investigating how these tools support the activities
for which they are used (e.g. in software engineering and project based learning). This
thesis looks into current usage in SE project teams of the following tools: internet
discussion forums (P2), instant messaging (P3), project wikis (P4) and issue trackers (a
type of lightweight project management tool) (P8). Also, the concerted use of tools,
including email, is addressed in several of the papers and is particularly central to the
main argumentation of P7.
Social web spaces like Facebook and MySpace are being actively used among students,
and researchers have recently begun looking into the potential of such tools to support
learning (Durkee et al. 2009; Idris and Wang 2009). From the empirical studies
underlying this thesis there is no indication, informally assessed, that these tools are
currently taken into use in the SE student projects to support project-related
collaboration, or that other use of social web spaces has any significant impact on the
project work. The use of social web spaces was defined to be outside the scope of this
thesis.
3.3 Supporting reflection on SE work
In educational contexts, the need to support reflection on learning processes (e.g. in
project work) has been addressed by traditional approaches such as the use of reflection
notes and diaries (Gleaves et al. 2007). Computerized tools designed to support
reflection have been developed, some focusing on individual action and some on social
discourse, with different ways of recording and representing the reflection and its
subject matter (Kim and Lee 2002). A more recent example is the „challenges
assessment workspace‟ developed for project groups doing distributed teamwork (Xiao
et al. 2008). After each project phase, the team had a structured assignment to
23

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
collaboratively identify and evaluate (using shared spreadsheets and whiteboards) the
challenges of teamwork in that phase of their project. The work of Xiao and colleagues
exemplifies the use of a structured approach to reflection with the aid of a defined
procedure, a shared workspace and specified external representations.
Turning to project work in industry, „lessons learned‟ – key project experiences with
some general business relevance for future projects – has recently been incorporated as
a concept in the Project Management Book of Knowledge (PMBOK)
(ProjectManagementInstitute 2008). This points to a general acknowledgement in the
project management field that learning from project experience needs to focus on a
useful outcome. Further, to identify lessons learned from a project process, it is not only
the „what‟, „where‟ and „how many‟ questions that need to be addressed, but also the
„why‟ and „how‟ questions (Schindler and Eppler 2003). To achieve this it may be
necessary to tap into participants‟ experience to try to learn from the tacit aspects of
project work (Eraut and Hirsch 2007).
Project debriefings are structured approaches to reflection intended to help projects and
organizations capture and use experience data from projects (Schindler and Eppler
2003). Schindler and Eppler examined a large body of empirical data on project-centred
learning in various types of organizations. They found that current practices, as
compared to current theoretical insights, require “significant improvements with regard
to the format, process, and use of lessons learned”. Schindler and Eppler list the
following success factors for gaining from lessons learned in project debriefing
workshops: 1) Have the entire project team regularly capture important project
experiences after major milestones, 2) Have an external, neutral moderator of the
debriefing workshops, 3) Aid the collection and structuring of lessons learned by a
graphical representation in the form of a timeline and document the workshop in a
poster format visible for all involved 4) Ensure that individual experiences are evaluated
and analyzed in collective and interactive sessions, and 5) Gain commitment from
gathered insights in the form of action consequences (i.e. what to do and by whom).
In the Software Engineering industry, the prevalence and cost of project overruns and
failures (Keil et al. 2000) means the motivation to have teams and organizations learn
from experience is strong. Efforts to achieve such learning may be integrated into
predefined work processes and supported by tool functionality for collecting and
providing experience data. The idea of experience factories (Basili and Caldiera 1995)
reflects a similar idea. In other settings, as in agile software development (Dybå and
Dingsøyr 2008), learning from experience may be more informal, taking place in faceto-face
project meetings and retrospective reflection sessions. Project retrospectives
(Derby et al. 2006; Kerth 2001) in agile SE, conducted at the end of project phases or
24

Software Engineering student projects: state-of-the-art
when projects are terminated, are an example of deliberate efforts to learn from
experience, formally integrated into the work process (i.e. the development
methodology). The major resources are participants‟ memory and collaborative effort to
reconstruct the project. Various facilitation techniques exist (Bjørnson et al. 2009;
Dingsøyr 2005) to help project participants gain an understanding of the „facts and
feelings‟ of their project process (thus approaching its tacit aspects), identify and
prioritize challenges and decide on how to address them.
Providing support for work and learning based on representations of the work processes
is a core idea in fields related to the management of work, for example in organizational
learning and knowledge management (Walsham 2001). In project based learning,
support for learning can to some extent be based on scaffolding (Wood et al. 1976) with
the aid of pre-defined approaches (such as those outlined in formally defined software
development process models (Bygstad et al. 2009; Rooij 2009)). Lin and colleagues
(Lin et al. 1999) point to four different ways of providing support for learners‟
awareness about their learning process: process displays, showing learners explicitly
what they are doing by making „normally tacit learning processes explicit and overt‟
(p.47); process prompting (making students explain and evaluate their actions before,
during or after problem-solving); process modelling focusing on how an expert would
approach the problem; and finally reflective social discourse where the individual gets
feedback from the community and accordingly modifies the practices. The last category
includes using collaboratively constructed, external representations of the learning
subject matter, to support students‟ and teachers‟ sense making of the learning process
(Pea 1994).
In project based learning with a high level of independence in organizing the projects,
and with project tasks that are not clearly defined in advance, the possibility to prespecify
the desired work or learning process is limited. Reflection and learning from
experience must to a large extent be based on participants‟ collaborative construction of
knowledge about their process and results. This can be achieved by the use of state-ofthe-art
project retrospective techniques. In the thesis, research papers P6-P8 describe
studies in which a project retrospective technique was adapted to, and applied in, a SE
project course. The timeline and satisfaction curve approach (Derby et al. 2006; Kerth
2001) supports the capturing and visualizing of participants‟ project experience,
including its affective aspects. The focus of P5 is also on retrospective reflection on the
project process, but in this case the reflection is facilitated by post-mortem interviews.
25

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
3.4 Using work tools as tools for learning
In project work in industry, lessons learned are frequently not elicited and documented.
Among the reasons are a lack of integration of experience recording into the project
processes and team members believing that coding their experience is ineffective and/or
not personally useful to them (Schindler and Eppler 2003). In the Software Engineering
field, Kasi and colleagues (Kasi et al. 2008) conducted a Delphi study to find what
experienced IT practitioners see as the most important barriers to conducting postmortem
evaluations. Among the types of challenges to successful post-mortem
evaluations found in the study was the lack of adequate data. „Hard data‟ about a project
can help focus discussion in post-mortem evaluations (Collier et al. 1996). These
findings point to the significance of capturing project data that is relevant for team
members‟ learning, and the need to facilitate reflection by having participants construct
knowledge representations in a way perceived as feasible and meaningful.
Information relevant to a work process can be captured from tools and artifacts already
used to support that work process. The use of logs from a code management system to
support coordination in a SD team has been proposed (Fitzpatrick et al. 2006). In that
study, an event notification system and a tickertape tool for CVS (Concurrent Versions
System, a popular version control system) messages was integrated with the code
management system, improving communication in the team. The project memory tool
Hipikat was designed to help new members of a distributed SD team learn from more
experienced colleagues (Cubranic et al. 2004). The memory contains the artifacts
produced during development and is built by the tool with little or no changes to the
existing work practices. Tools for systematically capturing design rationale have been
developed, e.g. IBIS (Conklin and Yakemovic 1991) and SEURAT (Burge and Brown
2008). SEURAT captures design rationale in software development and can also trace
requirements and decisions (Burge and Kiper 2008) in a project. (Omoronyia et al.
2009) outline an approach in which awareness support for software development is
provided through an awareness model derived from data originating in developers‟
interaction in the shared workspace. Some approaches to capturing data from a work
process involve users‟ deliberate tagging (performed as an integral part of the work
process) of information relevant to others (Dekel and Herbsleb 2008; Storey et al.
2005).
SE student projects represent a work and learning setting in which tool usage is
dominated by lightweight technology. As part of their use in day-to-day work, these
tools capture data (e.g. in logs, archives and various types of overviews) that may be of
use to aid reflection by the teams on their project process. The potential of collaboration
tools to aid project reflection through team members‟ examination of historical data can
26

Software Engineering student projects: state-of-the-art
be examined in an informal way by having team members browse historical data as part
of a post-mortem interview about their project. Such a study is reported in P5.
A recent study found that in SD projects, data repositories from the development
process (bug databases and email repositories) could help researchers reconstruct the
bug fixing „stories‟, but the data were not complete and correct enough to get the full
story deemed necessary to understand and support coordination of the SD work (Aranda
and Venolia 2009). One of the biggest problems with the repository data was missing
links from the bug records to the source code involved. Also, even with a bug database
and email repositories in combination, it turned out to be difficult to unveil the full
network of participants involved in the process, and clearly see who had been doing
what. The researchers got the full story of the bug fixing, with its social, organizational
and technical aspects, by eliciting the knowledge of the participants in interviews.
The use of historical data in collaboration tools to support reflection can be explored by
incorporating such use of data into reflection efforts organized as part of the work
practice. This is in line with the pedagogical rationale of project based learning. In this
thesis, the empirical study presented in P7 and P8 combines the timeline and
satisfaction curve project retrospective approach with the use of historical data in a
collaboration tool (i.e. an issue tracker) to investigate what may be the benefit of
introducing the tool in this setting. P5 also explores the use of historical data in
retrospective reflection, but framed within a tool-mediated post-mortem interview about
the project.
In the research literature on learning there are many models outlining learning cycles
from an educational or organizational perspective. Some of these models assign a key
role to knowledge representations (see Section 2.2). However, to the knowledge of this
author, none of these models outline the use of collaboration tools applied in the work
process as collectors and sources of data about the work process to be used in reflection
on that process. With the aid of a distributed cognition perspective, this challenge is
undertaken in P7 and P8 in the thesis.
27

4 Research approach and research process
I write impressions during the research, after each interview, for example.
I generate more organized sets of themes and issues after a group of
interviews or a major field visit. I then try to think about what I have learnt
so far from my field data. If this sounds a rather subjective and relatively
unplanned process, well it is. I believe that the researcher‟s best tool for
analysis is his or her own mind, supplemented by the minds of others when
work and ideas are exposed to them.
(Walsham 2006)
Simply observing learning and cognition as they naturally occur in the
world is not adequate given that learning scientists frequently have
transformative agendas
(Barab and Squire 2004)
Design experiments are conducted to develop theories, not merely to
empirically tune „what works‟
(Cobb et al. 2003)
The research of this thesis combines the use of interpretive case studies (Klein and
Myers 1999; Walsham 2006) and design research (Kelly 2003). As research approaches,
they have in common a focus on trying to understand specific settings; in the case of
design research with explicit intent to make an improvement. Interpretive case studies
can be used to build a thorough understanding of the setting to be designed for.
The combination of interpretive and design research in the thesis reflects an intent to do
exploratory studies in an early phase of the work and, on the basis of relevant theory
and empirically identified challenges, try out possible solutions and use the results to
contribute to theory and practice.
29

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
The two longitudinal field studies of single project teams (P2, P7) draw on naturalistic
observation and the collection of a wide array of qualitative data and can be considered
as ethnomethodologically informed (Randall et al. 2007).
For the studies presented in P1 and P3-P5 data has been collected across the teams of
project courses (all teams, or a selection based, for example, on the use of a specific
collaboration technology). The data collection was based on semi-structured interviews,
various types of project documentation and access to logs from collaboration tools.
The last part of the research involved tool design and intervention into the organization
of a project course, with the objective of answering research questions RQ3 and RQ4. A
prototype tool was developed, extending current functionality in a wiki tool (P4). It was
tested mainly through expert evaluation of scenarios of use. Interventions to the project
course were made with the aim of trying out approaches to retrospective reflection (P6-
P8) at the end of the projects. Seen as a step towards improving the organization of the
project course and informing the implementation of similar approaches to retrospective
reflection in other project based learning settings, this research can be considered as
initial steps of a design research process.
This chapter gives a chronological account of the research process. An evaluation of the
research as interpretive research and design research is provided in Chapter 7.
I start by briefly explaining my professional role with respect to the institutions in
which the research took place. In the period 2000-2004 I was affiliated with NITH as
academic staff, being responsible for, among other things, teaching and supervision in
project courses. At NTNU, starting my work as a PhD student in 2005, I was course
staff for the project course IT2901 in 2006 (as project supervisor), 2007 (as course
coordinator) and 2008 (as course coordinator in the first half of the semester).
Figure 6: Timeline indicating when the data collection for the empirical studies
took place
30

Research approach and research process
The diagram in Figure 6 gives an overview of the major research activities of the PhD
work (to be elaborated shortly) by showing how the data collection for the various
studies was distributed in time.
A more detailed overview of the research activities, showing their connection to the
research questions and publications, is given in Table 2. The table also outlines which
activities were carried out in the different educational institutions (NITH and NTNU).
Table 2: Empirical research activity vs. research questions and research papers
Institution
/ project
course
NITH/
PJ201
NTNU/
IT2901
Year of
empirical
study
Participants Research activity RQs Research
paper
2006 7 of 11 teams Interviews 1-2 P1, P7
6 of 14
customers
Interviews + analysis of
customer evaluation
1
All customers 1
Team V + After-the-project 1-2 P3, P7
customer interview + analysis of
project documentation
2007 Team A + Longitudinal study 1-2 P2, P7
project
stakeholders
4 teams (those
with project
wiki)
All teams
After-the-project
interviews + analysis of
project documentation
Interviews + analysis of
project documentation
2008 Development of wiki
walkthrough tool
All teams Interviews + analysis of
project documentation
Team F + Longitudinal study +
project retrospective reflection
stakeholders workshop
All teams Retrospective reflection
workshops
1-2-
3-4
P4, P5,
P7
1-2 P3, P5
3-4 P5, P7
1-2 P3
1-2-
3-4
P8, P7
3 P6, P7
As can be seen from Figure 6 and Table 2, the collection of empirical data took place in
the period 2006-2008. The research papers were published in the period 2007-2010.
The PhD research activity in 2005 mainly involved getting into the CSCW field and
included teaching a CSCW graduate course.
31

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
In 2006, empirical studies were conducted at NITH and at NTNU. A longitudinal study
of a NITH team involving some days of recorded observation over a semester was
useful because it informed the focus of further research, but the study itself did not
produce results that were published. The study reported in P1 addressed crosscommunity
collaboration in capstone SE projects with external customers, drawing on
data from semi-structured interviews with project students and customers in the
undergraduate SE project course PJ501 at NITH, and interviews with customers from
the NTNU project course IT2901 which was found to be similar enough to allow a
combination of data across the courses. The study was conducted in collaboration with
Bendik Bygstad at NITH.
Team V at NTNU was identified as a particularly interesting case of instant messaging
(IM) use, experiencing highly problematic team-customer collaboration over this
medium. Based on the IM log and follow-up interviews with the project manager and
the customer, the findings were reported in P3.
Data from the 2006 studies also informed a theoretical reflection paper (Krogstie and
Divitini 2007) on how the projects can be seen as a case of mobile learning, the
scaffolding of which should focus on boundary objects between interacting
communities (see Appendix B).
From 2007 on, all data were collected from the project course IT2901 at NTNU. A
longitudinal field study was conducted with Team A (Figure 7). It was made clear to the
team that I would have no role in the evaluation of their work and provide no
supervision. The study involved 75 hours of naturalistic observation of work sessions
and meetings with project stakeholders. Data collection included field notes, photos and
audio recordings from the observed work sessions, the team‟s email communication,
artifacts involved in the project work, project documentation at various stages of
development, follow-up interviews with the customer and the team right after the
project, and two team members reading through a draft version of P2. Brokering
towards an OSS development community was identified as a topic for P2, and as part of
the analysis the field notes were used to create a chronology of project events.
32

Research approach and research process
Figure 7: Team A having a meeting at the customer's site. The customer (second
from the right) has ordered pizza.
Data on the use of collaboration tools were collected across all teams in the 2007 cohort
of the project course through mid-semester interviews and through inspection of project
deliverables, particularly the reflection notes (for which a template had been made to
make sure that the use of collaboration technology was addressed). These data were
later used in P3 and P7.
Four 2007 teams were using project wikis, and three used them actively throughout
their project. Data on wiki usage was collected by examining the wikis and interviewing
students from the three active teams after project completion. Results on the use of
wikis in daily work were published in P4. Two of the retrospective interviews were
aided by the examination of wiki contents, a form of mediation turning out to be so
useful for project participants‟ recall and reflection that it was made a research topic
informing the analysis of the interviews. The results were presented in P5.
Pursuing the potential of wikis to aid retrospective reflection and the possibility to
improve existing wiki functionality in this respect, I took the role of customer for a
project team (Team W) in the spring semester of 2008. Team W developed the Wiki
Walkthrough Tool (WWT), a prototype tool implemented as an add-on to an existing
wiki tool (see Section 5.5) and presented in P4. The tool was tested in Team W‟s own
project wiki in the last phase of their project. Additionally, use scenarios for the tool
were evaluated by two expert teams, one of former project customers and one of project
supervisors.
In 2008, data on the use of collaboration tools in the teams were again collected through
mid-semester interviews with all teams as well as examination of the teams‟ reflection
notes. The results were used in P3 and P7. Continuing the work on ways to support
students‟ reflection on the project process, I introduced retrospective reflection
33

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
workshops at the end of the semester, adapting the timeline and satisfaction curve
approach from SE industry (Figure 8) (see 5.6). I was the workshop facilitator,
„external‟ by having no involvement with the evaluation of the projects and keeping the
outcomes confidential apart from anonymous use in research. The results were
published in P6.
Figure 8: The whiteboard after a reflection workshop with a student team. (The
photo has been manipulated to make the experience curves more visible in print).
Throughout the spring semester 2008, a longitudinal study was made of Team F. The
approach was similar to the one used for the study of Team A in 2007. I had good
access to project artifacts and the developing status of the project work, being updated
through the team‟s email list and by looking into the issue tracker (Trac) used to
manage the development work. Sample field notes are shown in Figure 9. During
observation sessions, field notes were made on a PC. Pictures were taken and included
directly in the electronic document as needed. After the sessions, the notes were cut and
pasted into a paper notebook and supplemented with handwritten comments.
A retrospective reflection workshop was conducted with Team F at the end of the
semester. This was an adapted version of the timeline and satisfaction curve approach
used in the workshops for all the other teams in the course. In the case of team F, the
use of historical data in Trac was incorporated into the workshop design to see if
participants could use the tool to recall project events they had not recalled by memory
alone, and if this had an impact on their reflection. The workshop lasted 3 days
altogether including individual and collective sessions (for details, see P7). The
workshop was video recorded in a usability lab, providing synchronized screen
recordings of the use of Trac and the team members‟ activity in the room. Analysis of
the workshop data acknowledged background knowledge of the team‟s project process
gained through the longitudinal study. Results were published in P7 and P8. P7 details
34

Research approach and research process
the use of Trac in the context of reflection (e.g. important tool features). In P8, the
results are generalized to a model of reflection incorporating the use of historical data in
collaboration tools.
Figure 9: Field notes.
35

5 Results
This chapter presents an overview of the results paper by paper. For each paper, the
original abstract is included, followed by a short description of the background for the
research and a brief elaboration on the results.
5.1 P1: Cross-Community Collaboration and Learning in
Customer-Driven Software Engineering Student Projects
Authors: Krogstie, Birgit and Bygstad, Bendik
I was the first author of this paper. Both authors participated in the research design, data
collection, analysis and writing process.
Published in: Proceedings of the 20th Conference on Software Engineering Education
and Training (CSEE&T) 2007
This paper explores collaboration and learning between stakeholders in customer-driven student
projects. The research objectives are to obtain empirically based knowledge on how students relate to
stakeholders in customer-driven projects, and to suggest implications for the pedagogical design of the
project courses. Empirical data was collected from two Bachelor courses in software engineering at two
learning institutions in Norway. To make sense of the interaction between the three stakeholders in the
project: the student groups, the university and the customer, we build on Wenger’s concept of
communities of practice and on the concept of boundary objects. Our analysis highlights that students,
through practical experience in the projects, learn to balance the requirements and expectations from
different stakeholders in designing a working technical solution - a valuable skill for software engineers.
We argue that for students to learn to balance stakeholders’ interests in the best possible way, visibility of
stakeholders’ goals should be focused throughout the projects. Explicit reference to the goals should be
incorporated into project artifacts serving as boundary objects. Collaboration technologies providing
standard shared workspace functionality are seen as adequate to support this.
Based on the authors‟ work experience with capstone SE projects as well as the data
collected in all teams of a capstone project course in 2006, the importance and
mechanisms of cross-community collaboration in the teams emerged as an interesting
issue. Relevant data were collected also from project customers, and the results of the
analysis were presented in P1.
In the paper, SE student projects are considered as a case of cross-community
collaboration. Project artifacts negotiated between the team and other stakeholders can
37

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
be seen as boundary objects. This perspective helps identify challenges and learning
potential associated with student teams‟ interaction with the different stakeholders, and
what support might be adequate to help the student teams in this interaction.
The paper ends with the recommendation that in the organization of the course and
supervision of the projects one should put explicit focus on the role and purpose of
project artifacts serving a role as boundary objects. Consciousness of the purpose of
such artifacts helps shed light on stakeholders‟ different objectives for project
participation and the potential role of the artifacts in supporting these objectives – two
issues that are, as shown in P1, frequently unclear to the students. As an example,
students typically regard some project artifacts as „school‟ deliverables and some as
„real‟ project artifacts relevant to the customer. Documentation of the product, for
example, conceptual models describing possible use scenarios, are frequently taken by
the students as belonging to the first category, but may in fact be useful when the
customer wants to continue developing the product. An open discussion between team
and customer about such issues may benefit collaboration and the project result. The
current use of a number of different collaboration tools in the projects point to a
potential for providing shared access among the collaborating stakeholders to boundary
objects and descriptions of the role and design of these objects
It is further proposed in P1 that if the students are fully aware of the purpose and role of
an artifact in collaboration, they may be allowed to determine for themselves (in
collaboration with the other stakeholder) what is the appropriate form and content of the
artefact. This contrasts with the mandatory use of pre-specified templates for, for
example, product requirements or status reports.
5.2 P2: Power through brokering. OSS participation in SE
projects
Author: Krogstie, Birgit
Published in: Proceedings of the International Conference on Software Engineering
(ICSE) 2008
Many software engineering projects use open source software tools or components. The project team’s
active participation in the open source community may be necessary for the team to use the technology.
Based on an in-depth field study of industry software engineering project students interacting with an
open source community, we find that participation in the community may affect the team’s work and
learning by strengthening the power of the broker between the team and the community. We outline
pitfalls and benefits of having student teams acquire development-related knowledge from open source
communities. The findings are relevant to the organization and supervision of software engineering
student projects interacting with open source communities.
38

Results
The background of P2 is a longitudinal study conducted with Team A in 2007. The
team‟s interaction with an open source software (OSS) developer community was
essential to the project‟s success. The paper gives a chronological account of the
project, focusing on the team‟s need for knowledge about an OSS development
framework they were required to use and how they approached the challenge of
acquiring this knowledge. This happened via confusion and hesitation through direct
contact with the developer community in the web forum and subsequent participation in
that community. The participation helped the team get the necessary resources for their
development and succeed with their project. The team‟s requests for functionality in the
development framework needed for their project led the OSS community to make
changes to the development framework itself.
The findings point to some pitfalls and benefits for SE education of having students
participate in OSS communities. Being a realistic aspect of modern SE work, OSS
community participation in a capstone project provides work experience highly relevant
to industry. The team member having the role as broker between the team and the
community is likely to be motivated by his/her role as an important contributor to the
team‟s work and possibly also as a contributor to the open source community.
Motivation and pride may be shared by the entire team, as seen in P2. On the other
hand, there are some pitfalls in projects depending on direct interaction with an OSS
community. Student teams may hesitate to try and participate actively, being in doubt
about their own skills and credibility, and this may cause problematic project delays.
Attempts to interact with the OSS community may fail if the team does not manage to
convince the community that they qualify as real users and potential contributors. The
OSS community becomes an extra project stakeholder to which the project has to relate,
which makes project management more challenging. For instance, participating on the
OSS arena and working on technical issues there – and receiving gratification for the
effort from other participants there – may move the focus from project tasks in need of
attention. Finally, brokering may be performed inadequately, for example if knowledge
is not properly shared within the team or the team‟s interests are not properly taken care
of in the OSS community.
Overall, P2 shows how cross-community collaboration in student projects extends the
traditional customer and course staff interaction, with implications for the dynamics and
learning of the team. In particular, successful interaction with an OSS community in an
Internet forum requires a certain communications competence.
39

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
5.3 P3: Do’s and Don’ts of Instant Messaging in Students’
Project Work
Author: Krogstie, Birgit
Published in: Proceedings of NOKOBIT 2009.
Instant messaging is a type of lightweight collaboration technology heavily used among students in
higher education for social interaction and for school work. This paper sheds light on students’ use of
instant messaging to support collaboration in software engineering student projects. We draw on several
years’ of research on the use of collaboration technology in such projects and present illustrative
examples of how instant messaging is used within the teams and for communication with other
stakeholders, e.g. project supervisor and customer. We find that the use of instant messaging in the
projects is generally successful, but in some cases it is inadequate, which may severely harm the project
outcome. Based on our findings, we discuss implications for the organization and supervision of SE
student projects..
The background of P3 was the finding that instant messaging (IM) is extensively used
in the SE student projects. Data on the use of collaboration tools by teams collected in
semi-structured interviews of project teams in 2006-2008 indicated that there are clear
patterns of use with this type of tool in the teams. These patterns are in line with the use
of IM in work life and in young people‟s lives as reported in research literature: IM is
used for informal communication, frequently as a substitute for face-to-face
conversation. My role as course staff and researcher provided me with knowledge of
particular IM-related issues in specific project teams in the 2006-2008 cohorts. Given
access to IM logs, I collected examples of successful as well as unsuccessful use of the
tool. Excerpts from such logs form the core of the paper. P3 concludes that generally,
IM is appropriate for informal, within-team collaboration and should be applied with
great caution (i.e. only under certain conditions) in formal stakeholder collaboration.
This has implications for the advice course staff should give to student teams about
collaboration.
5.4 P4: The wiki as an integrative tool in project work
Author: Krogstie, Birgit.
Published in: Proceedings of COOP 2008.
The paper provides insights on how wikis support project work and what characteristics of wikis make
them adequate for this purpose. The findings are based on a case study of software engineering projects
in an educational setting. Project wikis are found to serve an integrative role along several dimensions of
project work, enabled by the flexibility and automatic support for capturing history offered by the
technology. The findings demonstrate that a project wiki can serve as a knowledge repository, a means
for staging the project, a coordination mechanism, and a shared workspace. To many projects in need of
40

Results
project management and collaboration support, project wikis should be seen as an attractive, lightweight,
all-purpose alternative.
The background of P4 was the use of project wikis in many project teams in the 2007
cohort of the SE project course. The wiki seemed to be popular as a collaboration tool
and used for many purposes in the projects, and this led to my decision to investigate
the project wikis in more detail.
Figure 10: Screen shot from a project wiki main page as the project approached
final deadline. Note the strikethroughs of completed tasks in the to-do-list.
In P4 it is shown how lightweight project management tools can serve the needs of
small-scale SE project work. In particular, project wikis have a flexibility which makes
it possible to adapt them to a team‟s emerging needs, effectively providing integration
of work across several dimensions. It was found that the project wikis served to
integrate social and task-oriented aspects of work (see Figure 10, which includes
examples of both), integrate team-internal and team-external information, and integrate
work across artifacts. In this way, the wiki played different roles within the project: as a
knowledge repository for the team and other stakeholders, as a stage for the team to
41

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
provide a certain image of their project (internally and externally), as a coordination
mechanism, and as shared workspace.
5.5 P5: Using Project Wiki History to Reflect on the Project
Process
Author: Krogstie, Birgit
Published in: Proceedings of the 42nd Hawaii International Conference on System
Sciences (HICSS) 2009
In this paper, we address the potential of project wikis to support reflection on the project process
through participants’ reconstruction of the project trajectory. Drawing on a case study of software
engineering project work, we illustrate how information on project history can be found in a project wiki
and may be used to support recall of and reflection on the process. We discuss implications for
postmortem project reviews by considering how the utilization of project wikis could addresses some
challenges to successful reviews. We also propose extended wiki functionality that can be used to make a
more selective review of project history based on user-tagged contents.
The research presented in P5 was initiated based on findings from the research on dayto-day
use of project wikis in the capstone projects. In our interviews of project team
members about their use of project wikis, the wikis themselves were examined as an aid
to the interview. It turned out that recall and reflection was effectively aided by the
browsing of wiki contents. As a consequence I decided to focus not only on day-to-day
use of the wikis in the projects, but investigate the potential of the wikis to be used
retrospectively to aid recall of and reflection on the process.
P5 demonstrates that given the current use of wikis as lightweight project management
tools, there is a potential to utilize the historical information in the wikis for purposes of
recalling and reflecting on important aspects of the project process. Wiki contents (e.g.
the various revisions of the project wiki main page) can be chronologically traversed in
a „tour‟ of the project. By examining page contents and following hyper-links, the user
can inspect details related to the process and some of the project artifacts. The
combination of chronological traversal and inspection of details aids recall and
reflection.
P5 also identifies a potential for improvement of project wikis with the aim of providing
better support for retrospective reflection. More focused navigation of historical data
would be possible if the wiki allowed chronological browsing of the wiki pages
considered most relevant to retrospective reflection. What is relevant in this respect will
vary between projects and individual team members. Importantly, it should be possible
to chronologically browse relevant page revisions (e.g. of the main page, showing the
development of the team‟s frequently updated to do-list). The solution to this challenge
42

Results
proposed in P5 was to allow users to bookmark wiki page revisions with tags that are
individual or shared in the group. Tagging can be done as an integral part of day-to-day
work activity, for example tagging current page revisions, or retrospectively tagging
older page revisions based on current insights on what might be worth revisiting.
The wiki walkthrough tool (WWT) was developed, incorporating the above described
features. The WWT is a prototype plug-in to an existing wiki tool. It was integrated and
tested in the project wiki of the team developing the tool in the last phase of their
project. Various use scenarios of the tool were subject to expert evaluation (groups of
supervisors and customers). Some of the scenarios outlined use of the tool in project
management and some described pedagogical use (e.g. for project supervisors collecting
relevant information about a team‟s work throughout their project). The evaluation
indicated less enthusiasm for the project management potential of the tool and more for
its pedagogical potential. Also, it was suggested that the tool be extended to allow not
only bookmarking but also annotation of page revisions, e.g. to briefly explain what is
interesting about the pages. Figure 11 shows a screen shot from the tool. In the dialogue
window in the centre of the screen, the user is choosing to tag the present version of the
wiki page using the tag „gjennomgang‟ (in English: „examination‟ or „revision‟), which
is chosen from a list of recently used tags. In the window the user can also specify that
the tag is individual to that user (in this case „arnemart‟) and accordingly not shared
with, or visible to, the rest of the team.
Figure 11: Wiki Walkthrough Tool: adding a tag to the current version of a
page
43

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
5.6 P6: Shared timeline and individual experience: Supporting
retrospective reflection in student software engineering
teams
Authors: Krogstie, Birgit and Divitini, Monica
I was the first author of this paper and main responsible for setting up the study and
collecting and analyzing the data. The second author has provided feedback and
comments throughout the process.
Published in: Proceedings of the 22 nd Conference on Software Engineering Education
and Training (CSEE&T) 2009
To help SE student teams learn from their project process, we propose the use of facilitated postmortem
workshops in which each team reconstructs its project timeline. Individual team members’ experience of
the project is included by team members drawing individual ‘experience curves’ along the timeline. The
approach is based on current industry practice and adapted in accordance with theory on reflection and
learning. We present the detailed design of the workshops as they were implemented in an
undergraduate SE project course as well as some recommendations for future workshops. The
workshops are effective for promoting active participation, constructing a shared representation of the
project process, and revisiting project issues from multiple perspectives. Evaluation showed that students
were satisfied with the workshop and motivated to use the approach in later projects.
The background of the paper was the assumption that existing post-mortem analysis
approaches from SE industry may be successfully taken into use in SE student projects,
with some adaptation. This would give the students experience with a particular aspect
of SE practice and help them learn from their project experience. The feasibility of the
approach in terms of time (for students and staff) was a key constraint in the adaptation
to the educational setting.
The paper presents a concrete method for supporting retrospective reflection in student
SE teams. The method is adapted from state-of-the-art SE industry practice and is based
on the individual and collective construction of a timeline of project events and the
drawing of individual experience curves along the timeline, indicating how positive or
negative the experience was at different points in time. The timeline and curves serve as
a resource for the team‟s discussion of lessons learned. In the student projects, the
approach was used in a short 60 minute workshop at the end of the project. There was
no formal deliverable from the workshop, but the students were recommended to draw
on what they had learned when writing their reflection notes (one for each team) to be
handed in a few days later. The timetable of the workshop is shown in Table 3.
44

Results
Analysis of the results showed that the workshop helped students share their individual
perspectives (see Figure 12) and develop new, shared insights, i.e. knowledge, about
their projects.
The techniques applied in the workshop were chosen from existing approaches in SE
industry. Thus, the novelty of the design primarily lies in the integration of the
technique into an educational setting, requiring only minor adaptation of the workshop
design itself (e.g. aiming for a short duration and integrating questions relevant for the
reflection notes).
Task
name
Intro
(5 min)
Individual
timelines
(5 min)
Shared
timeline
(15 min)
Individual
experience
curves
(5 min)
Present
curves
(10 min)
Questions
about roles
& lessons
learned
(5 min)
Present
answers
(10 min)
Wrap-up
(5 min)
Description
Explain the purpose and agenda of the workshop. Clarify issues of
confidentiality and research
Each participant gets an A3 sheet of paper with a timeline reporting
common events in the course (mainly the deliverables). The participants
are asked to individually add events that they perceived had an impact on
their project.
Participants take turn in explaining the events they have listed The
facilitator marks the events on the whiteboard on a timeline similar to the
one on the individual sheets.
The team members each draw their experience curve (or „satisfaction
curve‟) on the A3 sheet. The smiley face on top of the sheet indicates a
level of great satisfaction. Down at the bottom is great dissatisfaction, and
the timeline itself marks a neutral position in the middle.
Each member in turn goes to the whiteboard, which holds the shared
timeline. The team member first draws her curve with her whiteboard
marker, next explains its shape. At the end of the session, all team
members‟ experience curves can be found on the whiteboard.
A sheet of incomplete statements addressing the project are uncovered,
and the students are asked to turn their A3 sheet and write their answers
on the blank page.
1) “In the project, my role was…” 2) “Through the project, I got better
at…” 3) “In a similar project, I would like to become more skilled at…”
4) “The most important thing I have learnt about software engineering in
this project is…”
The answers to the questions are presented around the table
The students may add further comments about their project on their sheet.
If formal evaluation of the workshop is not done on another occasion, this
is an opportunity to have feedback, oral or written. The A3 sheets are left
for the facilitator/course staff.
Table 3: Timetable of the retrospective reflection workshop conducted with
each of the teams in a SE project course. From P6.
45

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Figure 12: Snapshot of the whiteboard after the reflection workshop of one
project team, showing the shared project timeline with the individual satisfaction
curves. The numbers along Max’s curve indicate where he explained this
experience to the rest of the team. From P6.
5.7 P7: A model of retrospective reflection in project based
learning utilizing historical data in collaborative tools
Author: Krogstie, Birgit
Published in: Proceedings of EC-TEL 2009
In project based learning, learning from experience is vital and necessitates reflection. Retrospective
reflection is a conscious, collaborative effort to systematically re-examine a process in order to learn
from it. In software development student projects it has been empirically shown that project teams’
retrospective reflection can help the teams collaboratively construct new knowledge about their process
and that historical data in collaborative tools used in daily project work can aid the teams’ recall and
reflection on the different aspects of project work. In this paper, we draw on these results as well as other
findings on the use of collaborative tools in a similar setting. We use the framework of distributed
cognition to develop a model of retrospective reflection in which collaborative tools used as cognitive
tools for daily project work are utilized as cognitive tools in retrospective reflection, aiding the creation
of individual and shared representations of the project process.
P7 sheds light on the potential to use historical data in various types of collaboration
tools in retrospective reflection. A reason to utilize different tools is that they are
applied for various purposes in day-to-day project work and, as a consequence, contain
historical data on different aspects of the project process. The relevance of historical
information in project wikis and issue trackers was demonstrated in P5 and P8. Email
46

Results
archives are found to contain data that may shed light on the team‟s collaboration with
project stakeholders, which often entails challenges. The contents of email archives are
generally considered as formal documentation of the project, and it is reasonably easy to
search and retrieve contents related to some topic or stakeholder relationship. Some
tools are less adequate for use in retrospective reflection, partially due to lack of
features for systematic and efficient retrieval of the data and partially because they are
considered informal. For instance, the case studies of the thesis show that the instant
messaging logs of members of a project team may contain information with a potential
to shed light on important issues in the project (P3, P7). However, instant messaging in
the context of project work is considered a substitute for informal face-to-face
interaction and frequently interwoven with communication related to other activity
(school and private). The resulting IM logs are not intended or expected to be reexamined
by people other than their owners.
Figure 13: Retrospective reflection as distributed cognition. From P7.
On this background, P7 presents a model of retrospective reflection, taking into account
that different collaboration tools may contain historical data potentially shedding light
on different aspects, or sub-trajectories, of the project process. The model is based on
47

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
the theoretical framework of DCog and the model of the reflective process proposed by
Boud et al. (1985a) and highlights the construction and transformation of
representations as a core mechanism in retrospective reflection on a project process.
Also, the model is inspired by the approach to retrospective reflection applied in student
teams in the studies described in P6 and P8. The model explicitly includes steps of
individual and collective reflection as well as resulting representations of the project
process. Essential to the model, although not heavily explored in the empirical work of
the thesis, is the feedback loop in which lessons learned from the project (in the form of
various representations of the project process) are fed back into the project work
practice, which could be in the same project or in another one
Another important point made in the paper is that the dual role of a collaboration tool
(in day-to-day work and retrospective reflection) might impact on both aspects of the
practice through participants‟ awareness that historical data will be re-examined at a
later stage. This awareness might lead to better information sharing as well as to
information hoarding.
5.8 P8: Supporting Reflection in Software Development with
Everyday Working Tools
Authors: Krogstie, Birgit and Divitini, Monica
I was the first author of this paper and main responsible for setting up the study and
collecting and analyzing the data. The second author has provided feedback and
comments throughout the process.
To appear in: Proceedings of COOP 2010
Through their day-to-day usage collaboration tools collect data on the work process. These data can be
used to aid participants’ retrospective reflection on the process. The paper shows how this can be done in
software development project work. Through a case study we demonstrate how retrospective reflection
was conducted by use of an industry approach to project retrospectives combined with the examination
of historical data in Trac, an issue tracker. The data helped the team reconstruct the project trajectory
by aiding the recall of significant events, leading to a shift in the team’s perspective on the project. The
success of the tool-aided retrospective reflection is attributed to its organization as well as the type of
historical data examined through the tool and the tool features for navigating the data. These insights
can be used to help project teams determine the potential of their tools to aid retrospective reflection.
The work presented in this paper took as its starting point the previous findings (P5)
about the potential usefulness of historical data in wikis to aid retrospective reflection.
In the spring semester of 2008, project wikis were no longer the popular choice for
project management in the teams of our project course. Instead, many student teams
used Trac – an issue tracking tool built on top of the file versioning system Subversion
48

Results
(SVN). Trac provides lightweight project management functionality closely integrated
with the software development work, and fits well for projects with a strong focus on
the latter. For this reason, the course staff (including the author) decided to offer an
infrastructure with SVN and Trac to the 2008 project teams, accompanied by a brief
introductory lecture. Several teams choose to use Trac. The research presented in P8
results from a case study of one of these teams: team F.
Figure 14: Examination of historical data in Trac. From P8.
After following team F in a longitudinal study throughout the semester, we conducted a
three day workshop at the end of the semester to investigate the possibility of improving
the SD retrospective technique of constructing individual and shared timelines and
drawing experience curves (used in the other projects in the course as described in P6).
The workshop design was expanded to incorporate steps of exploring historical data in
49

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Trac, and to incorporate individual sessions preceding the collective one. In the
individual sessions, after the students had constructed their timeline of the project, we
asked them to chronologically browse the timeline on the Trac main page see to see if
this would make them remember events that had not been remembered by memory
alone. This step would make it possible for us to observe differences between team
members in their use of, and benefit from, the historical information; differences that
might be attributed to their different roles in the project.
Findings from the study showed that the historical data accessed through the tool helped
the students remember events or detailed information about events (e.g. their exact
time). Whereas all team members remembered something new to add to the purely
memory-based timeline, the benefit of using Trac retrospectively was bigger for the two
lead programmers than for the other three team members. The event „pre-study coding‟
(e.g. the onset of coding work) was not remembered by anyone prior to use of Trac. The
use of Trac led the two lead programmers to remember and acknowledge that coding
work took place early in the project. In the collective timeline construction, pre-study
coding was brought into the shared timeline and achieved „official status‟ as important
in the team. However, as the follow-up interviews from the study demonstrated, there
were viewpoints among the three other team members on how the pre-study activity
was conducted that were not brought up, probably due to the dominance of the two lead
programmers and their version of what pre-study programming was about. This is
further elaborated in the evaluation of the research process in 7.3.1.
The findings from the study are summarized in three main points about what was
important to make the tool-aided reflection successful: the organization of the workshop
into individual and collaborative steps of knowledge construction with the use of
adequate representations of the project process; the tool features for giving
chronological overview of project events; and the tool features for navigating between
overview and detail. The latter features permitted access from the chronological
overview to there-and-then versions of project artifacts, thus supporting the close
linking of process and product that is characteristic of SE work. Based on the findings,
P8 outlines a set of guiding questions that can be used to help assess the potential of a
collaboration tool to aid reflection on SE work. The questions are intended as a step
towards a framework relating various types of collaboration tools to the aspects of dayto-day
SE work supported by the tools and the potential for the tools to aid retrospective
reflection.
P8 also provides an example of how the reflection model from P7 can be instantiated so
as to comprise selected elements of a specific work-reflection-learning cycle, in this
case focusing on the role of Trac in the case study of P8 (see Figure 15).
50

Results
Figure 15: A model outlining how Trac supports project work and retrospective
reflection on the project process in the case study of P8. From P8.
51

6 Contributions and implications
This chapter summarizes the research contributions and their implications. The chapter
is structured in accordance with the four main research topics shown in the diagram in
Figure 3.
6.1 Project work as cross-community collaboration
Contribution 1 of the thesis is an increased understanding of cross-community
collaboration in SE student teams, conceptualized in terms of communities of practice
and related to the use of collaboration tools. The contribution is an extension of state-ofthe-art
research on how stakeholder communication and collaboration, as a crucial part
of SE practice, is currently approached in SE education, and how it can be more
carefully attended to in capstone projects.
In order for student teams to conduct their cross-community collaboration in a reflective
and successful way they need to relate and align to the stakeholders‟ practices and the
associated objectives for involvement with the student project. As students‟
understanding of these objectives may be limited (P1), project courses should be
designed to increase students‟ consciousness about this issue and encourage openness
among project stakeholders about their objectives in the collaboration processes. Also,
team members are likely to have different simultaneous goals for their project
participation, linked to their multiple memberships in different communities. Making
these objectives explicit within the relevant collaboration settings basically corresponds
53

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
to properly clarifying expectations for the collaborative work activity. In the thesis, the
variation in team members‟ objectives for their project work is also illustrated in the
reflection workshops described in P6. The individual satisfaction curves along with the
explanations provided by their creators illustrate how the „ups and downs‟ of project
work relate to the team members‟ goals and expectations and also those of other
stakeholders (e.g. when the curves of all team members drop at a point where negative
feedback was provided by the supervisor, or when a team member explains that she was
unhappy with her task over a long period because she would have liked to learn new
technology in the project rather than just applying a technology she knew in advance).
Apart from being a source of insight for the project teams about such issues in their
projects, data from the reflection workshops of all the projects in a course can be used
by course staff or researchers to gain insights about the effect of the current course
design, e.g. in terms of stakeholder collaboration in the projects.
Participants in SE student teams generally have a clear conception of the usage of
different collaboration tools in their project. The choice of tool for a particular instance
of collaboration appears to depend on whether the collaboration is within the team or
between team and stakeholders, and whether it is formal or informal. A similar
categorization can be used by the researcher (e.g. project course staff) investigating the
usage of particular tools in project teams with the ultimate objective of providing
guidance about the appropriateness of particular tools for different purposes. Exploring
the use of instant messaging in the student teams (P3), it is shown that team-stakeholder
collaboration by use of instant messaging poses particular challenges. More generally, it
is suggested that the appropriateness of a tool for team-stakeholder collaboration be a
question of negotiation between the stakeholders. Even with agreement between
stakeholders, using a tool in a setting for which it is not generally considered
appropriate (e.g. instant messaging for formal meetings) should be done with great
caution.
By looking at cross-community collaboration as achieved through brokering (P2), the
thesis provides an understanding of how the dynamics of cross-community
collaboration is closely related to the role and competence of the individual team
member. Not only the team as a community, but the individual‟s role in the team and
competence in adapting to the practice of both communities must be considered to
understand how cross-community collaboration takes place through brokering and how
it might be supported.
Contribution 1 can be a resource for practitioners and researchers within SE Education,
providing rationale and guidelines for developing project courses in the direction of
54

Contributions and implications
more consciousness, insight and reflection among students and course staff about
stakeholder objectives and tool use in cross-community collaboration.
6.2 Lightweight collaboration tools in project work
Contribution 2 of this thesis is an increased understanding of the use of lightweight
collaboration tools in the work practice of SE student teams. The contribution is a
response to the continuous need within CSCW and TEL for new knowledge about the
use of state-of-the-art technologies in current practices. In the case of SE student
projects, this is a need boosted by the spread of lightweight technologies in many areas
of modern life including project work, combined with the changes in educational and
SE industry practices. In the thesis, key issues related to tool usage in student projects
have been framed by considering the teams as communities of practice and learning and
by considering the work and learning in the teams as a case of distributed cognition.
The thesis provides specific insights on the use of instant messaging tools, internet
forums, project wikis and issue trackers, and also on the use of email as part of a
concerted use of tools in the project work. Through various case studies the thesis
shows how the current use of the tools is related to core challenges of SE projects and
project based learning (e.g. cross-community collaboration and the learning of new
technologies). Regarding issue trackers, the thesis primarily addresses their use from the
perspective of retrospective reflection, but in elaborating on the connection between
retrospective and day-to-day use of the tools (P7 and P8), the thesis also sheds light on
the role of issue trackers in day-to-day project work.
Research paper P6 does not provide insights on computer-based lightweight tools but
can be considered an illustration of the successful use of a different type of lightweight
tools in project teams‟ work: physical tools in the form of papers, pens and whiteboards.
The focus of the thesis is on the process in which the physical tools are used and the
representations that are created (P6-P8). The physical tools continue to play an
55

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
important role in the version of the reflection workshop which is aided by historical data
in a computerized collaboration tool (P7-P8).
For practitioners within SE education and PBL more generally, knowledge about the
typical and recommended use of collaboration tools in student projects is an aid to
understanding the challenges of specific projects and project courses. This
understanding is essential for course staff seeking to provide project students with the
appropriate scaffolding for learning. While recognizing the skills with which students
currently handle a variety of collaboration tools for work and social purposes, course
staff can make some recommendations and help the students be more conscious of their
tool use in different collaborative settings. The research on lightweight collaboration
tool use in student projects presented in the thesis can also serve as a basis for the
identification of issues for educational practitioners‟ own pedagogical research and
development of practice (e.g. addressing the role of a particular collaboration tool in
student projects or how a particular aspect of project work is supported by the use of
collaboration tools). The research methods as well as the results presented in the thesis
can inform this type of research.
For the organization of small scale SE project work in educational and other settings,
the findings on how a project wiki may support project management (P5) can aid the
decision of whether, and how, to use a project wiki as a lightweight project management
tool in a specific project. These findings can also be useful to project teams who are
already using project wikis and who may benefit from utilizing the tool in different
ways. The thesis research on issue trackers (P7 and P8) can inform considerations on
the choice of lightweight project management tool for a project and the comparison of
project wikis and issue trackers based on their qualities for day-to-day work and
retrospective reflection.
The thesis contribution on the use of project wikis and their potential integrative role in
small-scale projects adds to the knowledge within CSCW on tool support for project
work as well as the usage of wiki technology. For organizers of small-scale projects
the contribution can inform decisions about whether and how to make use of wiki
technology for lightweight project management.
For the TEL research field, the contribution adds to the body of literature on the use of
collaboration tools (e.g. wikis) in educational settings, offering a perspective that views
the tools primarily as aids to daily work practice (i.e. as tools for work rather than
learning technology) and indirectly supporting learning in the context of PBL.
Further, for the research communities in which the research papers have been published,
the thesis provides concrete examples of how daily work and tool use in a team can be
56

Contributions and implications
investigated retrospectively by the researcher. Specifically, the retrospective use of
project wikis (P5) and issue trackers (P7-P8) exemplifies how the researcher can gain
insights about daily work in a team with the aid of historical data contextualized
through participants‟ retrospective accounts of the process.
6.3 Supporting retrospective reflection
Contribution 3 of the thesis comprises new knowledge about how retrospective
reflection, seen as a part of the collaborative work practice, can be supported in SE
student projects and project work more generally. It also includes tools for supporting
such reflection.
A concrete and empirically tested design for retrospective reflection workshops, adapted
from current SE industry practice to the educational setting of SE student projects, is
offered (P6).
A new and dual use of existing collaboration tools in SE projects is proposed, extending
the current role of the tools in daily work to that of supporting participants‟ reflection on
the project process. Post-mortem interviews (P5) and retrospective workshops (P8) are
shown to be possible ways of integrating participants‟ examination of historical data
into a systematic reconstruction of the project process with the aim of identifying
lessons learned. For purposes of examining historical data, relevant tool features have
been identified along with important elements in the organization of the retrospective
reflection effort. Finally, the thesis offers a set of guiding questions for assessing the
potential of specific collaboration tools to aid retrospective reflection in project work
(P8).
In the case of project wikis, the PhD work contributes with a tool (the Wiki
Walkthrough tool) extending current wiki functionality to address identified
shortcomings of wikis with respect to their use in retrospective reflection (P5). The tool
57

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
has been developed as a prototype and tested through scenario evaluation by expert
groups (P5).
For course staff and others involved in the organization of SE student projects, the tools
for conducting retrospective reflection provide a way of implementing state-of-the-art
SE approaches (e.g. agile development methodologies) in SE student projects as well as
a way of improving the learning outcomes. The insights about the potential of
collaboration tools to support retrospective reflection can help course staff decide
whether to impose or encourage the use of particular tools in the projects, based on their
potential to aid retrospective reflection.
For organizers of PBL in domains other than software engineering, the thesis
demonstrates that an important part of the reflective activity of a project team can be
facilitated and organized in a workshop format. The design of the workshops as outlined
in P6 is focused on project work and project process independently of the SE domain
and is applicable to other work domains. As argued in the thesis, the ideal in PBL is that
the retrospective reflection of a team be part of the real work practice and not a separate,
educational exercise. Thus, for PBL-based courses within other domains the organizers
should find ways of making the reflection workshops an integral part of the work
practice, possibly by linking them to existing learning practices within the domain. As a
second option, the thesis research indicates that reflection workshops can be introduced
as more of an educational exercise: students appear to find it interesting to get the
opportunity to reconstruct and account for their story about the project and hear the
story of the other team members. The approach based on the drawing of individual and
shared timelines and satisfaction curves allows participants to address the emotional
aspects of project work in a form that does not put them off writing or talking „about
their feelings‟ or spending what they see as too much time elaborating on the project
process. The reflection workshop presented in P8 was adapted to a CSCW research
agenda and is not a readily applicable design for retrospective reflection in PBL
settings. However, the finding that steps of investigating historical data in collaboration
tools can be incorporated into an existing workshop approach and thereby improve it,
can be utilized by PBL organizers as in the development of other tool-aided reflection
workshop designs. For the organizations (in industry or education) in which PBL takes
place, the proposed approaches to retrospective reflections may contribute to
organizational learning. Organizers of PBL efforts consider using the results from the
reflection workshops as a way of informing the development of their courses, taking
some precautions as discussed in P6. This would support the educational practitioners as
researchers and developers of their own practice. Also, in formal education, successful
incorporation of retrospective reflection workshops in PBL can ultimately contribute to
industry practice if the students who learn the techniques find them useful and later
58

Contributions and implications
apply them in their professional career. This is another way of bringing about crosscommunity
learning through PBL.
To the CSCW field, the thesis provides new knowledge about how data stored in
collaboration tools through a work practice, can be utilized to support that work practice
through retrospective reflection. Framed in state-of-the-art industry practice for project
retrospectives in SE project work, current usage of lightweight collaboration tools in
project work (e.g. as manifest in Contribution 2), theory on human reflection and
learning, and a focus on the project process (e.g. trajectory and sub-trajectories) as the
main object of reflection, some key points are established. Firstly, day-to-day usage of
collaboration tools in a project determines what aspects (and challenges) of project work
become reflected in the tools. Secondly, particular tool features are useful for access to
and navigation of historical data stored in (or by) the tools in the context of
retrospective reflection. This includes chronological overview of the project process and
easy access from points in the process to project artifacts (e.g. the project product) in
their there-and-then state. Thirdly, the organization of the retrospective reflection
activity in which the historical data are used, should be designed to incorporate
individual as well as collaborative knowledge construction and attend to „facts‟ as well
as „feelings‟ about the project process. Fourthly, if collaboration tools are given a dual
role in a work practice, supporting day-to-day work as well as retrospective reflection
on that work, the effect on the work practice should be considered and preferably
empirically investigated (e.g. looking for signs of increased or improved information
sharing or information hoarding).
For tool designers, these key points can be used to inform the design of collaboration
tools with the aim of making the tools useful for retrospective reflection.
For researchers analyzing project work settings with the aim of identifying ways of
supporting the work practice, the key points illuminate the option of introducing or
redesigning retrospective reflection activities, e.g. by utilizing collaboration tools as a
resource for the reflection. In the field of Software Engineering in particular, the
proposed use of collaboration tools can be considered an answer to one of the
challenges to having SE professionals actually prioritize retrospective reflection on their
projects: getting access to relevant data to aid the reflection process.
59

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
6.4 A model of work and reflection
Contribution 4 of the thesis is the reflection model shown in Figure 13, conceptualized
in terms of distributed cognition and stressing the cognitive as well as the social aspects
of work and learning in project teams.
The model is presented in P7 as a contribution to the TEL research field. The model‟s
perspective on the work-reflection-learning cycle is novel in making explicit the
potential role of collaboration tools in supporting day-to-day work and the retrospective
reflection on that work and thereby strengthening the integration of these aspects of the
work practice. The model further incorporates individual and collaborative elements of
the reflective process, and makes explicit the use of representations of the project
process (ranging from the trajectories „in participants‟ heads‟ to explicit representations
developed through dedicated reflection activities) as means for, and outcomes of,
reflection. The model represents a practical view of the work-reflection-learning cycle
and its support in project based learning, incorporating elements from learning theory
and SE industry practices that have been combined and empirically tested in SE student
projects as part of the thesis research. While the workshops in the thesis research were
based on a SE retrospective approach using timelines and satisfaction curves as explicit
representations of the project process, the reflection model does not presume any
particular form of explicit representation of the project.
The model can be used to aid design for retrospective reflection in PBL settings. The
designers could be educational practitioners seeking to develop their own practice,
project managers, people responsible for learning activities on a project or organization
level, and designers of collaboration tools.
In the analysis of a PBL setting, the elements of the model and their interrelationships
can be used to identify a set of considerations about the work-reflection-learning cycle
and trigger relevant questions. Some such questions are: When should individuals‟
reflection on their work practice be explicitly encouraged or designed for, and what is
60

Contributions and implications
the desired outcome of this reflection? Similarly, when should collaborative reflection
efforts be encouraged and/or arranged, in what form and with what type of outcomes?
What is the intended use of the outcomes the reflection? For instance, how should
individual and collaborative reflection inform each other, and how should the outcomes
inform daily work? What collaborative tools currently in use are potential sources of
historical data that may enrich retrospective reflection? Should other tools be taken into
use in day-to-day work practice in order to gather data for use in reflection on work?
How do we know that the outcome of reflection is in fact useful, to the project and/or to
the organization? The number of possible questions is large. The work to address such
questions to aid analysis of, and design for, reflection in PBL should be seen as an
integral part of learning-related activities in the organization (e.g. under the heading of
course development, organizational learning or knowledge management). The reflection
model can be instantiated to outline the elements of a specific work-reflection-learning
cycle, as shown in P8 (Figure 15) where the historical data in an issue tracker were used
in a retrospective reflection workshop.
It may add to the practical usefulness of the reflection model to view it in light of
Contribution 3. This makes it easier to see how the ideas in the model are grounded in
practice and can be implemented in a concrete design for reflection in project work.
For TEL and CSCW researchers aiming to develop theory on reflection, the model can
serve as a basis for empirical or theoretical work.
Finally, the reflection model can serve as an aid to analysis of cases of work and
learning, helping the researcher determine how to gather data about the work process.
The historical data in collaboration tools contextualized through participants‟
reconstruction of the project process can be a resource for participants (team members
or other stakeholders (e.g. project supervisors) acting as researchers on their own
practice, for example through action research.
61

7 Evaluation
In this chapter, I evaluate the contributions of the thesis with respect to the research
questions and the state-of-the-art of the research fields. Also, I evaluate the research
approach and process and discuss my choice of theory.
7.1 Evaluation of the contributions with respect to the research
questions
RQ1: What characterizes software engineering student projects? The thesis answers
this question by providing new knowledge about two selected aspects of project work in
SE student projects: issues related to cross-community collaboration (Contribution 1)
and the use of lightweight collaboration tools (Contribution 2).
RQ2: What is the current usage of lightweight collaboration tools to support work in
software engineering student projects? This question is answered by Contribution 2,
which represents new knowledge on the use of a selection of commonly used
collaboration tools in SE student projects: instant messaging, internet forums, project
wikis, issue trackers, and also email as part of a concerted use of tools in the teams.
RQ3: How can retrospective reflection be supported in software engineering student
projects? The thesis answers the question in Contribution 3 by showing that a SE
industry approach to organizing reflection workshops can be successfully adapted to
student projects, providing adequate support for the reflective process as outlined in
relevant learning theory. A concrete workshop design is outlined. Also, the thesis shows
that collaboration tools and their historical data can be used as a resource for the
reflection.
RQ4: How can collaboration tools that are used in daily project work be utilized as
tools for project based learning? This is answered in Contribution 3, in which it is
proposed to have collaboration tools take on a dual role in the projects: as support for
day-to-day work and as support for retrospective reflection on that work by use of the
historical data in the tools. It is shown that certain tool features are important in this
63

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
respect, and that an existing retrospective reflection workshop approach can be used to
frame the exploration of historical data in the collaboration tools. The thesis does not
offer a ready-to-use workshop template in this respect, but offers guidelines for the
evaluation of the potential of tools to be used in retrospective reflection and a
recommendation to integrate the exploration of historical data into a timeline and
satisfaction curve workshop approach. In Contribution 4 the answer to RQ4 is framed
theoretically and generalized to project based learning at large through the model
outlining the work-learning-reflection cycle.
7.2 Evaluation of the contributions with respect to state-of-theart
of the SE Education, TEL and CSCW research fields
This section provides an overview of how the research contributions add to the literature
outlined in the state-of-the-art chapter (Chapter 3).
Contribution 1 adds to SE education and TEL research on the challenges of project
based learning (Section 3.1). The findings on how students address stakeholder
objectives and how we can help the students be more conscious about these objectives
(P1) contributes to the literature on stakeholder collaboration in SE project courses. The
study in P2 adds to the limited but growing literature on student participation in OSS
development. Additionally, the study in P2 provides a contribution to the TEL body of
literature on project based learning by applying the CoP concept of brokering to shed
light on the relationship between individual work and learning and that of the team.
Finally, the thesis research on instant messaging represents a small increment to stateof-the-art
TEL research on the use of collaboration tools in project based learning and to
the literature on instant messaging in educational contexts.
Contribution 2 brings forward state-of-the-art research on the use of lightweight
collaboration technology to support project based learning (Section 3.2). The thesis
research on project wikis (P4) in day-to-day project work also adds to the CSCW
literature on tool support for small-scale project work, in which the use of project wikis
has so far not been extensively addressed. The research on wikis can also be seen as a
contribution to the TEL literature, in which previous studies of wikis in educational
contexts have mainly been addressing wikis as learning technology and not – as in this
thesis – as tools for collaborative work.
Contribution 3 brings forward state-of-the-art research in SE education and TEL on
how to support project based learning (Section 3.3), focusing on how to support one
particular aspect of the work practice: retrospective reflection. Also, Contribution 3
64

Evaluation
adds to the CSCW literature of how collaboration tools in daily use in work practice can
be utilized to support that practice (Section 3.4).
With respect to the Software Engineering literature, the contribution meets two
recognized challenges to successful project retrospectives: the problem of inadequate
data (Kasi et al. 2008) and the problem of project participants finding it ineffective
and/or not personally useful to code their experience (Schindler and Eppler 2003) – the
solution of this thesis being to utilize existing historical data.
The study by Aranda and Veolia (2009) examining the potential use of repository data
from bug fixing processes to aid coordination of SE work (Section 3.4) complements
findings in the thesis. As the study was recently published and accordingly is not
discussed in the research papers of the thesis, it will be briefly elaborated here. The
study confirms that participants‟ meaning making is essential for the reconstruction of
project trajectories (e.g. stories of bug fixing). It is difficult to use repository data as a
source of immediately useful input for decision making in day-to-day coordination
because the data are insufficient and sometimes erroneous. For instance, repository data
may frequently fail to reflect who were actually involved in the work process. In this
PhD thesis, the focus is also on how to design for meaning-making by use of historical
data, but in a setting outside the immediate time pressure of day-to-day coordination. In
the case of retrospective reflection, the reliance on human memory makes it less
important that the historical data from collaboration tools alone can provide the basis for
reconstructing complete trajectories. Historical data providing erroneous information
about the project process might of course be a problem even if participants have the
time to check it against their memory and other data sources. The use of historical data
from a set of different collaboration tools reflecting various aspects of the project
process can help participants build sufficiently coherent knowledge about the process A
similarity between the findings of Aranda and Veolia and those underlying Contribution
3 is that both studies confirm the usefulness of historical data linking the development
process to the evolving product. Seen together, the studies strengthen the argument that
data in collaboration tools may serve a dual role in project work as shown in the
reflection model in P7, and that to achieve this, the process needed for participants to
actively give meaning to the data must be given due attention, whether in the context of
retrospective reflection or in the day-to-day coordination of work.
Contribution 4 brings forward state-of-the-art CSCW and TEL research on how to
support, integrate and conceptualize work, reflection and learning in collaborative work.
The reflection model in Figure 13 is a theoretical contribution mainly to the TEL field,
building on existing models considering learning as a cyclic process in which
experience is followed by reflection on the experience (Section 2.2). State-of-the-art SE
65

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
practice (Section 3.1), the theoretical framework of DCog (Section 2.1.3) and research
on the use of collaboration tools in collaborative project work (Section 3.2), as well as
Contributions 2 and 3 from the thesis research, were drawn upon to incorporate
elements specific to retrospective reflection and its support: the potential dual role of
collaboration tools combined with the inclusion of individual as well as collaborative
steps of knowledge construction by aid of various representations.
In the fields of CSCW and CSCL, the mainstream perspective on work and learning
considers meaning-making to be basically social by nature, and the unit of analysis is
typically the community or group in which the work and learning takes place. Whereas
some research has a focus on the relationship between the individual and the collective
(Engeström 2000; Stahl 2002), there is a need (Kaptelinin and Nardi 2006) for further
research emphasizing the interrelationship between the two. The research of this thesis
can be seen to contribute in this direction, both in the individual case studies (e.g. P2,
P6), and in the theoretical synthesis of the last papers (P7-P8). The use of concepts from
DCog enabled a clearer image of the collaborative and the individual as elements in a
reflective process.
With regard to the dual role of collaboration tools, the CSCW and SE bodies of research
(Section 3.4) on the use of data gathered from project work for purposes of supporting
the day-to-day work practice reveal similarities between use of historical data to aid
day-to-day work (e.g. awareness and coordination (Aranda and Venolia 2009;
Omoronyia et al. 2009)) and the use of the data to aid retrospective reflection. Both
types of settings entail meaning-making based on the reconstruction of project
trajectories, with the purpose of evaluating and acting upon the outcome. On a high
level of abstraction, it can be argued that the reflection model in P8 covers the gathering
and use of data from collaboration tools for support of any aspect of the work practice.
In its current form, the reflection model is particularly useful for shedding light on
support for retrospective reflection and how it can be integrated into the work practice.
The outcome of the reflective process (cf. Section 2.2, Figure 5) is intended to be
valuable for the work process. Exactly how this should be achieved has not been
thoroughly explored in the empirical research of this PhD work, but treated more on an
abstract level with reference to the role of retrospective reflection in SE practice and the
assumption that further research is needed on the use and usefulness of the reflection
outcomes. Continued research to validate and refine the model in order to improve its
value for process improvement and organizational learning should relate to the SE and
TEL research literature and also draw on state-of-the-art research within project
management, organizational learning and knowledge management.
66

Evaluation
7.3 Evaluation of the research process
This PhD work is situated on the boundary between several research fields, each with its
literature, approaches and research topics. In the case of SE education, TEL and CSCW,
there is significant overlap of research topics related to collaborative learning. Studies
of work and learning in SE student projects are a source of relevant research material
for conferences such as ICSE, EC-TEL and COOP. The fact that the research papers of
this thesis have been published within the different communities indicates a fairly wide
scope of the PhD work. While an aim to focus on publication within one of the research
communities might have resulted in a more extensive contribution within that field,
broader participation opens up for a larger set of perspectives on the research topics at
hand. For instance, the SE education research community favours an engineering- and
practitioner-oriented (albeit theoretically grounded) stance to SE student projects,
whereas the CSCW and TEL/CSCL fields generally require a stronger theoretical
contextualization. Being a computer scientist and a social scientist, I appreciate the
opportunity to approach my research in both ways.
The research of this thesis has been presented as a combination of interpretive and
design research. This section will address how the studies actually correspond to these
research approaches and, as part of the discussion, shed light on some methodological
challenges.
7.3.1 On the work in the thesis as interpretive research
As outlined in Chapter 4, the case studies of the thesis can largely be seen as
interpretive. The longitudinal studies are the ones that most clearly fit the
characterization. Klein and Myers describe IS research as interpretive “if it is assumed
that our knowledge of reality is gained only through social constructions such as
language, consciousness, shared meanings, documents, tools, and other artifacts.” It
“does not predefine dependent and independent variables, but focuses on the complexity
of human sense making as the situation emerges []; it attempts to understand
phenomena through the meanings that people assign to them” (Klein and Myers 1999,
p.69). This approach to research is in line with a basic constructionist perspective on
work and learning. The connection to the IS field does not make interpretive research
invalid for CSCW and CSCL, both of which are research fields with a focus on people‟s
sense making through their use of tools and artifacts.
According to Walsham, results from interpretive case studies can be generalized in the
following ways: by developing concepts, by generating theory, by drawing specific
implications, and by providing rich insights (Walsham 1995). In the thesis, the
67

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
contribution of research paper P2 („Power through brokering‟) can be seen as the
clearest example of „rich insights‟. The overall focus of the thesis research on current
practices in the SE student projects has been on providing insights about these practices
and in particular the use of collaboration tools. To make the insights more applicable,
they have in several cases been developed into guidelines and recommendations. This
seems to be particularly appropriate for contributions to the SE Education community
and should be a fair approach as long as the basis for providing the recommendations is
properly taken into account. In the case studies where data have been collected across
many project teams (P1,P3,P4), generalization has to some extent approached that of
positivist case studies (Yin 2003) by addressing what is more or less typical for a certain
population (e.g SE student projects).
In terms of use of theory, there are three main approaches in interpretive research
(Walsham 1995): using theory initially to guide design and data collection, using it as
part of an iterative process of data collection and analysis, and to using it as a final
product of the research. A restricted use of theory in an early phase resembles the use of
sensitizing concepts (Bowen 2006) in grounded theory. In this thesis, some theory (cf
Chapters 2-3) has been used from the start of empirical studies whereas other theory has
been introduced during data collection and analysis. Table 4 shows which main
theoretical concepts have informed the work presented in the various research papers.
The table indicates which theory was informing data collection and analysis (X), and
which theory was informing the analysis phase only (Xa). The choice and combination
of theory for the thesis is further discussed in 7.4.
Table 4: Theory informing the work presented in the research papers
68

Evaluation
In the longitudinal studies I have aimed for a balance between the initial use of theory
and an open attitude (“What‟s the story here”), seeking to build an understanding of
participants‟ interpretations of the phenomena at hand as well as my own interpretation.
I have made no “commitment to indifference” (Randall et al. 2007) but have tried to be
aware of my own preconceptions and ways of thinking: those originating in my
professional background (e.g. as course staff, as an engineer in computer science, and as
a social scientist who worked with activity theory in her Master thesis) and those that
are more situational/personal (e.g. developing a particular liking for a team member or
being tired during an observation session).
Klein and Myers (1999) present a set of seven principles of interpretive field studies
that can be used to guide the design and evaluation of such studies. In what follows, I
use examples from the longitudinal studies (Team A and Team F) to illustrate the
relevance of the principles as well as how I have followed them.
The fundamental principle of the hermeneutic circle says that the parts and the whole
should be seen in light of each other. In the longitudinal research, the writing,
elaboration and re-examination of field notes helped me move back and forth between
details (e.g. a particular utterance in a meeting or an instance of collaboration tool use)
and the whole picture (e.g. the project process). Another example is that throughout the
PhD research I have aimed to understand the use of specific collaboration tools in the
context of the project teams‟ concerted use of tools.
With respect to the principle of contextualization, one challenge in this PhD work has
been the question of whether the activity in the teams should be considered mainly as
learning or as work – or as both. One answer is provided by the choice to focus on work
as the core of project-based learning, which means the focus of empirical studies can be
more related to work whereas the connection to learning is theoretically underpinned.
Another answer is that the focus on work or learning depends on the intended audience
for the presentation of the research. The research of the thesis has been contextualized
in slightly different ways to provide relevant contributions to various communities (see
Figure 3). Another issue related to contextualization is that data collection and analysis
is contextualized not only by the chosen theory but also by the underlying interests of
the researcher. The latter in particular is not always opaque in the presentation of the
research. For instance, my interest in process trajectories and retrospective reflection
reflects a general research interest in work processes, which then become the context for
understanding other aspects of the SE student projects. Knowledge about the product
being developed has to me mainly been instrumental in making sense of the project
process, even though I believe the completion of the product to be a key motivational
factor for the project participants. Finally, a core topic in the thesis research is how the
69

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
students and course staff themselves contextualize the SE projects, and how this impacts
on work and learning (cf. Section 3.1).
The principle of interaction between the researchers and the subjects does not state that
no interaction should take place, but that the researcher be conscious about the possible
impact of the interaction on the research. This impact may be difficult to assess. In the
longitudinal studies, while I was trying to disturb the teams as little as possible in their
day-to-day work, I participated socially (e.g. by engaging in chit-chat prior to work
sessions). Sometimes I brought some chocolate or fruit. While observing work, I
occasionally asked clarifying questions, but only in situations where I did not
understand what the team members were doing and it seemed important to understand it
to make sense of their work and tool use. In such situations, the team members‟ answers
to my questions turned into meta-comments about their activity and were no longer part
of the team‟s ordinary activity; they were talking „about their world‟ instead of just
being in it (Sacks 1992). The distinction between meta comments and comments which
were part of normal activity could sometimes be blurred: to some extent, commenting
aloud about work was part of „normal‟ activity and provided other team members with
increased awareness of the ongoing work through consequential communication
(Gutwin and Greenberg 2002). This „thinking aloud‟ was particularly visible in
situations where students worked in pairs, one student at the keyboard and the other at
his side to assist. On some occasions the thinking aloud might have been intended to
inform me as a researcher. All in all, as I kept following the students, they seemed to
accept my presence but largely focused on their work task.
Importantly, my role as a researcher and not supervisor or examiner was made clear to
the study teams (as well as to the other teams in the course, who had to be informed that
the research subjects did not get any advantage over other teams). These conditions
were generally unproblematic. On one occasion I chose to leave a work session in the
computer lab as the team was in real need of help and started pushing me – while I
really felt like helping them. On another occasion, I decided that I would provide the
team with a small piece of advice to get them going in an ineffective communication
process – a decision rooted solely in my own impatience. This was however an
exception.
The principle of abstraction and generalization specifies that theoretical abstractions
always be related to “field study details as they were experienced”. This connection has
been maintained by looking at concrete instances of collaboration tool use in the case
studies and placing the observations in context of theoretical abstractions (e.g. brokering
(P2) and reconstruction of a process trajectory (P7, P8)).
70

Evaluation
According to the principle of dialogical reasoning it is important to make transparent (to
the reader and to oneself as researcher) “the historical intellectual basis of the research
(i.e. its fundamental philosophical assumptions)” – in other words, explain “the lenses
through which field data are construed, documented and organized” (Klein and Myers
1999, p.76). The research papers of the thesis always include a section outlining
theoretical background framing the empirical studies. Also, while the introduction of
relevant theory has happened at different stages of research for the different papers (see
Table 4), they are all grounded in an underlying constructionist view, which is made
explicit in some of the papers.
The principle of multiple interpretations and the principle of suspicion are important in
interpretive research, in which the identification of interesting phenomena may lead the
researcher to look for confirmatory data and findings while being less observant to
contradictory ones. As a case in point I will describe an experience related to the study
of Team F from which I gained important insights about the case and the research
process, in particular the importance of follow-up interviews. This could only be very
briefly addressed in P8. Figure 16 shows two diagrams originating in my analysis of the
data from the retrospective reflection workshop of Team F (see P7 and P8). They depict
my interpretation of how the team changed the story of their project during their
reflection workshop; the left diagram outlining the „before‟ version and the right
diagram showing the „after‟-.
Figure 16: Diagrams showing the researchers' interpretation of how Team F
changed their story of their project in their reflection workshop.
The diagrams were shown to the team members in individual follow-up interviews
about three months after the workshop (after the summer holiday). The students thought
the diagrams and my accompanying explanation fit with their conception of what
happened during the workshop. These viewpoints could be seen as strengthening my
interpretation, even taking into account the effect of the model power (Bråten 1981;
Buhl and Richter 2004) exerted by me as a researcher presenting finished diagrams.
However, the informal interviews with the students also brought about some new
71

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
viewpoints on the project process that had not emerged in the workshop – at least not
clearly enough to be explicitly addressed in the teams‟ discussion at the time. From the
interviews and my subsequent re-examination of the workshop data, there was reason to
believe that these viewpoints had already been there during the workshop (and were not
mainly results of maturation after the workshop), but they had been expressed only halfheartedly
then. The viewpoints were those of the less dominant team members. My
initial interpretation of how the workshop had changed the team‟s view of the process
reflected the perspective of the more dominant team members.
These insights served as an important reminder that I might have been attending more to
the activity and viewpoints of some team members than others during the many hours of
observation – maybe because they were the students most clearly influencing the team‟s
development work and the project process, and maybe because they were the ones most
able and prone to share their insights and viewpoints during day-to-day work. Apart
from providing methodological insights, the follow-up interviews made me refine the
findings (in P8) on the use of historical data in retrospective reflection (acknowledging
that data originating in a certain tool might be more relevant to some team members
than others and serve to strengthen already dominant views on the process) and on how
a reflection workshop should be conducted (e.g. by using explicit facilitation to make
everyone‟s voice heard).
Considering all the studies in the PhD research, a triangulation of data sources has been
applied throughout. This includes interviews with project participants and stakeholders,
contributing to multiple interpretations. Going back to participants to check if the
researcher‟s conceptions of the phenomena in question match those of the participants
has mainly been done for the longitudinal studies of single teams.
7.3.2 On the work in the thesis as design research
Design research in education is directed at “developing, testing, implementing, and
diffusing innovative practices to move the socially constructed forms of teaching and
learning from malfunction to function or from function to excellence” (Kelly et al.
2008a).
The improvement of practice should involve the participants (e.g. students and teachers
(Brown 1992; Collins 1992)) in the process. Brown describes her intentions with design
experiments in educational research as to ”transform classrooms from academic work
factories to learning environments that encourage reflective practice among students,
teachers, and researchers.” (p.174) The focus of the research in this thesis on better
equipping teams in SE student teams to reflect on their own practice could be seen as a
way of achieving this, although the focus is on reflection as part of work practice.
72

Evaluation
However, only some of the reflection seen in the teams (e.g. in P6) addresses how the
course is organized; the rest is (as intended) about work.
In the thesis, the interpretive research informing the research activities on reflection can
be seen as part of a design research effort starting with a more general exploratory
agenda and refining focus to reflection as an area for improvement. Guidelines for the
organization of SE student project are proposed (P1-P4). The thesis research involving
interventions by trying out new solutions (i.e. the wiki walkthrough tool in P4 and the
reflection workshops with and without the aid of historical data in collaboration tools in
P6-P8) result in standalone contributions (Contributions 2 and 3) but should
simultaneously be considered as first steps along a line of research aiming to improve
the solutions. If similar solutions are tried out in new projects within the SE project
course of study, there will be an iteration that can result in improvements from one year
to the next, with the researcher-as-course-staff as a change agent. To achieve
improvement of the project course within a semester, the reflection workshops will have
to take place during the semester as well as at the end of the semester. Research efforts
along these lines are considered further work in the thesis (see Section 8).
Students‟ reflection on their project work can be a resource for course staff aiming to
improve the course, as proposed in P6. To allow the students to maintain their focus on
the work practice, course staff will have to be brokers translating students‟ practicefocused
experience into issues of how to organize the course.
Design research should also contribute to the development of theory (Cobb et al. 2003,
p.9). In the thesis, development of theory on the basis of empirical studies of reflection
resulted in the reflection model presented in P7.
In sum, presenting the research of the thesis as design research is done with some
precautions and refers to the long-term research agenda as well as to the present results.
A full-fledged design research effort requires a systematic, iterative development of the
proposed solutions within a project course. I believe that the tools and theory developed
in this thesis on the basis of empirical work with projects in the selected SE project
course will be a useful basis for systematic improvement of the course and the specific
approaches to retrospective reflection.
7.4 On the choice of theory
So, my first piece of advice for new researchers is for them to choose
theories which they feel are insightful to them.
(Walsham 2006)
73

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Across educational, social and organizational psychology, there is a multitude of
frameworks describing structures of collectively created meaning emerging in, and
coordinating, groups‟ activities (Akkerman et al. 2007). Two of them are used for the
thesis: Communities of practice (CoP) (Wenger 1998; Wenger 2000), which is closely
connected to situated learning (Lave and Wenger 1991), and DCog (Hutchins 1995;
Rogers and Ellis 1994). Other relevant frameworks include actor network theory (ANT)
(Latour 2005) and activity theory (AT) (Engeström 1987; Kaptelinin and Nardi 2006;
Kuutti 1995; Leont'ev 1981).
There are several studies comparing different theoretical frameworks with respect to
their use in CSCW (Bratteteig and Gregory 1999; Halverson 2002; Nardi 1992; Nardi
2002; Randall et al. 2007). In this thesis, I will not go deeply into the discussions of the
pros and cons of the frameworks, but mainly stress the rationale for my choice of
theory.
My initial view of what was interesting in the cases was influenced by my background
in software engineering (with its challenges and methodologies) and educational theory
(having used activity theory to analyze an empirical case in my Master thesis, (Krogstie
2000)). I brought with me a general interest in what happens within a group (or
community) of people working and learning together and at the intersection of such
groups (or communities). To describe work and learning from the latter perspective,
CoP concepts can be used, including those of boundaries and brokering. AT, and in
particular the concept of activity system (Engeström 1987), are adequate for analysis of
similar settings, tensions/contradictions within and between activity systems serving as
a way of accounting for the static as well as dynamic aspects of interrelated activity
systems. ANT provides means of accounting for the dynamics within and between
networks, the concept of alignment (Latour 2005) being central. DCog can be used to
shed light on the same mechanisms by focusing on the transformation of representations
within and between functional systems (Hutchins 1995).
Halverson states that from a pragmatic view, one can identify four desired attributes of a
theory: descriptive power, rhetorical power, inferential power, and applicability to the
real world, the latter largely translating to the ability to inform design (Halverson 2002).
The above mentioned frameworks describing structures of collectively created meaning,
are all powerful means of analyzing and describing settings of work and learning when
systematically applied. When it comes to the power to inform design, DCog and AT
both have their proponents; Halverson (2002) arguing most strongly for the powers of
DCog and Kaptelinin and Nardi (2006) arguing in favour of AT. Independent of the
choice of theoretical framework, there is general agreement that the step from analysis
74

Evaluation
to design is problematic even if a theory of good descriptive power is used to
systematically outline the as-is situation (Dourish 2006).
In the early phases of the empirical work, I sought to use theoretical concepts mainly as
sensitizing concepts (Blumer 1954). In choosing theory, I aimed for „conceptual
simplicity‟ to gain the aid of some structure without having a framework too strongly
enforcing a preconception of what was interesting about the cases. A restricted use of
theoretical concepts creates an opening for the careful combination with other concepts
on an „at need‟ basis, to conceptualize findings and guide further research. I consider
CoP to be an analytical unit fitting these purposes, being adequate for shedding light on
collaborative work and the use of collaboration tools within teams and between the team
and other project stakeholders, and allowing for a combination with the „standard
repertoire‟ of CSCW (e.g. the concepts of awareness (Dourish and Bellotti 1992;
Gutwin and Greenberg 2002) and coordination (Carstensen and Schmidt 2002 (1999);
Schmidt and Simone 1996)).
At the point in the PhD work where attention was set on reflection there was a need for
theory shedding light on the process of reflection specifically. Strauss‟ concept of
project trajectories (Strauss 1993) and the model of the reflective process (Boud et al.
1985a) could be used as tools for analyzing the reflective process in a team. A strength
of the model of the reflective process is that it provides support for design by outlining
specific elements that may be included and supported in a reflection process. The choice
of drawing on SE project retrospectives approaches from industry in the PhD work as a
starting point for implementing the reflective processes in the project teams emerged
from personal communication with a colleague engaged in such practices.
The combination of the above theories and approaches led to a general research focus
on the role of representations in reflection and a need to theoretically frame the process
of reflection in the context of the team‟s overall activity. Activity theory was an option,
having a strength in accounting for the role of tools (“Its richness for CSCW lies in
principle in its approach to technology as a mediator of human activity, which in a
dialectic relationship with the cultural world produces activity.” (Randall et al. 2007,
p.90). However, the role of representations, internal and external, is more explicitly
addressed in DCog. DCog allows for a combination of a social and cognitive
perspective on collaborative work, being a framework “capable of capturing cognitive
activities as embodied and situated within the context in which they occur: social and
organizational” (Rogers and Ellis 1994). Combining this with theory accounting for the
process of reflection, I had a framework adequate for analyzing the case forming the
basis of P7 and P8. Concepts from DCog provided a language for the reflection model
75

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
in P8, enabling the generalization of results from the case studies to a model of
reflection in project based learning.
76

8 Conclusion and further work
This thesis has approached the challenge of supporting learning in SE student projects
by focusing on day-to-day project work and retrospective reflection on that work. The
underlying idea is that within project based learning, doing real work is the driving
force for learning. The students should accordingly be allowed to keep their focus on
the challenges of collaborative work and gain experience from addressing them as
Software Engineers. In the thesis, lightweight collaboration tools have been shown to be
valuable in day-to-day SE work in different ways and for different purposes. Also, the
lightweight collaboration tools have been shown to be potential resources for
retrospective reflection, in SE student projects and in project based learning more
generally, helping to integrate daily work and reflection on that work (Figure 17).
Figure 17: Support for learning in Software Engineering student projects as
approached in this thesis
The findings of the thesis point to a need for a conscious and reflective use of
collaboration tools in SE project courses; among students, course organizers and
supervisors. The thesis illustrates how handling of project challenges by project teams,
frequently involving cross-community collaboration, is closely connected to the use of
collaboration tools. The students would accordingly benefit from insights about the
strengths and weaknesses of various collaboration tools and tool usage, contextualized
77

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
in the typical challenges of SE and collaborative project work. Some initial ideas for
how this could be achieved include: training course staff to provide advice on this topic;
conducting introductory lectures; arranging workshops where students from different
groups compare their experiences, and addressing the topic of collaboration tool use in
retrospective reflection workshops during the semester. How the topic of collaboration
tool use may be more consciously addressed in SE project courses deserves systematic
and empirical research by SE education practitioners.
The thesis shows some cases of collaboration tool use in which cross-community
collaboration entails a relatively high degree of participation in the other community,
for example when a project customer gets his own page in a project team‟s wiki, or in
the case of OSS participation by a project team, through an internet developer forum. In
the context of SE student projects, further research could address the questions of when
such participation in the other community is appropriate and what it takes for it to
succeed, specifically addressing the role of collaboration tools. The concepts of CoP,
brokering and boundary objects might be useful in the analysis. The research could be
done within SE education, TEL or CSCW.
Another task for further research in SE education is to adapt the retrospective reflection
workshop approach outlined in the thesis, with or without the use of historical data in
collaboration tools, to the needs of specific courses. This should be considered and
conducted as design research and not merely „use‟ of research results. The outcomes
should be improvement of the specific project courses as well as guidelines and
templates for the organizing of retrospective reflection workshops in SE student
projects. The more systematic use of issue trackers in retrospective reflection in a larger
number of teams should be explored. An important aspect of adapting the approaches to
specific courses is the integration of reflection into the development processes (e.g. as
done in SCRUM and other agile development methodologies). This implies making
reflection part of a process improvement effort and conducting reflection workshops
several times during the project. In each workshop, not only should the teams
collaboratively determine what are main lessons learned, they should define a set of
action items to be implemented in their subsequent work. Accordingly, empirical
research on reflection in the projects should investigate the teams‟ actual use of, and
benefit from, the outcomes of reflection. Comparing data from several reflection
workshops in a project might provide useful data in this respect. This could be
supplemented by longitudinal observation of work in the team with a focus on the
follow-up of action items.
In PBL in other domains of education, a similar way of organizing retrospective
reflection can be adapted and tried out. The results of such studies should be best
78

Conclusion and recommendations for further work
practices and guidelines for PBL in the particular domains, and should also contribute to
the development of these work domains beyond educational settings.
The insights from this thesis about the potential of historical data in collaboration tools
to aid retrospective reflection should be developed into a contribution to the software
engineering community, as the rationale for the proposed use of historical data
originates in SE research and the needs of SE professionals. The empirical data on the
use of Trac in retrospective reflection, as well as new studies in which a similar
approach is introduced in a larger number of student projects, should result in
publications directed at the SE community with a focus on how the proposed approach
meets the needs of Software Engineering. Exposing the research to the software
engineering community (and not just SE education) would help ensure that the use of
historical data to aid retrospective reflection in SE student projects become not only an
improvement of education practices but, in line with the PBL core idea of doing „real
work‟, an improvement of SE practice.
The thesis outlined the first steps of an investigation of the connection between day-today
use of lightweight collaboration tools, tool features and the potential of the tools to
aid retrospective reflection. Further CSCW research along this line should aim for the
development of a framework that can be used by project teams and organizers as well as
designers of collaboration tools to consider the role of collaboration tools in
retrospective reflection. The development of the framework would require a survey of
state-of-the-art research on day-to-day tool use in teams combined with empirical
studies addressing the connection between day-to-day tool use and the use of the same
tools in retrospective reflection. Again, SE student projects may be used as test beds.
The reflection model proposed in the thesis can be used to inform further research on
reflective learning and the role of collaboration tools in supporting day-to-day
collaborative work and reflection on that work. Based on the results of such research,
the model can be further developed and refined. I propose three research directions that
can be pursued independently or in combination:
Firstly, the model can be used as a basis for exploring and comparing various types of
use of historical data in day-to-day work, including retrospective reflection. This
research should identify how current approaches and solutions to the gathering and use
of data for one purpose (e.g. coordination) – or the gathered data themselves - could be
applied for another purpose (e.g. reflection) within the work practice. The research
should draw on state-of-the-art research within SE and CSCW and result in
contributions (to the same communities) with practical applicability for the organization
of, and tool support for, collaborative project work. Also, the research should lead to a
79

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
refinement of the reflection model with respect to the role of collaboration tools in
integrating different aspects of the work practice.
Secondly, the model can be used to frame studies of retrospective reflection across
different domains. The studies may be based on use of the timeline and satisfaction
curve technique in a way similar to the workshops of the SE student projects described
in the thesis, but they may also be based other approaches instantiating the model in
different ways (e.g. with individual and/or collaborative reflection efforts, use of various
representations to aid reflection, and with the possible use of historical data from one or
more collaboration tools used in daily work). The different instantiations of the workreflection-learning
cycle should be used to shed light on similarities and differences
between domains and between approaches, and thus serve as a basis for refinement of
the model. Such a refinement would increase the value of the model as an aid to
designing for the work-reflection-learning cycle in the specific case, helping outline
options and unveiling the potential for supporting learning by use of reflection activities
and data originating in the work process.
Thirdly, and related to the previous point, to extend the value of the reflection model, it
should be elaborated to focus more strongly on the use of the results of reflection as an
aid to the work practice in question - within the project, team and/or organization. In
other words, the model should be refined as a tool for outlining core elements of process
improvement as well as the connection between project learning and organizational
learning. The research needed for this refinement should partially be empirically based,
exploring specific cases and trying out solutions (both of which can be informed by the
reflection model). The research suggested above, to adapt the use of retrospective
reflection workshops to specific SE project courses, can be considered part of this
effort. Also, the research should draw on the literature of SE practice and research,
organizational learning, knowledge management and project management, to utilize
state-of-the-art knowledge of process improvement and organizational learning.
In conclusion, this thesis brings forward the understanding of work and learning in SE
student teams and how work and learning can be bridged by the use of collaboration
tools aiding retrospective reflection. Further research is needed to explore the potential
of collaboration tools to support such reflection in project teams. This research should
consider the particular characteristics of the work/learning setting (e.g. issues of crosscommunity
collaboration), the collaboration tool features and usage (e.g. the concerted
use of tools), and the organization of the reflective activity (e.g. the use of techniques to
visualize the project process).
80

9 References
Abran, A., Moore, J. W., Bourque, P., and Dupnis, R. (2004). "Guide to The Software
Engineering Body of Knowledge ". City: IEEE.
Ackerman, M. S., and Halverson, C. (2004). "Organizational Memory as Objects,
Process, and Trajectories: An Examination of Organizational Memory in Use."
Computer Supported Cooperative Work, 13(2), 155-189.
Akkerman, S., van den Bossche, P., Admiraal, W., Gijselaers, W., Segers, M., Simons,
R.-J., and Kirschner, P. A. (2007). "Reconsidering group cognition: From
conceptual confusion to a boundary area between cognitive and socio-cultural
perspectives?" Educational Research Review, 2(1), 39-63.
Aranda, J., and Venolia, G. "The Secret Life Of Bugs: Going Past the Errors and
Omssions in Software Repositories " Presented at International Conference on
Software Engineering (ICSE'09), Vancouver, Canada.
Armarego, J. "Beyond PBL: Preparing Graduates for Professional Practice." Presented
at Software Engineering Education & Training, CSEE&T, Dublin.
Bandura, A. (1996). "Social cognitive theory of human development", in T. Husen and
T. N. Postlethwaite, (eds.), International encyclopedia of education. Oxford:
Pergamon Press.
Barab, S. A., and Squire, K. (2004). "Design-Based Research: Putting a Stake in the
Ground." Journal of the Learning Sciences, 13(1), 1-14.
Baron, N. S. (2004). "See you Online: Gender Issues in College Student Use of Instant
Messaging." Journal of Language and Social Psychology, 23(4).
Basili, V. R., and Caldiera, G. (1995). "Improving Software Quality by Reusing
Knowledge and Experience." Sloan Management Review, Fall 1995.
Beck, K. (2000). "Embracing change with extreme programming." IEEE Computer,
32(10), 70-78.
Berger, P., and Luckmann, T. (1966). The Social Construction of Reality, London:
Penguin Books.
Berglund, A., and Eckerdal, A. (2006). "What do CS students try to learn? insights from
a distributed, project-based course in computer systems." Computer Science
Education, 16(3), 11.
Bjørnson, F. O., Wang, A. I., and Arisholm, E. (2009). "Improving the effectiveness of
root cause analysis in post mortem analysis: A controlled experiment."
Information and Software Technology, 51, 150-161.
Blandford, A., and Furniss, D. (2006). "DiCoT: A Methodology for Applying
Distributed Cognition to the Design of Teamworking Systems", in S. W. Gilroy
and M. D. Harrison, (eds.), DSVIS 2005. Berlin Heidelberg: Springer-Verlag.
81

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Blumer, H. (1954). "What Is Wrong with Social Theory?" American Sociological
Review, 19(1), 3-10.
Blumer, H. (1986). Symbolic Interactionism: Perspective and Method, Berkley:
University of California Press.
Boehm, B. (1988). "A spiral model of software development and enhancement." IEEE
Computer, 21(5), 61-72.
Boud, D., Keogh, R., and Walker, D. (1985a). "Promoting Reflection in Learning: a
Model", in D. Boud, R. Keogh, and D. Walker, (eds.), Reflection: Turning
Experience into Learning. RoutledgeFalmer, pp. 18-40.
Boud, D., Keogh, R., and Walker, D. (1985b). Reflection: Turning Experience into
Learning: RoutledgeFalmer.
Bowen, G. A. (2006). "Grounded Theory and Sensitizing Concepts." International
Journal of Qualitative Methods, 5(3).
Brady, A., Johnson, R. R., and Wallace, C. (2006). "The intersecting futures of technical
communication and software engineering: Forging a multi-disciplinary alliance."
Technical Communication, 53(3).
Bratteteig, T., and Gregory, J. "Human Action in Context. A Discussion of Theories for
Understanding Use of IT." Presented at IRIS'22, Jyväskylä.
Brown, A. L. (1992). "Design Experiments: Theoretical and Methodological Challenges
in Creating Complex Interventions in Classroom Settings." The Journal of the
Learning Sciences, 2(2), 141-178.
Brown, J. S., Collins, A., and Duguid, P. (1989). "Situated Cognition and the Culture of
Learning." Educational Researcher, 18(1), 32-42.
Bruner, J. (1990). Acts of Meaning: Harvard University Press.
Bråten, S. (1981). "Quality of Interaction and Participation. On Model Power in
Industrial Democracy and Computer Networks", in G. E. Lasker, (ed.), Applied
Systems and Cybernetics. pp. 191-200.
Buhl, H., and Richter, A. (2004). "Downplaying Model Power in IT Project Work."
Economic and Industrial Democracy, 25.
Burge, J., and Brown, D. C. "SEURAT: integrated rationale management." Presented at
ICSE'08, Leipzig, Germany.
Burge, J., and Kiper, J. "Capturing Collaborative Design Decisions and Rationale."
Presented at Design, Computing, and Cognition, Atlanta, Georgia, USA.
Burnett, R. E. (2001). Technical Communication, Fifth Edtion, Boston, MA, USA:
Thomson Heinle.
Bygstad, B., Krogstie, B. R., and Grønli, T. M. (2009). "Learning from achievement:
scaffolding student projects in software engineering " International Journal of
Networked and Virtual Organizations, 6(2).
Carstensen, P. H., and Schmidt, K. (2002 (1999)). "Computer Supported Cooperative
Work: New Challenges to Systems Design", in K. Itoh, (ed.), Handbook of
Human Factors/Ergonomics. Tokyo: Asakura Publishing.
Cherry, S., and Robillard, P. N. (2008). "The social side of software engineering - A
real ad hoc collaboration network." International Journal of Human-Computer
Studies, 66, 495-505.
Coates, H., James, R., and Baldwin, G. (2005). "A critical examination of the effects of
learning management systems on university teaching and learning." Tertiary
Education and Management 11, 17.
82

References
Cobb, P. (1994). "Where is the Mind? Constructivist and Sociocultural Perspectives on
Mathematical Development." Educational Researcher, 23(7), 8.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., and Schauble, L. (2003). "Design
Experiments in Educational Research." Educational Researcher, 32(1), 9-13.
Cockburn, A. (2000). "Selecting a Project's Methodology." IEEE Software
(July/August), 64-71.
Cockburn, A. (2006). Agile Software Development: The Cooperative Game, 2nd
edition: Addison-Wesley Professional.
Collier, B., DeMarco, T., and Fearey, P. (1996). "A defined process for project post
mortem review." IEEE Software, 13(4), 8.
Collins, A. (1992). "Toward a Design Science of Education", in E. Scanlon and T.
O'Shea, (eds.), New Directions in Educational Technology. Springer-Verlag.
Conklin, E. J., and Yakemovic, K. B. (1991). "A Process-Oriented Approach to Design
Rationale." Human-Computer Interaction, 6, 357-391.
Cress, U., and Kimmerle, J. (2008). "A systemic and cognitive view on collaborative
knowledge building in wikis." Computer-Supported Collaborative Learning, 3,
105-122.
Cubranic, D., Murphy, G. C., Singer, J. S., and Booth, K. S. "Learning from project
history: a case study for software development." Presented at CSCW '04,
Chicago, USA.
Daradoumis, T., and Marques, M. (2002). "Distributed Cognition in the Context of
Virtual Collaborative Learning." Journal of Interactive Learning Research,
13(1/2), 135-148.
Decker, B., Ras, E., Rech, J., Jaubert, P., and Rieth, M. (2007). "Wiki-Based
Stakeholder Participation in Requirements Engineering." IEEE Software,
2007(March/April).
Dekel, U., and Herbsleb, J. D. "Pushing Relevant Artifact Annotationsin Collaborative
Software Development." Presented at CSCW'08, San Diego, USA.
Derby, E., Larsen, D., and Schwaber, K. (2006). Agile Retrospectives. Making Good
Teams Great: Pragmatic Bookshelf.
Dewey, J. (1933). How we think. A restatement of the relation of reflective thinking to
the educative process Boston: D. C. Heath.
Dewey, J. (1978 (1910)). "A short catechism concerning truth", in J. A. Boydston, (ed.),
John Dewey. The middle works 1899-1924.: Southern Illinois University Press.
Dewey, J. (2008 (1909)). Studies in Logical Theory: Bibliobazaar.
Dingsøyr, T. (2005). "Postmortem reviews: purpose and approaches in software
engineering." Information and Software Technology, 47, 293-303.
Dohn, N. B. (2009). "Web 2.0: Inherent tensions and evident challenges for education."
Computer-Supported Collaborative Learning, 4, 343-363.
Dourish, P. "Implications for Design." Presented at CHI, Montreal, Quebec, Canada.
Dourish, P., and Bellotti, V. "Awareness and coordination in shared workspaces."
Presented at ACM CSCW, Toronto, Ontario, Canada
Duim, L. v. d., Jesper, A., and Sinnema, M. "Good practices for Educational Software
Engineering Projects." Presented at 29th International Conference on Software
Engineering (ICSE'07), Minneapolis, USA.
Durkee, D., Brant, S., Nevin, P., Odell, A., Williams, G., Melomey, D., Roberts, H.,
Imafidion, C., Perryman, R., and Lopes, a. (2009). "Implementing E-Learning
83

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
and Web 2.0 Innovation: Didactical Scenarios and Practical Implications."
Industry and Higher Education, 23(4), 8.
Dybå, T., and Dingsøyr, T. (2008). "Empirical studies of agile software development: A
systematic review." Information and Software Technology, 2008(50), 833-859.
Engeström, Y. (1987). Learning by expanding. An activity-theoretical approach to
developmental research: Orienta-Konsultit Oy, Helsinki.
Engeström, Y. (2000). "From individual action to collective activity and back:
developmental work research as an interventionist methodology", in P. Luff, J.
Hindmarsh, and C. Heath, (eds.), Workplace Studies - Recovering Work Practice
and Informing System Design. Cambridge University Press.
Eraut, M. (2000). "Non-formal learning and tacit knowledge in professional work."
British Journal of Educational Psychology, 70, 24.
Eraut, M., and Hirsch, W. (2007). The Significance of Workplace Learning for
Individuals, Groups and Organisations. Oxford & Cardiff Universities.
Fitzpatrick, G. (2003). The Locales Framework: Understanding and Designing for
Wicked Problems: Kluwer Academic Publishers.
Fitzpatrick, G., Marshall, P., and Phillips, A. "CVS integration with notification and
chat: lightweight software team collaboration." Presented at CSCW'06, Banff,
Alberta, Canada.
Fuchs-Kittowski, F., and Köhler, A. "Wiki Communities in the Context of Work
Processes." Presented at 2005 international symposium on Wikis WikiSym '05
Gal, U., Youngjin, Y., and Boland, R. (2005). "The dynamics of boundary objects,
social infrastructures and social identities."
Germain, E., and Robillard, P. N. (2004). "Engineering-based processes and agile
methodologies for software development: a comparative case study." The
Journal of Systems and Software, 75, 17-27.
Gleaves, a., Walker, C., and Grey, J. (2007). "Using digital and paper diaries for
learning and assessment purposes in higher education: a comparative study of
feasibility and reliability." Assessment & Evaluation in Higher Education, 32(6),
631-643.
Grinter, R. E., and Palen, L. "Instant Messaging in Teen Life." Presented at CSCW'02,
New Orelans, Louisiana, USA.
Gutwin, C., and Greenberg, S. (2002). "A Descriptive Framework of Workspace
Awareness for Real-Time Groupware." Computer Supported Cooperative Work
11(3-4), 411-446.
Gutwin, C., Penner, R., and Schneider, K. "Group Awareness in Distributed Software
Development." Presented at CSCW'04, Chicago, Illinois, USA.
Gutwin, C., Penner, R., and Schneider, K. "Knowledge sharing in software engineering:
Group awareness in distributed software development " Presented at CSCW'04,
Chicago, Illinois, USA.
Halverson, C. (2002). "Activity Theory and Distributed Cognition: Or What Does
CSCW Need to DO with Theories?" Computer Supported Cooperative Work,
11, 243-267.
Helle, L., Tynjälä, P., and Olkinuora, E. (2006). "Project-based learning in postsecondary
education - theory, practice and rubber sling shots." Higher
Education, 51, 287-314.
84

References
Herbsleb, J. D., Mockus, A., Finholt, T. A., and Grinter, R., E. "An Empirical Study of
Global Software Development: Distance and Speed." Presented at International
Conference on Software Engineering, Toronto, Ontario, Canada.
Hickerson, C. A., and Giglio, M. (2009). "Instant Messaging Between Students and
Faculty: A Tool for Increasing Student-Faculty Interaction." International
Journal on E-learning, 8(1), 71-88.
Hildrum, J. M. (2009). "Sharing Tacit Knowledge Online: A Case Study of e-Learning
in Cisco's Network of System Integrator Partner Firms." Industry and
Innovation, 16(2).
Huang, A. H., and Yen, D. C. (2003). "Usefulness of instant messaging among young
users: Social vs. work perspective." Human Systems Management, 22(2), 63-72.
Hutchins, E. (1995). Cognition in the Wild, Cambridge, Massachusetts: The MIT Press.
Idris, Y., and Wang, Q. (2009). "Affordances of Facebook for learning." International
Journal of Continuing Engineering Education and Life-Long Learning
(IJCEELL), 19(2/3).
IEEE. (1990). "IEEE Standard Glossary of Software Engineering Terminology"std
610.12-1990. City: IEEE.
Isaacs, E., Camm, C., Schiano, D. J., Walendowski, A., and Whittaker, S.
"Characterizing Instant Messaging from Recorded Logs." Presented at CHI
2002.
Isaacs, E., Walendowski, A., Whittaker, S., Schiano, D. J., and Kamm, C. "The
Character, Functions, and Styles of Instant Messaging in the Workplace."
Presented at CSCW'02, New Orleans, Louisiana, USA.
Jonassen, D. H. (1995). "Computers as Cognitive Tools: Learning with Technology,
Not from Technology." Journal of Computing in Higher Education, 6(2), 40-73.
Kaptelinin, V., and Nardi, B. A. (2006). Acting with Technology. Activity Theory and
Interaction Design., Cambridge, Massachusetts: MIT Press.
Kasi, V., Keil, M., Mathiassen, L., and Pedersen, K. (2008). "The post mortem paradox:
a Delphi study of IT specialist perceptions." European Journal of Information
Systems, 17, 62-78.
Keil, M., Mann, J., and Rai, A. (2000). "Why Software Projects Escalate: An Empirical
Analysis and Test of Four Theoretical Models." MIS Quarterly, 24(4).
Kelly, A. E. (2003). "Research as design: The role of design in educational research."
Educational Researher, 32(1), 3-4.
Kelly, A. E., Baek, J. Y., Lesh, R. A., and Bannan-Ritland, B. (2008a). "Enabling
Innovations in Education and Systematizing their Impact", in A. E. Kelly, R. A.
Lesh, and J. Y. Baek, (eds.), Handbook of Design Research Methods in
Education. New York: Routledge.
Kelly, A. E., Lesh, R. A., and Baek, J. Y. (2008b). Handbook of Design Research
Methods in Education: Routledge.
Kerth, N. (2001). Project Retrospectives: A Handbook for Team Reviews Dorset House
Publishing Company.
Kim, B., and Reeves, T. C. (2007). "Reframing research on learning with technology: in
search of the meaning of cognitive tools." Instructional Science, 35, 207-256.
Kim, D., and Lee, S. (2002). "Designing Collaborative Reflection Support Tools in e-
project Based Learning Environment." Journal of Interactive Learning
Research, 13(4), 375-392.
85

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Kirschner, P. A., and Erkens, G. (2006). "Cognitive tools and mindtools for
collaborative learning." Journal of Educational Computing Research, 35(2),
199-209.
Klein, H. K., and Myers, M. M. (1999). "A Set of Principles for Conducting and
Evaluating Interpretive Field Studies in Information Systems." MIS Quarterly,
23(1), 67-94.
Kolb, D. A., and Fry, R. (1975). "Towards an applied theory of experiental learning", in
C. L. Cooper, (ed.), Theories of Group Processes. London: John Wiley, pp. 33-
58.
Kolb, D. A., Rubin, I. M., and McIntyre, J. (1971). "Organizational psychology: an
experiential approach". City: Prentice Hall: Englewood Cliffs, NJ.
Krogstie, B. (2000). Applying Activity Theory in Knowledge Management, University of
Oslo, Oslo.
Krogstie, B. "Power through brokering. OSS participation in SE projects." Presented at
International Conference on Software Engineering (ICSE) 2008, Leipzig.
Krogstie, B., and Bygstad, B. "Cross-Community Collaboration and Learning in
Customer-Driven Software Engineering Student Projects." Presented at
Twentieth Conference on Software Engineering Education and Training
(CSEE&T), Dublin.
Krogstie, B., and Divitini, M. "Practice-Based Learning as Mobile Learning: The Role
of Boundary Objects." Presented at Mobile Learning, Lisbon, Portugal.
Krogstie, B. R. "The wiki as an integrative tool in project work." Presented at COOP,
Carry-le-Rouet, Provence, France.
Krogstie, B. R. "Do's and dont's of instant messaging in students' project work."
Presented at NOKOBIT 2009, Trondheim, Norway.
Krogstie, B. R. "A model of retrospective reflection in project based learning utilizing
historical data in collaborative tools." Presented at EC-TEL 2009, Nice, France.
Krogstie, B. R. "Using Project Wiki History to Reflect on the Project Process "
Presented at 42nd Hawaii International Conference on System Sciences, Big
Island, Hawaii.
Krogstie, B. R., and Divitini, M. "Shared timeline and individual experience:
Supporting retrospective reflection in student software engineering teams."
Presented at CSEE&T 2009, Hyderabad.
Krogstie, B. R., and Divitini, M. "Supporting Reflection in Software Development with
Everyday Working Tools." Presented at COOP, Aix-en-Provence, France.
Kruchten, P. (2003). The Rational Unified Process: An introduction: Addison-Wesley.
Kuutti, K. (1995). "Activity Theory as a Potential Framework for Human-Computer
Interaction Research", in B. A. Nardi, (ed.), Context and Consciousness London:
MIT Press.
Kuutti, K., and Kaptelinin, V. "Rethinking cognitive tools: from augmentation to
mediation." Presented at Second International Conference on Cognitive
Technology Humanizing the INformation Age.
Latour, B. (2005). Reassembling the Social. An Introduction to Actor-Network-Theory:
Oxford University Press.
Lave, J., and Wenger, E. (1991). Situated Learning. Legitimate peripheral
participation, Cambridge: University of Cambridge Press.
86

References
Leont'ev, A. N. (1981). Problems of the development of the mind: Progress Publishers,
Moscow.
Lin, X., Hmelo, C., Kinzer, C. K., and Secules, T. J. (1999). "Designing Technology to
Support Reflection." Educational Technology, Research and Development,
47(3), 43-62.
Louridas, P. (2006). "Using Wikis in Software Development." IEEE Software, 23(2 ).
Lovejoy, T., and Grudin, J. "Messaging And Formality: Will IM Follow in the
Footsteps of Email?" Presented at INTERACT Zurich, Switzerland.
Lund, A., and Smørdal, O. "Is There a Space for the Teacher in a WIKI?" Presented at
Proceedings of the 2006 international symposium on Wikis WikiSym '06
Lyytinen, K., and Robey, D. (1999). "Learning failure in information systems
development." Information Systems Journal, 9, 17.
Majchrzak, A., Wagner, C., and Yates, D. "Corporate Wiki Users: Results of a Survey."
Presented at WikiSym, Odense, Denmark.
McCarthy, J., and Wright, P. (2004). Technology as Experience: MIT Press.
McMillan, W. W. "What Leading Practitioners Say Should Be Emphasized in Students'
Software Engineering Projects." Presented at Conference on Software
Engineering Education and Training, New Orleans, Louisiana, USA.
Mead, G. H. (1934). Mind, Self and Society, Chicago: The University of Chicago Press.
Muller, M., Raven, M. E., Kogan, S., Millen, D. R., and Carey, K. "Introducing Chat
into Business Organisations: Toward an Instant Messaging Maturity Model."
Presented at GROUP'03, Sanibel Island, Florida, USA.
Nardi, B. A. "Studying context: a comparison of activity theory, situated action models,
and distributed cognition." Presented at St.Petersburg International Workshop
on Human-Computer Interaction, St.Petersburg, USSR.
Nardi, B. A. (2002). "Coda and Response to Christine Halverson." Computer Supported
Cooperative Work, 11, 269-275.
Nardi, B. A., Whittaker, S., and Bradner, E. "Interaction and Outeraction: Instant
Messaging in Action." Presented at CSCW'00, Philadelphia, PA, USA.
Niinimaki, T. "Experiences of instant messaging in global software development
projects: a multiple case study." Presented at International Conference on
Global Software Engineering (ICGSE), Piscataway, NJ, USA.
Omoronyia, I., Ferguson, J., Roper, M., and Wood, M. (2009). "Using Developer
Activity Data to Enhance Awareness during Collaborative Software
Development." Computer Supported Cooperative Work, 18, 50.
OpenSourceInitiative. (2010). "The Open Source Definition". City.
Palincsar, A. S. (1998). "Social Constructivist Perspectives on Teaching and Learning."
Annual Review of Psychology, 49, 345-75.
Pea, R. (1985). "Beyond Amplification: Using the Computer to Reorganize Mental
Functioning." Educational Psychologist, 20(4).
Pea, R. (1994). "Seeing What We Build Together: Distributed Multimedia Learning
Environments for Transformative Communications." The Journal of the
Learning Sciences, 3(3), 15.
Phillips, D. C. (1995). "The Good, the Bad, and the Ugly: The Many Faces of
Constructivism." Educational Researcher, 24(7), 5-12.
87

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Pipek, V., and Wulf, V. (2007). "From Groupware towards Collaborative
Infrastructures. Panel presentation."European conference on Computer-
Supported Cooperative Work. City: Limerick, Ireland.
Polanyi, M. (1966). The Tacit Dimension: Doubleday & Co.
Poole, W. G. "The Softer Side of Custom Software Development: Working with the
Other Players." Presented at Conference on Software Engineering Education
and Training, Madrid, Spain.
Prince, M., and Felder, R. (2007). "The Many Faces of Inductive Teaching and
Learning." Journal of College Science Teaching, 36(5), 14-20.
ProjectManagementInstitute. (2008). "A Guide to the Project Management Body of
Knowledge (PMBOK Guide)". City: IEEE Society.
Quan-Haase, A., Cothrel, J., and Wellman, B. (2005). "Instant Messaging for
Collaboration: A Case Study of a High-Tech Firm." Journal of computermediated
communication, 10(4).
Radziwill, N. M., and Shelton, A. L. "TWiki as a Platform for Collaborative Software
Development Management." Presented at Advanced Software, Control, and
Communication Systems for Astronomy, Glasgow, Scotland, United Kingdom.
Randall, D., Harper, R., and Rouncefield, M. (2007). Fieldwork for Design. Theory and
Practice: Springer.
Riel, M., and Polin, L. (2004). "Online Learning Communities. Common Ground and
Critical Differences in Designing Technical Environments", in S. A. K. Barab,
Rob; Gray, James H., (ed.), Designing for Virtual Communities in the Service of
Learning. Cambridge: Cambridge University Press, pp. 16-50.
Rogers, Y., and Ellis, J. (1994). "Distributed Cognition: an alternative framework for
analyzing and explaining collaborative working." Journal of Information
Technology, 9, 119-128.
Rooij, S. W. v. (2009). "Scaffolding project-based learning with the project
management body of knowledge (PMBOK)." Computers & Education, 52, 10.
Sacks, H. (1992). Lectures on Conversation, Oxford, UK: Blackwell.
Salomon, G. (1993). Distributed Cognitions, New York: Cambridge University Press.
Schindler, M., and Eppler, M. J. (2003). "Harvesting project knowledge: a review of
project learning methods and success factors." International Journal of Project
Management, 21, 10.
Schmidt, K., and Simone, C. (1996). "Coordination Mechanisms: Towards a Conceptual
Foundation of CSCW Systems Design." Computer Supported Cooperative
Work, 5, 155-200.
Schön, D. (1983). The Reflective Practitioner: Basic Books, Inc.
Schön, D. (1987). Educating the Reflective Practitioner. , San Fransisco: Jossey-Bass.
Sharp, H., and Robinson, H. (2006). "A Distributed Cognition Account of Mature XP
Teams", in M. Marchesi and G. Succi, (eds.), XP 2006. Berlin Heidelberg:
Springer-Verlag, pp. 1-10.
Sommerville, I. (2001). Software Engineering, Sixth Edition: Addison-Wesley.
Stafford, T. (2008). "Introduction to the Special Issue on Instant Messaging in the
Workplace." IEEE Transactions on Professional Communication, 51(4), 350-
351.
88

References
Stahl, G. (2002). "Building collaborative knowing", in J.-W. Strijbos, P. A. Kirschner,
and R. L. Martens, (eds.), What We Know About CSCL And Implementing It In
Higher Education. Boston: Kluwer Academic Publishers, pp. 53-85.
Stahl, G., Koschmann, T., and Suthers, D. (2006). "Computer-supported collaborative
learning: An historical perspective", in R. K. Sawyer, (ed.), Cambridge
handbook of the learning sciences. Cambridge, UK: Cambridge University
Press.
Star, S. L. (1990). "The Structure of Ill-Structured Solutions: Boundary Objects and
Heterogeneous Distributed Problem Solving", in M. Huhns and L. Gasser,
(eds.), Distributed Artificial Intelligence. Morgan Kaufmann Publishers, pp. 37-
54.
Star, S. L., and Griesemer, J. R. (1989). "Institutional Ecology, 'Translations' and
Boundary Objects: Amateurs and Professionals in Berkley's Museum of
Vertebrate Zoology, 1907-39." Social Studies of Science, 19, 387-420.
Storey, M.-A. D., Cubranic, D., and German, D. M. "On the use of visualization to
support awareness of human activities in software development: a survey and
framework." Presented at 2005 ACM symposium on Software visualization,
St.Louis, Missouri, USA.
Strauss, A. (1993). Continual permutations of action, New York: Aldine de Gruyter.
Strijbos, J.-W., Kirschner, P. A., and Martens, R. L. (2004). What We Know About
CSCL And Implementing It In Higher Education: Kluwer Academic Publishers.
Suthers, D., and Hundhausen, C. "Learning by Constructing Collaborative
Representations: An Empirical Comparison of Three Alternatives." Presented at
European Conference on Computer-Supported Collaborative Learning,
Maastrict, the Netherlands.
Suthers, D. D. (2006). "Technology affordances for intersubjective meaning making: A
research agenda for CSCL." Computer-Supported Collaborative Learning, 1,
315-337.
Thomas, J. (2000). A review of research on project-based learning, Novato, CA: The
Buck Institute for Education.
Trentin, G. (2009). "Using a Wiki to Evaluate Individual Contribution to a
Collaborative Learning Project." Journal of Computer Assisted Learning, 25(1),
43-55.
von Glasersfeld, E. (1989). "Cognition, Construction of Knowledge, and Teaching."
Synthese, 80(1), 121-140.
Vygotsky, L. S. (1978). Mind in Society. The Development of Higher Psychological
Processes., Cambridge, Massachusetts: Harvard University Press.
Walsham, G. (1995). "Interpretive case studies in IS research: nature and method."
European Journal of Information Systems(4), 74-81.
Walsham, G. (2001). "Knowledge Management: The benefits and limitations of
computer systems." European Management Journal, 19(6), 599-608.
Walsham, G. (2006). "Doing interpretive research." European Journal of Information
Systems, 15, 11.
Wenger, E. (1998). Communities of Practice. Learning, Meaning, and Identity:
Cambridge University Press.
Wenger, E. (2000). "Communities of practice and social learning systems."
Organization, 7(2), 225-246.
89

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Wood, D., Bruner, J., and Ross, G. (1976). "The role of tutoring in problem solving "
Journal of Child Psychology and Psychiatry, 17(2), 89-100.
Xiao, L., Clark, S., Rosson, M. B., and Carroll, J. M. "Promoting Reflective Thinking in
Collaborative Learning Activities." Presented at Eighth IEEE International
Conference on Advanced Learning Technologies (ICALT), Santander,
Cantrabria, Spain.
Xu, L. (2007). "Project the wiki way: Using wiki for computer science course project
management." Journal of Computing Sciences in Colleges 22(6).
Yin, R. K. (2003). Case Study Research. Design and Methods. Third Edition.: SAGE
Publications.
Zuhrieh, S. (2009). "Learning with Technology: Using Discussion Forums to Augment
a Traditional-Style Class." Educational Technology and Society, 12(3), 15.
90

Glossary
B
Boundary object – artifacts, documents, terms, concepts etc. around which CoPs can
organize their interconnections (Wenger 1998). Boundary Objects are plastic enough
to adapt to local needs and constraints of the parties employing them, yet robust
enough to maintain a common identity across sites (Star and Griesemer 1989).
Brokering – connections provided by people who can introduce elements of one
practice into another (Wenger 1998)
C
Collaboration - to work jointly with others or together especially in an intellectual
endeavour (Meriam Webster Online). In the literature within the research fields of
CSCW and CSCL, there are definitions distinguishing between collaboration and
cooperation, some pointing to cooperation as involving the coordination of
independent tasks, whereas collaboration involves mutually dependent tasks.
However, the usage of these terms varies. In the thesis I do not apply a strict
distinction, and in the different research papers, I have been using the terms
cooperation technology, collaboration tools and collaborative tools interchangeably.
Computer supported cooperative work (CSCW) – the research field addressing
how collaborative activities and their coordination can be supported by means of
computer systems (Carstensen and Schmidt 2002 (1999)).
Computer supported collaborative learning (CSCL) – an emerging branch of the
learning sciences concerned with studying how people can learn together with the
help of computers (Stahl et al. 2006). See Technology-Enhanced Learning (TEL).
Community of practice (CoP) – A group of people characterized by a joint
enterprise, mutual engagement, and a shared repertoire. Another perspective on a CoP
is to consider it a shared history of learning. (Lave and Wenger 1991; Wenger 1998)
Cooperation – to act or work with another or others (Merriam Webster Online). See
Collaboration.
91

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
L
Lightweight collaboration tools – Collaboration tools that can be acquired and taken
into use at low cost (e.g. money, time to learn) for individuals and their organization.
A lightweight collaboration tool typically provides a limited set of features to support
one aspect of collaborative work and may thus be relatively easily integrated into
existing work processes (rather than imposing a certain process on the user). Many of
the lightweight collaboration tools are associated with Web 2.0.
O
Open source software (OSS) – computer software for which the source code and
certain other rights are provided under a software license that permits users to study,
change, and improve the software. The Open Source Initiative outlines a set of criteria
for an open source licence (OpenSourceInitiative 2010)
P
Post-mortem – In the context of project work, post-mortem refers to something
undertaken after project completion or after the completion of a project phase.
Problem based learning – see Section 3.1
Project based learning (PBL) – see Section 3.1
Project retrospective – an activity conducted after the completion of an entire project
or a major project phase to reflect on and learn from the project process in order to
improve it (Dingsøyr 2005) (or identify lessons learned to draw on in other projects).
Post-mortem review, post-mortem analysis, post-mortem evaluation, and also, for
short, post-mortem largely refer to the same type of activity, project retrospective
typically being used about post-mortem evaluation in the context of agile software
development (Derby et al. 2006; Kerth 2001).
S
Social software – a range of software systems used for interacting and sharing data,
comprising lightweight collaboration tools like instant messaging as well as web sites
like Facebook, LinkedIn, Flickr, YouTube, amazon and eBay allowing the users to
interact through the system and build and maintain a social profile. See Web 2.0.
Software development (SD) – the domain of work concerned with the development
of software. Can be considered a subset of software engineering.
Software engineering (SE) – A) An engineering discipline which is concerned with
all aspects of software production from early stages of system specification through to
92

Glossary
maintaining the system after it has been put into use (Sommerville 2001) – B) (1) The
application of a systematic, disciplined, quantifiable approach to the development,
operation, and maintenance of software; that is, the application of engineering to
software. (2) The study of approaches as in (1) (IEEE 1990).
T
Technology enhanced learning (TEL) – the support of any learning activity through
technology. Whereas CSCL (see above) can be considered a distinct research
community, it is also possible to consider it part of TEL. This has been done for
simplicity in the thesis, in which CSCL is referred to in cases where it is relevant to
point to that particular research community or body of literature.
Timeline – 1) In the reflection workshops described in the thesis, a T. is a line drawn
on paper or whiteboard along which are marked events perceived by project
participants to be of some importance to the project. 2) A collaboration tool (e.g.
Trac) may include functionality for chronologically displaying (often fine grained)
actions/events related to the work supported by the tool (e.g. all source code updates).
In Trac, the display of this chronology is named the Timeline.
Trajectory - “(1) the course of any experienced phenomenon as it evolves over time”
and “(2) the actions and interactions contributing to its evolution” (Strauss 1993,
pp.53-54).
U
Use – the act or practice of employing something (Webster Online).
Usage – firmly established and generally accepted practice or procedure (Webster
Online).
W
Web 2.0 – This is a concept for which many different definitions and explanations
exist. Dohn describes Web 2.0 as practice rather than a set of technologies, arguing
that a particular technology may be used in a way that is more or less Web 2.0. Web
2.0, according to Dohn, denotes activities that have most or all of the following
characteristics (the last one being necessary):
• “Collaboration and/or distributed authorship
• Active, open-access, “bottom-up” participation and interactive multi-way
communication
93

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
• Continuous production, reproduction, and transformation of material in use
and reuse across contexts
• Openness of content, renunciation of copyright, distributed ownership
• Lack of finality, “awareness-in-practice” of the “open-endedness” of the
activity
• Taking place on the WWW, or to a large extent utilizing Web-mediated
resources and activities” (Dohn 2009, p.345)
94

Appendix A: Research papers
P1
P2
P3
Krogstie, B. and B. Bygstad. Cross-Community Collaboration and Learning in
Customer-Driven Software Engineering Student Projects. Twentieth
Conference on Software Engineering Education and Training (CSEE&T)
2007, Dublin. IEEE Computer Society. (Krogstie and Bygstad 2007)
Krogstie, B. Power through brokering. OSS participation in SE projects. ICSE
2008, Leipzig. IEEE Computer Society. (Krogstie 2008a)
Krogstie, B.R. Do‟s and Don‟ts of Instant Messaging in Students‟ Project
Work. NOKOBIT 2009, Trondheim. Tapir. (Krogstie 2009a)
P4 Krogstie, B.R. The wiki as an integrative tool in project work. in COOP 2008.
Carry-le-Rouet, Provence, France. Institut d‟Etudes Politiques d‟Aix-en-
Provence. (Krogstie 2008b)
P5
P6
P7
P8
Krogstie, B.R. Using Project Wiki History to Reflect on the Project Process.
42nd Hawaii International Conference on System Sciences (HICSS‟42) 2009.
Big Island, Hawaii: IEEE Computer Society. (Krogstie 2009c)
Krogstie, B.R. and M. Divitini. Shared timeline and individual experience:
Supporting retrospective reflection in student software engineering teams.
CSEE&T 2009, Hyderabad. IEEE Computer Society. (Krogstie and Divitini
2009)
Krogstie, B.R. A model of retrospective reflection in project based learning
utilizing historical data in collaborative tools. EC-TEL 2009, Nice. Springer.
(Krogstie 2009b)
Krogstie, B.R. and M. Divitini. Supporting Reflection in Software
Development with Everyday Working Tools. COOP 2010, Aix-en-Provence,
France. Springer. (Krogstie and Divitini 2010)
95

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
96

Research paper P1
Title:
Cross-Community Collaboration and Learning in Customer-Driven Software
Engineering Student Projects
Authors:
Birgit Krogstie and Bendik Bygstad
Published in:
Proceedings of CSEE&T 2007
Pages:
8
Complete reference:
Krogstie, B. and B. Bygstad (2007). Cross-Community Collaboration and Learning in
Customer-Driven Software Engineering Student Projects. Twentieth Conference on
Software Engineering Education and Training (CSEE&T), Dublin, Ireland, 3-5 July.
IEEE Computer Society.
97

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
98

Cross-Community Collaboration and Learning in Customer-Driven
Software Engineering Student Projects
Birgit Krogstie
NTNU
birgitkr@idi.ntnu.no
Bendik Bygstad
NITH
bendik.bygstad@nith.no
Abstract
This paper explores collaboration and learning between stakeholders in customer-driven
student projects. The research objectives are to obtain empirically based knowledge on how
students relate to stakeholders in customer-driven projects, and to suggest implications for
the pedagogical design of the project courses.
Empirical data was collected from two Bachelor courses in software engineering at two
learning institutions in Norway. To make sense of the interaction between the three
stakeholders in the project: the student groups, the university and the customer, we build on
Wenger’s concept of communities of practice and on the concept of boundary objects.
Our analysis highlights that students, through practical experience in the projects, learn to
balance the requirements and expectations from different stakeholders in designing a working
technical solution - a valuable skill for software engineers. We argue that for students to
learn to balance stakeholders’ interests in the best possible way, visibility of stakeholders’
goals should be focused throughout the projects. Explicit reference to the goals should be
incorporated into project artifacts serving as boundary objects. Collaboration technologies
providing standard shared workspace functionality are seen as adequate to support this.
1. Introduction
Software Engineering (SE) project courses provide students with arenas for project
based learning [1, 2] and serve as a stage of transition from the student role to that of a
SE professional. Projects with external customers and “real” problems provide
authenticity [3] and have been found to be useful in preparing the students for work in
the IT industry [4]. A pedagogical ideal for the projects can be seen to have the students
complete a tour of “educational refinement” in the SE “real world”, making use of their
knowledge and skills to their best of their ability while having their beliefs challenged.
Through the projects, we want students to make a first step towards practicing the SE
‘methodology-in-action’ [5] which requires enough experience to allow for a critical
distance to, and application of, the formalized SE methodology taught at the university.
In SE projects, success depends on the team recognizing and reconciling the different
goals, knowledge and practices among major stakeholders [6, 7]. In customer-driven
student projects, there is one stakeholder not found in professional SE: the university as
represented by the course staff. The latter serves as a resource assisting the students in
meeting realistic challenges of SE work, e.g. managing the customer relationship, but at
the same time, they impose complexity and constraints that are not there in real SE
work. The research objectives of the work presented in this paper are to obtain
empirically based knowledge on how students relate to stakeholders in customer-driven
projects, and to suggest implications for the pedagogical design of the project courses.
In what follows, we first introduce our theoretical understanding of stakeholder
communities and goals, and how students’ reconciliation of them is essential to
learning. Particularly, we point to the role of artifacts as boundary objects between
communities. In section 3 we present our case study. In section 4 we provide a number
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
99

of examples from the case illustrating how students reflect on their relationship to
different communities in the project. In section 5, we discuss our findings in the light
of theory and suggest implications for the pedagogical design of project courses.
Finally, we address limitations and further work, and conclude the paper.
2. Cross-community collaboration: the role of goals and boundary objects
In this section we elaborate on three generic aspects of customer-driven SE student
projects that can be seen as central to students’ learning: stakeholders as learning
communities, stakeholders’ goals for project engagement, and project activity as a
question of relating to and reconciling the goals and interests of interrelated learning
communities.
2.1. Project stakeholders represent different kinds of learning communities
Learning communities are communities designed to support learning. Riel and Polin
[8] describe three, overlapping types: Task-based learning communities are groups of
people who are organized around a task, working together for a specified period of time
to produce a product. Practice-based learning communities are larger groups with
shared goals and provide members with richly contextualized and supported arenas for
learning. Knowledge-based learning communities resemble the practice-based ones but
are focused on producing external knowledge about the practice.
A university is officially designated to serve a role in society as a practice-based and
knowledge-based learning community. Its practices include research and
teaching/learning. A part of the university organization, i.e. a SE study program, can be
considered a learning community of its own. A customer organization may also be
considered both practice-based and knowledge-based, the weight on the latter
depending on the actual policies and practices (e.g. knowledge management efforts). A
sub-community in the customer organization may typically be serving as the learning
community most directly relating to the student project. The customer-driven SE
student project exists for a much more limited time than the other two communities, but
it can be considered as a type of learning community focused on the achievement of a
task: a task-based learning community.
University
SE study program
SE profession
learning
Customer
organization
SE student
project
Figure 1: The SE project at the intersection of learning communities,
Several overlapping communities will potentially learn from the project (see Figure 1). The
impact is likely to be smaller, and the learning cycle longer, the more general the community.
E.g., the project supervisor may learn about project supervision from one session to the next;
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
100

the course staff may improve the course between semesters, and the success of the SE study
program may finally influence university decisions. The customer representative may learn
about technical issues whereas her department benefits from knowledge about the viability of
an idea after project completion.
2.2 Project stakeholders have different goals for their engagement in the projects
Students want formal qualifications making them attractive on the job market, which
includes knowledge and skill e.g. in project management and programming. The project
customer becomes part of the students’ network in the business world, perhaps leading
to a job offer. Students also want to enjoy the social aspects of project work.
The course staff seeks to improve the course. Such improvement partially rests on
evaluation of the projects as reflected in the grades. Learning objectives are formalized
in course descriptions. Students’ viewpoints on the projects (e.g. as found in reflection
notes) provide feedback on the pedagogical quality of the project course. Additionally,
the project course must meet relevant needs in the business world. If customers express
that their needs are met, it is an acknowledgement of the relevance of the project course
and courses on which the project is based. SE projects thus serve as a test bench for the
SE study program. The university-customer relationship further contributes in building
a business network for the university, to be utilized in new student or research projects.
2.3. Cross-community interaction as essential to project work and learning
Two important types of connections between communities are boundary objects and
brokering [9]. Boundary objects are “artifacts, documents, terms, concepts, and other
forms of reification around which communities of practice can organize their
interconnections.” [10-12]. The artifacts central to the SE project are important because
they support the goal attainment of one or more stakeholders. Reaching agreement on
the development of an artifact important to more than one community is a question of
making alignments based on the views, knowledge and practices of the communities.
An example from the SE world explicitly referring to negotiation in development work
is the WinWin methodology [13]. In SE student projects, there are artifacts to be
negotiated with the course staff, with the customer and with both of them. The
requirements specification is a particularly central artifact, negotiated between the
student team and the customer and subject to course staff’s supervision and evaluation.
For a boundary object to work well, communities should be able to use it in making
sense of their own activity as well as the activity of the other community. The boundary
object contributes to making visible stakeholders’ objectives, as well as possibly the
negotiation and development process, e.g. by documenting arguments, decisions and
versions. This corresponds to the boundary object being socially translucent [14],
providing awareness, accountability and visibility of the development process.
Brokering is “connections provided by people who can introduce elements of one
practice into another.” [9] (p.105). Students in a SE project can be considered brokers
between the university, the customer organization and the project team to the extent that
they are members of all communities and work to align the perspectives and practices
from each community in the project. In the capacity of being students, the developers
are participants in the teaching/learning practices of the university. Also, they are
participants in the task-oriented practice of the project team. Further, the students can
be seen as novice professionals engaging in legitimate peripheral participation [15] in
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
101

the SE professional community, but the team members generally have no formal
membership in the project customer organization.
3. The case: two project courses in Norway
In an interpretive qualitative study we collected data from the spring semester of
2006 in two, fairly similar bachelor level project courses in Norway. The authors are
engaged in teaching and supervision of these project courses. Project Course 1 (P1) is a
project course at NITH, which is a small, private university college delivering study
programs within IT. Project Course 2 (P2) is a course at NTNU, which is a large public
university with a main focus on technical study programs.
Although there are some differences between the courses, they are structurally
similar enough (see Table 1) for us to use empirical data from both of them combined to
analyze the relationship between the stakeholders.
Table 1: Characteristics of the two project courses of our case
Characteristics Project Course 1 (NITH) Project Course 2 (NTNU)
Level
Bachelor, 6 th (final) semester
Number of students 50
46
Size of project Approx.5 students, self-formed Approx.5 students, mostly self-formed
groups/duration groups, one semester, 60% of groups, one semester, 50% of semester
semester workload
workload
Form of course Only introductory / occasional lectures. Supervisor from staff. Appointed group
delivery
contact in the customer organization.
Location Work mainly at the customer’s site. Work mainly on the university campus.
Some groups work at customers’ site.
Educational
Mostly common courses, including Some common courses, including basic
background within basic courses in Java and
system analysis and programming.
the study program RUP/UML
Previous project 2 nd year, case ready-made for the 2 nd year, case ready-made for the
course
course, RUP-based, Java/UML, no course, partly XP-based, Java/UML, no
external customer
external customer
Case Real case, external customer, software product to be developed
Imposed structure on Final project report required. Method (e.g. process model) freely chosen by the
part of the school group. Groups required to report regularly on progress/status to supervisor.
Mid-term report required.
Assessment One grade for the whole group. Both product, process and presentation count.
Data for this paper were collected through semi-structured interviews with P1
groups, done as part of an exploratory case study on work and collaboration support in
the projects. One-hour interviews were conducted ca 3 weeks before project delivery
with 7 out of 11 groups. We also made brief (10-20 min) telephone interviews with P1
and P2 customers 2-3 months after project completion, addressing customers’ rationale
for, and outcome from, project participation. 6 out of 14 P1 customers responded, as did
9 out of 9 P2 customers (of 10 projects). We further draw on reflection notes in
students’ project reports. Also, we made extensive use of available written documents:
customers’ evaluation forms, formal course specifications, project reports, etc.
Main viewpoints from the customer interviews were categorized and summarized in a
table. Data on the project teams was analyzed in the following steps. First, the
interviews and the collected written materials were summarized for each group. Then
we used our theoretical lens, described in the previous section, to identify central
attributes of the experiences for each group. Finally, we analyzed the data across
groups, identifying patterns and explanations.
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
102

4: Case findings: students’ view on relating to different communities
Our analysis resulted in four main findings, the first one relating to customers’
viewpoints on the projects and the others relating to students’ viewpoints.
4.1. Customers have several different goals for their project participation
Customer interviews indicated satisfaction with the project; products having been
developed according to specifications and expectations and at very low cost. Many
customers reported that used the project to learn about and/or evaluate their technology
and/or ideas. Also, many said they had learnt something that would influence their daily
work. Further, many reported learning about playing the role of customer, e.g. in terms
of needs for involvement. The requirements specification was generally regarded by
customers as the most important project document. Finally, many customers were
looking for new employees, and some ended up recruiting from the project groups.
4.2. Project teams relate to communities and community membership
We find that SD project students actively relate to the various communities involved
and the balancing of requirements from different project stakeholders
In our interviews we asked the students how they mainly saw themselves working in
the project (e.g. at the customer’s site): as students or consultants/professionals.
Students generally saw themselves as students. There was some indication that groups
who most strongly considered their work to be of real value to the customer, also
considered themselves more as consultants. The customer’s expressed confidence in the
group, expectations for the result and willingness to provide necessary resources
seemed to boost students’ self confidence about their role as novice professionals.
Most groups considered their project to be close to a real SE project, with the
exception of the course staff’s requirements for documentation, the lack of
consequences if the project failed and the general absence of economic considerations.
“This is no Mickey Mouse project”, as the project manager in one group expressed it.
4.3 Awareness of stakeholder goals is often insufficient in the teams
Students are not necessarily aware of all stakeholder goals for the project work (e.g.
as revealed in our customer interviews), or they have an interpretation of stakeholder
goals that might have been questioned by the stakeholders themselves if they had
known students’ reasoning.
Students are aware that different stakeholder goals should be met, some related to
completing work as a SE project and others to completing the project as a university
course: “In a real project there would have been money involved. Now focus is very
much on us making a project following the school’s requirements, even if some of the
school’s requirements is that it should be good for the customer, but here in a way we
work to have the best possible grade, apart from working on a product that sort of leads
to further contracts; the customer is happy.”
We found a general opinion among the students that the role of the university is to
provide and test students’ use of “by the book” SE knowledge. There is a largely
implicit requirement that guidelines and practices (e.g. of modeling, diagramming and
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
103

programming) taught in other courses be followed. Students contrast the SE practices
prescribed by the course staff to the customer’s requirements and ‘the real world’: “I do
not think companies [..] need so much. [..] as what we do [..] in school. Because as it
turned out, we were writing a lot of reports, and then our supervisor from
said I do not need all these pages, I just want concrete things”.
4.4. Potential boundary objects are not actual boundary objects in the projects
Project artifacts intended by staff to be boundary objects are not always perceived by
students as relevant to stakeholders’ goals. For instance, certain models and associated
diagrams are typically perceived as unnecessary or purely 'school knowledge'. Students
do however reflect on the difference between models supporting development work and
models for cross-community communication. “..[].. the models and the documentation
we make is mainly to give us an understanding; they are not so much for them [..]. It is not necessarily the same models, it is not necessarily done in the
same way.” Also, the course staff’s requirements are contrasted with the group’s
needs:.”[I] have on several occasions felt that ’this model, we do not really need it’, but
we have to spend time on it because the school requires it to be there.”
5. Implications for the pedagogical design of customer-driven SE projects
It is not problematic that customers have several goals for their project participation,
as long as students’ learning is not as a consequence being downgraded in the projects.
The involvement of course staff usually ensures a focus on the formal learning goals.
Customers’ various goals, in our view, underscore the potential of customer-driven SE
projects to serve a role beyond the confines of the SE study program.
It is however a challenge to have all stakeholder goals made explicit. Our findings
indicate that students’ interpretation of stakeholder goals is inadequate, making it
difficult for students to decide what project artifacts are important to what stakeholders,
and in what way. Customers and course staff also have an interest in knowing about the
goals and students’ understanding of them; this allows for elaboration on the meaning
of the goals and negotiation over their importance when priorities must be made.
One aspect which should be kept in mind as we stress whether students give
mandatory or recommended project artifacts a role as boundary objects in their work, is
their purpose: effective cross-community collaboration to reach a good project result. If
students find a better way of collaborating with their customer, e.g. over a different
type of boundary object, course staff should give due credits to the approach. In terms
of pedagogy, the project supervisor must be able to see when advice should be given to
adhere to standard artifacts in a prescribed way, and when students should be ‘let loose’
to follow their own approach. Underlying our viewpoint here is the assumption that
students’ learning benefits both from their active development of their own solutions
and from an experience of mastery in their project [16].
A related issue is the perceived discrepancy between real life and school knowledge
among the students in our study. This could be interpreted as a useful insight on part of
the students, but also as a lack of ability to see the applicability of particular tools and
techniques in a real project. To have students experience the usefulness of various
project artifacts, it may be pedagogically necessary to make mandatory the development
of certain artifacts. It is however essential to have students actively relate to the link
between artifacts and stakeholder goals, which implies having students provide their
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
104

own account of why each artifact is of importance to one or more of the stakeholders.
Theis account should be based on communication with the project stakeholders.
The set of connections between artifacts and goals is in itself a boundary object,
typically manifest in documents such as a team/customer collaboration agreement and a
Software Development Plan, and typically developing throughout the project.
5.2. Implications for the pedagogical design of customer-driven SE project courses
Building on the previous discussion, we suggest that the following guidelines be
followed in the design of customer-driven project courses:
Stakeholders should be encouraged to make their goals explicit to themselves and
each other. Students should be aided in eliciting customer requirements. Also, they
should be encouraged to discuss team members’ objectives for project participation as
well as the goals for the team as a whole. Developing a set of team-internal project
rules may be useful. Course staff needs to make sure that learning goals and evaluation
criteria are clearly defined and easily available to students and project customers.
Course staff should also check on students’ understanding of the goals. Customers need
guidance about their role and about conveying their objectives as clearly as possible to
the group: ‘learning to be a customer’ should occur as early as possible in the project.
For every project artifact serving a role as a boundary object in the project, its role
for the attainment of project goals as well as the rationale behind its current state should
be made explicit, opening up for scrutiny and negotiation among project stakeholders.
A representation of the relationship between artifacts and stakeholder goals should be
incorporated into the artifacts themselves. This can be achieved by imposing the use of
standards and templates explicitly referring to stakeholder goals. Also, there are
standard functionality shared workspace tools adequate for providing the necessary
persistence, content structuring, workspace awareness and access for the communities
involved. Such tools should be utilized in the projects.
5.3. Limitations to our study
Our case material is limited in scope and time, although we draw on materials from
two different learning institutions. In-depth interviews with all relevant stakeholders,
combined with a rich written material should lead to satisfactory internal validity of
findings. External validity is mainly determined by our research approach. An
intensive, qualitative approach tends to produce results strong on accuracy, but weaker
on generalization [17]. This indicates that our findings will be most relevant in contexts
similar to our cases, i.e. for customer driven projects in SE at the Bachelor level.
6. Conclusion
Our empirical findings indicate that students consciously balance stakeholder goals
in their projects, but that the understanding of these goals and their connection to
various project artifacts often is inadequate. We suggest that in the pedagogical design
of the courses, a stronger focus should be placed on the connections between
stakeholder goals and project artifacts. These connections should be made visible and
negotiable in connection with boundary objects: artifacts playing a role in the goal
attainment of more than one stakeholder Scaffolding for effective development of these
artifacts implies a focus on project process as well as on technological infrastructure.
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
105

In our project courses we currently see the successful use of various shared
workspace tools: wikis, forums, simple file hierarchies, and more comprehensive tools
(e.g. Sharepoint), used for within-team purposes and for sharing contents and
interacting with other stakeholders. We suggest that this infrastructure be utilized in
efforts to provide effective representations of, access to, boundary objects. In future and
ongoing work, we investigate more closely the potential of particular collaboration
tools to support the development of boundary objects in customer-driven SD projects.
7. Acknowledgements
[13]
We wish to thank Glenn Munkvold for useful discussion related to this paper.
8. References
[1] P. C. Blumenfeld, E. Soloway, R. W. Marx, J. S. Krajcik, M. Guzdial, and A. Palinscar, "Motivating
Project-Based Learning: Sustaining the Doing, Supporting the Learning," Educational Psychologist, vol.
26, pp. 369-398, 1991.
[2] L. Helle, P. Tynjälä, and E. Olkinuora, "Project-based learning in post-secondary education - theory,
practice and rubber sling shots," Higher Education, vol. 51, pp. 287-314, 2006.
[3] J. S. Brown, A. Collins, and P. Duguid, "Situated Cognition and the Culture of Learning," Educational
Researcher, vol. 18, pp. 32-42, 1989.
[4] P. J. A. van Vliet and L. R. Pietron, "Information Systems Development Education in the Real World -
A Project Methodology and Assessment," Journal of Information Systems Education, vol. 17, pp. 285-
293, 2006.
[5] B. Fitzgerald, "The use of systems development methodologies in practice: a field study," Information
Systems Journal, vol. 7, pp. 201-212, 1997.
[6] J.-H. Ahn and A. E. Skudlark, "Resolving conflict of interest in the process of an information system
implementation for advanced telecommunication services," Journal of Information Technology, vol. 12,
pp. 3-13, 1997.
[7] A. Pouloudi, "Aspects of the Stakeholder Concept and their Implications for Information Systems
Development," presented at 32nd Hawaii International Conference on Systems Sciences, 1999.
[8] M. Riel and L. Polin, "Online Learning Communities. Common Ground and Critical Differences in
Designing Technical Environments," in Designing for Virtual Communities in the Service of Learning,
S. A. K. Barab, Rob; Gray, James H., Ed. Cambridge: Cambridge University Press, 2004, pp. 16-50.
[9] E. Wenger, Communities of Practice. Learning, Meaning, and Identity: Cambridge University Press,
1998.
[10] U. Gal, Y. Youngjin, and R. Boland, "The dynamics of boundary objects, social infrastructures and
social identities," 2005.
[11] H. Karsten, K. Lyytinen, M. Hurskainen, and T. Koskelainen, "Crossing boundaries and conscripting
participation: representing and integrating knowledge in a paper machinery project," European Journal
of Information Systems, vol. 10, pp. 89-98, 2001.
[12] S. L. Star, "The Structure of Ill-Structured Solutions: Boundary Objects and Heterogeneous Distributed
Problem Solving," in Distributed Artificial Intelligence, vol. 3, M. Huhns and L. Gasser, Eds.: Morgan
Kaufmann Publishers, 1990, pp. 37-54.
H. In, B. Boehm, T. Rodgers, and M. Deutsch, "Applying WinWin to quality requirements: a case
study," presented at ICSE'01, Toronto, Ontario, Canada, 2001.
[14] T. Erickson and W. A. Kellogg, "Social Translucence: An Approach to Designing Systems that Support
Social Processes," ACM Transactions on Computer-Human Interaction, vol. 7, pp. 59-83, 2000.
[15] J. Lave and E. Wenger, Situated Learning. Legitimate peripheral participation. Cambridge: University
of Cambridge Press., 1991.
[16] A. Bandura, "Self-efficacy," in Encyclopedia of human behaviour, vol. 4, V. S. Ramachaudran, Ed. New
York, USA: Academic Press, 1994, pp. 71-81.
[17] A. Langley, "Strategies for theorizing from process data," Academy of Management Review, vol. 24, pp.
691-710, 1999.
20th Conference on Software Engineering Education & Training (CSEET'07)
0-7695-2893-7/07 $20.00 © 2007
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 08:55 from IEEE Xplore. Restrictions apply.
106

Research paper P2
Title:
Power Through Brokering: Open Source Community Participation in Software
Engineering Student Proejcts
Authors:
Birgit Krogstie
Published in:
Proceedings of ICSE 2008
Pages:
10
Complete reference:
Krogstie, B. (2008). Power through brokering. OSS participation in SE projects.
International Conference on Software Engineering (ICSE) 2008, Leipzig, Germany,
10-18 May. IEEE Computer Society.
107

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
108

Power Through Brokering: Open Source Community
Participation in Software Engineering Student Projects
Birgit R. Krogstie
Norwegian University of Science and Technology
Sem Sælands vei 5-7, 7034 Trondheim, Norway
birgitkr@idi.ntnu.no
ABSTRACT
Many software engineering projects use open source software tools
or components. The project team’s active participation in the open
source community may be necessary for the team to use the
technology. Based on an in-depth field study of industry software
engineering project students interacting with an open source
community, we find that participation in the community may affect
the team’s work and learning by strengthening the power of the
broker between the team and the community. We outline pitfalls and
benefits of having student teams acquire development-related
knowledge from open source communities. The findings are
relevant to the organization and supervision of software engineering
student projects interacting with open source communities.
Categories and Subject Descriptors
K.3.2. [Computing Milieux]: Computer and Information Science
Education
General Terms: Human Factors
Keywords: FLOSS, open source, software engineering,
software engineering education, computer science education,
communities of practice
1. INTRODUCTION
Student SE industry projects, also known as customer-driven
projects, are designed to provide SE students with realistic
experience from software development work. The projects require
that students relate to various stakeholders and engage in crosscommunity
interaction [1]. Desired learning outcomes span both
social and technical skills. The latter includes being flexible to
customers’ technology requirements, which may be outside the
team’s current experience and preference [2]. Through the
pedagogical organization of SE project courses, student teams are
provided with scaffolding for their learning process [3, 4], which
might include guidance on the use of particular technology.
Sometimes in the case of industry projects, neither course staff nor
the customer is familiar with technology required in the project. The
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
ICSE’08, May 10-18, 2008, Leipzig, Germany.
Copyright 2008 ACM 978-1-60558-079-1/08/05…$5.00.
students have to demonstrate independence in coping with such
situations.
Requirements to learn and apply specific technologies may force SE
teams to interact with user and/or developer communities through
different degrees of community participation. For instance,
involvement with open source software (OSS) communities may be
needed. In what follows, we will talk about open source software
communities for simplicity – tacitly assuming that similar
considerations apply to free, libre and open source software
(FLOSS) communities.
OSS communities are a relatively new arena for information
acquisition and knowledge building in SE student projects.
Supervisors might be in doubt about what is going on and what
advice to give in such cases, feeling that their possibility to impact
on the project process is diminished. On part of the students, having
to participate in a different community in accordance with the
practices of that community may result in increased knowledge as
well as self confidence. Involvement with an OSS community in the
pursuit of knowledge may thus benefit the SE student team.
However, with an increased numbers of stakeholders to relate to, the
complexity of project work [5] increases, and coordination becomes
more challenging. Also, the introduction of unfamiliar, third party
OSS components introduces difficulties in terms of providing a
system design early in the SD lifecycle [6]. This makes planning
and estimation harder. Generally, there is very limited experience in
student teams in respect of project management. Programmingrelated
tasks, as will be argued, tend to be given priority, and
programmers given strong influence on the project process. We
believe that project managers as well as supervisors of SE student
teams may benefit from insights on how OSS participation can
affect the projects, in order to be prepared to take adequate
measures.
With this as our point of departure, in this paper we explore why
and how the interaction between a group of students, involved in an
SE industry project, and an OSS developer community evolves over
time. We further address how such interaction might influence the
project process, particularly in respect of the power structures in the
team.
The paper is based on an in-depth exploratory study of the work and
learning of a student SE team during the spring term of 2007. We
demonstrate how the interaction of the team with an OSS developer
community can be paramount to the success of a SE project. In our
case, we found that the OSS community readily responded to the
students’ requests for assistance on the use of the OSS, and that
issues of contribution from the team to the OSS community quickly
arose. Also, we found that the team member responsible for
791
109

communicating with the OSS community through his brokering role
maintained a position of power in the SE team.
Based on our findings, we provide an account of (i) mechanisms
enabling a SE team to successfully interact with an open source
community and (ii) possible pitfalls and benefits of the engagement
in terms of project success.
First, we provide a theoretical background for the issues focused in
our study. Next, we present our case, research method and findings.
We further discuss the findings in the context of the above outlined
research objectives, before concluding the paper.
2. THEORY
This section starts by providing a theoretical grounding for
understanding power structures in SE teams, viewing teams as
communities of practice. We then address how collaboration across
communities can be achieved, before briefly outlining relevant
research on how participation in OSS communities is typically
structured and how OSS projects may serve as a resource in SE
education.
A student SE team can be viewed as a task-based learning
community [7], a form of community of practice [8] oriented
towards learning and the time-limited effort of solving a particular
task. The building of knowledge in the team is likely to be
combined with the application, through trial-and-failure, of various
solutions to the development task in question. Learning within the
team may be based on expertise within the team, as seen in peer
programming [9]. A student SE team may learn through interaction
with course staff, e.g. a supervisor, and with the external customer,
if the project is an industry project. In addition, there will typically
be a need to acquire information from other sources, e.g. textbooks
or resources on the Internet.
Power structures in student SE teams can be seen as having several
dimensions, for instance between the customer’s and the
university’s goals, between experienced and inexperienced
programmers, between various roles within the team, as well as
between the team’s needs and individuals’ needs. A team typically
regards it as the most important goal of their project to deliver
working software in line with the customer’s requirements. Even
when the project report weighs heavily in the formal assessment and
grading of the project, documentation is typically conceived as
something which is required by the university, but which is not too
important for the customer – which might be a realistic observation
[1] – and is not the real challenge of the project. With students’
orientation towards the ‘real’ development task follows that
programming is regarded as the core of the practice of the
community constituted by the SE team. Accordingly, the skilled and
talented programmers get authority as core members of the team.
The ongoing programming tasks tend to be both a major constraint
and a driving force in the projects. As a result, whatever
programmers decide to spend time on tends to be regarded as
adequate use of time. Other team members, including the project
manager, might not be able to judge whether it actually is. Team
members good at activities regarded as secondary to programming
will be closer to the periphery of the team community [10]. From
the author’s professional experience as SE project course staff over
several years, it is common in the teams that the role of project
manager is seen as important, but less central to the project than the
programmer role, mainly because good programmers are a scarce
resource.
Sometimes, due to convenience or availability, information needed
for the team’s work is collected from outside the communities of the
team, the university, and the customer organization. The
information may be found in a more or less pedagogically organized
form, and its gathering may be of a more or less interactive nature.
Useful information sources include textbooks and tutorials,
handbooks, FAQs, case descriptions / examples, and templates.
Often, such resources are freely available on the Internet and can be
found by the aid of a search engine. More interactive sources of
information include communication channels belonging to a virtual
community, e.g. associated with the use and/or development of a
particular technology. Email lists, forums, wikis and blogs are
frequently used to support virtual communities [11]. Also,
synchronous communication may be supported through instant
messaging and chat rooms. Gutwin et al. found that developers in
OSS communities, who generally are distributed in their work,
maintain awareness of each other primarily through text-based
communication (mailing lists and chat systems) [12].
In order to cross the border to a (virtual) community, a certain
understanding of the rules of interaction and the community as a
social system is required [13, 14]. Lave and Wenger described how
the mechanism of legitimate peripheral participation (LPP) allows
newcomers to gradually become members of a community of
practice [10]. Not only is LPP seen a way of entering the
community; it is also seen as the basic way of learning the practices
of the community. Wenger explained how two main mechanisms
support the interaction between two communities: brokering and
boundary objects [8]. A broker is an individual who is a member of
both communities and is able to contribute to one community based
on the understanding of the practices of the other community.
Boundary objects, originally described by [15], are artifacts, abstract
or concrete, that retain an identity across the communities they
connect and play a role in the meaning-making of each community.
A boundary object thus works as a means for two communities to
coordinate their practices, even if the communities understand the
boundary object in different ways and use it differently.
Often, it will not be necessary for members of a SE team to become
full-fledged members of a community to get necessary information
from that community. In respect of communities related to a
particular technology, there might be an opening for outsiders to
participate as users, but not as developers.
In the case of OSS development (OSSD), there is typically an
openness that allows for participation in development for those who
are able and willing to contribute. OSSD takes on many different
shapes, but some general characteristics pertain to many developer
communities [16]. The participation structure tends to be layered,
with a core of active developers, intermediate layers of codevelopers,
active users and, constituting the outer layer, passive
users [17, 18]. Roughly, the model postulates that active users report
defects, co-developers repair defects, and new and updated
functionality is provided by core developers. Passive users may
contribute by answering questions from other users. Contributions
to the development of the OSS code may be seen as based a form of
gift economy where the giver achieves status from giving away. At
the same time, these mechanisms assure a certain quality of the
code [13]. The threshold for being admitted as a developer in an
OSS community may vary between projects, large OSS projects
based on the sponsorship of major corporations being far more
restrictive than small OSS projects in terms of accepting
development contributions.
792
110

Due to the openness of OSS communities, students may use them as
an environment for learning which is interactive, ‘real’ and outside
the university. OSS development has been acknowledged both as a
means and as an objective of learning in SE education. Whereas the
number of studies reporting lessons learned from students’
participation in OSS development is still limited, experience and
lessons learned from student projects have been reported (e.g. [19,
20]). Toth [20] lists the following benefits of using and extending
OSS tools in student SE projects: a baseline of such tools
automatically gives a ‘critical mass’ to the projects, the tools can
freely be extended, using OSS is engaging for students, students
may evolve their own tools (‘eating their own dog food’), and
students increase their value in the job market. Spinellis [21] points
to the following benefits of working with OSS in general: increased
industry contacts, professional maturation, improved skills in
written communication, experience with system administration, and
exposure to management approaches. These are benefits equally
relevant to SE and CS students as to SE professionals. Jaccheri and
Østerlie [22] demonstrated that students may learn from an action
research approach to participation in OSS development, resulting
both in familiarity with action research and improved capabilities in
programming and design. Ellis et al. [19] mention possible
disadvantages of using OS projects as a basis for SE education: lack
of documentation, inconsistent coding standards, and inconsistent
quality.
Lessons learned and guidelines emerging from current studies on
student projects engaging in OSS communities generally address the
pedagogical organization of SE project courses in which students’
primary development task is itself OSS development. There is
however a lack of research on (i) ways in which the OSS
community might be an important source of knowledge relevant to
the students’ development task without being its main arena, and (ii)
pedagogical issues arising from such a setting. This is where this
paper aims to contribute.
3. OUR CASE
The particular SE project investigated in this work is part of an
undergraduate level course at a Norwegian technical university. The
students take the course in the final semester of their Bachelor
program, i.e. their 6 th semester. The students work in mostly selfformed
teams of 3-5 students, developing software for an external
customer. Students make a prioritized wish list from a number of
available project descriptions mainly provided by industry, and
teams are assigned tasks by course staff according to students’
preferences and skills. The workload of the course is half of the
semester, but students tend to spend more time on the project, giving
it higher priority than parallel courses. In addition to the software
product, the students must hand in a project report (both a
preliminary and a final version) to the customer and course staff.
Also, the teams give oral presentations of their projects halfway
through the semester and at the end of the semester. There are no
lectures in the course except an introductory lecture in which
available projects are presented. Each team receives supervision on
their project process from a member of course staff, usually a
teaching assistant. The customer is responsible for providing
supervision on technical issues when necessary, as well as providing
resources otherwise unavailable to the students (e.g. particular
hardware, software, and physical or virtual workspace). Based on
the course staff’s evaluation and customer feedback through an
evaluation form, each team is given one grade (except in rare cases
where major variation in individual work effort is documented by
the team). The grade is based on a combination of the software
product, the documentation and the project process. The teams are
provided with some templates and guidelines for their work, e.g.
related to project planning, status reporting and contents of the
project report.
There are no general requirements that any particular process
model, analysis/design methodology, development tool or
collaboration technology be used in the projects. Students are
expected to relate to the customer’s requirements and draw on their
skills and resources from previous and parallel courses at the
university. The teams use tools available at the university, provided
by the customer, or otherwise available. When needed and feasible
within the time frame of the project, the teams are expected to get
into new technology as part of their project work.
The student team followed in the case study presented in this paper
consisted of five male students: Ethan, George, Sam, Owen, and
Morgan (names have been altered), all in their twenties. Their
development task was to make an auctioning system for an IT
consultancy company, here called Anniva. The company wanted to
use the system for in-house purposes and as a substitute for an
existing system used by employees to sell items (such as PCs and
bicycles) to colleagues on a private basis. As a secondary objective
for the project, the company wanted the team to make use of a
particular open source Java development framework, here denoted
PLENTI, in making the auctioning system, to see how the
framework could be utilized. The team succeeded in developing the
auctioning system, receiving the grade B on their project. (B is
regarded as a good grade; of the nine project teams in the 2007
semester of the course, there were 2 As, 4 Bs and 3 Cs.) Crucial to
the team’s development work, as will be seen, was their interaction
with the PLENTI developer community.
4. RESEARCH METHOD
The student team of our study was observed during the spring
semester 2007, i.e. over a period of four months. Given the real-life
context, focus on a contemporary phenomenon, lack of control over
events and our intent to explore “how” and “why” aspects of the
setting, a case study was appropriate [23]. Part-time participant
observation was conducted with adherence to principles of
interpretive field research [24]. For the research project at large, data
(mainly documents and interviews) was collected across all teams in
the course. Data for the study reported in this paper originate from
an in-depth study of one particular project team and includes
recordings, field notes and pictures from 15 hours of meetings
between the team and customer/supervisor and 60 hours of teaminternal
work sessions and meetings (mainly in the computer lab),
copies of pages from the team’s wiki, some logged msn
conversations, various project documents from the team’s
workspace, the preliminary and final version of the project report,
all email correspondence going through the team’s email list,
recorded interviews with the team, the supervisor and the customer,
and the threads in the PLENTI user forum (listserv) resulting from
the team’s requests to the forum.
Generally, the researcher was given free access to work sessions and
meetings (participation and recording) as well as to the team’s
server workspace, wiki and email list. The researcher attended all
meetings with the customer, with the exception of a couple of
meetings which were audio recorded for the researcher by a team
793
111

member. Field notes were made during and immediately after the
sessions, to the extent possible. The team’s supervisor was
interviewed during and after the project, and the team’s customer
was interviewed after the project. The team was interviewed once
during the middle of the project and once after completion of the
project.
Most of the meeting recordings and interviews have been fully
transcribed. Recordings from work sessions have been partly
examined and summarized based on their perceived relevance in the
analysis process. Quotations have been translated into English by
the researcher at need. Analysis of the data started out with some
theoretical concepts being perceived as central (cf. the Principle of
Abstraction and Generalization, [24]). Coding was performed on the
basis of these concepts, by the aid of a computerized qualitative
analysis tool (Atlas.ti). New codes were added as themes developed
during interpretation and writing. For instance, PLENTI was used as
a code to mark and organize data related to the team’s use of the
framework and interaction with the community. Further, the data
have been chronologically structured within selected themes.
The researcher was the coordinator of the project course, grading
the projects, but not supervising any group. In the process of
observing the Anniva team, great care was taken to avoid mixing up
the roles of researcher and course staff and to be aware of the
possible effects of ‘the teacher being present’ (cf. the Principle of
Interaction Between the Researcher and the Subjects, [24]). As part
of the initial agreement between the researcher and the team, it was
decided that another member of course staff set the grade on their
project. Further, it was agreed that the researcher was not to
supervise them, but restrain interaction to being social without
causing too much disturbance. The reason for not supervising was
not mainly that supervision would influence the case. The influence
of the researcher is an inevitable consequence of participant
observation, even if the degree of influence can, and should, be
limited. The team was not, however, to be given advantages as
compared to the other teams in the course, or at least the sum of
advantages and disadvantages of being research subjects should be
perceived as close to zero by the team and by the other teams. These
considerations impacted on the possibility to have the researcher’s
interpretation validated with the research subjects during the study
(cf. the Principle of Multiple Interpretations, [24]). As all teams in
the course were however interviewed about their project work and
viewpoints on the course in the middle of the semester, and on that
occasion it was possible to have some prompted viewpoints from
the Anniva team. Also, the day after the completion of the project, a
three hour interview was conducted with the team to have their
feedback on the researcher’s preliminary interpretation of the
project.
In reflective discussions within the team towards the end of the
project and in the interview conducted after the project, the team
expressed that they had perceived the researcher as non-interfering
and agreeable, and that they had not received project supervision
from her.
To validate findings reported in this paper, a draft version has been
reviewed by two of the team members, Owen and Ethan. Ethan
expressed that he liked the paper. Owen provided more detailed
viewpoints which have been incorporated in the Analysis and
Findings and Discussion sections.
5. ANALYSIS AND FINDINGS
In presenting the findings of our study in terms of the team’s
interaction with the PLENTI community, we have made a
chronological structuring of the material, distinguishing what we see
as four partly overlapping phases.
The first phase is a (dis)orientation phase in which the team strives
to understand what PLENTI is and how to obtain adequate
knowledge about the framework. In the second phase, PLENTI is
regarded by the team as an obstacle, but starts taking on the role of a
tool (albeit not quite mastered by the team). In the third phase, the
team is communicating with PLENTI users and developers through
the listserv of the community, actively making use of the framework
as a tool in the project development task. In the fourth phase signs
of a feeling of identity related to the PLENTI community
participation can be found in the team, and the team is contributing
to the PLENTI development.
In the subsequent discussion (Section 6), we will argue that the
transition to the third phase was essential to the project, and address
how this transition came about in our case. In the present section,
we characterize each phase in more detail, giving some illustrative
excerpts from the data where appropriate.
5.1.1 First phase (January-February): What is
PLENTI?
In the original project description as well as in the initial customer
meeting, it was specified by the customer that PLENTI should be
used as a development framework. The reason was that one of the
two customer representatives had attended a seminar with an
enthusiastic PLENTI developer and thus caught interest in the
framework. During the first customer meeting, the team was offered
the alternative of using another framework, but not PHP, which was
the only alternative with which they were familiar. This left
PLENTI as the de facto option. Adding to the arguments to use
PLENTI, the customer expressed interest in having it tried out to see
how it worked for this type of application development.
None of the students in the team had any advance knowledge about
PLENTI. Further, no one had any experience with the programming
of web applications with servlets. The web pages of the PLENTI
community were the only resource about the framework known to
the team.
The team decided that every team member was to learn PLENTI
and participate in the programming tasks. Two team members, Sam
and Owen, were assigned the task of starting the knowledge
acquisition about PLENTI. Ethan and George, accepted as the lead
programmers in the team, took on the task of developing a prototype
to demonstrate to the customer as a basis for refining the functional
requirements. Ethan worked on the web user interface whereas
George focused on the underlying business logic, none of them
making use of PLENTI for their tasks. Both programmers
participated in team-internal discussions on the role and use of
PLENTI, though. The fifth team member, Morgan, took the role as
project manager, which made him mainly responsible for overall
project planning, generating and handing in bi-weekly status reports
and activity plans to the supervisor, scheduling meetings, and
making sure that work on the project report progressed in
accordance with the deadlines.
794
112

5.1.2 Second phase (February-May): PLENTI as an
obstacle gradually becoming a tool
It soon became evident to the team that getting into the workings of
the framework was not an easy task. Sam and Owen spent many
hours in the PC lab reading through descriptions and examples, Sam
doing some exploratory coding and Owen reflecting overview level
information on the framework in the Software Development Plan
which had to be handed in early in February.
The web documentation of the framework turned out to rely on the
reader’s prior knowledge of the development of web applications,
which the team lacked. The examples provided in the material were
not relevant enough for the team to be able to easily draw on them.
The team explained their challenges with PLENTI to the supervisor
and the customer in meetings with them.
Exhibit 1: Excerpt from February 16, meeting between the team
and the customer:
Ethan: ”We had a bit of trouble with, with.. that is, we have a small
problem with the PLENTI documentation. It is a bit.. They have a
lot of examples, but there is sort of nothing that is.. the most basic
stuff. They assume that you know quite a lot.” Owen:”The examples
and documentation found there are mainly presentations, mostly of
how PLENTI is better than earlier and similar tools. We have some
problems with, sort of.. the totally basic stuff, they aren’t explained
anywhere, so it becomes a lot of trying and failing.”
Ethan and George continued focusing on their preferred parts of the
prototyping work, and in the larger part of this phase, the team had
little progress with their understanding of PLENTI.
Gradually some knowledge of the workings of PLENTI developed,
however, as the programmers started using the framework in their
prototyping. The team was however aware that they did not yet
utilize the powerful mechanisms that would be the main reason for
using the framework. Attempts to do the latter were postponed,
awaiting the completion and demonstrating of the prototypes. Also,
work on design and architecture models of the auctioning system
was postponed, again with reference to the lack of understanding of
how PLENTI should be used. In the eyes of Ethan, it would be more
or less meaningless to draw a design model: “It would just be a big
box named PLENTI with a number of arrows coming out of it”.
In the midterm version of the project report, handed in on March 7,
as well as in the related oral presentation, PLENTI was still
presented by the team as something they needed to learn properly.
The auctioning system prototype demonstrated was mostly a mockup
and hardly made use of any of the functionality in the
framework.
In the customer meeting on March 28, it was clear to the customer
as well as to the team that the progress of the project was not
satisfactory.
Exhibit 2: Excerpt from March 28, meeting between the team
and the customer:
Ethan: ”Ok, we have made several prototypes, so that.. we have
learnt PLENTI by programming. But we haven’t made any
completely robust solutions yet.” Owen: ”And then we have used a
lot of time to learn, study PLENTI, then, because the documentation
is quite bad, I think.”
5.1.3 Third phase (April-May) Gradual PLENTI
community participation
In the customer meeting on March 28. Ethan reports “having been a
bit on the IRC” communicating with the PLENTI community,
reporting that there was “this guy” that he communicated with.
Exhibit 3: Excerpt from March 28, meeting between the team
and the customer. Ethan talks, quoting his IRC communication
with Bernhard:
’Why do you do it that way?’ And we just: ‘What? That’s how we do it in JSP.’ ’Don’t do it that way!’
No good, no good, so…”
Expressing the above, explaining how Bernhard had reacted
negatively to their approach, Ethan still appeared satisfied with the
IRQ dialogue. He had in fact reached a PLENTI developer with his
request, and the dialogue resulted in relevant directions for the team.
After the Easter holiday, following up on the success with IRC,
Ethan turned to the PLENTI users forum, a listserv service linked to
the PLENTI web pages. The subsequent interaction between Ethan
and the community in this forum amounted to eight threads, most of
which with several postings, during the course of the project.
A condensed thread from the forum is shown in Exhibit 4. Other
requests from Ethan were addressed by Bernhard partly in parallel,
in different threads.
Exhibit 4: Excerpt from April 13.-18., PLENTI users forum on
the Internet:
April 13.: Ethan: ”We’re trying to make a credentials manager for
authenticating against LDAP. We’ve written some code but we’re
unsure about what to do next, or if the code is correct. Any ideas? ..
Code: ”
April 14.: BG “Hi Ethan, using the upcoming PLENTI 1.6, it’s very
easy to plug in your own credentials manager ”
April 16. Ethan: “Hi, Bernhard, In addition to , what files do
we need? ”
April 18.: BG: “Just should suffice ”
April 18.: Ethan: “Thank you Bernhard! That worked perfectly. We
have now functioning LDAP authentication in our webapp. Has
integrating LDAP authentication directly in PLENTI been
considered? Sincerely, Ethan Lake”
April 18.: BG: “Wonderful! Several people have asked for it in the
past, it would be a very welcome contribution to the project.”
Whereas the communication shown in Exhibit 4 took place in the
PLENTI users forum (as opposed to the developer forum, which
also could be found on the web site), changes to PLENTI itself were
discussed in the last two postings. The initial posting was the first
request from Ethan on the forum. Bernhard, the chief developer of
PLENTI, was, with one exception, the one who answered Ethan’s
(and a number of other users’) requests on the listserv. He gave
quick response, often on the same day (or night). These observations
together may be taken as an indication that the OSS community is a
very small one, with small distance between user and developer as
well as between user issues and development issues.
At one point, Ethan was asked by Bernhard to send large pieces of
code by private email instead of pasting the code directly into the
forum. Ethan sent code by email as requested. On a couple of
795
113

occasions, Bernhard asked for more information or clarification to
understand Ethan’s questions, on which Ethan immediately
followed up. Generally, communication between the two appeared
effective, to-the-point, polite and friendly.
The interaction with the PLENTI community was conveyed to the
team through the team’s mailing list. Ethan made sure that his
requests and Bernhard’s answers were in this way shared with the
rest of the team. The new knowledge was used by the programmers
- Ethan, George and Sam - to develop and modify their code. In
discussions within the student team, new issues were identified that
required more consultation with Bernhard. Ethan took on the job of
formulating and submitting postings, which he sometimes did in the
PC lab and sometimes in his home at night. Ethan’s role in this was
never questioned by the team.
In terms of the division of project work, Owen produced use cases
and other parts of the documentation, coordinating with the others
as needed. Morgan continued performing project management,
more by following up on what the others were in fact doing than by
pushing the process. Sam largely assisted with programming tasks.
Ethan and George were in effect driving the process, everyone
realizing that their success with the coding was essential to the team.
Morgan and Owen took on the main responsibility of making the
final report, leaving the design and programming-related contents to
the programmers. Until the last weeks of the project, and with the
exception of a data model which was created early in the project and
had been heavily modified after the customer meeting on March 28.
the creation of models and diagrams to be used in the project report
was largely ignored by Ethan and George, who were mostly
concerned about coding and debugging. Ethan and George were
regarded as the ones having control of the totality of the system
under development. George was highly estimated for his
programming skills. The rest of the team, in his absence, referred to
how they needed him to get going on particular tasks for the project
to succeed.
Ethan made the architecture model asked for on several occasions
by customer and supervisor. Suggestions to Ethan from Owen and
Sam in the direction of a layered design model were turned down by
Ethan, who continued to regard PLENTI as somehow noncompliant
with a layered architecture and came up with a model
following his own notation.
The following comment to a draft version of this paper was made by
Owen: “In the absence of a thorough elaboration phase in which one
builds a well defined software architecture and decide on the design
of the system, a lot of the development and design of the product
was up to those actually writing the larger part of the code”.
5.1.4 Fourth phase (April-May): Contribution to
PLENTI
The point at which Ethan started discussing with Bernhard about
possible contributions to PLENTI might be seen as the start of the
phase in which the team were seen as, or saw themselves as,
contributors to the PLENTI community.
Three main events constituted the contribution part of the team’s
interaction with PLENTI. First, there was a nightly build of
PLENTI in which the changes directly resulted from requests from
the Anniva team and a bug subsequently discovered by Bernhard.
Second, the team wanted to integrate the system with a MySQL
database, for which there was no support in the framework. In this
case, Bernhard indicated how long it would take to implement the
functionality (i.e., if they wanted to do it); at least a week’s work.
The team took this to mean a lot more work for them as
inexperienced programmers, and the idea was abandoned. Third,
there was the LDAP functionality referred to in Exhibit 4, suggested
by Ethan and applauded by Bernhard. Even after the completion of
the project, the team continued discussing the possibility of
contributing such a module. That development job was however
never carried out.
The team’s account as given in the common reflection note shows
how PLENTI is given a very central role in the team’s
understanding of their project process. We note that the framework
still plays a role as an excuse and explanation for insufficient project
management.
Exhibit 5: Excerpt from the common reflection note in the final
project report
What did not work so well in the project when it comes to project
management? It has been problematic to plan the use of time in a
realistic way. Our lack of experience with PLENTI and web
application development resulted in frequent ad-hoc estimates. This
was particularly difficult in the beginning of the project. During the
development process we naturally gained more insight into PLENTI
and web application development, which made it easier to make
realistic estimates.
In respect of achieving good project management: What would
you do differently if you later on participate in a similar project?
If we had worked more intensively in the beginning of the project
with our understanding of PLENTI, and the more advanced
functions that we ended up using, project management and
scheduling would probably have been a bit easier.
The formerly mentioned period of inactivity was redeemed by a
good customer meeting, and this might have been held earlier. Also
we could have used the PLENTI email list earlier, as we got good
user support from the PLENTI developers there.
In the final oral presentation on May 25, the team spent much time
describing the role of PLENTI in their project: how PLENTI had
caused trouble, how the solution was based on PLENTI, and how
interaction with the PLENTI developers had been essential. A
certain pride with the process was observable in the team.
In the individual reflection note, written by each team member as a
supplement to the common reflection note, Ethan does not make a
point of his role in respect of PLENTI when explaining his role and
tasks in the project. He mentions his main responsibility for coding
related to the user interface (CSS and HTML), for making diagrams,
and for reviewing and addressing language issues in the report. In
the note, responding to the question about whether he would have
liked to have a different role in a similar project on a later occasion,
he states that: “I am very happy about what I have been working
with in the project”.
6. DISCUSSION
From the chronological presentation of findings, we note that the
transition from the second to the third phase of interaction with the
PLENTI community was essential to the project. It was in the third
phase they got access to the knowledge they needed to be able to
utilize the framework. The fourth phase, contributing to the
development of PLENTI, was not strictly necessary for the project
team to succeed with their development task.
796
114

Getting from the second to the third phase can be seen mostly as a
result of the team’s realization that progress was too slow and that
something must be done. The means for the transition to take place,
was the interaction with the OSS community, the only place where
the necessary knowledge could be found. In what follows, we will
focus on an important mechanism enabling this interaction: the
brokering performed by one team member.
In our case, Ethan took on the role of broker between the team and
the OSS community. In the community, he presented the team’s
challenges in a way which was meaningful to the OSS developers.
Back in the SE student team, Ethan explained what Bernhard’s
feedback meant to their work and translated it into Java code in
collaboration with the other programmers.
In the brokering process, there were artifacts that can be seen as
boundary objects [15] enabling meaning-making across
communities. The auctioning system, from the outset unknown to
Bernhard, gradually became more useful in mediating interaction as
specific modules were discussed and changed. Pieces of code were
described but also submitted in ‘raw’ format, explored, and perhaps
modified and returned. Further, the PLENTI framework itself
mediated the interaction: From the start, PLENTI was well known
to Bernhard but not sufficiently mastered by Ethan and the team. As
the interaction proceeded, the team’s knowledge improved to the
point that it became meaningful to discuss changes to PLENTI
itself. The roles of the boundary objects in each of the communities
were of course different: For the SE team, PLENTI was a means to
develop a working auctioning system. To Bernhard, having
someone doing development by the use of PLENTI was a means to
improve PLENTI itself.
We leave the elaboration on boundary objects at this point, noting
that they should be seen as integral to the brokering and enriching
our understanding of the process. Having argued that Ethan was in
fact a broker, we turn the discussion to how the brokering empowers
the broker, with possible effects on the SE project.
6.1 The power of the broker
We suggest that the broker between a SE team and an OSS
community gains increased power in the team in three ways: 1)
through his position as gatekeeper of knowledge needed by the
team, 2) through strengthening his existing authority as a
programmer, and 3) through increased credibility in both
communities through the feedback provided in the OSS community
In what follows, we pursue these arguments referring to our case.
6.1.1 The power of the gatekeeper
As the broker translates knowledge from one community to the
other, he simultaneously controls which information flows between
the communities. It is the broker’s interpretation which will reach
each community, even if he faithfully aims to represent the
community’s interests and not his own vested ones. When the
receiving community depends on the information to succeed with
their core activities, controlling the information is a great source of
power and an opportunity to control the agenda. Some
characteristics of our case point to the gatekeeping role of the
broker.
First, it is only Ethan in the team that interacts with the PLENTI
community. Thus, no one else actually passes the only gate towards
the PLENTI community, namely the listserv, apart from reading the
resulting threads there and taking part in the interpretation of the
new knowledge in the context of the team’s work. Nothing would
have prevented the other team members from communicating with
Bernhard if they wanted to. However, the division of labour letting
Ethan take on the role as broker and gatekeeper seems to be
generally agreed-upon in the team, perhaps because they
acknowledged his abilities to attend to the interests of the team.
Second, Ethan is fairly active in taking and keeping the role as
broker, thus in a sense defending it as his domain. For instance, in
meetings, when unresolved, PLENTI-related issues arise, he is
quick at suggesting that he pose a request to the community.
Third, Ethan’s communication with the PLENTI community is done
through postings partly sent outside the hours of collocated work in
the SE team. The questions asked and the interpretations made in
the OSS community are therefore withheld from, or at least
distanced from the collaborative activity of the team. No one in the
team ever expressed any dissatisfaction over this. This might point
to the team’s general appreciation of members’ flexibility of work
(time and place), but may also indicate satisfaction with the way
Ethan reflected their discussions and represented their interests in
the OSS community.
6.1.2 The programmer increasing his power
The role as broker towards a developer community must be held by
an individual having enough knowledge about development to be
able to effectively communicate over issues important to the
community. In practice, this means a programmer. As argued in
section 2, programming skills give authority in the SE teams. The
programmer brokering towards a developer community gets more
programming skills and thus more power.
The following points from our case serve to illustrate how being a
programmer is highly central to the broker:
First, in the OSS community, the language of interaction is largely
source code as well as considerations on aspects of source code, on
the level of single lines of code and on the level of modules and
versions of the entire system. Ethan makes his requests
understandable to Bernhard by giving excerpts from their source
code, and receives answers directly addressing aspects of the code
or even containing modifications of the code itself. In addition to
being able to ‘speak code’, there is a need to follow the implicit
rules of communication in the forum. It is an open question whether
programmers generally have a better starting point for
communicating in virtual developer forums, but it is not unlikely
that many programmers are experienced in gathering information
from this type of site. In all his contributions to the PLENTI users’
forum, Ethan demonstrates a high degree of literacy in terms of this
communication channel. His contributions appear friendly, polite,
concise and goal-directed, and always appropriate to the context.
Second, being a broker, you need to represent your community.
Representing a SE team in the context of a developer community
means being accountable for the software produced by the team. It
is difficult to see how team members other than Ethan and George,
who had the hands-on experience with the current version of the
code and its unresolved bugs as well as the overview of the current
version of the system, could have communicated with PLENTI
developers over development issues. In face-to-face meetings with
project stakeholders, e.g. the customer, Ethan frequently took on the
task of articulating the status of work and the current issues and
priorities of the team. This worked only partially, as Ethan’s role
and conduct was not quite accepted by the customer. In the OSS
797
115

community, however, Ethan always convincingly represented the
team and their work.
Third, illustrating the previous point, no one in the team but Ethan
actually took any initiative towards the PLENTI community. It was
not until the lead programmers decided that something must be done
about the lack of PLENTI skills and usage in the project, that
effective interaction happened. In our case, it took a programmer not
only to undertake the role as broker, but also to understand that
someone needed to take that role.
6.1.3 The credibility gained through OSS participation
Being acknowledged as a participant in a developer community is
likely to add to the self consciousness of the participant. When a
contribution is met by a response in a forum, the community
membership of the contributor is acknowledged. When a
contribution is openly praised in the forum, the skills of the
participant are implicitly valued, and his credibility in the forum is
maintained or strengthened. If the participant is relatively
inexperienced and do not already participate actively on several
related arenas, the newly earned credibility is likely to be a source
of pride and increased self confidence. Also, the credibility won can
be demonstrated to others outside the community, adding to
credibility and strengthening authority there as well. From our case,
we draw some illustrative points:
In Exhibit 4, the last response from Bernhard to Ethan includes the
following: “Wonderful! Several people have asked for it in the past,
it would be a very welcome contribution to the project.” The
contribution of Ethan and the team is thus publicly acknowledged as
something relevant and welcome. The positive response was
referred to by Ethan on later occasions, e.g. in customer meetings.
In team-internal conversation, Ethan on some occasions talked of
Bernhard almost as if he was a buddy, using his first name. Ethan
also pointed out to the team how quick Bernard was at providing
them with response, and often outside of normal work hours.
Ethan’s pride in the engagement of the team with the PLENTI
developer community was very visible whenever the project was
accounted for in meetings with supervisor or customer in the last
part of the project. On one hand, referring to ongoing successful
interaction with the PLENTI community served as a way of
ensuring project stakeholders that the project was in fact
progressing. On the other hand, it may be seen as a way of assuring
the stakeholders that the team were competent, doing development
work of a standard acceptable to the framework developer
community.
Finally, whereas the fourth phase of OSS interaction, contributing to
the framework, might not have been strictly necessary for the team’s
development task, it might have benefited the development task
indirectly: Ethan’s positive attitude towards contributing to the
PLENTI development might have inspired Bernhard to give better,
faster or more elaborate response.
6.2 OSS community participation aided by a
broker: Benefits and pitfalls for SE projects
Turning focus to what we see as benefits and pitfalls of having
student SE teams acquire knowledge from OSS communities, we
give a final account of the impact of OSS community involvement
on the Anniva team, looking at their project more broadly.
The team succeeded with their development task, producing a
working system by active use of advice received from the OSS
community. On the other hand, the project report suffered from a
lack of attention from the programmers. The poor quality of the
report was a main reason for the project receiving a B and not an A,
according to course staff in the grading meeting.
It is only a guess that the team in our case would have been no more
concerned about the project report if they did not have to interact
with an OSS community. The focus on programming (at the cost
e.g. of developing models and documentation) might be seen as a
consequence of a lack of experience and maturity in terms of project
management, or as a legal choice of prioritizing what was to the
team the essence of the project. In respect of the team postponing
the work with design and architecture modeling, it is worth noting
that the PLENTI framework can be seen as an example of a third
party software component, unfamiliar to the team and poorly
documented, which implies a realistic SE challenge of determining
when to attempt to develop an overall design and when to start using
the component [6].
Communicating in the OSS forum appeared to be a gratifying
experience for the broker. The team was vulnerable to his absence,
but that problem never occurred. Also, our observations indicate
that the broker shared his knowledge from the OSS forum with the
rest of the team in an open and effective way. It is difficult to say if
he spent too much of the project time in the forum, or if the team
spent too much time on programming tasks based on the results of
the interaction. The fact that the interaction took place only in the
last part of the project, indicates a limited use of project time. Also,
the team’s interaction with the OSS community having to do with
changes to PLENTI constituted only a small part of the interaction.
Finally, these discussions were closely related to pertinent
development tasks in the project.
The fact that Ethan did not mention in the individual reflection note
his role in the communication with the PLENTI community, might
indicate that he did not consider it as something requiring a
substantial part of his time or being very important to the project,
but there may be other reasons why he did not mention his
brokering role. As for the power of the broker, Owen (on reading a
draft of this paper) expresses uncertainty about how much power
Ethan gained through his gatekeeping role. Owen had not until now
reflected much on the importance of the interaction with the
PLENTI community. It may be the case that Owen and Ethan take
their successful OSS community interaction for granted,
underestimating the communication skills actually required.
Finally, balancing our previous focus on the significance of OSS
participation to the team, we should note that communicating with
the PLENTI community was one out of several activities important
to the project. The power structures in the team may be seen to have
mirrored the complementary skills and interests of the two lead
programmers, which ensured a certain balance in power between the
two. Nevertheless, given our findings, our interpretation is that the
team, and the broker in particular, were both challenged and
inspired by the fact that they were participants in, and contributors
to the PLENTI OSS community and considered it an essential part
of their project experience.
Leaving the story of the auctioning system project, we now turn to
what we see as more general benefits and pitfalls – in terms of a
good project result - for student SE teams in need of interacting with
OSS communities as part of their development work.
798
116

6.2.1 Benefits for SE student projects interacting with
OSS communities
In line with arguments from others’ research, we hold that OSS
community participation is a realistic aspect of modern SE work.
The management of such interaction should be regarded as highly
relevant industry SE experience. If the team interacts with an OSS
community, part of the experience is shared by the whole team.
The broker in particular may learn a lot about how to achieve
successful communication and acquire relevant knowledge in an
OSS community. A broker whose authority and importance in the
team is acknowledged by the team is likely to take pride in his
position as a knowledge provider and thus be strongly motivated to
contribute to the project result. If the broker further experiences that
he earns credibility on the OSS arena, being recognized as a proper
participant there, he might want to make his contribution more
substantial and visible, which may again benefit his team.
6.2.2 Pitfalls for SE student projects interacting with
OSS communities
Student SE teams may hesitate to embark on interaction over
technically challenging issues in a type of community unknown to
them, the students being in doubt about their own skills and
credibility. This may cause problematic project delays. Also,
attempts to interact with an OSS community may turn out to be nonsuccessful
if the team does not manage to convince the community
that they represent ‘real’ users and development issues and thus
qualify as potential contributors.
To a team inexperienced in project management, interaction in an
OSS community may pose great challenges because it involves an
extra external stakeholder and may be perceived as even more out of
control than other programming related activity.
OSS community participation can be used as an excuse to spend
time on the wrong tasks from a project management point of view.
Having an alternative arena for gratifying response and
acknowledged participation might lead a programmer to focus too
narrowly on certain programming tasks related to that arena.
Brokering may be performed in an inadequate way: A broker unable
or unwilling to share relevant knowledge with the team, or mainly
representing other interests than those of the team in the OSS
community, stands in the way of a good project result and
everyone’s learning. Further, there is a risk associated with having
only one broker – both in case of inadequate brokering and in the
case of sudden absence of the one broker.
6.3 Implications for the pedagogy of SE
project courses
An implication of the work presented in this paper is that course
staff responsible for SE project courses should pay particular heed
to SE student projects requiring interaction with OSS communities
for development-related knowledge acquisition. These projects hold
a potential for industry-relevant experience but at the same time
hold some challenges. Course staff should be aware of benefits and
pitfalls inherent to this type of project. Further, awareness of the
mechanism of brokering and the potentially influential role of the
broker(s) in the projects may aid a supervisor in determining what
advice to provide on project management and process issues.
Part of the charm of industry projects is that they are very diverse.
Having used our in-depth single case study to derive issues we
believe to be generally relevant to student projects interacting with
OSS communities as a resource in their development work, we
would like to stress that the particular challenges of OSS
participation in one project may be very different from those in
another. Considerations over project management and supervision
must always be made in the light of the particular characteristics of
the specific project. In this respect, the project supervisor will
always benefit from having experience with the approaches and
domains relevant to the project. In the type of project of our current
focus, some background within OSS development or research is
likely to be an advantage to the supervisor.
In a situation in which many of the projects in a course include OSS
community participation, the overall organization of the course may
be designed to reflect this particular focus. For instance, lectures
may be given on how OSS communities work. Each project’s
interaction with the OSS community may be more explicitly used as
a learning resource, e.g. by being examined and discussed within
and across teams. Further, the OSS community interaction of a team
should be considered as a learning result (or documented part of the
project process) and assessed and credited by course staff, e.g. in
the grading of the projects.
6.4 Limitations to the study
Drawing on a single case, our findings may be seen as closely
related to its particular characteristics. Still, we believe findings
from our study on the role of the broker have general relevance to
SE student projects interacting with OSS communities.
An important characteristic of the OSS community of our case is its
small size, which may be seen as a condition for the rapid entry of
the student team into OSS development contribution. Arguments
about the empowerment of the broker however apply even if OSS
community participation is restricted to the user role.
Finally, our research explored a student team. Our findings may
have some relevance to teams of SE professionals, but differences
between these categories of SE teams should be considered. These
include the level of competence in project management in the team,
and the formalized learning goals of a project course which are not
found in industry and which impact on how we wish to guide the SE
student teams through project supervision.
7. CONCLUSION
In this study, we set out to gain insights on mechanisms enabling a
SE student team to successfully interact with an open source
community and on possible pitfalls and benefits of the engagement
in terms of project success. We have done so by pointing to the role
of the broker and how this role implies increased authority and
influence in the team for several reasons and with several possible
consequences, positive and negative.
We hope that our contribution will inspire SE course staff and others
involved with SE student projects to build on our work with
reference to their own experience. Continued focus on work and
learning in SE student projects involving OSS community
participation should result in empirically based research
contributions for us all to share, with the aim of improving current
pedagogical practices.
Further work may look at the implications of the type and sizes of
the OSS communities for student teams’ involvement there. Also,
the relevance of OSS participation in student SE projects to similar
799
117

situations in real-life industry projects – and vice versa - are topics
worth pursuing.
8. ACKNOWLEDGMENTS
The author wishes to thank Glenn Munkvold, John Krogstie, Owen
and Ethan for feedback on draft versions of this paper. Monica
Divitini and Thomas Østerlie provided valuable advice.
9. REFERENCES
[1] B. Krogstie and B. Bygstad, "Cross-Community Collaboration
and Learning in Customer-Driven Software Engineering
Student Projects," presented at CSEE&T, Dublin, 2006.
[2] L. v. d. Duim, A. Jesper, and M. Sinnema, "Good practices for
Educational Software Engineering Projects," presented at
ICSE'07, Minneapolis, USA, 2007.
[3] B. Bygstad, B. Krogstie, and T.-M. Grønli, "Scaffolding
Project Based Learning with the Rational Unified Process.
Experience from 5 years of Student Projects in Software
Engineering," presented at NOKOBIT, Molde, 2006.
[4] D. Wood, J. Bruner, and G. Ross, "The role of tutoring in
problem solving " Journal of Child Psychology and Psychiatry,
vol. 17, pp. 89-100, 1976.
[5] P. H. Carstensen and K. Schmidt, "Computer Supported
Cooperative Work: New Challenges to Systems Design," in
Handbook of Human Factors/Ergonomics, K. Itoh, Ed. Tokyo:
Asakura Publishing, 2002 (1999).
[6] J. Li, R. Conradi, C. Bunse, M. Torchiano, O. P. N. Slyngstad,
and M. Morisio, "Development with Off-The-Shelf
Components: 10 Facts," IEEE Software, 2007.
[7] M. Riel and L. Polin, "Online Learning Communities.
Common Ground and Critical Differences in Designing
Technical Environments," in Designing for Virtual
Communities in the Service of Learning, S. A. K. Barab,
Rob; Gray, James H., Ed. Cambridge: Cambridge University
Press, 2004, pp. 16-50.
[8] E. Wenger, Communities of Practice. Learning, Meaning,
and Identity: Cambridge University Press, 1998.
[9] C. McDowell and L. Werner, "The Impact of Pair
Programming on Student Performance, Perception and
Persistence," presented at ICSE'03, Portland, Oregon, USA,
2003.
[10] J. Lave and E. Wenger, Situated Learning. Legitimate
peripheral participation. Cambridge: University of
Cambridge Press., 1991.
[11] M. Huysman, C. Steinfeld, C.-Y. Jang, K. David, M. Huis in
't Veld, J. Poot, and I. Mulder, "Virtual Teams and the
Appropriation of Communication Technology: Exploring the
Concept of Media Stickiness," Computer Supported
Cooperative Work, vol. 12, pp. 411-436, 2003.
[12] C. Gutwin, R. Penner, and K. Schneider, "Knowledge
sharing in software engineering: Group awareness in
distributed software development " presented at CSCW'04,
Chicago, Illinois, USA., 2004.
[13] M. Bergquist and J. Ljungberg, "The power of gifts:
organizing social relationships in open source communities,"
Information Systems Journal, vol. 11, pp. 305-320, 2001.
[14] E. Wenger, "Communities of practice and social learning
systems," Organization, vol. 7, pp. 225-246, 2000.
[15] S. L. Star and J. R. Griesemer, "Institutional Ecology,
'Translations' and Boundary Objects: Amateurs and
Professionals in Berkley's Museum of Vertebrate Zoology,
1907-39," Social Studies of Science, vol. 19, pp. 387-420,
1989.
[16] W. Sacchi, J. Feller, B. Fitzgerald, S. Hissam, and K.
Lakhani, "Understanding Free/Open Source Software
Development Processes," Software Process Improvement and
Practice, vol. 11, pp. 95-105, 2006.
[17] K. Crowston and J. Howison, "The social structure of free
and open source software development," First Monday, vol.
10, 2005.
[18] A. Mockus, R. T. Fielding, and J. D. Herbsleb, "Two Case
Studies of Open Source Software Development: Apache and
Mozilla," ACM Transactions on Software Engineering and
Methodology, vol. 11, pp. 309-346, 2002.
[19] H. J. C. Ellis, R. A. Morelli, T. R. deLanerolle, and G. W.
Hislop, "Holistic Software Engineering Education Based on a
Humanitarian Open Source Project," presented at CSEE&T,
Dublin, Ireland 2007.
[20] K. Toth, "Experiences with Open Source Software
Engineering Tools," IEEE Software, 2006.
[21] D. Spinellis, "Open Source andProfessional Advancement,"
IEEE Software, 2006.
[22] L. Jaccheri and T. Østerlie, "Open Source Software: A
Source of Possibilities for Software Engineering Education
and Empirical Software Engineering," presented at First
International Workshop on Emerging Trends in FLOSS
Research and Development (FLOSS'07), 2007.
[23] R. K. Yin, Case Study Research. Design and Methods. Third
Edition., vol. 5: SAGE Publications, 2003.
[24] H. K. Klein and M. M. Myers, "A Set of Principles for
Conducting and Evaluating Interpretive Field Studies in
Information Systems," MIS Quarterly, vol. 23, pp. 67-94,
1999.
800
118

Research paper P3
Title:
Do’s and Don’ts of Instant Messaging in Students Project Work
Authors:
Birgit Krogstie
Published in:
Proceedings of NOKOBIT 2009
Pages:
14
Complete reference:
Krogstie, B. R. (2009). Do's and dont's of instant messaging in students' project work.
NOKOBIT 2009, Trondheim, Norway, 23-25 Nov. Tapir.
119

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
120

DO‟S AND DON‟TS OF INSTANT MESSAGING IN STUDENTS‟
PROJECT WORK
Birgit Krogstie, IDI, NTNU, birgitkr@idi.ntnu.no
Abstract
Instant messaging is a type of lightweight collaboration technology heavily used among students in
higher education for social interaction and for school work. This paper sheds light on students’ use of
instant messaging to support collaboration in software engineering student projects. We draw on several
years’ of research on the use of collaboration technology in such projects and present illustrative
examples of how instant messaging is used within the teams and for communication with other
stakeholders, e.g. project supervisor and customer. We find that the use of instant messaging in the
projects is generally successful, but in some cases it is inadequate, which may severely harm the project
outcome. Based on our findings, we discuss implications for the organization and supervision of SE
student projects.
Keywords: instant messaging, synchronous chat, student projects, software engineering student projects.
1 INTRODUCTION
“But in the case I have a technical question, and want to ask somebody in the group, I
first look at MSN.” (Member of a student software engineering project team)
Observing software engineering (SE) students in a project meeting or in the computer lab, you might
notice chat windows frequently popping up on the laptop and terminal screens. Different actions are taken
by the students in response to the pop-ups. Some windows are ignored (at least for the present), some are
discreetly attended to with a brief textual exchange, and some (if the ongoing activity in the room allows
it) appear to call for the immediate attention of the neighbour or all the students in the room. Some
messages appear to be project or otherwise school related, others belong to the private sphere in which a
social network appears to be incessantly maintained mainly through this channel.
What is going on, is instant messaging. Instant messaging tools are easily available, lightweight
collaboration tools that provide the user with the opportunity to communicate with one or more contacts
through textual chat. This channel of communication is constantly open and present in the students‟ work
environment; mostly in the background and sometimes in the foreground. From the viewpoint of course
staff in a project course, it is highly interesting to know whether, and in what way, the use of instant
messaging is supporting, or perhaps getting in the way of, the intended learning.
SE student projects is a case of project-based learning (Blumenfeld, Soloway et al. 1991). This
pedagogical approach is based on student projects that are central to the curriculum, focused on questions
or problems that “drive” students to encounter the central concepts and principles of a discipline,
involving students in a constructive investigation, student-driven to a significant degree, and realistic, not
school-like (Thomas 2000). Through SE student projects, students are being exposed to challenges of SE
work while receiving necessary guidance and support (Bygstad, Krogstie et al. 2006; Bygstad, Krogstie et
al. 2009). Figure 1 shows a SE student team in various settings of everyday work.
121
13

Figure 1.
A SE student team at work
SE is complex design work (Carstensen and Schmidt 2002 (1999)), and project teams have to manage
project tasks under constraints such as deadlines, team members‟ competence and customers‟ changing
product requirements. While some of the project challenges are technical, many are mainly about
interpersonal issues. Collaboration with project stakeholders (e.g. customer, supervisor, technology
providers) is frequently challenging to the teams (Krogstie and Bygstad 2007; Krogstie 2008).
Stakeholder communication is an area in which industry points to a need for SE students to gain
competence through experience in student projects (McMillan 1999).
The work in SE projects is supported by collaboration technology. In the case of many SE industry
projects, and typically in student projects, lightweight collaboration tools are used. Lightweight
collaboration tools can be acquired and taken into use at low cost (e.g. money, time to learn) for
individuals and their organization. A lightweight collaboration tool typically provides a limited set of
features to support one aspect of collaborative work and may thus be relatively easily integrated into
existing work processes (rather than imposing a certain process on the user). Many of the lightweight
collaboration tools are associated with Web 2.0, e.g. wikis, discussion forums, and instant messaging.
In this paper, we shed light on the use of instant messaging tools in SE student teams. Our main questions
are: when IM is used to aid project work, for what purposes and in what way is IM used? And what are
the implications for the organization of SE project courses? The findings reported in the paper originate in
qualitative studies of three cohorts of SE project students, 2006, 2007 and 2008, in which we investigated
the students‟ use of collaboration technology in their project work.
The paper is organized as follows: In Section 2 we briefly describe the functionality of IM tools and
present related work on the use of instant messaging among students and in work life. In Section 3 we
outline our research approach. Findings on the use of IM in the SE student projects of our study are
presented in Section 4. In Section 5 we discuss the findings in light of implications for the the
organization and supervision of SE student projects. Section 6 concludes the paper.
2 BACKGROUND
We start this section by briefly outlining the main functionality of IM tools. There are many different IM
platforms. They include AOL‟s Instant Messenger and Microsoft‟s Windows Messenger (MSN). Key
attributes of IM tools are (Garrett and Danziger 2008):
Near-synchronous communication that can be initiated by either party
Presence awareness showing if other users are connected and/or available
Notifications of incoming communication, typically with pop-up windows
Many tools support the exchange of speech and video as well, but in what follows we will focus on what
can be considered the basic functionality: textual chat. This is the functionality used by the SE students in
our case.
When a user logs on to the IM service, a „buddy list‟ appears on the screen. The list shows the user‟s
contacts and updated information on their status: whether they are logged on to the service, and any
message they might have left indicating their availability and/or current activity. “Busy reading for exam”
could for instance be such a message. If the user chooses to initiate a chat with a contact in the buddy list,
122
14

a chat window is opened. As soon as the user has written something and clicked „Enter‟, a chat window
appears on the contact‟s screen. The chat may also include more than two participants.
Once the user has written one or more lines of text in the chat window and clicks „Enter‟, the text will be
sent to the other party/parties, who may choose to answer directly, in the same way. In this sense, instant
messaging can be seen as a synchronous communication medium. On the other hand, instant messaging is
„semi-asynchronous‟ in the sense that the message will remain in the recipient‟s open chat window and
may be answered at a later point, e.g. if the recipient is away for a while or if he wants to finish other
tasks first. Further, many instant messaging services support offline messaging, which means that a
message may be sent to a contact who is offline to be delivered when he logs on to the system. The user
may choose to save the conversation in a log. Instant messaging can thus be seen as a „semi-persistent‟
communication medium.
Research shows that IM tools have generally come to play an important role in the life of young people.
IM tools provide a means for developing and maintaining peer group membership and social identity
(Grinter and Palen 2002; Grinter and Eldridge 2003; Lewis and Fabos 2005). Young people “meet on
MSN”. They are also conscious about how IM supports both social and work-related interaction and what
features of IM are desirable for different purposes (Huang and Yen 2003). Young people‟s use of IM
tools, and the links to other members of the social network contained in the tools, reflect a „networked
individualism‟ and participation in a networked society (Castells 2000). A study addressing American
college students‟ perception of the differences between instant messaging and email (Lancaster, Yen et al.
2007) concludes that IM is seen by the college students to be the best option in the context of personal
and social relationships, IM being better for conveying emotions, building relationships and in ease of
use. From this research, we may infert that SE project students are likely to be skilled users of instant
messaging, utilizing the opportunities that have opened up by the popularity and extensive use of these
tools, and being aware of some of their limitations. However, we cannot assume that the students are
equally familiar with the use of IM in a context of work, for which other considerations might apply.
In the workplace, IM has become a mainstream informal collaboration tool (Cherry 2002; Stephens
2008) and has been found to support a number of different communication tasks as well as the process of
enabling information exchange, i.e. reaching out to others to initiate communication (Nardi, Whittaker et
al. 2000). Among the types of IM use is complex work discussions (Isaacs, Walendowski et al. 2002).
Research on open source development (OSD) has shown that a purely textual medium (e.g. IRC) which is
not integrated with the development tools may be sufficient for effective work-related communication
between developers (Gutwin, Penner et al. 2004). When our SE project students use instant messaging,
they are, in other words, getting experience with collaboration technology in industry use.
Whereas IM is in itself interruptive, a recent study found that overall it helps people manage their
interruptions rather than increasing the overall amount of work disruption (Garrett and Danziger 2008).
This has to do with the ways IM allows users to manage availability. One way of doing this is to flag it in
the buddy list (e.g. “working hard, will not answer”). Further, it is generally socially acceptable to
postpone IM response, because non-response means that you might be away from the computer. This is
what Nardi (Nardi, Whittaker et al. 2000) calls “plausible deniability”. The new communication patterns
afforded by IM also support interruption management (Garrett and Danziger 2008): quick answers can be
obtained without expectation of longer conversation, and a chat window can be kept open among the
communicating parties for as long as desired, allowing for queries on at-need basis with the expectation
of having an answer when it is convenient for the other party. The findings on IM tools as tools for
interruption management can be taken as an argument that in the context of project-based learning we
should not be too concerned about the popping up of chats windows irrelevant to the project work at
hand. The students are likely to be capable of managing their use of time, helped by the social norms and
sanctions within the team, and having to manage their time is part of the desired project work experience.
Rather than focusing on the use of IM that is not relevant to project work, thus, in what follows we focus
on the use of IM that is related to the project work.
In the SE projects of our study, different collaboration tools were used to support different aspects of
work. While the specific selection and usage of tools differed among teams, there was a pattern (Krogstie
2009): Lightweight project management tools (typically project wikis or issue tracking tools) were used
123
15

for managing team-internal task coordination and provide links to project documentation. Development
tools were used to write, test and integrate source code. Storage and versioning of project artifacts were
managed in a file versioning system that may or may not be integrated with the project management tool.
Email was used for formal and documented team-internal and external communication. Project teams
typically had their own mailing list. Internet sites were used to get necessary information about
technology. This included FAQ lists and discussion forums. IM was used for informal messages within
the team and to support synchronous, distributed work. Less frequently, it was used for communication
with other stakeholders.
In this paper, we will go into detail about the teams‟ use of instant messaging.
3 RESEARCH APPROACH
The paper is based on empirical research on SE student teams in the period 2006-2008 focusing on the
teams‟ use of collaboration technology (Krogstie and Bygstad 2007; Krogstie 2008; Krogstie 2008;
Krogstie 2009). The data have been collected from two project courses offered in the last (6th) semester
of undergraduate programs in IT at two different learning institutions (NITH and NTNU). In the courses,
teams of 3-5 members develop software for genuine customers. The main learning objective is to have
students get experience with SE work in a team for a customer. Project deliveries include a software
product and a report. The teams choose which collaboration and development tools to use, depending on
customer requirements, team members‟ prior experience, team members‟ wish to learn, and the
availability of the technology. The NITH and NTNU project courses are considered similar enough for
data on the teams‟ use of collaboration technology to be combined in our study.
The research on instant messaging is based on triangulation of data sources. They include 30-60 minutes‟
interviews about the use of collaboration technology in the teams, performed with all 7 teams in the NITH
course in 2006, all 9 teams in the NTNU course in 2007, and 10 out of the 11 teams in 2008. They also
include longitudinal observation of two NTNU teams (one in 2007 and one in 2008) over the entire
semester, and project deliveries and customer evaluation for all teams in the NITH course in 2006 and the
NTNU course in 2006-2008. Further, some NTNU teams in 2006-2008 have been interviewed after their
project based on interesting characteristics of their use of collaboration technology. In some cases, we
have asked teams for IM logs. Overall, the data from NITH have been used in combination with the
NTNU data to identify general patterns of collaboration technology use in the SE student teams. In-depth
study of IM use in single teams, with access to IM logs, has been conducted only with data from the
NTNU course.
The researchers‟ role as course staff at both learning institutions provided a background for identification
of interesting data and for their interpretation.
During data collection and analysis, some themes and concepts have been pursued from the start (e.g.
Instant Messaging), and some have emerged through the analysis. Summaries and transcripts have been
produced at need, and translation to English performed as necessary for publication.
The research for the paper can be considered mainly as an interpretive case study in which data collection
and analysis have been guided by a constructivist view of learning and guidelines for interpretive field
studies (Klein and Myers 1999). However, in the paper, rather than focusing in detail on “the complexity
of human sense making as the situation emerges” and understanding “phenomena through the meanings
that people assign to them” (Klein and Myers 1999), which typically implies an in-depth elaboration of
single cases, we wanted to present several examples to illustrate how instant messaging is used in
different ways in the projects and link the examples to general patterns of IM use seen from the
interviews. By drawing some inferences across the cases we thus approach a positivist type of case study
research (Yin 2003).
4 FINDINGS
Findings are presented in the following way: First, team-internal use of IM is addressed and next, we turn
to the use of IM in collaboration with project stakeholders. In our presentation we have given priority to
124
16

three examples in the form of excerpts from real IM logs, to provide the reader with a flavor of the actual
IM usage in the teams. While our research approach does not enable us to quantify the representativeness
of the examples, we address more informally in the below sections what can be considered typical and
untypical.
4.1 Team-internal use of instant messaging
A type of IM usage very frequently seen in the teams, is brief coordination and information sharing
within the team, synchronous and asynchronous. IM is a channel constantly open to the other team
members and used as a convenient way to get an indication about others‟ presence (in the buddy list) or
sharing information even when the other team member is present and sitting nearby. Frequently, small
amounts of workspace contents are shared by pasting it directly into a chat window, which may be very
effective to provide context for a question or with concrete information needed for a specific task. In the
world of software engineering, exact syntax is often essential to make things work, which makes „cut and
paste‟ very useful.
In general, from our observations, the use of IM for these brief exchanges appears to be well integrated
into the students‟ way of communicating with their peers and presents few problems to the projects. In
line with the work on interruption management (Garrett and Danziger 2008) we note that the students
cope with always being „on‟ and available.
Team-internal IM use is however not limited to brief exchanges. Example 1 shows an excerpt from a
conversation between Dave and Chuck, two out of the three members in a 2007 team, and the ones most
involved in the team‟s development work. Dave and Chuck were generally working very well together,
for several reasons preferring a distributed work arrangement and using IM to discuss ongoing work.
They had access to a shared server and worked on different, but integrated parts of the system. With this
partially shared workspace, Dave and Chuck could immediately see some of the results of the other‟s
work, but not all. Results could however be communicated through the IM chat window.
In the excerpt shown in Example 1, the project is approaching deadline, and Dave and Chuck are working
from their respective homes late at night. The conversation takes place between 03:57:58 and 04:10:53
a.m. The overarching task is debugging: identifying and correcting errors in the code.
The excerpt contains several references to the development work going on in parallel, both the current
events in the shared workspace (e.g. “pop, a notepad”, Dave referring to a text-editing window appearing
on his screen) and ongoing tasks for which the two students have distributed responsibility (e.g. Dave:
“but what about conditionals? do they run correctly”. Chuck: “don‟t think move runs correctly”. Dave:
“ok”). A shared understanding of the current state of work is thus constantly maintained and re-created.
New tasks are identified, negotiated and assigned. (Dave: “can you debug it?”. Chuck:”I‟ll just have a
look at the installer first”. Dave:”ok”).
Humor is extensively used in the conversation, see for instance the emoticons like “:OOO” and the
outburst “give me that much work because of nothing!”. The generally positive tone and close
relationship between the two students permits remarks like the last one to be given without resulting
tension. Also, the tone of the conversation should be seen in light of the ongoing final project spurt and
fact that the work and the conversation takes place in the middle of the night.
Incidentally, the excerpt in Example 1 does not include the sharing of textual elements like pieces of
source code or error messages, but the whole chat from which the excerpt is taken contains much of this
type of material.
The entire log shows that the chat window was kept open for about 29 hours. This includes breaks when
the team members went to get some sleep, food, coffee, or take a shower. The log contains comments
addressing these activities.
125
17

Example 1.
Excerpt from an IM log illustrating conversation over ongoing, distributed debugging
work among the students Dave and Chuck. Translated from Norwegian by the author.
In sum, the open IM window substituting face-to-face communication in cooperative development work
among the two students provides both workspace awareness (Gutwin and Greenberg 2002) and social
awareness to the parties. As a note, we may add that Chuck and Dave‟s team delivered an outstanding
project result.
4.2 Use of instant messaging in collaboration with external stakeholders
Generally, we see that when team and stakeholder communicate over IM, it is successful for quick and
informal exchanges. Example 2 shows a case of this in the form of an IM exchange between a team and a
supervisor. The chat takes place during work hours, lasts for 1.5 minutes and addresses an upcoming
project delivery. The conversation takes up on previous conversation on what is to be handed in by the
team. Besides answering Michael‟s initial question whether the activity plan for the last two-week period
is to be delivered, Thomas (the supervisor) makes sure that the team remembers to hand in both the
original plan and a revised version showing the actual work done. Michael informs that the team is going
to revise the delivery document internally and send the final version during the same evening, which
Thomas accepts. The exchange can be seen as efficient, friendly, informal and task-oriented, and reveals
an accommodating attitude on part of the supervisor.
126
18

Example 2.
Excerpt from an IM log showing a brief exchange between a project manager (Michael)
in a student SE team and their supervisor (Thomas). Translated from Norwegian by the
author.
The extent to which instant messaging is used for team-supervisor communication in the projects of our
study depends on teams‟ and supervisors‟ preferences. Supervisors who communicate with project teams
over IM, typically communicate with all their teams on this channel. When (most) supervisors do not use
IM for communication with the teams, it may be because they are not heavy IM users or because they
restrict availability to other channels. In general, the teams appear not to expect from the beginning that
the supervisor be available through IM. The main thing for the students is that there is an agreement on
how communication should take place and the supervisor is available and responds as agreed, whether
face-to-face, over IM or email. In cases where communication should be documented, as is often the case,
email and meetings with minutes are favored by teams and supervisors.
For communication between teams and customers, email and face- to-face meetings are the most
frequently used channel. In cases of remote customer, telephone meetings and video conferencing are also
used.
However, particularly in the case of remote customers, instant messaging is sometimes being used for
informal team-customer communication, typically over technical issues. An example of the latter is a
2008 team whose customer was located in another city. The following excerpt is from the team‟s
reflection note addresses the issue of which type of cooperation technology was used for which purpose:
“MSN: The group created a separate MSN account that was used for customer contact. The
customer was always available on MSN when he was at work, which was very useful
whenever we had questions that needed to be answered quickly.”
The customer of this team expresses similar satisfaction in his final evaluation of the project, commenting
that
“We have had good communication over email and MSN. The team asked for input whenever
they were uncertain about something.”
A 2007 team also had a customer who was located in another city for large parts of the project
period, occasionally visiting for face-to-face meetings with the team. While remote, he used instant
messaging actively in communication with the team. In their common reflection note, the team writes:
“MSN has been the instant messenger application of choice for all group members, and
enabled us to easily stay in touch with each other and the customer at all times of the day
(and night, for that matter).”
127
19

In his formal evaluation of the process after the project, the customer particularly expresses appreciation
of the „continuous communication‟ with the team during the period when he was in another city.
In both of the above cases the customer took a strong personal interest in the development process and
appreciated being involved with technical issues and have them resolved quickly. Formal meetings were
however not conducted over IM.
A team in the 2006 cohort did use IM for formal meetings with their remote customer. While this is not
typical of our projects, we will go into some detail on the case because it demonstrates a communication
breakdown which might in part be due to the inadequacy of IM in the particular coollaboration setting.
The project group in question had taken on the task of making a game for a collaborative virtual
environment. The customer role was held by a team of two researchers, one at the students‟ university and
one in an academic institution on the other side of the globe. The team never met the remote customer
face-to-face. Meetings between him and the group were conducted via IM. Being the one proposing the
use of IM for meetings, the customer was willing to have meetings at late hours to cope with the time
difference. As the project process was being subject to formal evaluation, the project group wanted to
document their communication with the customer. They therefore chose, with their customer‟s consent, to
log the meetings conducted via IM. Some weeks into the project, the group was frustrated because of
what they conceived to be misunderstandings and vagueness related to needs, requirements,
responsibilities and expectations in the project. Deadline was steadily approaching without the necessary
progress of work. Example 3 shows an excerpt from a meeting over IM conducted at this rather critical
point in the project. We have called the remote customer Johnatan/Johnny, and his Norwegian partner
Anja. Frode and Per are two of the four students in the project group. During the meeting, the students are
collocated in the PC lab. Anja is on another location in the students‟ city whereas Johnny is on the other
side of the globe. The communication is taking place in English, Johnny‟s mother tongue. We lack
timestamps for the comments in the log, but the conversation shown in the excerpt should be understood
as continuous and without major pauses.
From Example 3 it can be noted that highly critical questions in the project are at stake: what are the
requirements, who is doing the planning and resource allocation (the group feels it is their job and that the
customer is interfering), and access to important resources for the work (a particular server). In the heat of
the conversation, Frode, who was in practice the project manager, starts commanding the remote
customer in a mix of anger and desperation: “Read the storyline document! Get us access to the server”.
Not surprisingly, the remote customer simply leaves the conversation at that point, and communication
breaks down. The team expresses uneasiness with the situation. The local customer, Anja, stays in the
conversation and tries to moderate and explain the situation. Frode goes on to explain what he sees as the
major problems.
128
20

Example 3.
Excerpt from an IM log showing a communication breakdown between a student SE team
and their remote customer. Originally in English.
5 DISCUSSION.
In this section, we discuss the findings and outline implications for the organizing of SE project courses.
Research referred to in Section 2 indicated that young people are likely to be competent users of instant
messaging tools. Our findings confirm that this is the case among the SE project students. Being a simple,
lightweight tool already part of the students‟ personal infrastructure as a means for communication and
social networking, it appears that IM is easily incorporated into project based learning in the SE projects
as a way of supporting informal communication; synchronous and asynchronous, distributed and
collocated. Further, IM provides a limited but easy way of sharing workspace content by cut-and-paste
(and even the exchange of files via shared folders). The fact that instant messaging is also in use among
developers in industry SE projects stresses the value of using and developing students’ IM competence.
From the assumption that understanding the students‟ work process is a prerequisite to providing good
supervision of the process, we believe that increased insight about the current and potential use of IM
tools to support work can help course staff provide students with guidance on some aspects of the
students‟ work process. General findings on the use of instant messaging in student teams as presented in
this paper are one source of knowledge about IM usage. Much more important in the specific case of
supervision are the particular practices of the team in question. As opposed to project activities being
formally documented e.g. in email or deliverables, students‟ IM use is typically inaccessible to the
supervisor. Leaving out the option that the supervisor be provided access to, and examine, the team‟s IM
logs, the teams must provide their supervisor with information about their IM collaboration practices in
order to get advice on these practices. Where problems of collaboration occur in the projects, it should be
129
21

considered whether current use of IM plays a role. Where IM is successfully supporting project work, it
might still be worthwhile for the team to account for the tool usage, to increase their understanding of
their own work practice and to become aware of potential problems.
Distributed project work can be successfully supported by instant messaging as a communication channel
combined with shared workspace access, as seen in Example 1. The example illustrated how the social
and the task-oriented aspects of working together were both present in the communication taking place
through the tool. Such an arrangement should be considered an option in projects that for some reason
depends on work to be conducted in a distributed way (e.g. when one or more students have to partially
work from home). It should be noted, however, that in our projects, the team members‟ work is partially
collocated. The likelihood of success of IM chat as a substitute for face-to-face interaction may be
different with students who do not know each other from face-to-face interaction, but we cannot draw
conclusions about differences between IM use in virtual and partially collocated teams from our study.
The practice of letting the student teams choose the collaboration technology for their projects is in line
with the independence seen as important in project-based learning. Also, it can be seen as a way of
encouraging students to make use of their existing competence with certain tools, e.g. IM tools, and
consider how they should be applied in a new and challenging context (e.g. that of coordinating complex
work tasks and collaborating with external stakeholders). Being allowed to choose the collaboration
technology for their projects, we assume that most teams will choose to use IM tools for some aspects of
their work, given the ground that these tools has gained as personal communication infrastructure among
the students. However, whereas most students in our study were users of a particular IM service,
individual preferences vary. In the projects, we occasionally saw individual students refusing to use
certain tools on principle (e.g. by being against proprietary software) while the rest of the team wanted to
use those tools. In such cases, the teams might benefit from their supervisor‟s mitigation to find good
solutions.
For the organizing of SE project courses, we see the following implications of students‟ competence with
IM tools:
The student teams should be allowed to choose the collaboration technology for their projects, within
the constraints of availability, the team‟s competence and the customer‟s requirements.
Teams should be required to present their supervisor with an account of their practices of using
instant messaging, and the supervisor should use this information as a basis for providing process
guidance.
In projects heavily depending on distributed work, the use of instant messaging chat to aid such work
should be discussed with the team.
Example 2 illustrated that informal collaboration between the team and the supervisor may be
successfully conducted over IM. We believe that keeping IM open as a communication channel between
team and supervisor can encourage students to get early in touch when they are in need of help, which
makes it possible to solve problems before they escalate and before too much time is spent on being
stuck. The viability of this solution depends on the supervisor‟s willingness to use IM in communication
with the student and to be available for consultation on a fairly short, or at least explicitly agreed-upon,
notice. We see the following general implication for the organizing of SE project courses:
Course staff may use instant messaging as a collaboration tool for providing informal supervision or
otherwise communicate informally with the project students. Expected availability (e.g. response
time) should be explicitly addressed as the use of IM for collaboration is agreed with the team.
Example 3 illustrates the general challenge of stakeholder communication and, in particular,
communication with the customer, in project work. Successful stakeholder communication depends on
project teams‟ ability to share the right information in the right format at the right time. Where there are
language and/or cultural barriers, the challenges of communication increase. As Frode from Example 3
put it in an interview after completion of their project: “We did not know if the customer was joking or if
he was angry with us!”. The choice and use of collaboration technology strongly impacted on the
communication. IM chat with the use of standard functionality (i.e. text only) provided opportunities for
130
22

immediate feedback in the heated discussion while conveying a sense of informality and providing
limited opportunities for the communicating parties to monitor each other‟s reactions (e.g. through tone of
voice and body language). In Example 3, the mismatch between the communication needs and the
affordances of the communication medium was particularly severe. The Molotov cocktail of problematic
factors included the team and customer never having met, not knowing each other, having different native
language and cultural background (including organizational/university culture – one being hierarchical
and the other flat), having different conceptions of responsibilities connected to the customer role in the
project, and the team being in a situation of severe problems (being far behind schedule while important
resources were missing).
The team may be unable to identify a potential mismatch between the choice of communication medium,
the purpose of communication and the form of communicating. In some cases, as in Example 3, the
customer might have suggested an inadequate arrangement for communication. A similar and typical
example, related to another type of collaboration tool, is when the customer insists on the use of email for
all student enquiries but turns out to be very slow in answering his email.
The biggest and most general problem illustrated in Example 3, however, is that IM is being used for
formal collaboration. Many project issues require a formal meeting, properly prepared and documented,
or formal, thought-through, written communication, with copies and attachments. Instant messaging can
be seen to encourage an informal tone and comments which are not meant to be revisited for purposes of
documentation, even if logging is possible. If IM is used in team-customer collaboration, the logs should
be kept by the team for purposes of process documentation, and the customer should be informed about
this usage. For instance, the IM log from which Example 3 is drawn was an important part of the
documentation of the project process and used by course staff in the evaluation of the project. The IM log
illustrated some reasons for the disappointing project result.
In contrast to Example 3, our findings from the SE project course also included examples of successful
use of IM in team-customer collaboration, enabling teams to have quick feedback from a remote customer
(See Section 4.2) on mainly technical aspects of software development. In this case, the purpose and
issues of IM conversation were limited and clear.
Implications for the organizing of SE project courses of the challenges of using IM for customer
communication include:
Given the high project risk associated with unsuccessful customer communication, project
supervisors should make sure that they are aware of, and accordingly require the team to account for,
how collaboration with stakeholders is conducted in the team, including the use of collaboration
technology, e.g. instant messaging tools. If the customer has asked for an arrangement of
communication highly likely to cause problems for the team, the supervisor might provide the team
with advice on how to change the arrangement or point out what are the pitfalls.
With respect to formal communication, in general, instant messaging should be used with care. The
participants should ensure that the needs of formal communication are met, for instance that there is
enough information to provide the social awareness necessary for the parties to understand each
other‟s views and feelings (e.g. satisfaction, anger) when important issues are discussed. This might
not be possible over instant messaging alone if the parties are not familiar with each other from
previous face-to-face interaction. Also, the parties should make sure that the communication is
documented in a way suitable for archiving and later retrieval.
If any collaboration with the customer takes place over instant messaging, the conversations should
be logged, in agreement with the customer
If the customer is motivated to use instant messaging to be able to give rapid feedback on technical
issues, the team should use the opportunity.
Many of the implications outlined in this section address issues that apply to the use of collaboration
technology in general to support work in the SE student teams, not only to the use of instant messaging.
Even if most teams in our study seemed to be capable of choosing adequate media for various aspects of
cooperative work, we suggest that it be made a topic in the introductory presentation of the project course.
131
23

Also, initial sharing of experience among students in class may spread good ideas across teams and
further help course staff understand students‟ current practices. In the project course of our study we have
good experience with conducting a mid-term workshop in which students meet across teams to discuss
various aspects of their project work. The more or less successful use of collaboration tools is an
appropriate issue for such discussion. To facilitate fruitful exchange of experience, students may be asked
to provide scenarios of collaboration from their projects, accounting for how issues were resolved and
how collaboration technology was used. To illustrate the actual collaboration practices, logs documenting
the collaboration may be used. IM logs are an example of documentation that for reasons of privacy may
be inadequate for publicly presenting a team‟s collaborative practices. However, for the teams internally,
the logs may be a valuable source of information if they want to reflect on and learn from their own
project process (Krogstie 2009).
6 CONCLUSION
In this paper, we have presented findings on the use of instant messaging tools in SE student projects.
Used in the projects, the tools serve to support aspects of the work process and are means of providing
realistic work experience as intended in project-based learning. Findings from our studies indicate that the
teams‟ use of instant messaging for team-internal collaboration, whether for quick coordination or lengthy
exchanges over ongoing, distributed work, is frequently successful.
In organizing and supervising SE student projects we should be aware of students‟ existing competence in
using IM tools and the type of collaboration typically happening with the aid of these tools. To gain the
necessary understanding of the project process required for guidance of the specific team, we should ask
the team to account for its collaboration practices, including the use of collaboration tools such as IM
tools.
The use of IM to support collaboration with project stakeholders is potentially challenging. Based on our
findings, we propose that the student teams be advised to restrict the use of instant messaging to informal
collaboration. Much of the customer collaboration in the SE projects may be considered formal and
requires the use of other types of collaboration technology.
Finally, we have pointed to the use of collaboration tools such as IM in the SE projects as a topic worth
discussion and reflection. On the level of the project course, this may be done in introductory lectures and
workshops. In addition, the use of IM and other collaboration tools should be discussed internally in each
project team to help the teams gain insights about their own work.
While we have limited the focus of this paper to student projects in the domain of software engineering,
our results have relevance for project-based learning in other domains in which instant messaging tools
are used to aid the project work.
A limitation to our study is the lack of data combining observation of IM use with access to the resulting
logs. While IM logs in combination with post-hoc interviews with project participants provided rich data
about the role of IM in supporting work processes, observation of the work taking place as the logs were
produced would have provided valuable additional insights. However, we have been able to obtain IM
logs reflecting collaboration processes identified by the teams as important to their projects. Another
limitation to the study lies in the analysis of the interviews forming the basis for the findings on patterns
of IM use in the student teams. While the material has been thoroughly examined, the analysis would
have benefited from systematic categorization of the answers about instant messaging, organized e.g. in a
table. For instance, in the paper, we combined the data from the 2006-2008 interviews based on the
informal finding that there was no change in the patterns of IM use over these years. This finding and
approach could have been better substantiated with quantification of the data.
As further work we plan to introduce the topic of collaboration tool use in reflection workshops in a SE
project course to increase students‟ consciousness of this aspect of their work practice. We will evaluate
the outcomes to see if this deliberate focus leads to insights in the student teams about how the different
collaboration tools, including instant messaging, should be used to support their project work.
132
24

Acknowledgements
Thanks to Monica Divitini and John Krogstie for feedback on ideas and drafts, and to the SE students for
sharing their views and instant messaging logs.
References
Blumenfeld, P. C., E. Soloway, et al. (1991). "Motivating Project-Based Learning: Sustaining the Doing,
Supporting the Learning." Educational Psychologist 26(3 & 4): 369-398.
Bygstad, B., B. Krogstie, et al. (2006). Scaffolding Project Based Learning with the Rational Unified
Process. Experience from 5 years of Student Projects in Software Engineering. NOKOBIT 2006,
Molde, Norway, Tapir.
Bygstad, B., B. R. Krogstie, et al. (2009). "Learning from achievement: scaffolding student projects in
software engineering " International Journal of Networked and Virtual Organizations 6(2).
Carstensen, P. H. and K. Schmidt (2002 (1999)). “Computer Supported Cooperative Work: New
Challenges to Systems Design”. Handbook of Human Factors/Ergonomics. K. Itoh. Tokyo,
Asakura Publishing.
Castells, M. (2000). The Internet Galaxy: Reflections on the Internet, business, and Society. Oxford,
Oxford University Press.
Cherry, S. M. (2002). "IM means business." Spectrum, IEEE 39(11): 28 - 32
Garrett, R. K. and J. N. Danziger (2008). "IM=Interruption Management? Instant Messaging and
Disruption in the Workplace." Journal of Computer-Mediated Communication 13(1).
Grinter, R. E. and M. Eldridge (2003). Wan2tlk?: Everyday Text Messaging. CHI2003, Ft.Lauderdale,
Florida, USA, ACM.
Grinter, R. E. and L. Palen (2002). Instant Messaging in Teen Life. CSCW'02, New Orelans, Louisiana,
USA, ACM.
Gutwin, C., R. Penner, et al. (2004). Group Awareness in Distributed Software Development. CSCW'04,
Chicago, Illinois, USA, ACM.
Huang, A. H. and D. C. Yen (2003). "Usefulness of instant messaging among young users: Social vs.
work perspective." Human Systems Management 22(2): 63-72.
Isaacs, E., A. Walendowski, et al. (2002). The Character, Functions, and Styles of Instant Messaging in
the Workplace. CSCW'02, New Orleans, Louisiana, USA, ACM.
Klein, H. K. and M. M. Myers (1999). "A Set of Principles for Conducting and Evaluating Interpretive
Field Studies in Information Systems." MIS Quarterly 23(1): 67-94.
Krogstie, B. (2008). Power through brokering. OSS participation in SE projects. ICSE 2008, Leipzig,
IEEE Computer Society.
133
25

Krogstie, B. and B. Bygstad (2007). Cross-Community Collaboration and Learning in Customer-Driven
Software Engineering Student Projects. Twentieth Conference on Software Engineering
Education and Training (CSEE&T), Dublin, IEEE Computer Society.
Krogstie, B. R. (2008). The wiki as an integrative tool in project work. COOP, Carry-le-Rouet, Provence,
France, Institut d‟Etudes Politiques d‟Aix-en-Provence.
Krogstie, B. R. (2009). A model of retrospective reflection in project based learning utilizing historical
data in collaborative tools. EC-TEL 2009, Nice, France, Springer.
Krogstie, B. R. (2009). Using Project Wiki History to Reflect on the Project Process 42nd Hawaii
International Conference on System Sciences, Big Island, Hawaii, IEEE Computer Society.
Lancaster, S., D. C. Yen, et al. (2007). "The selection of instant messaging or e-mail. College students'
perspective for computer communication." Information Management & Computer Security 15(1):
5-22.
Lewis, C. and B. Fabos (2005). "Instant messaging, literacies, and social identities." Reading Research
Quarterly 40(4): 470-501.
McMillan, W. W. (1999). What Leading Practitioners Say Should Be Emphasized in Students' Software
Engineering Projects. Conference on Software Engineering Education and Training, New
Orleans, Louisiana, USA, IEEE Computer Society.
Nardi, B. A., S. Whittaker, et al. (2000). Interaction and Outeraction: Instant Messaging in Action.
CSCW'00, Philadelphia, PA, USA, ACM.
Stephens, K. K. (2008). "Optimizing Costs in Workplace Instant Messaging." IEEE Transactions on
Professional Communication 51(4): 369-380.
Thomas, J. (2000). A review of research on project-based learning. Novato, CA, The Buck Institute for
Education.
Yin, R. K. (2003). Case Study Research. Design and Methods. Third Edition, SAGE Publications.
134
26

Research paper P4
Title:
The wiki as an integrative tool in project work
Author:
Birgit Krogstie
Published in:
Proceedings of COOP 2008
Pages:
12
Complete reference:
Krogstie, B. R. (2008). The wiki as an integrative tool in project work. COOP, Carryle-Rouet,
Provence, France, 20-23 May. Institut d’Etudes Politiques d’Aix-en-
Provence.
135

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
136

The wiki as an integrative tool in project work
Birgit R. Krogstie
1
Norwegian University of Science and Technology,
Sem Sælands vei 5-7, NO-7491 Trondheim , Norway
birgitkr @idi.ntnu.no
Abstract. The paper provides insights on how wikis support project work and
what characteristics of wikis make them adequate for this purpose. The findings
are based on a case study of software engineering projects in an educational
setting. Project wikis are found to serve an integrative role along several
dimensions of project work, enabled by the flexibility and automatic support for
capturing history offered by the technology. The findings demonstrate that a
project wiki can serve as a knowledge repository, a means for staging the
project, a coordination mechanism, and a shared workspace. To many projects
in need of project management and collaboration support, project wikis should
be seen as an attractive, lightweight, all-purpose alternative.
Keywords: wikis, cooperation technology, project work, software engineering
1 Introduction
”When we had a wiki up and running in the first place, we wanted to use it for as
much as possible” (Project manager, Team A)
Wikis are lightweight technology supporting easy creation and maintenance of web
pages. The flexible structuring of contents is a particular feature making wikis
adequate and popular as knowledge repositories. A wiki may be used for keeping the
shared information resources of a group such as an organizational unit or and internet
community, allowing the members add and modify contents. The contents of a wiki
may be shared with the world or restricted to a group or community [1].
Lightweight project management (PM) solutions are often sought when ‘heavy’
PM is not feasible, as seen in open source software development. Louridas [2] points
to potential advantages of wikis to support software engineering (SE) work:
comprehensive versioning and change control, the provision of a shared workspace
for producing and storing project documentation, and the possibility of having input
from many participants in a more effective way than e.g. over email. Louridas argues
that wikis are especially useful as discussion media, the advantages over discussion
lists being general user-friendliness and the fact that discussion items can be placed in
context. Further, he points to the democratic aspect: a project wiki gives a voice to
everybody in the project. Chao [3] presents survey findings of innovative forms of
wiki use in student SE teams.
137

As course staff in SE project courses we have seen many student teams using wikis
during the last couple of years. The project wikis contain or provide links to various
project artifacts, convey team-internal as well as information for external stakeholders
and generally seem to reflect a mixture of administrative, development-related and
socio-emotional aspects of project work. Existing research literature does not address
in depth how project wikis aid different aspects of SE practice. For instance, the work
of Chao on wiki use in SE projects identifies different types of use, but only on an
overview level based. Louridas addresses the potential of wikis in SE projects without
presenting empirical findings on actual use.
The objective of the work presented in this paper has been to gain insights on the
actual work practices involving wiki use in SE teams and the purposes for which the
wikis are used in these teams. In what follows, we provide an account of related
research. Next, we describe our case and research method. We present findings on
how the wikis in our case serve to integrate aspects of project work along several
dimensions, providing illustrative examples. Finally, we relate this to a set of different
roles played by the wikis in the projects, before concluding the paper.
2 Related work
Whereas our focus is on the role of wikis in SE work, it is worthwile to examine
the body of research on wikis in educational contexts. Much of this research turns on
the use of one wiki for an entire course [4] and/or possible pedagogical extensions of
wiki functionality addressing teacher-student interaction, e.g. [5, 6]. We believe that
there are important differences between course wikis and project wikis (while some
wikis may be hybrids). The analytic distinction provided by Riel and Polin [7]
between practice-based, knowledge-based and task-based learning communities
captures a major point: A SE student project is mainly task-based. In the type of
project considered in our case, which is industry projects with a high degree of
independence on part of the project teams, the project wikis are established by the
team to support their work towards project completion and the cooperative
development of a product. The wiki is typically not intended to be re-used across
projects (as might be the case if knowledge building or sustained practice within a
course or organization were the objectives). In other words, to understand the usage of
a SE project wiki with particular heed to the perspective of its users, it is reasonable to
view it as collaboration technology used by a project team even if the particular
setting is that of a university course.
Brown et al. [8] addressed how wikis can be used as a collaboration tool among
students of ethnography. The type of wiki in question resembles a course wiki more
than a project wiki, but there are aspects of its use which resembles project wiki usage
in SE teams. In particular, Brown et al studied the use of wikis for collaboration over
the writing of field notes, students sharing, comparing and discussing their notes in
the wiki. Findings address e.g. the willingness of wiki users to share personal
fieldnotes on an open wiki and the very limited editing of what is perceived as others’
documents in the wiki. A finding of particular relevance to SE work is the combined
distributed and co-located wiki use found in the groups of ethnography students. This
138

type of use can be seen as an example of how teams in a situation of partly
distributed, nomadic work [9] find ways to utilize the flexibility of the wiki to serve
their situated needs.
In the context of project work in general, the usefulness of wikis has increasingly
been recognized [2, 10]. Singh et al. [11] show how interdisciplinary projects can use
social bookmarking to support personalized access to wiki content. As mentioned in
the introduction, Louridas sees a number of advantages of wikis in SE work [2].
Wikis can be seen as a form of collaboration technology fitting the philosophy and
practices of free and open source development [12]. Mullick and colleagues [10]
introduced a wiki in a simulated global software development project to improve
coordination among distributed teams, finding that a common portal to project
information was necessary to coordinate the design work across teams.
Chao’s study of wiki use in student SE teams [3] unveiled satisfaction with the use
of wikis as a lightweight collaboration tool. In what follows, we expand on Chao’s
work to find what it is about project wikis that serves SE teams so well.
3 Case and research method
We conducted a field study during the spring 2007 among software engineering
student teams developing software solutions for external customers. Teams of 3-5
students work half time on their projects throughout their sixth (and last) semester,
drawing on and integrating knowledge from other courses in the software engineering
program. The overarching learning objective is to gain experience with development
work in a team for a customer, the management of the team-customer relationship
intended to be a major challenge and source of learning for the students. Each team
has an external customer providing a development task unique to that team. The
customer is responsible for providing necessary information on requirements and
feedback on the development of the product. Further, each team has a supervisor from
course staff providing guidance on the project process. Each team receives one grade,
which is set in cooperation between the supervisor and an external examiner.
Status reporting and bi-weekly meetings with the supervisor are mandatory, as are
the delivery of two preliminary versions of a project report before the final project
delivery. Apart from this, strong independence on part of the students is encouraged.
It is up to the teams to organize themselves, to manage the communication and
relationship with project stakeholders, and to choose and adapt an appropriate
development methodology / process model as well as collaboration and development
tools. The choices made by the teams depend on the customer’s preferences or
requirements, team members’ prior knowledge and experience, team member’s wish
to learn, and the availability of the technology (e.g. through university license, as
freeware, or provided by the customer).
Four out of the nine teams in the 2007 cohort made use of project wikis, and three
teams (hereby denoted team A, team T and team D) were active wiki users throughout
the entire project. The project deliverables included a report (electronic and on paper)
and source code, but the wiki was not itself a deliverable.
139

Following an interpretive approach [13], we collected data on these teams through
participant observation, interviews with the various stakeholders (teams, supervisors
and customers), examination of project documentation and project wikis. The data has
been coded based on themes emerging through the analysis, without explicit reference
to one theoretical framework, but with the use of core CSCW concepts as deemed
useful. Students from the teams A, T and D were encouraged to provide feedback on
the full version of the paper, but we did not succeed in having their opinion.
4 Findings and analysis
In our analysis, we investigate wiki usage in teams A, T and D, assuming the
projects to be similar enough to make it possible to draw a picture of wiki usage
across cases. All the teams had the necessary technical competence to solve their task.
They were successful with the project in terms of achieving a good grade and
delivering a product to their customer in accordance with specifications. They had a
project manager being a project blog sponsor/enthusiast, creating the blog and taking
the main responsibility for updating it (while all other team members also contributed,
to varying degrees). They used their project wiki as a shared project workspace and
main channel for updated and official project status information, as will be elaborated
They regularly updated a to-do-list and other memo/task-oriented lists on the wiki.
They password-protected the wiki or avoided informing widely about the web
address, but made the wiki accessible to customer and supervisor. Finally, they used a
version management system separate from the wiki for the software development
Some more detail on each of the teams illustrates the differences and provides
context for some of the findings presented in this section:
Team D appeared very structured and ambitious. There were 3 members, one of
which was absent during a large part of the project period due to serious illness. The
two others were the lead developers and relatively old students, experienced with
project management and software engineering. The project work was almost solely
done in a distributed fashion. The customer was located in the same city as the
students and the university. The development task was technically challenging. The
application used for generating the project wiki was DokuWiki. After a while during
the project period the team included a wiki plug-in to manage project activities.
Team A had 5 members who were mostly collocated in their project work. Their
customer was located in a different part of the country during the first part of the
project. The application used for the project wiki was MediaWiki. After the mid-term
project report delivery, the project manager developed a script for collaboratively
writing the final project report in the wiki and having the report auto-generated.
Team T had 5 members. Their project work was largely collocated. The customer
was located in the same city as the team and the university. T. Among the three teams
in our study, team T had the greatest number of personal and emotional expressions in
their wiki, which was a MediaWiki. The team used only standard wiki functionality.
In the three project teams, we found that the project wikis serve a role in the teams’
efforts to address and balance considerations typically competing for project
resources and attention. The overarching perspective of our analysis thus evolved to
140

ecome integration, and we identified several dimensions along which this integration
seemed to receive wiki support. The dimensions are, as will be seen, interwoven.
4.1 Wiki usage: Integrating across the social and the task-oriented
In this section we elaborate on the use of project wikis to convey task-oriented as
well as social/emotional information. We address how to-do-lists are used, how
individual project contributions are made visible, and how project status information
is conveyed through the structuring of the main page.
The wiki main page is used for reflecting project status, in particular remaining
tasks, but also the social and emotional state of affairs.
“The wiki quickly proved itself to be an invaluable tool for information management
and collaboration, with the front page being used primarily as an editable bulletin
board describing the current messages/tasks, and sub pages holding the actual
content.” [..] (interview, team D)
The teams were very selective in terms of what should go into the main page.
Issues that could be resolved elsewhere, e.g. in weekly meetings or over msn, need
not, and should not – according to the students - be put in the wiki. To-do-lists and
news bulletins on the main page were for conveying and reading the project status.
Each team kept a to-do-list on their main page. In addition, there were various
other lists, their number, position and purpose varying across teams and between
phases of the project. Examples include individual to-do-lists, a project news bulletin,
and separate to-do-lists for the project report and the testing.
“What is put on sub pages tends to be forgotten. What is current, what is to be done
now, should be on the main page. What is most recent should be on top of the list.”
(interview, team A)
The lists of remaining tasks continuously changed throughout the projects, growing
and shrinking depending on the phase of the project and the distance to the next
deadline. At need, a dedicated list (e.g. for the final report) was branched out.
The process of assigning tasks to team members was typically done in face-to-face
work settings, but also through the wiki: from the list of currently unresolved tasks, a
team member could just pick one and work on it, in case of one of the teams by
moving the item to his/her individual list in the wiki. To-do-lists were thus not only
supplementing (on a day-to-day level) but in fact partly substituting other project
planning or development artifacts such as a ticket system for managing software
changes. Team T saw the wiki as “an easy way out” in terms of change management:
“The ticket system does have some features that might be better; you can.. integrate it
with SVN, then, the versioning control, when you make a new change, you may write
something particular that makes the ticket system pick it up, and see that things have
been changed, but it becomes so advanced, that ..people do not bother to get into it,
you know. And then it is easier only to have a wiki-page.” (interview, team T)
Team D, on the other hand, integrated a wiki plug-in for the management of
activity lists, thus taking a more structured approach to software changes in their wiki.
Strikethroughs were extensively used as a means for making visible the completion
of tasks in the wiki to-do-lists. An item with a strikethrough typically remained in the
list until the overall task (e.g. a preliminary report) had been completed.
141

Project logos and other visual material were used to achieve a certain
personalization of the web page, related to a team identity, but also to individuals. An
example of the latter is that the wiki responsible team member of team T included a
link to a website which provided a new random picture of a cat with each click – cats
being a strong personal area of interest to the team member.
Emotional expressions in the wiki relating to the management of tasks ranged from
individual team members’ comments by the descriptions of tasks (“Finally!”, “What
to do with the graphics?”) via the project manager’s use of red, large-size fonts for
urgent messages to pictures representing the general feeling of a deadline approaching
(Fig.1):
Fig. 1: Part of the wiki main page of team T towards final deadline.
Strong outbursts were often quickly diminished or removed by their creators in
subsequent wiki versions. The flexibility to easily remove material thus possibly
encourages the (sometimes brief) exposure of such material on the wiki.
Individual to-do-lists indicated that individuals had, or had taken, responsibility for
specific tasks. Initials by tasks in a common task list served the same purpose. Effort
made was indicated by strikethroughs and comments, e.g. on difficulties with the task.
A cake list was launched on team T’s main page as a kind of pillory. Members
being late for meetings had their names put in the list. The team thus made visible that
it was not acceptable to break team rules, and the rule breakers received a small
penalty by having their name exposed. At the same time, they were offered the
opportunity to make amends through a social contribution (making a cake for the
team). Viewed as part of the account of the status of the project, names on the cake
list indicated (minor) project management challenges, and their being addressed.
A wiki main page can easily be re-structured while other pages remain unchanged.
The main pages in our study were resized along with the phases and needs of the
142

project. Typically, a clean-up was performed after a completed deadline, and spin-offs
(e.g. new sub-pages or separate headings on the main page) resulted from increased
activity in certain areas or the recognition of new needs and objectives.The
restructuring thus reflects a coordination of tasks, but it also has an emotional side,
e.g. making visible that something has been accomplished, or that the team is turning
over a new leaf. A fully packed main page, maybe slightly more chaotic than usual,
expresses something different about how the project is currently experienced by its
members than a sparsely populated page with short to-do-lists.
4.2 Wiki usage: Integrating team-internal and team-external information
From the above account of task-oriented and social/emotional use of the wiki, it is
evident that the project wikis were used for team-internal purposes. The wikis were
however also used for providing external stakeholders (customer, supervisor and
possibly others) with information about the status of the project (e.g. project plan,
status report, latest revision of requirements specification).
Making the wiki externally available means, to some extent, providing a desirable
image of the project. A nicely structured wiki with tasks gradually getting their
strikethroughs convey an image of a process under control. The emotional
expressions found on the pages, however, reveal emotional sides of the project work
to external stakeholders. The teams in our study found this acceptable, but allowed for
different amounts of informal and emotionally laden contents on their wikis.
Team D saw the wiki as very central in communicating project status to the
customer. In their final report, the wiki was placed in the middle of a figure showing
the collaboration structure of the project. The wiki was shown as the link between the
team and the other stakeholders. To the surprise of team D, however, the customer, in
his evaluation report on the project, expressed satisfaction with everything in the
project except that he would have wanted more face to face meetings instead of
updates only through the wiki. In the case of teams A and T, the supervisor told the
team that he wanted them to use a wiki and give him access to it, which most likely
influenced the teams’ choice to use wiki technology.
When asked if they would have liked to share wiki contents with other teams in the
course, the teams in our cases were generally positive. The project manager of team A
however thought that they might not want to display task lists full of uncompleted
tasks because it would reveal a lack of control. This comment reflects the general
competitive attitude among the project teams in the course.
Communication through the wiki seemed to happen largely one-way, from the
team to other stakeholders. Customer and supervisor generally provided feedback
through other channels. In the case of team A, however, the customer was situated in
another city during the first half of the project and frequently visited the wiki. If there
was something of particular importance in the wiki, the team would send him email
about it. The team had a couple of conferences with the customer over msn and
Skype, and, the customer had been visiting the wiki before the meetings, examining
for instance the current version of the requirement specification. Further, the customer
maintained a separate page in the project wiki containing non-functional requirements
for the system under development. In the case of team T, one of two customer
143

contacts once made comments directly in the requirements specification in the project
wiki. He later explained that he found it interesting to try out new technology for this
purpose. A typical customer attitude to project wiki pages, however, seems to be that
they are the students’ area, even when other stakeholders are given write permission.
4.3 Wiki usage: Integrating across artifacts
Major project artifacts, e.g. minutes, status reports, plans and implementation
related information, were typically accessible from the main page. Also, the wikis
contained links to relevant web pages with data e.g. about the customer or a tool.
The project task of the teams in our study was a SE task, and coding was done in
separate environments. Wiki integration with code versioning was very limited. None
of the teams were familiar with wikis providing support for such integration.
Lightweight and more manual solutions were however helpful. Team A did not use a
separate versioning system, but used their customer’s code repository, to which access
was restricted. The project manager uploaded modified source code to wiki pages for
each member to download locally (as the team could not consecutively run their code
on a shared server), using msn to notify when a new download should be made.
Text formatting in wikis is not compatible with commonly used text editors. The
teams accordingly ‘wikified’ documentation by adding the necessary tags to
unformatted text. This was time-consuming, annoying and infeasible for the full
project report. None of the teams were familiar with wiki functionality for generating
printable pages from wiki pages. The low-threshold alternative to integrating the
report into the wiki was to make it accessible through a link. One team developed a
script converting report chapters in the wiki into Latex, making it possible to perform
joint editing within the wiki and generate a printable version at any time.
The wikis are used in combination with other collaboration tools to work on and
discuss project artifacts. The teams are very clear about which tools are used for
which purpose. Discussions typically happen face-to-face or over instant messaging.
Chat among distributed programmers is done in parallel with programming work, and
unresolved work issues might be added to the wiki to-do-list during the work session.
Email is preferred for formal communication, particularly when the customer is
involved and there is a need for documentation.
5 Discussion
One way of accounting for the richness and versatility of the observed integrative
wiki usage in the SE projects is to distinguish between different roles served by the
wikis in the projects. We find that the project wikis simultaneously serve as
knowledge repositories, stages, coordination mechanisms, and shared workspaces. In
the light of these roles, we will elaborate on current wiki usage.
144

5.1 The wiki as a knowledge repository
Our findings indicate that the project wikis serve a role as knowledge repositories
for the teams and other stakeholders. In all the wikis of our study, project artifacts and
useful resources were included or linked in, “so we know where to find it”. We may
regard this as ‘traditional’ usage of wiki technology.
Apart from the current version of the wiki at any point, historical information is
stored in the wiki. Each wiki version can be seen as forming a point, of greater or
lesser significance, along a project trajectory. Trajectory has been described as “(1)
the course of any experienced phenomenon as it evolves over time [..] and (2) the
actions and interactions contributing to its evolution” [14](p.53-54). The development
over time of certain aspects of the project, or artifacts such as the software product,
may be seen as forming sub-trajectories that are, to some degree, represented in the
wiki. Ideas developing into artifacts can be seen as trajectories of interaction,
reflecting social as well as technical/task-oriented aspects of the project process.
5.2 The wiki as a stage
The teams provide certain types of information on the wiki while withholding other
information. This can be seen as providing a certain image of the project, internally
and externally. This is particularly evident from the main page, being loaded with
status information of a technical as well as social and emotional nature. The latter
type of material becomes somewhat restricted through the teams’ practice of not
conducting discussions in the wikis (in spite of the potential benefits [2]), and through
the fairly quick moderation or removal of some material.
Internally in the team, the ‘staged’ picture of the project not only provides a status
update but also potentially serves a role in conveying a project identity for team
members to share. The inclusion of project logos and informal material such as
humorous pictures may contribute to a feeling of project identity.
For external use, a professional appearance might be of greater importance.
Information on the wiki constitutes part of the controlled information flow between
project stakeholders. This is important e.g. in managing customer expectations. The
team’s version of project status as shown on the wiki should be regarded as one-way
information in need of some channel for feedback (whether in the wiki or in another
channel), as indicated by customer D’s wish for more face-to-face interaction.
In the projects of our study, the project wikis were not deliverables, whether to
customer or course staff. The wikis were however accessible to the supervisors, who
played an important role in the formal evaluation of the team. Our data are not
sufficient for making inferences about how this particular aspect of the course design
affected wiki usage, e.g. the allowance for informal and unstructured contents.
5.3 The wiki as a coordination mechanism
We have seen how the to-do-lists provide a means for planning, monitoring and replanning
project work in a very flexible way. The linkability and malleability of the
145

tool [15] may be seen as major enablers of this usage, in combination with the history
function which makes it possible to capture a picture of the process over time as part
of the current status.
The to-do-lists have a team-internal coordinative role, but also play a role in
coordination with external stakeholders who monitor project status in order to act
upon it. We argue that is not only the strictly task-oriented aspects but also the social
and emotional side of the lists that constitute and inform articulation work in the
teams, providing cues about how to appropriately approach current challenges.
Louridas suggested that wikis are good for project democracy. In our study, we see
that the project manager, being the major wiki sponsor in the team, uses the wiki to
set or visualize the agenda for the project, thus manifesting his power. All team
members are wiki users, to varying degrees, making visible their contributions. We
may only speculate that the wiki offers better visibility for the viewpoints of those
who fail to get their voice through in face-to-face meetings.
5.4 The wiki as shared workspace
The wiki constitutes a workspace providing shared access to various artifacts. The
inherent wiki functionality for handling access conflicts to pages or page sections is
appreciated by the teams in our study. However, we have seen that the current use of
wikis for collaborative writing is so far limited. Taken from team members’
viewpoints on how they would want to use their wikis, it seems that lack of
knowledge of, or access to, adequate technology (e.g. for converting between formats)
was a major reason why e.g. the project reports were not written in the wikis directly.
Some workspace awareness [16] is achieved as the wiki is used in combination
with other development and collaboration tools, e.g. instant messaging. Also, the
history of the wiki and information on the main page on who recently did what,
provides awareness of team members’ work. The project wikis may be seen as
socially translucent [17], providing social cues about team members and their effort –
both what they actually did and, to some extent, how they felt about it. A wiki main
page with its to-do-list, individual comments and strikethroughs provides visibility
and accountability of team members as well as of the team as a whole.
Turning to the project trajectory in context of the shared workspace, the concept of
locale [18] is useful. Fitzpatrick made interaction trajectory one of the core aspects of
the locale – “the place constituted in the ongoing relationship between people in a
particular social world and the ‘site and means’ they se to meet their interactional
needs, i.e. the space together with the resources and things available there” (p.9) A
project wiki may be regarded as part of these ‘site and means’ in the project, and the
history function of the wiki as a potentially valuable resource for reviewing the
project trajectory in order to learn from it and/or work on its future course.
A final remark should be made on the emotional and social contents of the wikis in
our study. These contents are mostly about reflecting and supporting a task-oriented
project process by providing some social awareness. In the teams, social interaction
generally takes place on other arenas. Accordingly, the project wikis should not be
considered as ‘social software’.
146

6. Conclusion
We have shown how project teams make use of wikis in a way integrating aspects
of their project work. The wikis are simultaneously knowledge repositories, means of
staging the projects, coordination mechanisms and shared workspaces.
We believe that a core issue in determining the appropriateness of using a project
wiki in a specific project is how much predefined structure is sought for the
management of the project. This relates to the choice of development methodology
and lifecycle model. Agile development and open source projects tend to favour
lightweight and flexible tools [12]. Further, project size, heterogeneity and degree of
distributed vs. collocated work impacts on the overall complexity and need for
computerized coordination mechanisms [19]. The development of safety-critical
projects require a more rigid development process than projects involving less risk.
We suggest that the use of a project wiki as a lightweight project management tool
is appropriate for projects that are not safety critical. We see project wikis as relevant
for SE student projects, small SE projects (particularly if work is at least partially
distributed), teams within larger SE projects, and open source SE projects.
Our findings demonstrated the potential of a wiki to serve many roles in SE project
work besides that of coordination mechanism. If the wiki were to be supplemented
with a project management tool in which coordination was handled, it is difficult to
predict how the wiki might serves its other roles (knowledge repository, stage, shared
workspace). This topic might be pursued through empirical research.
Whereas our main research agenda has been to provide a descriptive account of
current wiki usage, we would like to point to alternative usage and further research.
First, there is a potential for utilizing the knowledge repository of the individual
projects across projects. Students express interest in looking at earlier project reports
to learn from them, and this is relevant to industry settings as well. If a project is
staged to be accessible and intelligible to all stakeholders, it is likely to be of potential
value to other project teams. Project wikis may generally have a place in the broader
picture of knowledge management in an organization. Second, given the effort
currently made by the teams to create and maintain informative wiki pages, there is a
potential to conduct project-external communication through this channel to a larger
extent than what we see in the teams today. Generally, the practice of using the
customer’s preferred channel (typically email) to convey wiki contents by including
links to the relevant wiki pages, seems to be successful as a way of providing a slight
information push. Third, better opportunities for collaborative writing were requested
by our teams. Having the wiki tool manage versioning and access conflicts and offer
functionality to generate nicely formatted reports would significantly increase the
value of wikis as shared workspace. Some existing wikis provide such solutions.
Taken from our findings, there is only limited use of historical information in the
SE project wikis. History stored in the wiki can be utilized for purposes of reflection,
e.g. in post-mortem project reviews. Our research continues along this line.
147

Acknowledgements
This work was funded through the MOTUS2 project at NTNU. Monica Divitini and
John Krogstie provided valuable comments on draft versions of the paper.
References
1. Leuf, B. and W. Cunningham, The Wiki Way - quick collaboration on the Web. 2001:
Addison-Wesley.
2. Louridas, P., Using Wikis in Software Development. IEEE Software, 2006.
3. Chao, J. Student Project Collaboration using Wikis. in 20th Conference on Software
Engineering & Training (CSEET'07). 2007. Dublin, Ireland.
4. Xu, L., Project the wiki way: Using wiki for computer science course project management.
Journal of Computing Sciences in Colleges 2007. 22:(6).
5. Lund, A. and O. Smørdal. Is There a Space for the Teacher in a WIKI? in Proceedings of
the 2006 international symposium on Wikis WikiSym '06 2006.
6. Reinhold, S. WikiTrails: augmenting Wiki structure for collaborative, interdisciplinary
learning. in International Symposium On Wikis 2006.
7. Riel, M. and L. Polin, Online Learning Communities. Common Ground and Critical
Differences in Designing Technical Environments, in Designing for Virtual Communities in
the Service of Learning, S.A.K. Barab, Rob; Gray, James H., Editor. 2004, Cambridge
University Press: Cambridge. p. 16-50.
8. Brown, B., et al. Seeing Ethnographically: Teaching ethnography as part of CSCW. in
ECSW'07. 2007. Limerick, Ireland.
9. Bødker, S., et al. Technology for Boundaries. in GROUP'03. 2003. Sanibel Island, Florida,
USA: ACM
10. Mullick, N., et al. Siemens Global Studio Project: Experiences Adopting an Integrated GSD
Infrastructure. in International Conference on Global Software Engineering. 2007. Munich,
Germany: IEEE Press.
11. Singh, A.V., A. Wombacher, and K. Aberer. Personalized Information Access in a Wiki
Using Structured Tagging. in OTM. 2007: Springer.
12. Gutwin, C., R. Penner, and K. Schneider. Group Awareness in Distributed Software
Development. in CSCW. 2004. Chicago, Illinois, USA: ACM.
13. Klein, H.K. and M.M. Myers, A Set of Principles for Conducting and Evaluating
Interpretive Field Studies in Information Systems. MIS Quarterly, 1999. 23:(1) p. 67-94.
14. Strauss, A., Continual permutations of action. 1993, New York: Aldine de Gruyter.
15. Schmidt, K. and C. Simone, Coordination Mechanisms: Towards a Conceptual Foundation
of CSCW Systems Design. Computer Supported Cooperative Work, 1996. 5 p. 155-200.
16. Gutwin, C. and S. Greenberg, A Descriptive Framework of Workspace Awareness for Real-
Time Groupware. Computer Supported Cooperative Work, 2002. 11:(3-4) p. 411-446.
17. Erickson, T. and W.A. Kellogg, Social Translucence: An Approach to Designing Systems
that Support Social Processes. ACM TOCHI, 2000. 7:(1) p. 59-83.
18. Fitzpatrick, G., The Locales Framework: Understanding and Designing for Wicked
Problems. The Kluwer International series on Computer Supported Cooperative Work, ed.
R. Harper. 2003: Kluwer Academic Publishers.
19. Trac home page, http://trac.edgewall.org/. Last accessed 17.April 2008.
20. Carstensen, P.H. and K. Schmidt, Computer Supported Cooperative Work: New Challenges
to Systems Design, in Handbook of Human Factors/Ergonomics, K. Itoh, Editor. 2002
(1999), Asakura Publishing: Tokyo.
148

Research paper P5
Title:
Using Project Wiki History to Reflect on the Project Process
Author:
Birgit Krogstie
Published in:
Proceedings of HICSS 2009
Pages:
10
Complete reference:
Krogstie, B. R. (2009). Using Project Wiki History to Reflect on the Project Process
42nd Hawaii International Conference on System Sciences (HICSS’42), Big Island,
Hawaii, 5-8 Jan. IEEE Computer Society.
149

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
150

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
Using Project Wiki History to Reflect on the Project Process
Birgit R. Krogstie
Norwegian University of Science and Technology
birgitkr@idi.ntnu.no
Abstract
In this paper, we address the potential of project
wikis to support reflection on the project process
through participants’ reconstruction of the project
trajectory. Drawing on a case study of software
engineering project work, we illustrate how
information on project history can be found in a
project wiki and may be used to support recall of and
reflection on the process. We discuss implications for
postmortem project reviews by considering how the
utilization of project wikis could addresses some
challenges to successful reviews. We also propose
extended wiki functionality that can be used to make a
more selective review of project history based on usertagged
contents.
1. Introduction
Software engineering (SE) work is acknowledged
to be complex, as elaborated in the literature [1, 2] on
computer-supported cooperative work (CSCW) and
demonstrated by the frequent SE failures [3].
Understanding the history of a SE project is essential to
SE practice in two ways: In day-to-day coordination
and project management, and in learning from the
project experience. Such learning can be seen both as
an intrinsic aspect of work [4, 5] and as means for
achieving organizational learning within and across
projects [6-8]. We might also add that in the field of
education, project work is acknowledged to be a
particularly effective setting for learning [9].
This paper addresses how project wiki history can
support postmortem reflection on the project process.
The paper does not address why a team should choose
to use a project wiki, or how project wikis compare to
other project management tools. We take as a starting
point the acknowledgement that many teams currently
do use project wikis for lightweight project
management. We argue that there is more to these
wikis than what can be seen in day-to-day project
management: there is a lot of historical information
with a potential to shed light on the project history.
In proposing this approach to improving SE
postmortem reviews, we address some recognized
challenges to making such reviews successful. We also
present the design of a wiki extension which can be
used to support the approach. A prototype of the wiki
extension has been developed and is briefly presented
in the paper along for a scenario of its use. We do not
provide complete guidelines for a postmortem review
utilizing wiki history.
Theoretically, we lean on the work of A. Strauss
and the concept of trajectory. We see an SE teams
reconstruction of its project trajectory as essential to
reflection on the project process.
The paper is structured in the following way: The
Background section addresses some core concepts and
related work. In Section 3 we present our case and
research method. In Section 4 we draw on a case study
to show that project wikis have a potential to support
reflection on the project process. In Section 5, we
discuss how project wikis can be used to address some
challenges to postmortem reviews. A tool to improve
the usefulness of project wikis in such reviews is
described in Section 6. Section 7 addresses limitations
and further work and concludes the paper.
2. Background
Three issues provide a background for the
following chapters and are addressed in this section:
How trajectories play a role in reflection, the practice
and challenges of postmortem reviews in SE, and the
use of wikis as lightweight project management tools
in SE projects.
2.1. The role of trajectories in reflection
Dewey defined reflective thought as 'active,
persistent, and careful consideration of any belief or
supposed form of knowledge in the light of the grounds
that support it and the further conclusions in the
direction of which it tends' [10, p.133]. In this paper,
we consider reflection as a process of assessing
something (e.g. a product or a sequence of events) in
978-0-7695-3450-3/09 $25.00 © 2009 IEEE
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
151
1

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
respect to some quality and with respect to alternatives.
For instance, in reflection on a process, the congruence
with, or deviation from, some other process, whether
real (e.g. the process of a previous project) or imagined
(e.g. an alternative course of the same process, or the
ideal process as prescribed in a model) will typically
be considered.
We subscribe to the view that human activity is
basically social [10, 11] and that reality is socially
constructed [12]. Individual members of a social world,
such as a project team, belong to different social
worlds and have varying degrees of involvement in
these worlds over time [11, 13]. Reflecting on project
experience involves the alignment of different
participants perspectives.
Strauss [11] refers to Mead on the issue of how a
social world reflects on its history: The temporal
spans of group life mean that the aims and aspirations
of group endeavor are subject to reviewal and
recasting. Likewise past activities come to be viewed
in new lights, through reappraisal and selective
recollection. ..History, whether that of a single person
or of a group, signifies a coming back at self (Mead
1936). Strauss uses the term trajectory to denote
the course of any experienced phenomenon as it
evolves over time. Events as they unfold, e.g. in a
project, form a trajectory, and from each point in time
there are many possible trajectories onwards.
Taking the experienced phenomenon into
consideration, different individuals trajectories will be
different even when they largely comprise the same
events. What actually happened, is a question of
interpretation and reconstruction. Also, trajectory is
given meaning in a particular situation of use [14]. For
instance, the same sequence of information elements
presented after an intermediary deadline in a project
will be seen in a different way if used for purposes of
coordination rather than to learn from the process. In
sum, there is a lot of trajectory context which is not
available from its representation (e.g. a textual
description). Strictly, then, a trajectory is not a
sequence of single elements of information as
(re)presented, but as used. For practical purposes in
what follows, however, we will frequently refer to
trajectory as the trajectory as represented, e.g. a set of
information elements, chronologically ordered. Doing
so, we keep in mind that the representation will gain
meaning through situated sense-making the essence
of the sought-after reflection.
2.2. A case of reflection: SE postmortem
reviews
We now turn to reflection as part of software
engineering (SE) work practice. SE work is typically
performed in teams and structured as projects. The
trajectory of a SE project is typically guided by a
lifecycle model incorporating phases and iterations.
Options range from strictly and moderately structured
models (e.g. traditional waterfall, RUP) to agile
approaches (e.g. XP, SCRUM) [15]. A process model
typically specifies or recommends a division of labor
and a set of artifacts to be used and produced.
Generally, deadlines for project deliverables heavily
impact on the day-to-day experience of SE project
work.
To cope with the complexity of work and avoid the
all too frequent failures in SE projects, efforts are
needed to understand project history beyond the status
information needed for day-to-day articulation work.
We will point to three major activities for which it is
useful to review project history for purposes of
reflection. First, capturing and making explicit the
design rationale has been seen as important to
developers understanding of the system and as a
source of learning for new team members [16, 17].
Second, postmortem reviews are conducted to learn
from experience within and across projects in an
organization [18]. Third, supervision of SE project
teams in educational settings involves gaining
understanding of, and providing advice for, the project
process [19].
In what follows, we will focus on postmortem
reviews. What is commonly known as a postmortem
review is a collective learning activity organized for a
project that has either finished a major activity or phase
or has ended. Postmortems are conducted to reflect on
what happened in the project in order to improve
practice for individual participants and for the
organization as a whole. A post mortem report is part
of the review result. Several approaches to
postmortems have been proposed in the literature and
taken into use in industry. The process typically takes
from half a day to three days and is guided by a
facilitator. Different approaches diverge over issues
like homework or not, structured vs. open discussion,
presence of management, appropriate output (e.g. how
to make a useful report) and learning focus (tacit or
explicit knowledge). The approaches provide lists of
steps in the review process [18]. A problem with
postmortem reviews is that they are not heavily used in
industry despite the recognized need to learn from
software engineering failures [20]. IT (Information
Technology) specialists point to a number of barriers
relating to set-up, data collection, data analysis and
knowledge sharing. Some of these challenges relate to
the availability of relevant information on the project
process and the means available for structuring that
information. Three of the barriers [20] (pp.77-78) are:
2
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
152

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
1) Insufficient integration with existing learning
systems: .[..] (e.g. reporting systems, quality control
systems, and informal knowledge sharing).
2) Simple information and complex issues:
Information used in post mortem evaluations is limited
and simplified compared to the complexity of the
issues involved in system development.
3) Information overload: Post mortem evaluations
are overwhelming because there is too much to reflect
on.
2.3. The use of project wikis as lightweight
project management tools
To address current usage of project wikis and the
types of information contained in the wikis, we refer to
a case study reported in [21]. The case is an
undergraduate project course in software engineering
(SE). The projects are organized to be as close to
industry SE projects as possible. Teams of 3-5
students work half time on their projects throughout
their sixth (and last) semester in a Bachelor of
Informatics study program. Each team has an external
customer providing a development task unique to that
team. The customer is responsible for providing
necessary information on requirements and feedback
on the development of the product. Further, each team
has a supervisor from course staff providing guidance
on the project process. The project deliverables
includes a report in three versions, as well as a working
software solution for the customer. Strong
independence on part of the students is required. The
teams are responsible for managing stakeholder
communication and project organization as well as
choosing an appropriate process model and
development and collaboration tools. Four out of the
nine teams in the 2007 cohort of our case made use of
project wikis, and three teams were active wiki users
throughout the entire project. Two different wiki tools
were used in the 2007 teams: DokuWiki and
MediaWiki.
An exploratory case study was made by this author
on the role of the wikis in these projects [21]. The
researcher was course coordinator, but did not evaluate
the projects. Access to the wikis was granted for
purposes of research. Data for the study also comprised
interviews with the teams as well as various project
documentation. In the study, the project wikis were
found to serve an integrative role along several
dimensions of project work, enabled by the flexibility
and automatic support for capturing history offered by
the technology [21]. These findings demonstrated that
the project wikis simultaneously served as knowledge
repositories, means for staging the projects,
coordination mechanisms, and shared workspaces.
The more purposes for which a project wiki has
been used, the greater the potential to find useful
information there about the corresponding aspects of
project work. For instance, if day-to-day coordination
and allocation of tasks to team members is done with
the aid of to-do-lists on the wiki main page, it is
possible to use the wiki to reconstruct who did the
work in various areas over time. If emotional
expressions relating to the experience of project work
have been added to the wiki, the wiki may be used to
reconstruct at least part of the story of what it was like
to work in the project over time.
A content page from a project wiki is shown in
Figure 1. This is the upper part of the main page of the
wiki referred to in Section 4, and the particular page
revision is from April 16. 2008 at 12:13. A project
main page typically contains overview information
about the project, e.g. news bulletins and to-do-lists,
and links to various information useful to the project
team.
Below the header there are links to previous and
next versions of the page, making it possible to
chronologically traverse page versions. Surrounding
the user-created contents of the page are menus
providing access to various functionality, e.g to switch
to editing mode, contribute to a discussion of the page,
or view its history. Whereas this type of functionality
is representative of wikis in general, the exact design
shown is specific to DokuWiki.
Figure 1. Wiki content page (Main page of team A)
On basis of contents pages, the links between them,
and the various page revisions, wiki tools offer metapages
with synthesized information. This includes
overviews of all pages, the history of changes to one
3
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
153

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
page, all revisions of a page, and the difference
between two specified page revisions. The upper part
of a history page is shown in Figure 2. The page
revisions are listed chronologically, with links to each
of them. From the list, it is possible to see who edited
the page, when it was done, and what part of it was
edited (if specified). Also, it is possible to see the
difference, e.g. changes, between any two versions of
the page.
Figure 2. History page of the main page in Figure 1
For purposes of day-to-day project management,
for which timely status information is essential,
functionality for extracting synthesized information
from the project wiki is useful. For instance, all the
teams in our 2007 study reported that the history
function in the wiki was occasionally used during the
project to identify team members contribution.
When the purpose of extracting information is to
have a review of project history and reflect on the
project process, the numerous content page revisions
and the history pages providing overviews of these
revisions are an interesting resource. In what follows,
we will investigate the potential to use this resource for
such a purpose.
3. Case and research method
Two of the post-mortem interviews conducted in
the case study referred to in Section 2.3 were
facilitated by the use of the teams project wiki. These
interviews turned out to shed light on the wiki as a
means to support recall and reflection on the process
after project completion. This issue was singled out as
a separate research topic, and the resulting research is
described in this paper.
To present and substantiate our findings on wikifacilitated
reflection, we have chosen to focus on one
of the wiki-aided interviews. By focusing on a single
case, we aim to provide a coherent account of how the
traversal of wiki contents can shed light on a particular
project process. The interview was conducted with the
team manager of Project A one month after project
completion. An interview guide with a number of
questions was used as a checklist in the interview, but
issues were generally explored as they emerged in the
conversation. The 96 versions of the wiki main page
were chronologically traversed during the interview,
starting with the first revision dated February 1. 2007,
and ending with May 23. 2007, the day of project
delivery. The project wiki was a DokuWiki [22].
As the main page was the central page in the wiki
in terms of project status information and coordination,
the revisions of the main page became the basis for the
interview. Links to other pages were followed
whenever suggested by the interviewee and in some
cases the interviewer. The interview lasted for 2 hours
and was partially summarized in detail, partially
transcribed. Wiki screenshots were added to the
summary/transcript to make a coherent basis for
analysis. Interview excerpts considered relevant for
publication were translated from Norwegian to English
at need. The wiki contents were originally in English.
Data sources used in the analysis include the
researchers thorough examination of the project wiki,
various other project documentation from the team, a
post-mortem interview with the team supervisor, the
customers project evaluation, and an interview
conducted with the entire team earlier in the semester.
During analysis of the interview, we identified excerpts
reflecting the interviewees recall of, or reflection on,
the project trajectory with reference to wiki contents.
The findings reported in the next section are based on
these excerpts and the referred to wiki pages.
4. Recall and reflection through project
wiki traversal
In the interview with the Project A manager
(hereafter called TB), the interviewer went
chronologically through revisions of the wiki main
page, visiting other pages (content and history) at need.
Interviewer and interviewee were sitting next to each
other in front of the interviewers computer screen.
In this section, we provide examples from the
interview illustrating how information from the wiki is
actively used in a process of recalling and reflecting on
the project process. In each example, one or more
4
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
154

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
content pages, history pages or a combination of both
are used.
The following contextual information may aid
comprehension of the first example: The customer was
located in another city throughout the project. As the
team was to develop a module in a larger system, it
was important that the team follow the customers
coding conventions.
4.1. Example 1: The introduction and use of
the customers page
On the wiki main page, under the headline News
bulletin, there is a bullet point with a link stating:
Useful information and tidbits! Step right up and get it
while its hot, folks. As TB clicks on the link, he
enters the customers page. On top of the customers
page is a contents list with the header Information
from . The contents mainly address
technical issues such as coding conventions. For
instance, the first section is called If, foreach etc. and
starts with Always use curly braces when writing
blocks. A code example is given in the section.
TB goes to the history page. Looking at the
information there, it can be seen that the page was
created by TB in February, and that it has been updated
by the customer only, in the period until early March.
TB explains that he created the first version of the page
based on an email from the customer with some
guidelines for development. Then he had provided the
customer with access rights to the wiki and a link to the
customer page. After this, the customer continued
using the page to provide the team with information,
mostly non-functional system requirements and
guidelines about how to use a framework written by
the customer. The interviewer asks why there is no
more information in the later phases of the project, and
TB explains that the team knew how things were to be
done then.
In this example, TB uses the content and history of
the customer page to account for a form of
collaboration with the (remote) customer and how it
was successfully initiated by the team. The
interviewers follow-up question leads to reflection on
the status of the project halfway in the project period:
in TBs opinion, the team at that point was in control
of non-functional requirements and mastered the way
of doing development required by the customer.
4.2. Example 2: Conveying urgency
TB looks at the main page on May 3. at 22:36. The
first item in the News bulletin starts with the words
UN-FREAKING-BELIEVABLY URGENT in upper
case, red, boldface letters immediately drawing
attention to that part of the page. In the next page
revision, created 23:31, the font size is even larger.
When the interviewer comments on this, TB says that
the change shows the development and a few
psychological reactions, sort of.
In this example, changes between page revisions
are a source of recall and reflection. Changes made
from one hour to the next conveys a shapshot of the
project; a taste of the experience of doing project work
under conditions of perceived urgency. In the example,
TB points out how the web site reflects the feelings
involved his feelings at the time, as it were.
4.3. Example 3: A developing to-do-list
TB looks at the main page as it looked on May 3. In
the news bulletin, there is a new item on top with a link
to a Final Report to-do list. TB explains that they had
to branch out a task list. He goes to the Final Report
TODO page and then to its history page. He notes that
all revisions have been created by another team
member and comments that this team member had
main responsibility for updating the page. The
interviewer traverses many revisions of the page from
the period May 3.-8. In these versions, new to-do-list
items are steadily added and at one point the list is
restructured to contain sub tasks. The tasks gradually
become marked with strikethroughs. TB comments
that some structural changes can be seen from the
development of the list. He comments on the adding,
removal and crossing out of tasks, and laughs while
explaining that it isnt that funny when things get
added.
On the page of May 9. a list of team members
names followed by their initials have been added at the
top of the list. In subsequent page revisions, the tasks
become appended with initials. TB explains that this is
about distribution of responsibility.
As the interviewer browses more page versions, it
is apparent that there are many versions from May 9.
TB says that maybe the team at the time had some
mock-arrangement to hand in. He explains that their
supervisor liked to have the team make deliveries of
preliminary versions to him before the big deadlines,
and that the team appreciated this because there was
an extreme time squeeze towards the end when we had
to work on the project, the code, and we had as good
as finished the report then. And that was very good.
Or else we would not have been able to finish. The
interviewer asks if this means that their supervisor had
stressed them a lot to get going with the report. TB
confirms this and says that he would in fact
recommend to supervisors to stress the handing in of
5
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
155

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
intermediary reports. Also, he would recommend that
teams conduct mock presentations.
This example shows how the wiki documents
project management practice (e.g. the active use of todo-lists)
and the status of task completion over time.
The distribution of responsibility indicates how the
management of remaining tasks became more
meticulous towards project deadline. Some feelings
associated with this process and phase are reflected
upon by TB: the frustration of having to add new tasks
at a point when one would wish to see the list shrink.
The number of revisions of the page on a particular
date, and the amount of changes done then, reflect to
the team member that there was something particular
going on. He reconstructs (rather than recalls) that
most likely there was an intermediate delivery going
on, explaining this as an established collaboration
practice between the team and their supervisor. TB
reflects on the adequacy of this practice, arguing why it
was beneficial to the project. He also draws the
perspective further, generalizing to supervised project
work (e.g. in the course) when recommending
supervisors to require intermediate, mock-up deliveries
to help project teams complete important work on time.
4.4. Example 4: Project work during final
spurt
TB and the interviewer are looking at the main
page from May 23. This is the very last phase of the
project, two days before delivery. There is an item on
top of the News bulletin list, with a link: Smarty stuff
contains some Smarty functionality to be used in our
templates. The interviewer asks what this is. TB starts
explaining what he thinks it means, but then
remembers and corrects himself: TB: It is in fact.. It
was the last night, that one. When we sat fixing the
code. And added functionality that we had not had the
time for yet. So it is really just.. things that should have
been there some months before. To put it that way. Not
a day before. But what to do
In the page version from 04:41 on the same night,
the item Sudden realizations has been included in the
News bulletin, with the following sub item:
person_event.paid is currently not considered! How
would this fit into our design at all?!. The interviewer
asks a bout this, and TB explains that it refers to a very
important thing that they had forgotten. But luckily
they managed to fix it on that night. The interviewer
asks if they were sitting together in the team at the
time. TB says yes, but then corrects himself based on
the time of the page revision: No, that sudden
realizations, that, lets see, 04:41, it must have been
me remembering No! Shit! before I was about to go
to bed, and then I went into the wiki quickly, because I
had to remember it for the meeting the next day.
So I guess theres no more activity that night.. The
interviewer goes on to the next revision of the page, in
which another bullet point has been added: TB laughs
and says that that, too, was remembered suddenly and
later fixed.
On the page version of May 23, 22:22, even more
Sudden realizations have been added. TB comments
to the page:Yes, now we sit, here we are
..21:56.. B:Yes, right, there are the things you
remembered, now you have had a meeting.
TB:Yeseh.. looks like it.. [].
In this example, events during the last night and
day of the project are reconstructed. TB reflects about
work that should have been done earlier, on stress,
effort and luck. He uses detailed information from the
wiki page (exact time of the revision) to reconstruct
how he used the wiki at the time: as a means for
capturing his ideas in the early morning hours (in his
bed, from his home), to remember for the next days
team meeting. TB thus accounts for the work practices
in the project under the extreme conditions of final
deadline approaching. Trying to reconstruct this
exceptional phase, TB actively uses the time
information on the page revisions. Occasionally the
information requires some thinking to make sense.
(E.g.,May 23, 04:41 is the workday/night before May
23, 22:22).
4.5. What can be seen from the examples?
The findings show that at project team members
facilitated traversal of a project wiki leads to recall of,
and reflection on the project process. The wiki does not
contain a coherent account of the process, but the
contents are rich enough for the project manager to
reconstruct a project trajectory piece by piece.
Information in the wiki triggering recall and
reflection include time data (date and time of day of a
revision, sequence of revisions, frequency of
revisions), authorship, page content, and page
appearance.
Issues being recalled and reflected upon include
particular project events, project phases, collaboration
within the team and with other stakeholders (e.g. roles
and responsibilities, and the use of collaboration
technology).
Some more specific observations are:
− Sequences of events are reconstructed with the aid
of information about the time of each revision.
− Comments and reflection about the use of the wiki
in the project often leads to reflection on the project
more generally, e.g. events, phases, practices and
collaboration structures.
6
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
156

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
− The development of to-do-lists over time is a good
source of information on project management.
− In periods in which the density of wiki revisions is
high, the changes provide a picture of the there and
then experience, as in the final project spurt.
− Facts as well as feelings are addressed in the review.
Contents that appear to have been emotionally laden
as they were added to the wiki (e.g. the red, boldface
and growing urgent message), evokes
emotional reaction (in our example: laughter) in the
interview situation.
− Reflection may extend to SE project work at large,
e.g. lessons learned and suggested guidelines for
projects or project courses.
5. Benefits of reviewing project wiki
history in postmortem reviews
Having seen how a postmortem interview can be
aided by the use of a project wiki, we now return to the
selected challenges of postmortem reflection listed in
Section 3.2. We will have a look at each challenge in
turn to see how it can be addressed through suitable
use of a project wiki. In doing so, we propose some
wiki functionality that would be useful in this respect.
5.1. Using project wikis to integrate with
existing learning systems
The project wiki itself as used in our case study can
be considered a part of the organizations learning
system. The information about the project process
found in the wiki includes elements of evaluation of
the process itself, formally or informally produced and
conveyed. Examples are project status reports and
emotional expressions of state-of-affairs. Later use of
this information in a postmortem review is a way of
turning to account learning that has already taken
place, developing insights further as they are viewed in
hindsight and by different team members.
In addition to utilizing the history resulting from
day-to-day use of the project wiki, it is possible to use
the wiki to provide closer integration between the work
process and the identification of issues important to
understand and reflect upon the process. A quick and
easy way of tagging and/or annotating wiki contents
relevant to a particular issue would be useful in this
respect. To capture history, the tagging should apply to
page revisions and possibly sections of page revisions.
For a postmortem review, it should be possible to
generate chronological views of the tagged contents to
support the reconstruction of a trajectory.
To support learning across projects in an
organization, the result of a postmortem review could
be stored in the project wiki itself, if the wiki is kept as
a knowledge repository about the project. The
postmortem review report in the wiki should contain
links to wiki elements (page/section revisions) deemed
particularly interesting to the project trajectory.
5.2. Using project wikis to avoid
oversimplification of complex issues
Whereas a wiki page revision represents a snapshot
in time, showing only a particular view of some
aspects of the project, it simultaneously represents a
recognizable scene from a complex work setting. The
associations following the encounter with a familiar
surrounding may help the team recall events and
emotions and the meaning they were given at the time,
as shown in our case study. Also, links in the tool may
be followed between different pages and back and
forth in time, thus allowing for the exploration of how
events are connected, e.g. causalities. In this way, it
may be possible to reconstruct a more complex picture
than what is possible based on memory-based recall.
The proposed tagging of contents by team
participants and chronological traversal of the marked
contents may be used to ensure that issues important to
the team are actually addressed in the wiki-aided
postmortem review. The opportunity to individually
tag information during day-to-day work may help in
ensuring that everyones voice is heard in the common
postmortem review. All team members active
participation in the tagging is necessary for the
approach to succeed in capturing the full complexity of
the project.
5.3. Using project wikis to help avoid
information overload
The organization of wiki contents across pages and
revisions can help team members access and sort out
important information and structure their review of that
information. The focus on the wiki main page in the
postmortem interview of our case is an example,
showing a simple way to gain some focus in the review
process. Also, the example shows that allowing the
team member to explore promising tracks as they turn
up during the review, is a good way of spurring further
investigation of the process.
On the other hand, given the context of a
postmortem review, some structure might be needed
for purposes of efficiency in terms of covering key
issues in a limited amount of time [23]. There are many
ways of approaching historical data on a project, and it
may be desirable to focus on different aspects of the
project in turn (e.g. individual team members
7
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
157

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
perspectives, team-customer relationship, development
of requirements.) Again, team participation in the
tagging of contents, whether in the there-and-then
work situation or in reflective hindsight, would help
ensuring that information considered important by
those involved is given priority.
6. The wiki walkthrough tool: reviewing
selected aspects of project history
In Section 5 we have argued that user-defined
tagging of wiki page revisions combined with the
possibility to review the set of tagged revisions, could
make a project wiki even more useful for postmortem
reviews. In this section, we demonstrate how a wiki
can be extended in this way. Our wiki walkthrough
tool (WWT) has been implemented as a prototype
extension of DokuWiki. An elaborate presentation of
the tool is beyond the scope of the paper, but we
provide a brief use scenario and a description of the
tool functionality.
6.1. A scenario for the use of a wiki
walkthrough tool
The following scenario is intended to show in a
compact way how a project wiki can be used for
postmortem reflection, aided by some extended wiki
functionality.
The context for the scenario is as follows:
There is a SE student project team with five
members doing development work for a customer. The
team has a project wiki in which they include or link in
artifacts related to the project process and product.
The wiki is actively used in project management:
Various to-do-lists are updated by all team members,
and there is a news bulletin mainly updated by the
project manager. The chapters in the project report
are included in the wiki and converted to a coherent,
printable report at need. In addition to the wiki, the
team makes use of development tools, including a
versioning system, and collaboration technology such
as a shared calendar and instant messaging.
The postmortem review scenario:
Throughout the project every team member tags elements
in the project wiki reflecting what she considers to be
important events. The team member decides if the tag is to be
visible to other team members. The following postmortem
review takes place twice: just before the delivery of a
preliminary project report, and before delivery of the final
report:
Each team member spends an hour going through her
tagged wiki contents, modifying the trajectory made up of
those elements by visiting wiki pages and adding or removing
tags. In a common workshop supervised by a project-external
facilitator, each team member gets 15 minutes to present her
trajectory, using a projector in the meeting room. The team
next discusses the project and how their respective
understandings differ.
After the mid-term postmortem review, results of the
discussion (e.g. conflicting viewpoints, new insights) are
documented and actively used in project planning (e.g.
changed priorities, changed roles) After the final postmortem
review, the team decides on some elements in the wiki that
should be included in their common documentation of the
project process.
The postmortem review in the scenario is designed
to reflect acknowledged practice [23], e.g. encouraging
individual recall and reflection followed by plenary
presentation and discussion, and the creation of a
project management report which is later actively used
to achieve improvement in the project.
The scenario has been evaluated by two expert
groups experienced in the supervisor and customer
roles in the projects of the course described in Section
3. Here, we only report the main conclusions from the
evaluation: The expert groups were positive about the
potential of wiki history to be utilized in postmortem
reflection on the project process. They were also
positive about the potential of the WWT to help a team
achieve a more focused review.
6.2. The wiki walkthrough tool functionality
A prototype of the WWT was developed during
spring 2008. In providing a brief description of the tool
functionality in what follows, we take two screenshots
as our starting point.
Figure 3. Tagging a content page with an existing
tag ('todo_historikk')
Figure 3 shows a page from a project wiki. This is a
DokuWiki into which the WWT functionality has been
integrated. The user is browsing the wiki and decides
that she wants to tag the particular page revision. She
has clicked the Add tags button at the top of the page.
In the pop-up window appearing in the middle of the
page, the user gets the option of using an existing tag
or creating a new one. She picks an existing one,
todo_historikk, from the list. When she clicks Save,
8
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
158

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
the page revision will be tagged with todo_historikk.
While in the example it is the current revision of the
page being tagged, the procedure would have been the
same if the user was viewing an older revision of the
page and wanted to tag it.
When the user chooses All tags or Tag search at
the upper right of a contents page like the one in Figure
3, she can proceed to choose a tag and get a report
showing the page revisions tagged by it. In Figure 4,
the report shows contents tagged with todo_historikk
In the report view, for each page, tagged revisions
are shown together chronologically. The user may
view one or more of the revisions. In Figure 4, three
wiki pages have been tagged with todo_historikk.
Start has three revisions tagged, but the user has
chosen to view only the last one. For the page
TODO, the user has chosen to view two of the tagged
revisions. In the small window on top of the upper
TODO revision, visibility of the revisions can be
changed. The third page displayed in this report is the
presentasjon_final page revision tagged in Figure 3.
Figure 4. Report view showing wiki contents tagged
with 'todo_historikk'
In the report view there is an option of expanded or
compact views of the pages. The user may go directly
from to a page revision to explore it further, e.g. follow
one of its links. She may later return to the report view.
Also, from the report view, the report may be printed.
The tool has not yet been subject to evaluation
through full-fledged use in a real project.
6.3. Related work on wiki tagging
Tagging wiki contents is not a new idea as such.
Many existing wiki extensions allow users to impose
extra structure on a wiki in order to make it useful in
new ways.
The use of semantic web enabled technologies to
improve navigation and search in a wiki, so called
social tagging, has been implemented in SweetWiki
[24] and Semantic MediaWiki [25]. Gnowsis [26], a
semantic desktop including a wiki, implements the idea
of collecting relevant information to reflect a
personalized view. The use of structured tagging,
including free, personal tagging, to provide
personalized access to wiki information for purposes of
document management, is discussed in [27].
History or chronology is not in particular focus in
the aforementioned systems. However, whereas the
opportunities to tag earlier page revisions are only
partially addressed in the associated literature, we
assume that some existing tagging tools largely cover
the functionality proposed for the WWT. Accordingly,
from a technical point of view, some existing tool
could be used for our purposes, maybe after some
minor tailoring. From a usability perspective, we hold
that alternative usage of a tool, perhaps involving only
a subset of the available functionality and requiring
substantial adjustments, might introduce too much
noise to meet our high-level requirement for minimal
intrusiveness into the work process in which tagging
should be a background feature.
A tool that deserves particular consideration for its
related functionality is WikiTrails [28], designed to
help build context and structure around existing wiki
contents in an educational setting. The tool makes it
possible for a wiki user to leave a trail while browsing
the wiki, enabling other users to follow that trail later
on. The aim is thus related to ours: building and
sharing a story through a sequence of wiki pages. A
manual mode is provided, in which pages for a trail
may be picked and added by the user, e.g. from a list.
Two aspects of WikiTrails make it inadequate for
the intended use of the WWT. First, WikiTrails is not
designed to support the marking of potentially
interesting information element during work activity
for possible later incorporation in a trail. Second,
WikiTrails provides no support for the generation of a
chronological sequence; focus is on generating a trail
that will amount to a pedagogically sound presentation
of existing wiki contents. A third and minor objection
to WikiTrails in our context is that is seems to lack
functionality for the tagging of sections within a page,
a drawback it shares with the WWT prototype.
However, in WikiTrails the opportunity to annotate
might compensate for this fairly coarse granularity.
In sum, while acknowledging the potential of
existing tools to aid the structuring and traversal of
wiki contents, we advocate the use of a designated tool
for the reconstruction of a project trajectory.
9
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
159

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009
7. Conclusion and further work
In this paper, we have drawn upon a detailed study
of a postmortem interview from a capstone SE student
project to demonstrate the potential of project wikis to
be used as a means for the recall of, and reflection on,
a project process, thus making the project wikis already
in use even more useful to the project teams.
Some limitations to our research provide directions
for further research. The interview forming the core of
our empirical data was partially exploratory, and
adequate for pointing out the potential for project wikis
to aid in postmortem reflection. Our findings indicate
how various types of information may be used to recall
and reflect on various types of issues. However, we do
not have sufficient data to reasonably generalize about
the connections between the two. We suggest that
wiki-aided postmortem reviews be conducted in other
projects and the interaction and tool use of the
participants during the sessions be analyzed. The
interviews should follow a well founded design for a
postmortem review and be video recorded. In this way
it might be possible to identify what type of
information in a project wiki promotes reflection in
certain ways or on certain topics. Also, collective, toolmediated
postmortem reflection should be investigated.
Our wiki walkthrough tool needs to be evaluated
through use in real projects to see if it is deemed useful
by the project teams.
The idea of utilizing history stored in a
collaboration tool in reflection on the process is
relevant to other tools. E.g., source code management
and bug tracking tools contain a history of commented
project artifact revisions. The student teams at our
university increasingly prefer this type of tool (e.g.
Trac) to manage their SE projects. A next step in our
research is to conduct and evaluate postmortem
reviews mediated by these tools to see if they, too, can
help students learn from their SE project experience.
7. Acknowledgements
Thanks to Monica Divitini, Arne Martin Aurlien,
Håvard Håskjold and William Young. The research
was funded by the MOTUS2 project at NTNU.
8. References
[1] P. H. Carstensen and K. Schmidt, "Computer Supported
Cooperative Work: New Challenges to Systems Design," in
Handbook of Human Factors/Ergonomics, K. Itoh, Ed.
Tokyo: Asakura Publishing, 2002 (1999).
[2] G. Fitzpatrick, W. J. Tolone, and S. M. Kaplan, "Work,
Locales and Distributed Social Worlds," presented at CSCW,
Stockholm, Sweden, 1995.
[3] M. Keil, J. Mann, and A. Rai, "Why Software Projects
Escalate: An Empirical Analysis and Test of Four Theoretical
Models," MIS Quarterly, vol. 24, 2000.
[4] J. Dewey, Democracy and education. New York: The
Free Press, 1997 (1916).
[5] D. Schön, The Reflective Practitioner:Basic Books, 1983.
[6] C. Argyris and D. Schön, Organizational Learning II.
Reading, MA, USA: Addison Wesley, 1996.
[7] P. Senge, The Fifth Discipline. London, United Kingdom:
Doubleday, 1990.
[8] E. Wenger, "Communities of practice and social learning
systems," Organization, vol. 7, pp. 225-246, 2000.
[9] P. C. Blumenfeld, E. Soloway, R. W. Marx, J. S. Krajcik,
M. Guzdial, and A. Palinscar, "Motivating Project-Based
Learning: Sustaining the Doing, Supporting the Learning,"
Educational Psychologist, vol. 26, pp. 369-398, 1991.
[10] J. Dewey, How we think. A restatement of the relation
of reflective thinking to the educative process (Revised edtn.)
ed. Boston: D. C. Heath, 1933.
[11] G. H. Mead, Mind, Self and Society. Chicago: The
University of Chicago Press, 1934.
[12] A. Strauss, Continual permutations of action. New York:
Aldine de Gruyter, 1993.
[13] P. Berger and T. Luckmann, The Social Construction of
Reality. London: Penguin Books, 1966.
[14] G. Fitzpatrick, The Locales Framework: Understanding
and Designing for Wicked Problems: Kluwer Academic
Publishers, 2003.
[15] J. Lave and E. Wenger, Situated Learning. Legitimate
peripheral participation. Cambridge: University of
Cambridge Press., 1991.
[16] I. Sommerville, Software Engineering, Sixth Edition:
Addison-Wesley, 2001.
[17] E. J. Conklin and K. B. Yakemovic, "A Process-
Oriented Approach to Design Rationale," Human-Computer
Interaction, vol. 6, pp. 357-391, 1991.
[18] A. H. Dutoit, R. McCall, I. Mistrik, and B. Paech,
"Rationale Management in Software Engineering: Concepts
and Techniques," in Rationale Management in Software
Engineering, A. H. Dutoit, R. McCall, I. Mistrik, and B.
Paech, Eds.: Springer, 2006.
[19] T. Dingsøyr, "Postmortem reviews: purpose and
approaches in software engineering," Information and
Software Technology, vol. 47, pp. 293-303, 2005.
[20] B. Bygstad, B. Krogstie, and T.-M. Grønli, "Scaffolding
Project Based Learning with the Rational Unified Process.
Experience from 5 years of Student Projects in Software
Engineering," presented at NOKOBIT, Norway, 2006.
[21] V. Kasi, M. Keil, L. Mathiassen, and K. Pedersen, "The
post mortem paradox: a Delphi study of IT specialist
perceptions," EJIS, vol. 17, pp. 62-78, 2008.
[22] B. R. Krogstie, "The wiki as an integrative tool in
project work," presented at COOP, Carry-le-Rouet,
Provence, France, 2008.
[23] S. M. Smith and E. Vela, "Environmental contextdependent
memory: A review and meta-analysis,"
Psychonomic Bulletin & Review, vol. 8, pp. 203-220, 2001.
[24] "DokuWiki home page," vol. 2009.
[25] E. Derby, D. Larsen, and K. Schwaber, Agile
Retrospectives. Making Good Teams Great: Pragmatic
Bookshelf, 2006.
10
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:00 from IEEE Xplore. Restrictions apply.
160

Research paper P6
Title:
Shared timeline and individual experience: Supporting retrospective reflection in
student software engineering teams
Authors:
Birgit Krogstie and Monica Divitini
Published in:
Proceedings of CSEE&T 2009
Pages:
8
Complete reference:
Krogstie, B. R. and M. Divitini (2009). Shared timeline and individual experience:
Supporting retrospective reflection in student software engineering teams. Conference
on Software Engineering Education and Training (CSEE&T) 2009, Hyderabad, 17-19
Feb. IEEE Computer Society.
161

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
162

Shared timeline and individual experience: Supporting retrospective
reflection in student software engineering teams
Birgit R. Krogstie and Monica Divitini
Norwegian University of Science and Technology
{birgitkr, divitini}@idi.ntnu.no
Abstract
To help SE student teams learn from their project process, we propose the use of
facilitated postmortem workshops in which each team reconstructs its project timeline.
Individual team members’ experience of the project is included by team members drawing
individual ‘experience curves’ along the timeline. The approach is based on current industry
practice and adapted in accordance with theory on reflection and learning. We present the
detailed design of the workshops as they were implemented in an undergraduate SE project
course as well as some recommendations for future workshops. The workshops are effective
for promoting active participation, constructing a shared representation of the project
process, and revisiting project issues from multiple perspectives. Evaluation showed that
students were satisfied with the workshop and motivated to use the approach in later projects.
1. Introduction
Learning from experience is essential in project work, whether in education or in industry.
Such learning is about the day-to-day awareness of one’s own learning and about reflection
with some distance to events, i.e. retrospective reflection. In software engineering (SE),
retrospective reflection on the project process can be conducted in postmortems reviews, “a
collective learning activity which can be organized for projects either when they end a phase
or are terminated.” [1] p.295. The main purpose is to reflect on what happened in the project
in order to improve future practice, for the individual participants and for the organization as
a whole. A postmortem report is produced. The postmortem is typically facilitated.
In industry SE projects, it is often a challenge to have teams prioritize postmortems [2],
other tasks competing for resources. In student SE projects, reflecting on and writing about
the process is not seen as ‘real’ project work (like coding and testing) and conflicts with other
activities within or outside the project. Also, in a student team, it may be difficult to make
everyone participate actively. The power hierarchy in the team (e.g. [2]) might impact on who
is heard, who bothers to get involved, or who is allocated for the task. Thus, having a team do
the process reflection, with full participation, is a challenge welcoming practical approaches.
In student SE projects, successful postmortems should help students learn from the specific
project and serve as a hands-on introduction to standard SE practice. Further, from a course
administration point of view, resource requirements (e.g. course staff time) must be feasible.
To find a postmortem approach meeting these challenges, we look to theory on reflection and
learning and to current SE industry practice, as described in Section 2. Section 3 provides a
step-by-step design of a postmortem workshop in an undergraduate SE course. Findings on
students’ reflective process are presented in Section 4. In Section 5 we discuss the adequacy
of our approach, making some recommendations. Section 7 concludes the paper.
2. Background
22nd Conference on Software Engineering Education and Training
Learning from project experience involves looking back at the process, identifying events,
their sequence and their meaning. This may be considered as retrospectively reviewing the
978-0-7695-3539-5/09 $25.00 © 2009 IEEE
DOI 10.1109/CSEET.2009.20
85
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
163

project trajectory and involves facts as well as feelings. Anselm Strauss [3] uses the term
‘trajectory’ to describe: “(1) the course of any experienced phenomenon as it evolves over
time” (e.g. an engineering project) and “(2) the actions and interactions contributing to its
evolution” (pp.53-54). Events in a SE project can be understood in terms of the action and
interaction taking place within the team and between the team and other stakeholders. While
Strauss uses the concept ‘arc of action’ to describe a trajectory as retrospectively understood,
we apply the term ‘reconstructed trajectory’ to the retrospective understanding of the process.
From a social constructivist perspective [4, 5], project reality is interpreted through the
individual’s unique history of social interactions. Team members assign different meaning
and significance to events based on their previous experience and can be expected to
remember different facts as significant to the project. Making a shared, reconstructed
trajectory can be seen creating common ground [6] for communicating about the project
process. Possible disputes over facts may be settled by checking available evidence. In the
case of the interpretation of events, completely shared understanding among team members is
unattainable in principle, but participants may get a richer understanding of the process
through the encounter with others’ perspectives. Contradictions or discrepancies between
interpretations may serve as a source of reflective thinking [7]. To support reconstruction of
the shared trajectory and its use in reflection, it may be given a visual representation
containing elements that are considered shared and agreed-upon as well as elements
representing individual and possibly conflicting interpretations.
Boud and colleagues present a model of reflection outlining three major components:
experience(s), reflective process, and outcome [8]. Experience involves behavior, ideas and
feelings. In the reflective process addressing the experience, there are three steps: 1)
Returning to experience, 2) attending to feelings, and 3) re-evaluating experience. Essentially,
this means reconstructing the trajectory of the experience. Boud et al.’s model is adequate for
guiding the reconstruction of a project trajectory in a postmortem review. In particular, the
process of reconstruction can be approached by decomposing it into the steps of returning to
experience, attending to feelings and re-evaluating experience. As Boud et al. stress, the steps
should not be understood as a fixed template. Reflection is situated and creative, moving back
and forth between steps and sometimes omitting them.
Among the many SE postmortem techniques [1] there is an approach which reflects well
the idea of reconstructing a project trajectory: the timeline exercise as outlined by Derby et al.
[9]. An example of its use in industry can be found in [10]. The exercise is based on industry
best practice for postmortems. While the context of [9] and [10] is agile development, the
timeline technique is not bound to any particular software lifecycle model. Derby et al.
outline a basic exercise with some possible variations. The purpose of the exercise is to
stimulate memories, make a picture from many perspectives, examine assumptions about who
did what when, and see patterns. The exercise may be used for ’just the facts’, or facts and
feelings.
In the next section, we show how we have adapted and detailed the timeline exercise in
line with the previously presented theory.
3. Design of a postmortem workshop in a SE project course
The case of our study is an undergraduate project course taking up 50% of students’
workload in the last (6 th ) semester of the Bachelor program. The teams develop software for
genuine customers. Each team receives one grade and has a supervisor from course staff.
Deliveries include a software product, a project report in several versions and an oral
86
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
164

presentation. In 2008, there were 11 teams of 3-5 students; 46 students altogether. The grade
distribution in the course was 3 As, 4 Bs, 3 Cs and one D. B is a good grade that all but the
most ambitious students would be satisfied with. The overarching learning goal of the course
is to “get experience with SE project work in a team with a customer”, and postmortem
reflection is seen as important to the learning outcome. In 2008 it was decided to provide
support for this reflection by running a facilitated workshop with each team, to achieve active
participation by all team members. The workshops were obligatory and scheduled between
the final delivery and the oral project presentation.
The author of this paper was the workshop facilitator. She started the semester as course
coordinator, and after the course startup she continued only as researcher on the teams’ work.
At the outset of the workshop, the students were informed that the facilitator was not involved
in project evaluation, and that nothing said or written would be referred to outside the room,
except made anonymous and for research purposes. All teams accepted audio recording and
having pictures taken of the resulting work on the whiteboard. The photos were sent to each
team at the end of the workshop as a resource for their work on the reflection notes.
The workshop setting was as follows: The participants were sitting by a table in a
room with a large-size whiteboard. Each participant was provided with an A3 paper
form containing a timeline marked with some major project events (e.g. main
deliveries). On top of the sheet was a smiley face, and at the bottom a sad face (See
Figure 1). On the table, there were pens and whiteboard markers in different colors.
The workshop lasted 60 minutes and was divided into eight tasks (see Table 1). Task
1 is the introduction, in which background information is provided. Tasks 2-3
comprises individual and shared work to draw the project timeline. Individual timelines
are made on the A3 sheets and the shared timeline is drawn by the facilitator on the
whiteboard. Tasks 4-5 comprise individual drawing of experience curves on the A3
sheets and the drawing and presentation of the curves along the timeline on the
whiteboard (see Figure 1). With reference to Boud et al.’s reflection model, tasks 2-5
can be seen as a ‘Returning to experience’ reflection step. Tasks 4-5 explicitly prescribe
‘attending to feelings’. In tasks 6-7, the team is ‘re-evaluating experience’ by taking
different perspectives: those of tasks, roles and lessons learned. In this way, issues that
have emerged in tasks 2-5 are re-examined, and issues that have so far been missed may
be brought up. The wrap-up in task 8 encourages an overall perspective on the process
and can be used by the facilitator to have participants’ feedback on the workshop itself.
Task name
1 Intro
(5 min)
2 Individual
timelines
(5 min)
3 Shared
timeline
(15 min)
4 Individual
experience
curves
(5 min)
5 Present
curves
(10 min)
Description
Explain the purpose and agenda of the workshop. Clarify issues of confidentiality
and research
Each participant gets an A3 sheet of paper with a timeline reporting common
events in the course (mainly the deliveries). The participants are asked to
individually add events that they perceived had an impact on their project.
Participants take turn in explaining the events they have listed The facilitator
marks the events on the whiteboard on a timeline similar to the one on the
individual sheets.
The team members each draw their experience curve (or ‘satisfaction curve’) on
the A3 sheet. The smiley face on top of the sheet indicates a level of great
satisfaction. Down at the bottom is great dissatisfaction, and the timeline itself
marks a neutral position in the middle.
Each member in turn goes to the whiteboard, which holds the shared
timeline. The team member first draws her curve with her whiteboard
marker, next explains its shape. At the end of the session, all team
members’ experience curves can be found on the whiteboard.
87
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
165

6 Questions
about roles
& lessons
learned
(5 min)
7 Present
answers
(10 min)
8 Wrap-up
(5 min)
A sheet of incomplete statements addressing the project are uncovered,
and the students are asked to turn their A3 sheet and write their answers on
the blank page.
1) “In the project, my role was…” 2) “Through the project, I got better at…”
3) “In a similar project, I would like to become more skilled at…” 4) “The
most important thing I have learnt about software engineering in this project
is…”
The answers to the questions are presented around the table
The students may add further comments about their project on their sheet.
If formal evaluation of the workshop is not done on another occasion, this is
an opportunity to have feedback, oral or written. The A3 sheets are left for
the facilitator/course staff.
Table 1: Schedule of the postmortem workshop
Our data collection is based on various observation data from the workshops, and on
students’ course evaluation. The evaluation form, filled in by the students after the oral
presentation, included two items on the workshop. Our observation data include the workshop
audio recordings, some of which were summarized and partially transcribed. Excerpts
were translated into English at need. The data also includes pictures of the workshop
results on the whiteboard, and individual A3 sheets. The author’s knowledge of the course as
researcher and staff is a backdrop for interpretation of the data. The whiteboard pictures were
prepared for analysis and presentation using standard photo-editing software, and were
visually inspected and compared to look for observable patterns. Pictures were manipulated
with felt pen on printouts to make the curves more visible.
4. Findings
In this section, we present our findings. We have selected two teams, coined P and Q, to
provide some illustrative examples.
Figure 1: Timeline and individual experience curves (originally in different colors) for team P.
We have numbered the points along Max's curve where he explained the up- or downturn.
4.1. Perspectives on the same process are different among team members
Looking at the individual timelines within each team, we find that team members
mention partially overlapping, but markedly different sets of events. This applies to all
the teams in our study. Looking at the satisfaction curves within each team, we see that
no teams have completely congruent curves. The curve sets range from ‘more or less
88
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
166

congruent’ to sets that seem to originate in totally different projects. In 9 out of the 11
projects, the curves appear to reflect very different experiences within the team. In
Figure 1, for example, it can be seen that the student we have called Max has a curve
with many turns, reflecting shifting spirits throughout the project. Ellis’ mood seems to
have been fairly stable and positive. It can be seen from the curves that Ellis was the
only team member who had a good time in the period just before the midterm report
delivery (the timeline event by number 9). Sean and Max experienced a steep upturn in
connection with the delivery. Finlay took longer to reach above-average spirits.
We have numbered the points along Max’ explanation curve where he accounted for the
up- or downturns. To illustrate the richness of an explanation of an individual curve, we
provide Max’ explanation at the points 10, 11 and 12 (Figure 1):
10) ”Then there was a lot of frustration because of the team member who disappeared, and I
personally got completely p** off [..] that we could not get in touch and he didn’t gave a damn.
[…] We approached deadline, I felt that I was nagging like crazy and was rather stressed
because I feared we would not reach our goal with all that needed to get finished ”. 11) ”Then
we approached delivery and then I saw that this was going to go really well, and up went my
spirits.”. 12) ”Spirit-wise I ended up approximately on Finlay’s level. We were rather happy
when we had delivered.”
Excerpt 1: A part of Max’ explanation of his curve. Team P.
From this excerpt it can be noted that Max addresses his personal feelings about the
challenges of organizing the project, in this case the issue of a member who quit the project
without informing the team. Max also accounts for his own behavior at the time: he was
project manager (not formally, but in practice) and pushed the other team members towards
their deadline – which he perceived as ‘nagging’ and very stressful. He explains how the
subsequent success of the project work affected him, comparing his final level of spirit with
that of another team member. He also summarizes the team’s feeling at the end of the project.
4.2. Making a shared timeline is more than a mere adding of individual timelines
The construction of the shared timeline in task 3 often leads to discussion of details,
significance, sequence and timing. When an event occurred in more than one timeline,
discussion of the naming of the event often revealed discrepancies in the interpretation
of elements of the project process. In many teams, events that had not emerged in task 2
(drawing individual timelines) turned up in task 3. These new events might be given as
reasons for, or consequences of, other events.
The most important finding from the shared timeline construction is that the
facilitated ‘return to experience’ results in a shared representation different from the
sum of individual timelines. Some events might be added or canceled, others renamed.
In any case participants build common ground for reflection in the following tasks.
4.3. The satisfaction curves give new insights about the project
We have observed that the students glance at each other’s curves with interest in task
4. Visible differences between curves appear to trigger puzzlement and amusement. In
task 5, curve explanations in some cases trigger team members’ there-and-then response
and reflection. In team Q, Magda explains a low part of her curve. She says she had
been depressed about being assigned to a project using technology she already knew: “it
was only project writing and project work that I learned anything about.” Peter supports
and elaborates on Magda’s account, suggesting that it might be hard to be the only team
89
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
167

member skilled in the technology. He thus acknowledges the view that improving
technical skills is essential to a positive project experience.
4.4 Through the workshop tasks, different perspectives are taken on important issues
We see that the teams, by going through the 8 tasks, revisit issues of importance to
their project. In task 7, answers frequently can be seen as ‘re-evaluating experience’,
e.g. as major issues from the curve explanations are addressed in the lessons learned.
The examination of issues from different perspectives sometimes reveals tensions
and what appear as contradictions. These add to the picture of the dynamics and
development of the project and fuel reflection on the process. An example from team P
illustrates this. During task 5, Max explains that he was ‘nagging’ the team (Excerpt 1).
Later, Sean explains a downturn in his curve during the same period: “Max was away
for a week, and that was good for everyone. After that, it just went steadily upwards.“
During task 7, explaining about roles in the project, Finlay says that he and Max, as an
early bird and a B-person, respectively, had been quarreling a lot. Max says he had
unofficial roles like ‘A****le-Max’ and ‘Fusser-Max’. Answering what he had become
better at, Max says “accepting that people are different”, e.g. in terms of preferred work
hours. About lessons learned, Sean says: “Don’t make conflicts that can be avoided”,
explaining that sometimes, people had been “on the verge of psychotic outbursts”. His
comment leads to discussion and agreement in the team that they had never let the sun
go down on their quarrels, which were never personal. The facilitator refers to the
whiteboard, asking if it reflects who had been quarreling the most, Sean looks at the
curves (Figure 1), pauses, and says: “Yes, Finlay and Max” – which evokes laughter.
During task 8, the team summarizes: Max: “ was awesome, really.”
Finlay: “We reached our goal.” [..] Max:”It would have been worse if we had had a
team member that we did not get along so well with” [..] ”We are largely similar, the
whole bunch.” Ellis:”I personally could not have had it better with the team really.”
4.5 Students perceive the workshop as useful
Students’ perception of the insights gained through the workshop can be read from
the course evaluation, in which agreement to the following two statements was rated: 1)
“The postmortem workshop gave me insights about the project.” 2) “In another project
in the future, I would suggest the use of timeline and experience curves to help the team
reflect on the process”. 44 out of 46 students handed in the form, giving a 96% response
rate. To statement 1, 2% of the students responded “strongly disagree” or “disagree”,
25% responded “neither agree nor disagree”, and 73% responded “agree” or “strongly
agree”. To statement 2, the respective percentages were 2%, 39% and 59%. From this,
we may conclude that the students generally considered the postmortem approach
useful to their project and worthwhile applying in another project.
5. Discussion
In this section, we discuss the relevance of our postmortem approach to SE education
and the adequacy and possible improvements of the specific workshop design.
5.1 Relevance to SE education
Our findings suggest that the timeline and experience curve technique is adequate for
postmortem reflection in SE student teams. Workshop observation indicated that the aspects
of reflection outlined by Boud et al. are generally supported. The individual perspectives of
90
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
168

team members and the co-constructive effort of making a shared timeline results in a ‘return
to experience’ richer than would have been possible for an individual, reflecting team
member alone. Feelings are attended to in the drawing of satisfaction curves. Experience is
re-evaluated through the revisiting of issues in the different workshop tasks, particularly the
last ones. Further, the students deemed the workshop approach useful for later projects.
Turning from student’s learning to organizational learning, three findings address the
potential for course staff to use the workshops to learn about the course:
First, students’ discussions provide insights about the effect of the course design. I.e.,
the example from team Q in Section 4.3 points to different stakeholders having
different project objectives [11]. Magda primarily wants to use the project to improve
her technical skills, while the formal course objective is to have students get experience
with project work for a customer, with a focus on the organization of teamwork and
management of stakeholder relationships. Such insights on students’ objectives and
motivation may be used to better align course objectives and actual learning outcome,
e.g. by presenting the course differently or changing incentives or course requirements.
Second, in our data, no patterns can be found relating curve sets to project grades.
While data from a much larger number of teams might have revealed some significant
correlation, a team’s curve set cannot be a diagnostic tool in project evaluation. The
curves reveal only part of a complex reality. Our curve sets may however be seen as
disproving possible preconceptions about successful collaboration. For instance, a
project receiving an A might have discrepant (Figure 1) or largely congruent curves
(another team in our data). Rich information is needed to understand the connection
between experience curves and project result, e.g. why discrepancies might represent
‘fruitful dynamics’ rather than subversive conflicts (cf. Section 4.4).
Third, it is possible to find patterns relating satisfaction curves to elements of the course
design. These patterns can be interpreted in light of students’ explanations of their curves. We
found two patterns in our data: a) Many students account for what can be denoted ‘report
dips’: down-periods before report deliveries, associated with stress and comments like
‘writing report is simply boring’. b) Feedback from the supervisor (e.g. report versions)
has great impact on the team members’ experience. For instance, one of our
supervisors, known to be ‘strict’, had been giving feedback making students in several
teams very disappointed and even demotivated for some time. This did however not
correlate with a negative grade. Patterns thus relating course events to individual
experience – whether revealing surprising connections or providing empirical evidence of
prior assumptions - can be used as a resource for improvements to the course design.
We see a potential for course staff to gain insights about the course from the workshops,
but students’ reflection and learning might suffer if an additional agenda is introduced.
Recommendation 1: If the workshop is used to learn about the course, agree with students that
data will be anonymous and used for improvement of next year’s course. Recommendation 2:
Make sure, and make students aware, that the workshop facilitator is not grading the project.
5.1 Suitability of the proposed workshop structure
Observations indicate that the workshop tasks were easily understood and willingly
performed. There were few questions, seemingly no misunderstandings and no apparent
reluctance to draw and explain curves on the whiteboard. All turns were explained and
linked to events, and the presenter was rarely interrupted. Explanations were generally
given in a straightforward way, and the audience seemed sensitive to the information.
91
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
169

Having the facilitator, and not the students, write down events on the timeline in task
3 was time-efficient, but requires sensitiveness: the students should not be overrun. A
facilitator in charge of the timeline gets an opportunity to support the reconstruction of
a shared timeline, e.g. when two students name the same event differently.
Recommendation 3: use the proposed sequence of workshop steps. Shifting between
individual and collective effort furthers active participation and multiple perspectives.
We see that some teams might have benefited from a longer discussion. Need for
more time typically increases with the size of the team. Recommendation 4: Use 90
minute time slots, to have more flexibility to adjust to the specific team.
In terms of feasibility for project courses of different sizes, the approach has a per-project
cost of one staff hour plus preparation. Cost of preparation is likely to decrease with the
number of teams facilitated by each staff member. While our workshop design is based on an
agile development approach, it is adequate for other approaches. Recommendation 5: Use the
reflection workshop as part of agile or more structured development approaches. Finally:
Recommendation 6: If the workshop is to be conducted during the course of a project and not
only at its end, the workshop should focus on measures for change, e.g. process improvement.
6. Conclusion and further work
We have proposed a design for postmortem workshops in SE student teams, based on
industry best practice and a theoretical framework accounting for the reflection process.
Based on its implementation in a SE course, we recommend our design, with additional
recommendations made in Section 5. In our continued research and course development
we will introduce a postmortem workshop in the middle of the projects, using a similar
design with stronger focus on process improvement and specific change measures.
7. Acknowledgements
Thanks to the NTNU project MOTUS2, the SE students, Torgeir Dingsøyr,and John Krogstie.
7. References
[1] T. Dingsøyr, "Postmortem reviews: purpose and approaches in software engineering," Information and
Software Technology, vol. 47, pp. 293-303, 2005.
[2] V. Kasi, M. Keil, L. Mathiassen, and K. Pedersen, "The post mortem paradox: a Delphi study of IT specialist
perceptions," European Journal of IS, vol. 17, pp. 62-78, 2008.
[3] A. Strauss, Continual permutations of action. New York: Aldine de Gruyter, 1993.
[4] P. Berger and T. Luckmann, The Social Construction of Reality. London: Penguin Books, 1966.
[5] G. H. Mead, Mind, Self and Society. Chicago: The University of Chicago Press, 1934.
[6] H. H. Clark and S. A. Brennan: Grounding in communication. In L.B. Resnick, J.M. Levine, & S.D. Teasley
(Eds.). Perspectives on socially shared cognition . Washington: APA Books, 1991
[7] J. Dewey, How we think. A restatement of the relation of reflective thinking to the educative process (Revised
edtn.) ed. Boston: D. C. Heath, 1933.
[8] D. Boud, R. Keogh, and D. Walker, Reflection: Turning Experience into Learning: RoutledgeFalmer, 1985.
[9] E. Derby, D. Larsen, and K. Schwaber, Agile Retrospectives.: Pragmatic Bookshelf, 2006.
[10] N. B. Moe and T. Dingsøyr, "Agile development and teamwork: A case study of a Scrum team and teamwork
organization," Work in progress, 2008.
[11] B. Krogstie and B. Bygstad, "Cross-Community Collaboration and Learning in Customer-Driven Software
Engineering Student Projects," in CSEE&T, Dublin, 2007, pp. 336-343.
92
Authorized licensed use limited to: Norges Teknisk-Naturvitenskapelige Universitet. Downloaded on February 5, 2010 at 09:01 from IEEE Xplore. Restrictions apply.
170

Research paper P7
Title:
A Model of Retrospective Reflection in Project Based Learning Utilizing Historical
Data in Collaborative Tools
Author:
Birgit Krogstie
Published in:
Proceedings of EC-TEL 2009
Pages:
15
Complete reference:
Krogstie, B. R. (2009). A model of retrospective reflection in project based learning
utilizing historical data in collaborative tools. European Conference on Technology
Enhanced Learning (EC-TEL) 2009, Nice, France, 29 Sept – 2 Oct. Springer.
171

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
172

A Model of Retrospective Reflection in Project Based
Learning Utilizing Historical Data in Collaborative Tools
Birgit R. Krogstie
Norwegian University of Science and Technology (NTNU)
birgitkr@gmail.com
Abstract. In project based learning, learning from experience is vital and necessitates
reflection. Retrospective reflection is as a conscious, collaborative effort
to systematically re-examine a process in order to learn from it. In software
development student projects it has been empirically shown that project teams’
retrospective reflection can help the teams collaboratively construct new
knowledge about their process and that historical data in collaborative tools
used in daily project work can aid the teams’ recall and reflection on the different
aspects of project work. In this paper, we draw on these results as well as
other findings on the use of collaborative tools in a similar setting. We use the
framework of distributed cognition to develop a model of retrospective reflection
in which collaborative tools used as cognitive tools for daily project work
are utilized as cognitive tools in retrospective reflection, aiding the creation of
individual and shared representations of the project process.
Keywords: project based learning, reflection, distributed cognition, cognitive
tools.
1 Introduction
In project based learning [1] learning from experience is essential [2-4]. To achieve
this learning, reflection is necessary, both during day-to-day work and with some
distance to the activity reflected upon [4]. In this paper, we will focus on what we will
call retrospective reflection, a form of reflection-on-action in which participants in a
collaborative process systematically re-examine their process.
There exist many techniques and tools to support reflection in project based learning.
Examples include the writing of diaries and reflection notes, and user annotation
of information in the collaborative tools used to support their work. Computerized
tools can be specifically introduced to support learning. Reflection supporting tools
comprise tools showing users their learning process and its result, tools providing
guidance for users’ monitoring of their learning process, and tools providing scaffolding
for comparison with expert thinking [5]. In an organizational setting, advanced
collaboration platforms may include knowledge management functionality [6], but
such tools are untypical in formal education.
The project based learning to be focused in this paper is that which is based on project
work involving the development of artifacts and aided by lightweight collaborative
U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 418–432, 2009.
© Springer-Verlag Berlin Heidelberg 2009
173

A Model of Retrospective Reflection in Project Based Learning 419
tools. The latter includes tools frequently associated with Web 2.0, e.g. wikis, instant
messaging tools, and discussion forums. Today’s students regularly use such tools to
coordinate and perform their activities, whether imposed by school or not [7, 8]. Adding
to the picture, students in higher education generally expect to have flexibility of
time and place for work. From the point of view of course organizers, lightweight tools
are inexpensive options providing flexibility of use and functionality well known to
students and course staff.
A core argument in what follows is that by supporting various aspects of work
throughout a project, lightweight tools collect historical data with a potential value to
help the students recall what happened in the project and reflect on it.
The objective of the research in this paper is to provide a general account of retrospective
reflection in the above described type of project based learning setting. We
draw on empirical research on software development (SD) student projects: mainly
published work [9-11] and some additional findings. We provide a substantial new
contribution by situating the results in a theoretical framework, generalizing to project
based learning independent of work domain and presenting a model which can be
used to aid the organization of retrospective reflection in project based learning.
In Section 2, we outline the empirical research on SD student projects. We present
our theoretical framework in Section 3, and in Section 4 use it to analyze findings
from the student projects. Section 5 outlines our conceptual model of retrospective
reflection. Section 6 concludes the paper.
2 Empirical Basis: Research on SD Student Projects
In this paper we draw on findings from research on SD student project teams in the
period 2006-2008 focusing on the teams’ use of collaboration technology. The project
course is arranged in the last (6th) semester of an IT undergraduate program. Teams
of 3-5 members develop software for genuine customers. The main learning objective
is to get experience with SD work in a team for a customer. Project deliveries include
a software product and a report. The teams choose which collaboration and development
tools to use, depending on customer requirements, team members’ prior experience,
team members’ wish to learn, and the availability of the technology.
The data collection in the studies on these teams includes in-depth, longitudinal
non-participant observation, semi-structured interviews across the cohorts, and examination
of project documentation. A constructivist view of learning as well as
guidelines for interpretive field research [12] guided the data collection and analysis.
2.1 Retrospective Reflection Based on Participants’ Memory
The adoption of a retrospective reflection technique from SD industry [13] in the SD
project course was subject to a qualitative study [10]. Facilitated reflection workshops
were conducted with 10 student teams at the end of their project. Each workshop
lasted about an hour and started with participants’ individually drawing project timelines
incorporating events they perceived as important to the project, and along the
timelines, ‘satisfaction curves’ indicating their feelings about the project over
time. From the individual timelines, the teams constructed shared timelines on a
174

420 B.R. Krogstie
Fig. 1. Shared timeline from a reflection workshop (One event emphasized by the authors)
whiteboard. Individual satisfaction curves were drawn along it (see Fig. 1) and explained
by the participant. Next, the participants individually answered questions
about tasks, roles and lessons learned and presented their answers. After the workshop,
the teams made reflection notes.
The study showed that individual timelines and satisfaction curves reflected different
perspectives on a project process. Fig. 1 illustrates how experience curves may
differ within a team. The study also showed that shared timelines often reflected
views of the project not found in any individual timeline. Closer examination of the
individual timelines used in the creation of the shared timeline in Fig. 1 revealed that
most of the events from the individual timelines had been included in the shared one,
and some had been transformed through the co-constructive effort. For instance, the
event ‘a bit ineffective work’ in an individual timeline was modified into an event
marking a point in the project process when the team realized that they had to work
more efficiently (‘insight: need to work more efficiently’). The study [10] concludes
that the satisfaction curves gave the students new insights, that the workshop helped
them take new perspectives on important issues, and that they considered it useful.
2.2 Retrospective Reflection Aided by Historical Data in Collaborative Tools
In SD industry it is acknowledged [14] that project retrospectives would benefit from
better data to help participants create a shared understanding while avoiding oversimplification
and time-consuming examination of unimportant information. This was
addressed in a study of SD student projects in which historical data in project wikis,
used as lightweight project management tools by several teams [15], were used to aid
project participants’ reflection on their project process [9]. In retrospective interviews
with project members, wiki contents were chronologically examined, and particular
types of information was seen to trigger recall of and reflection on project events,
project phases, and collaboration within the team and with other stakeholders.
In [11] it was shown that historical data in an issue tracking tool, a lightweight tool
for SD project management, was useful to aid retrospective reflection. Historical
data was accessed by traversal of a timeline showing team members’ updates to development
artifacts. The reflection effort was organized in line with the approach of
175

A Model of Retrospective Reflection in Project Based Learning 421
creating timelines and satisfaction curves [10], but conducted as a two day, video
recorded workshop with one team to investigate in depth the use and role of historical
data in the reflection process. Examination of the historical data helped team members
individually identify project events that had not been included in the timelines made
from (individual) memory alone and that were later included in the collaboratively
constructed timeline and accounts of lessons learned. Historical data in the issue
tracker were also used by the team to adjust details in the shared timeline.
Fig. 2. Historical data in a collaborative tool aiding recall of a project event [11]
Some features for navigation and retrieval of historical data were found to be important
in retrospective reflection [9, 11]. These features included chronological
overview and traversal, and the possibility to switch between overview and details.
Interviews with the 2007-2008 teams and examination of their reflection notes
showed that the collaborative tools used were generally lightweight. The teams were
clear about what tools were used for what purpose. While the specific tool selection
and usage differ among teams, we see a general pattern:
Lightweight project management tools (typically project wikis or issue tracking
tools) are used for managing team-internal coordination of tasks, e.g. create and follow
up on a project plan, and define, assign and track the status of tasks. The tool
provides links to project documentation. Development tools are used to write, test and
integrate source code. The storage and versioning of project artifacts are managed in a
file versioning system that may or may not be integrated with the project management
tool. Email is used for formal and documented communication internally and in
communication with other stakeholders. Typically, the project team has its own mailing
list. Instant messaging chat is used for informal, team-internal messages and substitute
face-to-face conversation over synchronous, distributed work. Less often, it is
used for communication with other stakeholders. Internet sites are used to get
176

422 B.R. Krogstie
information about technology; in most cases, simple web search or FAQ lists provide
answers, but occasionally project members participate in discussion forums [16].
These patterns of use, combined with the functionality of tools, imply that data resulting
from various types of project work is generally logged. Wiki revisions are
automatically stored and the email clients store all mail unless otherwise specified by
the user. It is tacitly expected that an email user keeps an archive of work-related
email. With instant messaging, team members frequently choose to enable logging.
On an internet forum, postings remain as long as the community hosting the forum
wants to keep them. Looking into the historical data stored in these tools, they can be
seen as a trace of the work undertaken with the aid of the tools. The studies investigating
reflection aided by historical data [9, 11] showed that data in tools used for project
management and coordination reminded participants of events related to those aspects
of project work and thus helped them reflect on that type of issues.
To examine the potential for historical data to shed light on other types of issues,
we started out by issues that, according to the teams’ own reflection. had been of great
importance in their projects (e.g. misunderstandings in team-customer collaboration,
problems of getting timely information from a service provider). Examining historical
data in collaborative tools used by those teams, including email archives, instant messaging
logs, and discussion forums, we looked for historical data that could shed light
on those issues The data found were, as seen by the researcher, rich enough to have
such a potential, partially because the data shed light on the use of the collaborative
tool itself, which was often at the core of the project challenge.
We end this section by outlining some more characteristics of the SD projects relevant
to our agenda of understanding and supporting retrospective reflection in the
teams. Work in the projects is typically diversified, project participants having different
roles, dividing work and using different tools to address different tasks. Team
members’ roles affect their day-to-day use of collaborative tools. Consequently, historical
data in a tool typically reflects work in which some team members have been
more involved than others. Project artifacts (e.g. requirements specifications and project
plans) frequently play a role in collaboration with project stakeholders (e.g.
customer [17] and course staff) having different goals for their project involvement.
Project artifacts in various states can be seen as more or less well defined versions.
These may be deliberately saved by the user (as when a text document is renamed and
saved) or automatically stored in a tool. A file versioning system stores the contents
of and differences between every file version ‘checked in’ by the users. In our studies
of retrospective reflection aided by historical data in collaborative tools, going into
detail often meant exploring specific artifact versions.
3 Theoretical Background
Taking the view of constructivism, seeing learning as integral to activity that is basically
social and situated, and focusing on the role of tools, several theories [18, 19]
may shed light on project based learning. They include activity theory (AT), actor
network theory (ANT), symbolic interactionism, situated learning, and distributed
cognition [20, 21]. In [11] it was shown that distributed cognition is an adequate
framework for understanding retrospective reflection in SD student projects. It has
177

A Model of Retrospective Reflection in Project Based Learning 423
been used to analyse learning in educational [22-24] as well as work [25, 26] settings.
In the present study, the main reason for choosing distributed cognition among the
candidate frameworks is its focus on transformation between representations. Such
transformations can be seen as a core element in a process of retrospective reflection
incorporating construction of timelines and examination of historical data. The concept
of cognitive tools [23, 27] can shed light on how such representations aid work
and learning. Selecting distributed cognition as a framework we focus on its descriptive
power and what we want to achieve by applying it [18].
We want to develop a model which not only descriptively accounts for the elements
and dynamics of retrospective reflection in project based learning but which
also informs its organization. To this end we augment our theoretical framework with
theory addressing how the process of learning and reflection may be facilitated.
Reflection can be seen as active and careful consideration of any belief or supposed
form of knowledge [28], implying the reviewing and judging of present knowledge or
beliefs [29]. A model of the learning process linking reflection to the experience reflected
on in a reciprocal relationship is provided by Kolb and Fry [30]. Boud et al.
[31] present a model of the reflective process in which the returning to experience is
essential. The experience comprises the behaviour, ideas and feelings involved in the
situation reflected upon. In the reflective process, feelings about the experience
should be attended to and the experience re-evaluated. The desired outcomes of reflection
include new perspectives on the experiences, change in behaviour, readiness
for application and commitment to action. The model is descriptive with respect to
steps typically occurring in reflection but is also meant to provide guidance about how
to achieve successful reflection.
Project work experience can be understood in terms of the project trajectory [32], a
concept in line with the idea of temporarily distributed cognition. Strauss refers to
Mead on the issue of how a social world reflects on its history: “The temporal spans
of group life mean that the aims and aspirations of group endeavor are subject to reviewal
and recasting. Likewise past activities come to be viewed in new lights,
through reappraisal and selective recollection. ..History, whether that of a single person
or of a group, signifies a ‘coming back at self’ (Mead 1936)”. By ‘trajectory’
Strauss means “the course of any experienced phenomenon as it evolves over time”.
Distributed cognition “implies that the learners are enabled to think deeply and
create certain types of artifacts that represent their thinking by working with cognitive
tools” [23] p.209. Cognitive tools are tools that “help users represent what they know
as they transform information into knowledge and are used to engage in, and facilitate,
critical thinking and higher order learning.” [27]
Stahl [33] explicitly addresses the collaborative aspect of learning in his model of
collaborative knowledge building, “a cyclical process with no beginning or end”
(p.75). At the heart of the model are processes of building personal and collaborative
knowing. The model comprises the transformation of cultural and cognitive artifacts
and sheds light on the interplay of individual and collective reflection and learning.
Our aim is to develop model that sheds light on the elements and dynamics of retrospective
reflection in the particular setting of project based learning in which lightweight
collaborative tools are used to support work and retrospective reflection. We
want the model to account for the return to experience in the light of project trajectory
178

424 B.R. Krogstie
and to express how work and learning involves individual as well as collective reflection
and the use of cognitive tools.
We clarify our use of some concepts: Activity is used as a generic, commonsense
term and not as a reference to activity theory. By project artifact we mean something
used and produced in project work, e.g. a diagram or a report. In distinguishing between
work and the retrospective reflection on that work, we deliberately ‘hide’ the
reflection and learning in day-to-day project work inside the concept of project work.
This is not to pretend that project based learning only happens in ‘chunks’ of retrospective
reflection, but it is our agenda to shed light on the latter.
We proceed with an analysis of retrospective reflection on the above theoretical
basis, from the empirical grounding described in Section 2 and with the objective of
developing a model. With a sidelong glance at the work on organizational memory in
[25], we structure our analysis in terms of retrospective reflection being socially distributed,
temporally distributed, and involving transformation of representations.
4 Analysis
We use findings from the research on SD teams outlined in Section 2 to shed light on
how retrospective reflection may be supported in similar settings of project based
learning from the perspective of distributed cognition.
4.1 The Social Distribution of Cognition in Retrospective Reflection
The social distribution of the cognition involved in retrospective reflection can be
seen as having two main components: the social distribution of the process reflected
upon, and that of the retrospective reflection activity itself. The social distribution of
the process reflected upon in the case of SD student projects was largely described in
Section 2 and illustrates the complexity of the experience to be returned to in retrospective
reflection We noted that tasks relating to different aspects of work were
distributed in the teams, resulting in a distribution of tool use. Historical data were
generally being stored in the tools as a result of the work. The data reflected aspects
of project work in which the tool has played a role, including the tool use itself. E.g.,
in Fig. 2, the historical data reflects software development work, more specifically
coding. Crucially, in our context, a new version of a project artifact stored in the tool
represents different data than the previous version, this distinction serving to capture
the temporal and partially also the social distribution of the project work.
We now take a closer look at the social distribution of the retrospective reflection
with reference to the SD students teams. The timelines in [10] (see Fig.1) were drawn
with the aid of simple physical tools; paper and pencil, whiteboard and pens. These
tools had a flexibility in the situation of knowledge sharing appearing to make them
adequate for externalizing and transforming participants’ representations.
When historical data in collaborative tools were introduced in similarly organized
retrospective reflection [11], particular tool features for retrieving and navigating data
were found to be important to the utility of the tool. For instance, the view in Fig.2
allows easy chronological traversal with direct access to project artifacts in there-andthen
versions. The likelihood of being able to retrieve interesting data from a specific
tool also depends on the actual tool usage in the team’s work. Further, what is
179

A Model of Retrospective Reflection in Project Based Learning 425
worthwhile examining retrospectively depends on what the team considers important
issues. These might have been identified prior to retrospective reflection, but may also
emerge from the examination of the historical data as seen in [11] and Fig.2.
Synchronous, distributed work among pairs of SD team members is frequently
supported by instant messaging chat. Such conversation has an oral flavor, often
combining there-and-then problem resolution with social chit-chat. Instant messaging
logs exemplify historical data that for privacy reasons may be inadequate for retrospective
reflection even if the contents are of potential interest to the team.
The social distribution of retrospective reflection is also about which participants
are involved, e.g. whether reflection is done individually or by the whole team
together. In the SD teams, recall of events was essential in the construction of individual
and shared timelines [10, 11]. Research on transactive memory, “a set of individual
memory systems in combination with the communication that takes place between
individuals” [34], shows that in collaborative remembering, comparing groups
of people who have a history of remembering together (e.g. as colleagues or as family
members) with groups people who have not, those who are used to repeatedly remembering
together have the most efficient systems for doing so [35, 36]. In close
relationships, responsibility for knowing is distributed, with the effect that some information
is differentiated and non-redundant whereas other is shared. However,
research indicates that the collective remembering skills that a group of people has
developed through experience are only an advantage in a situation of remembering if
the group is allowed to use these skills [37]. Translated to the context of project based
learning, we may take these results to indicate that recall of project events benefits if
project members can draw on the collaboration expertise developed through their
project work. The retrospective use of collaborative tools well known from day-today
work may be part of this.
Findings from the research field of social contagion of memory indicate that there
is a tendency for participants in collaborative remembering to be influenced by others
in the remembering process, individual memory thus being distorted [38, 39]. This
research is however based on experiments far from real-life work. Recent research on
memory of events that are real, complex and significant in people’s lives show that
collective memory is qualitatively different from individual memory in these settings
[40, 41]. In project based learning, the inclusion of individual as well as collective
steps of recall and reflection may serve as a way of exploiting the positive effects of
recalling alone as well as those of doing it in concert.
The empirical research on SD student projects outlined in Section 2 indicated that
creating a shared representation based on individual ones results in more knowledge
than that found in the individual, external representations [10]. However, those who
had the most expertise with a certain aspect of work tended to dominate the team’s
shared, externalized view of that aspect of work [11]. Research stating that expertise
is a combination of the expert, his environment and the tools he uses [23] underpin
these findings. Apart from being a source of power differences, however, expertise is
an important resource for using collaborative tools as cognitive tools for reflection.
We conclude from this section that the historical data in collaborative tools are important
resources in unveiling different aspects of project work involving different
team members, although the value of using a particular tool in a specific case must be
considered. A combination of individual and collective reflection seems feasible.
180

426 B.R. Krogstie
4.2 The Temporal Distribution of Cognition in Retrospective Reflection
Retrospective reflection in project based learning is part of a trajectory of project
based learning spanning the entire project, but also constitutes a separate sub-activity
within the overall project. The distributed cognition of retrospective reflection can
thus be seen as situated in two different contexts with different objectives for the
ongoing activity: There-and-then project work focused on completing project tasks,
and here-and-now retrospective reflection focused on making sense of there-and-then
work in the light of knowledge of the entire process.
Events of there-and-then work may be interpreted as belonging to different subtrajectories,
and can be seen as the core of what is reflected upon. We illustrate this by
going into some detail about the findings reported in [11] and illustrated in Fig.2. In the
retrospective reflection workshop of the team in question, there was a project event not
remembered by any team member in the individual reconstruction of the project trajectory
based on memory alone, i.e. not included in any individual timeline. The event
marked the onset of an activity in the project, more specifically coding work done to get
familiar with technology to be used later in the real development. As such, the event
could be seen both as a turning point in the trajectory of the entire development activity
and as the starting point of a sub-trajectory: that of the early-investigation of technology.
The team members’ individual examination of historical data in the issue tracking
tool made two of them recall the event as important to the project (Fig.2) and include
it in their timelines. Later, as the team created a shared project timeline, the event was
included and brought up into the discussion of how things might have been done
differently in the project. The event was discussed in the light of the specific activity
for which it marked the starting point and in the light of the entire development project.
The trajectory of the project as represented in the shared timeline was compared
to a process described in SD literature as ideal for a certain type of development
work. These findings illustrate that comparing trajectories is at the core of the
retrospective reflection process.
Considering retrospective reflection as part of a trajectory of project based learning
we see a possible effect on tool usage if it is known to the team that historical data
will later on be systematically examined. The dual role of the tool may lead project
members to adjust their day-to-day tool usage, e.g. by providing more frequent or
elaborate comments associated with ongoing tasks, which may affect project work in
a positive way. However, changes may have adverse effects, e.g. if participants cease
to communicate issues in a tool because the logged data may give an unfavourable
impression in retrospect. Adjusting tool use to meet the needs of both retrospective
reflection and day-to-day work may be seen as part of project based learning itself.
We conclude from this section that the ‘return to experience’ of retrospective reflection
involves examination and comparison of sub-trajectories of project work and
that retrospective use of tools may affect project work through participants’ awareness
of the dual role of the tools.
4.3 Transformations of Representations and the Use of Cognitive Tools
Looking at the timeline technique as used in [10], and illustrated in Fig.1 we see many
examples of use of representations as cognitive tools. The internal, individual
representations of the project process were transformed into externalized individual
181

A Model of Retrospective Reflection in Project Based Learning 427
timelines on paper. Both types of representations were used in the collaborative session
through which the knowledge captured in the representations was transformed
into a shared, external representation in the form of a timeline, a process likely to
make the individual, internal representations change. The internal and external representations
created and modified through retrospective reflection served as cognitive
tools for the teams’ later writing of collaborative reflection notes. The experience
curve helped participants reflect on a particular sub-trajectory corresponding to the
‘attending to feelings’ in the reflective process [31].
The above outlined transformations were achieved by use of the timeline as a cognitive
tool. The successful use of timelines in the study indicates that it is a good form
of representation to aid both individual and collective recall and reflection.
Salomon [24] argues that “even in the most radical formulations of activity-insetting
[..] there is no way to get around the role played by individuals’ representations.”
p.134. The timeline approach as conducted in the SD studies pays heed to this
by including the steps of creating external, individual representations.
We see the following as favourable characteristics of the timeline: the metaphor is
easily grasped by participants, it maps well to the conception of process as trajectory,
it is easily understood when presented to others, and different timelines representing
the same process are easily combined. The timeline provides a framework for contextualizing
historical data in collaborative tools, in which the timing and chronological
sequence of events associated with the data are generally available.
In the collaborative tools used in project work, some of the data originating in the
work and stored in the tools are used as cognitive tools in the work process, thus contributing
to the transformation of participants’ internal project representations..
Turning to the step of retrospective reflection, the same data may partially be accessed
in the same way as in day-to-day work (as when previous events in the
timeline of the issue tracker was examined; this is a tool use similar to day-to-day
coordination work) but may also be retrieved in a way not typically used in day-today
work (as when early revisions of the project wiki main page were chronologically
examined in retrospective reflection). Thus, the representations of the project
trajectory retrieved from the tool vary with the purpose and procedure of retrieval.
When collaborative tools are used as cognitive tools for retrospective reflection,
externalized representations originally transformed through day-to-day project work
take part in the transformation of other representations of the project trajectory;
internal and possibly external ones (e.g. timelines), individual and/or shared ones.
In the studies outlined in Section 2, tools used to aid retrospective reflection were
of two types: those created for the purpose of retrospective reflection, i.e. the timelines
and curves on paper and whiteboard, and tools primarily supporting day-to-day
project work and being re-used for the purpose of retrospective reflection, e.g. the
project wiki and the issue tracker. In case of the latter category of tools, even if the
process of accessing historical data had some transformative effect on the representations
retrieved, the historical data in the tool remained fixed. The timelines and curves
created for the purpose of retrospective reflection, on the other hand, underwent
continuous transformation during the retrospective reflection activity.
Whereas the empirical studies of SD projects indicate that the two types of tools
independently benefit retrospective reflection [9, 10], there appears to be added value
182

428 B.R. Krogstie
in combining them [11], using the historical data as a means for transforming the
timeline and the timeline as a means for making sense of the historical data.
By (re-)using a computerized tool that has been used in one context, e.g. that of
(some aspect of) day-to-day project work, and employing it as a cognitive tool for
retrospective reflection, some of the expertise inherent in the learner’s there-and-then
usage of the tool may be applicable to the tool use in retrospective reflection. This can
be understood as utilizing the expertise of the joint learning system [23] of day-to-day
work in the transition to retrospective reflection on that work.
There are limits to the insight that can be gained by the use of one type of representation
of a project, e.g. a timeline. This is the case even if the representation is made
more sophisticated (for instance by including representation of sub-trajectories like
the individual experience curves). Not all aspects of project work fit within a temporal/linear
perspective, even if it is possible to revisit most issues along a timeline.
There may be aspects of project work that are better expressed in other ways, e.g.
with textual descriptions, diagrams or even role play. For instance, a representation
outlining the structure of an artefact, showing who contributed to what part, may be
useful to aid reflection on the work process. Some representations providing synthesized
information about the project are likely to be available in the historical data of
collaborative tools; project plans are an example. The timeline should be seen not
only as a valuable project representation in itself, but also as a good starting point for
identifying and creating other representations by helping participants get an overview
and create a context for exploration of particular issues.
In the studies outlined in Section 2, retrospective reflection was conducted when
the projects were more or less finished. Taken from the learning objectives of the
course, the students were meant to be able to use their experience from the project in
other projects, but implicitly this was taken to happen through individual learning.
Revisiting the model of the reflective process [31], its outcomes comprise new perspectives
on experiences, change in behaviour, readiness for application, and commitment
to action. We would expect a successful process of retrospective reflection to
result in all of these being somehow incorporated in participants’ internal representations
of the project experience, and some of it to be expressed in the external representations
resulting from reflection. However, the actual learning from experience is
best seen in further project work, and retrospective reflection should be an elements
of a learning cycle [30]. If process improvement is part of the purpose of retrospective
reflection, lessons learned should be captured in representations that are applicable in
later project work. How to make lessons learned applicable in practice is a challenging
issue at the core of organizational learning and knowledge management.
Our conclusion from this section is that many types of transformations of representations
may be involved in, and add value to, retrospective reflection. The transformations
we have discussed were achieved by the use of tools primarily supporting
day-to-day work as well as tools introduced for retrospective reflection.
5 Retrospective Reflection in Project Based Learning: A Model
Based on the previous analysis, we outline a model of retrospective reflection in
project based learning and briefly illustrate with examples from Section 2.
183

A Model of Retrospective Reflection in Project Based Learning 429
The model in Fig. 3 illustrates retrospective reflection incorporating the main elements
elaborated in Section 4, and serves as a generic description of retrospective
reflection utilizing individual and shared timeline representations as well as historical
data in various collaborative tools used in the project. The rectangles in the diagram
are representations (internal or external), the ovals are processes in which the representations
are transformed. These processes can be seen as learning processes in the
sense that new knowledge is constructed. A representation having an outgoing arrow
pointing to a process is a cognitive tool in that process. Where arrows point in both
directions between a representation and a learning process, the representation used in
the process is itself changed through the process. The finer, dashed arrows show the
use of historical data to identify events and sub-trajectories.
In more detail, process (1) is the daily project work. Collaborative tools are used as
cognitive tools, resulting in data remaining in the tools as historical data. The tools in
daily work can be seen as continuously updated ‘representations’ of the project by
containing data reflecting aspects of the project work. Internal representations of the
project in participants’ heads also impact on, and result from, the work.
Turning to the retrospective reflection, there is an individual step (2) in which individuals
make an external representation of the project process in the form of a timeline.
We have indicated the possibility to represent sub-trajectories (e.g. a satisfaction
curve). In this, the collaborative tools with their historical data potentially serve as
cognitive tools, as illustrated in Fig.2. In making sense of the historical data, the
learner recognizes them as associated with project events and sub-trajectories.
Participants’ individual timelines are cognitive tools for collaborative reflection (3)
in which a shared timeline is created (Fig. 1), mediated also by the individual, internal
representations. Elements from the individual timelines are included and may be
transformed (see Fig.1, the emphasized event). Again, the collaborative tools with
their historical data may be used as cognitive tools. In addition to the shared timeline,
the team may create other representations (e.g. textual reflection notes). Some of the
resulting representations may serve as cognitive tools in further project work, closing
the cycle of project based learning.
The model is a simplification with respect to the number of representations and
transformations. For instance, the step of orally presenting the contents of individual
timelines to the team and the transformation of historical data that happens through
retrospective use of the tool has been only indirectly assumed.
The model outlines main elements in the distributed cognition of retrospective reflection
and the dynamics among these elements. It can be used to guide the organization
of retrospective reflection in project-based learning. While our analysis addressed
the benefit of various representations and transformative processes, retrospective
reflection may be arranged without the inclusion of all the elements. Depending on
constraints and objectives, a specific organization of the process may leave out the
individual or the collective step of creating external timeline representations. Also, the
use of collaborative tool history can be omitted. To decide whether a specific collaborative
tool should be used in retrospective reflection, a team should take into account
the tool features for data navigation and retrieval, tool usage in daily work, and issues
seen as important to the team and thus worth reflecting on.
184

430 B.R. Krogstie
Fig. 3. A model of retrospective reflection in project based learning
6 Conclusion
We have used empirical data from SD student projects to analyse retrospective reflection
in the context of project based learning and develop a model outlining its
elements and dynamics. On basis of distributed cognition, the model describes retrospective
reflection as a set of transformations of representations of the project process,
internal and external. A timeline of events is used as a cognitive tool. It fits the idea of
process trajectories, is good for situating historical data in context, and is good for
being shared and combined into a collaboratively constructed representation.
The scope of our analysis and model is project based learning in which lightweight
collaborative tools are used to support various aspects of work. While the domain of
our empirical studies is software development, the model may be used to aid the
organization of retrospective reflection in project based learning within any domain.
Taking a constructivist view of knowledge, there is no single, ‘real’ story of a project
that can be revealed through the use of timelines and collaborative tool history.
Our approach can help participants unveil issues of importance to them and aid individual
and collective sense making of various aspects of the project. We believe that
the systematic approach, the explicit incorporation of individual contributions into
collective, co-constructive learning activity, and the utilization of available resources
in the form of historical data together result in reflection outcomes that are likely to be
useful with respect to the learning objectives and valid in the sense that more stones
have been turned.
185

A Model of Retrospective Reflection in Project Based Learning 431
The analysis presented in this work would have benefited from empirical data on
the use of more types of lightweight collaborative tools in retrospective reflection in
SD student projects or similar settings of project based learning. Further research
should examine the use of different collaborative tools within the framework of the
model. Results of this research can be used to validate the model.
References
1. Blumenfeld, P.C., et al.: Motivating Project-Based Learning: Sustaining the Doing, Supporting
the Learning. Educational Psychologist 26(3 & 4), 369–398 (1991)
2. Boud, D., Keogh, R., Walker, D.: Reflection: Turning Experience into Learning.
Routledge Falmer (1985)
3. Dewey, J.: Democracy and education - an introduction to the philosophy of education. The
Free Press, New York (1997) (1916)
4. Schön, D.: Educating the Reflective Practitioner. Jossey-Bass, San Fransisco (1987)
5. Lin, X., et al.: Designing Technology to Support Reflection. Educational Technology, Research
and Development 47(3), 43–62 (1999)
6. Edwards, J.S., Shaw, D., Collier, P.M.: Knowledge management systems: finding a way
with technology. Journal of Knowledge Management 9(1), 113–125 (2005)
7. Garrett, R.K., Danziger, J.N.: IM=Interruption Management? Instant Messaging and Disruption
in the Workplace. Jnl. of Computer-Mediated Communication 13(1) (2008)
8. Grinter, R.E., Palen, L.: Instant Messaging in Teen Life. In: CSCW 2002, New Orelans,
Louisiana, USA. ACM, New York (2002)
9. Krogstie, B.R.: Using Project Wiki History to Reflect on the Project Process. In: 42nd
Hawaii International Conference on System Sciences, Big Island, Hawaii. IEEE, Los
Alamitos (2009)
10. Krogstie, B.R., Divitini, M.: Shared timeline and individual experience: Supporting retrospective
reflection in student software engineering teams. In: CSEE&T 2009, Hyderabad (2009)
11. Krogstie, B.R., Divitini, M.: Collaboration tools as a resource for retrospective reflection
(submitted, 2010)
12. Klein, H.K., Myers, M.M.: A Set of Principles for Conducting and Evaluating Interpretive
Field Studies in Information Systems. MIS Quarterly 23(1), 67–94 (1999)
13. Derby, E., Larsen, D., Schwaber, K.: Agile Retrospectives. Making Good Teams Great.
Pragmatic Bookshelf (2006)
14. Kasi, V., et al.: The post mortem paradox: a Delphi study of IT specialist perceptions.
European Journal of Information Systems 17, 62–78 (2008)
15. Krogstie, B.R.: The wiki as an integrative tool in project work. In: COOP 2008, Carry-le-
Rouet, Provence, France. Institut d’Etudes Politiques d’Aix-en-Provence (2008)
16. Krogstie, B.: Power through brokering. OSS participation in SE projects. In: ICSE 2008,
Leipzig. IEEE Computer Society, Los Alamitos (2008)
17. Krogstie, B., Bygstad, B.: Cross-Community Collaboration and Learning in Customer-
Driven Software Engineering Student Projects. In: CSEE&T 2007, Dublin. IEEE, Los
Alamitos (2007)
18. Halverson, C.: Activity Theory and Distributed Cognition: Or What Does CSCW Need to
DO with Theories? Computer Supported Cooperative Work 11, 243–267 (2002)
19. Nardi, B.A.: Studying context: a comparison of activity theory, situated action models, and
distributed cognition. In: St. Petersburg International Workshop on Human-Computer Interaction,
St. Petersburg, USSR, Int.Centre Sci. & Tech. Inf, Moscow, Russia (1992)
186

432 B.R. Krogstie
20. Hutchins, E.: Cognition in the Wild. MIT Press, Cambridge (1995)
21. Salomon, G.: Distributed Cognitions. Cambridge University Press, New York (1993)
22. Karasavvidis, I.: Distributed Cognition and Educational Practice. Journal of Interactive
Learning Research 13(1/2), 11–29 (2002)
23. Kim, B., Reeves, T.C.: Reframing research on learning with technology: in search of the
meaning of cognitive tools. Instructional Science, 35 p., 207–256 (2007)
24. Salomon, G.: No distribution without individuals’ cognition, in Distributed Cognitions. In:
Salomon, G. (ed.) Psychological and educational considerations. Cambridge Univ. Press,
Cambridge (1993)
25. Ackerman, M.S., Halverson, C.: Organizational Memory as Objects, Process, and Trajectories:
An Examination of Organizational Memory in Use. Computer Supported Cooperative
Work 13(2), 155–189 (2004)
26. Sharp, H., Robinson, H.: A Distributed Cognition Account of Mature XP Teams. In: Abrahamsson,
P., Marchesi, M., Succi, G. (eds.) XP 2006. LNCS, vol. 4044, pp. 1–10.
Springer, Heidelberg (2006)
27. Kirschner, P.A., Erkens, G.: Cognitive tools and mindtools for collaborative learning.
Journal of Educational Computing Research 35(2), 199–209 (2006)
28. Dewey, J.: How we think. A restatement of the relation of reflective thinking to the educative
process (Revised edn.). D. C. Heath, Boston (1933)
29. Kim, D., Lee, S.: Designing Collaborative Reflection Support Tools in e-project Based
Learning Environment. Journal of Interactive Learning Research 13(4), 375–392 (2002)
30. Kolb, D.A., Fry, R.: Towards an applied theory of experiental learning. In: Cooper, C.L.
(ed.) Theories of Group Processes, pp. 33–58. John Wiley, London (1975)
31. Boud, D., Keogh, R., Walker, D.: Promoting Reflection in Learning: a Model. In: Boud,
D., Keogh, R., Walker, D. (eds.) Reflection: Turning Experience into Learning, pp. 18–40.
Routledge Falmer (1985)
32. Strauss, A.: Continual permutations of action. Aldine de Gruyter, New York (1993)
33. Stahl, G.: Building collaborative knowing. In: Strijbos, J.-W., Kirschner, P.A., Martens,
R.L. (eds.) What We Know About CSCL And Implementing It In Higher Education, pp.
53–85. Kluwer Academic Publishers, Boston (2002)
34. Wegner, D.M.: Transactive memory: A contemporary analysis of group mind. In: Mullen,
M.B., Goethals, G.R. (eds.) Theories of group behaviour, pp. 185–208. Springer, New
York (1987)
35. Hollingshead, A.B.: Retrieval processes in transactive memory systems. Journal of Personality
and Social Psychology 74, 659–671 (1998)
36. Wegner, D.M., Guiliano, T., Hertel, P.T.: Cognitive interdependence in close relationships.
In: Ickes, W. (ed.) Compatible and incompatible relationships. Springer, Heidelberg (1985)
37. Hollingshead, A.B.: Communication, learning, and retrieval in transactive memory systems.
Journal of Experimental Social Psychology 34, 423–442 (1998)
38. Gabbert, F., Memon, A., Allan, K.: Memory conforminty: Can eyewitnesses influence
each other’s memories for an event? Applied Cognitive Psychology 17, 533–543 (2003)
39. Roediger, H.L., Meade, M.L., Bergman, E.T.: Social contagion of memory. Psychonomic
Bulletin & Review 8, 365–371 (2001)
40. Alea, N., Bluck, S.: Why are you telling me that? A conceptual model of the social function
of autobiographical memory. Memory 11, 165–178 (2003)
41. Barnier, A.J., et al.: A conceptual and empirical framework for the social distribution of
cognition: The case of memory. Cognitive Systems Research 9, 33–51 (2007)
187

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
188

Research paper P8
Title:
Supporting Reflection in Software Development with Everyday Working Tools
Authors:
Birgit Krogstie and Monica Divitini
Published in:
Proceedings of COOP 2010
Pages:
20
Complete reference:
Krogstie, B. R. and M. Divitini (2010). Supporting Reflection in Software
Development with Everyday Working Tools. COOP 2010, Aix-en-Provence, France,
19-21 May. Springer.
189

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
190

Supporting Reflection in Software Development with
Everyday Working Tools
Birgit Krogstie, Monica Divitini
Norwegian University of Science and Technology,
Sem Sælands vei 7-9, NO-7491 Trondheim, Norway
{birgitkr, divitini}@idi.ntnu.no
Abstract. Through their day-to-day usage collaboration tools collect data on
the work process. These data can be used to aid participants’ retrospective
reflection on the process. The paper shows how this can be done in software
development project work. Through a case study we demonstrate how
retrospective reflection was conducted by use of an industry approach to project
retrospectives combined with the examination of historical data in Trac, an
issue tracker. The data helped the team reconstruct the project trajectory by
aiding the recall of significant events, leading to a shift in the team’s
perspective on the project. The success of the tool-aided retrospective reflection
is attributed to its organization as well as the type of historical data examined
through the tool and the tool features for navigating the data. These insights can
be used to help project teams determine the potential of their tools to aid
retrospective reflection.
Keywords: reflection, project work, software development, issue tracker.
1 Introduction
Software development (SD) is highly cooperative work which has received
considerable attention in CSCW [1-3]. In SD, mistakes are common as well as costly
[4], and learning from experience is essential [5].
In SD organizations using large integrated systems to support and coordinate work,
efforts to have the organization learn from experience may be integrated into
predefined work processes and supported by tool functionality for collecting and
providing experience data. The idea of experience factories [6] reflects a similar idea.
In other settings, as in agile software development [7], learning from experience may
be more informal and deferred to face-to-face project meetings and retrospective
reflection sessions. Project retrospectives [8-9] in agile SD are an example of what is
known in the project management literature as project debriefings [10], which cover
various methods for recording experiences from projects. Project retrospectives are
conducted at the end of project phases or when projects are terminated. The major
resources are participants’ memory and collaborative effort to reconstruct the project,
and various facilitation techniques exist (e.g. [11]). In this paper we generically refer
to retrospective reflection to indicate activities conducted at the end of a project phase
191

to rethink the working process with the goal of learning from experience.
Retrospective reflection can be seen as an aspect of reflective practice [12]
comprising reflection-on-action in context of the project.
One challenge, found to be among the reasons why many organizations choose not
to conduct retrospective reflection [13], is the lack of adequate data to help
participants reconstruct the process. Human memory is fallible in recalling a process
of weeks or months. For other data sources to be useful in practice, however, they
should provide access to data with an appropriate level of detail and enough context
to help avoid oversimplification and examination of unimportant details [13].
The objective of this paper is to investigate the potential to support project teams’
retrospective reflection by the use of historical data in lightweight collaboration tools
[14]. These are tools typically providing basic functionality for collaborative activity
without imposing much structure on the process. This allows flexible incorporation of
the tool into the specific process. Taking lightweight collaboration tools into use
requires little resources (e.g. acquisition cost, time for training) on part of users and
organizations. Lightweight tools are in use in a large number of SD projects, some of
which have very limited resources for organizing retrospective reflection sessions.
Lightweight tools may be the choice of small organizations, teams doing crossorganizational
work, teams basing their selection of tools and methodologies on local
needs and preferences, or teams in organizations running ‘lean’ processes. As a sideeffect
of day-to-day work, the lightweight collaboration tools collect data originating
in the work process. Such historical data may have the appropriate detail and context
to aid participants’ retrospective reconstruction of the process.
The CSCW literature has addressed the use of lightweight cooperation technology
to support day-to-day project work, e.g. in software development [15-17]. Also, there
is research on the use of data captured in collaboration tools to gain insights about
collaborative work activity, as will be elaborated in the Background section. The
potential to aid retrospective reflection on an entire project process by use of
historical data in the various lightweight collaboration tools supporting that process
was identified in [18].
For a project team to be able to determine the potential of their collaboration tools
to aid retrospective reflection, they need knowledge about what it is that makes a tool
useful for such a purpose. Such knowledge is also of value to collaboration tool
designers who might want to extend the area of application of their tools to
retrospective reflection.
Research contributions in this area should include empirical investigation of tool
use as it unfolds in the context of retrospective reflection. This paper provides such a
contribution by going into detail on how retrospective reflection is aided by the use of
historical data in a collaboration tool in a SD project followed in an in-depth case
study.
In the background section we outline relevant theory and related work from the
CSCW field. The case section describes the SD project of our study and the project
management tool used in that project. The research method section includes a
description of the organization of the team’s reflection workshop. Findings from the
workshop are next described, followed by a section on discussion and implications,
a section outlining a set of guiding questions regarding the potential of collaboration
tools to support reflection in SD work, and finally our conclusion.
192

2 Background
In this section we briefly provide a background on distributed cognition as a way of
understanding SD work and on the reconstruction of the process trajectory as
important to retrospective reflection. We also address how the day-to-day use of
collaboration technology results in the creation and storage of historical data.
In the analysis we have used distributed cognition [19-20] to theoretically frame
the problem. Distributed cognition is a perspective which has been used to understand
learning [21], software development [22-23], and cooperative work more generally
[24]. Cooperative work can be seen as distributed across the people involved in the
activity, through coordination of the processes between internal and external
representations, and through time in such a way that results of earlier activity can
transform later activity. Individual participants’ mental representations [25] of the
project process mediate day-to-day project work as well as reflection on the work.
Strauss [26] argues that when people make sense of a process, they think of it as a
trajectory. On this basis, retrospective reflection can be seen to involve the individual
and collective reconstruction of the process trajectory, which includes identifying its
significant events. The trajectory can have an internal or external representation (e.g.
a diagram or a textual description). In sum, distributed cognition is a particularly
useful framework for analyzing and designing for retrospective reflection because it
helps shed light on the role of representations that mediate the reflective process,
being used and transformed by participants individually and collaboratively.
In the day-to-day project activity shaping the project trajectory, cooperative work
entails coordination, access to artifacts in a shared workspace, and formal as well as
informal communication – aspects of work typically supported by cooperation
technology. Formal communication, for which archiving is important, may for
instance be conducted over email [27] and informal communication, e.g. about
ongoing work, by synchronous [16, 28] and asynchronous chat. A shared workspace
to hold artifacts under development must be provided, as well as ways of providing
co-workers with workspace awareness [29-31], including social awareness [32].
Coordination of complex work, e.g. its planning, monitoring and re-planning, requires
coordination artifacts [1, 33], computerized tools ranging from simple to-do-lists and
bulletin boards to advanced enterprise solutions. Among the tools used to aid the
coordination of work are issue trackers [34-35] used in Software Development.
As a consequence of the activity along the trajectory, data get stored. Depending
on the usage of each tool, its data will reflect different aspects of the SD work, e.g.
development and bug-fixing, stakeholder collaboration [36], and team-internal,
informal communication [37-38]. From a distributed cognition perspective, the data
are external representations originating in there-and-then distributed cognition. The
data may have the form of logs (events, communication), versions of project artifacts
(e.g. code, documentation), and plans and status reports. Data are generally recorded
in repositories that are accessible through the tools. The idea of capturing information
relevant to a work process from tools and artifacts already used to support that work
process has been explored in several ways in the context of SD. Some approaches
involve the deliberate tagging of information relevant to others [39-40]. The use of
logs from a code management system to support coordination in a SD team has been
proposed [41]: An event notification system and a tickertape tool for CVS messages
193

was integrated with the code management system, improving communication in the
team. The project memory tool Hipikat is designed to help new members of a
distributed SD team learn from more experienced colleagues [42]. The memory
contains the artifacts produced during development and is built by the tool with little
or no changes to the existing work practices. The SEURAT system [43] is designed to
capture design rationale in software development and was originally intended to
support software maintenance. The system can also trace requirements and decisions
[44] in the project. Aranda and Venolia [45] found that for purposes of coordinating
SD work, data repositories from the development process (bug databases and email
repositories) could provide some information about the bug fixing ‘stories’, as
reconstructed by the researchers having access to all the repository data, but the data
in the repositories were incomplete and sometimes erroneous. The study showed that
the reconstruction of the stories, including the social, organizational and technical
knowledge, required the participants in the bug fixing process to account for the
stories. Aranda and Venolia also found that among the most problematic types of
missing data in the repositories were missing links from the bug records to the source
code involved.
In [18] it was argued that data collected in various collaboration tools in daily use
in a project may be systematically utilized to aid retrospective reflection in the project
team. From a distributed cognition perspective, the historical data in the tools can be
considered external representations of the project process. Together with team
members’ internal representations of the project process and external representations
constructed in the team’s collaborative reflection effort, the historical data in the tools
are a potential resource in the reconstruction of, and reflection on, the project process.
The outcome of the reflection (i.e. lessons learned, manifest in internal and external
representations) subsequently informs the work practice, including the use of the
collaboration tools from which the historical data were drawn. In the paper, a
lightweight project management tool in the form of an issue tracker (Trac [46]) was
used as an example of a tool with a potential to support retrospective reflection.
The aim of this paper is to demonstrate in detail how retrospective reflection can be
aided by the use of historical data in Trac. Outlining how the historical data are
utilized as an integral part of a systematic retrospective reflection approach with
individual and collective steps, we stress how the collaboration tool plays a role in
knowledge building as well as which tool features are particularly useful to support
retrospective reflection in the team.
3 Case
Our case is a SD student team in an undergraduate, capstone project course. The
projects have external customers, require a high level of independence in terms of
choice of process, technology and solution, and are designed to provide a work setting
as close to industry SD as possible. Meeting customer requirements is a primary goal,
along with meeting the university’s requirements for certain project deliveries, most
notably a project report to be delivered in a preliminary, mid-term and final version.
194

There are 3-5 students in the teams. The project takes up half the workload in the final
(6 th ) semester of a Bachelor of IT program.
As part of the course, a retrospective reflection workshop is arranged for each team
at the end of the project based on a technique drawn from industry SD practice [8-9].
With this technique, participants individually and collectively construct a timeline of
project events. Opinions and feelings about what is positive and negative are added,
e.g. by use of post-it notes in different colours or by use of curves indicating the ‘ups
and downs’ of the process. The final steps of the workshop are directed at determining
future action based on the analyzed experience. In the workshop of our course, we use
the following main steps: the participants draw a timeline of their project, adding
events considered significant. First, timelines are drawn individually and next a
shared one is collectively constructed. The emotional aspects of the process are
addressed by having participants draw curves (that we call ‘satisfaction curves’) along
the timeline, indicating their satisfaction with the project at different times. Next
follows a discussion of issues related to lessons learned. It was shown in a study that
the workshops can help the teams share knowledge and reach new insights about their
projects [47].
The project of our study had five team members (Fig. 1). Their task was to develop
an application allowing people to use their mobile phone to keep track of the
geographical location of their friends and engage in textual chat with them. The
required solution was technically advanced, including a server, an administration
client and a mobile client and the use of services from a provider of geographical
positioning data.
Fig. 1: The project team of the case study: snapshots from everyday project work
To support the management of their development work, the team chose to use an
issue tracking tool called Trac (http://trac.edgewall.org/). Trac is a web application
that supports the organization of work into tasks and phases with milestones, and is
lightweight in the sense that it contains only the basic features necessary to manage
SD work and requires little resources to be taken into use. Tasks are defined by tickets
that are assigned to the users, e.g. team members. Trac also contains a wiki, allowing
for any contents to be flexibly added. Trac provides a set of views into the files
managed by an underlying file versioning system (SVN) containing e.g. source code
and project documents. When a team member uploads (i.e. commits) new or modified
files to the file management system, e.g. from the development environment in which
source code is produced and tested, it can be seen in Trac as a new changeset.
Usually, the user adds a comment on what the changeset is about. All versions of all
files that have been committed to SVN, as well as the differences between file
versions, can be viewed through Trac.
195

The Trac timeline (see Fig. 2, screenshot (a)) contains an item for each changeset
(and its comment), wiki update, ticket update, and milestone completion. Each item
has a date and time of the update; an identifier linking to the particular version of the
project artifact(s) affected; and the name of the user doing the update. In Fig. 2 (a), for
example, the comment associated with changeset [86] reflects an update of source
code to the SVN server made by Matthew on 13 February. Apart from conveying
status updates, comments are sometimes used more directly for coordination, as in
“be sure to update before doing any changes” (comment made by Justin on 5 May.).
The timeline is thus central to the coordination of SD work in the project, particularly
when team members are not collocated. It provides an instant overview of state-ofaffairs
as well as an entry point to the project artifacts.
Fig. 2: Three screenshots from Trac as Justin examines the timeline and identifies
pre-study coding
196

4 Research Method
The research reported in this paper is interpretive [48]. The main source of data is a
retrospective reflection workshop conducted with the selected SD project team in a
usability lab over a period of two and a half days at the end of their project. In
addition, a longitudinal study of the team’s work had been conducted throughout the
project (one semester), including 45 hours of non-participant observation of meetings
and work in the computer lab and frequent examination of project artifacts (including
the contents of Trac) as well as the team’s internal and external email communication.
The study provided knowledge important to our interpretation of data from the
workshop. Selection of the project was based on a combination of the technically
complex project task and the team’s decision to use Trac.
Table 1. Organization of the reflection workshop
Session type Step Explanation
Individual 1) Reconstructing
The team member was given a sheet of paper with a timeline
sessions
the containing a few fixed project events (e.g. delivery deadlines)
(Days 1-2, project and was asked to add important events. Next, for each of these
each trajectory by events, the researcher added it to a similar timeline on the
session 1,5-
2 hours)
The
whiteboard
was photographed
memory whiteboard and asked the team member to explain the
importance of the event. Next, the team member was asked to
draw on the paper sheet a curve showing how he had felt about
working in the project. Finally he was asked to draw the curve
along the whiteboard timeline, explaining its shape.
2) Using The team member was asked to use the laptop and Trac to freely
after these collaboration explain the project. When general browsing and explanation
sessions tool history to appeared exhaustive, the team member was asked to do a
aid further chronological walkthrough of the Trac timeline, exploring links
reconstruction at need. As new events deemed important to the project were
identified, they were added to the whiteboard timeline by the
interviewer, who took care to add events based on what the team
member considered important.
Common 3) Reconstructuring
The project timeline was reconstructed again, the researcher
session
the writing down events on the whiteboard as the team members
(Day 3, half
day)
project
trajectory
using
collaboration
tool at need
listed them in a round-robin fashion. A PC with access to Trac
was available to the team members, the screen projected onto the
wall next to the whiteboard. When no more events were
suggested, the team members came to the whiteboard one by one
to draw their satisfaction curve and comment on it.
4) Discussion There was a common session addressing questions of project
of tasks, roles, tasks and lessons learned, each question answered by all
lessons team members in turn. Discussion was allowed and encouraged.
learned, roles
The workshop steps were based on the timeline and satisfaction curve approach
described in the Case section. Compared to the short retrospective workshops
conducted with the other teams in the course, the design incorporated an extra step of
using historical data in Trac to aid recall. Further, individual timeline construction
was done in a sequence of individual sessions (rather than in parallel) to allow the
197

collection of rich data about each team member’s timeline construction and explore
how the use of historical data in Trac might be helpful in different ways or to different
degrees among the team members. The lab was equipped like a small meeting room
with a whiteboard and a PC. Several movable cameras were installed in the ceiling,
and all sessions were videotaped. The use of Trac was recorded with screen capture
and synchronized with the video recording. Photos were taken of the whiteboard.
In line with the principle of considering multiple interpretations [38], individual
follow-up interviews were made five months after the workshop, i.e. when analysis of
the data had resulted in findings to be presented to the participants and discussed in
light of their interpretation.
Two questions guided our focus in the workshop and the subsequent data analysis.
First, we looked for indications that the retrospective use of Trac benefited reflection,
e.g. helped the team return to experience, re-evaluate it and create new perspectives.
More specifically, we looked for events recalled only by some team members, events
recalled only after use of Trac and, among those, events of general importance in
later discussion, e.g. addressing lessons learned. Second, we wanted to know more
about how Trac was used by the team members to aid their recall and reflection, i.e.
what particular functionality and data in the tool seemed relevant and useful. In the
data analysis we systematically examined the individual timelines and the shared
timeline and the recordings of the sessions in which they were made, making a table
of all events that were mentioned, noting who had mentioned them and whether they
were mentioned before or after examination of the data in Trac. We picked one event,
namely pre-study coding, based on the following criteria: the event was recalled only
after use of Trac, it was recalled by some team members only, and it was important in
the team’s final discussion of lessons learned, The selection of an event guided further
analysis, including close examination of all workshop data relating to the event with
the aim of to providing a coherent account of how the event was elaborated in, and
impacting on, the reflection process in the team, Data were transcribed and translated
into English at need. In the presentation, names (including user names in screenshots)
are made anonymous.
Our study can be seen as a “fine-grained, field-based empirical study” useful for
designing systems and understanding the nuances of practice [24]. Even though the
reflection workshop in our case was organized with a clear research agenda,
impacting e.g. on its duration, the participants did real reflection on their SD project.
There are two main limitations to the study. First, it was conducted in the context
of SD education. The project was however close to industry practice, e.g. with a
customer and requirements for a working software product, and the team had some
members who were relatively experienced as developers. Second, a single case study
limits the possibilities to generalize from results. However, our purpose was to
understand the details of a team’s reflection on their project and investigate the
potential to support reflection through certain steps and with certain resources
198

5 Findings
In this section we draw on our workshop data and describe the project team’s recall of
and reflection on ‘pre-study coding’, an example of a project event recalled only
through the aid of historical data in the collaboration tool. To provide some context,
we start by describing the team’s project process on basis of the longitudinal study.
5.1 The Process and Organization of the Case Project
The members were generally satisfied with their team and the assigned task. The
relationship with supervisor and customer worked well throughout the project.
Requirements for the product were more or less clear after the second customer
meeting. Prior to midterm report delivery, time in the project was spent partially on
trying out technology, later to be characterized by the team as ‘pre-study coding’,
partially on writing the project report. After four iterations, the team delivered a
product with which both they and their customer were satisfied.
The team had decided on a flat organization, formally with Eric as project manager.
In practice, Justin had that role, taking main responsibility e.g. for deliveries and
meeting minutes. Matthew was seen as the technical expert, taking responsibility for
the server application and the overall design. Justin was in charge of the client
application, and Travis for the map functionality implemented late in the project. All
five team members actively participated in programming and report writing. Fig. 3
(created by the authors processing information from the Trac timeline) shows that all
team members participated and that Matthew and Justin did the most ticket updates.
Fig. 3: A summary overview of the Trac timeline contents
5.2 Pre-study Coding in the Individual, Memory-based Accounts
The five individual whiteboard timelines created on the basis of memory alone
(workshop Step 1, see Table 1) vary in terms of the selection of events and the shape
of the satisfaction curves. The events remembered to some extent reflect the roles in
the project. For instance, Matthew is the only one who remembers choices of certain
server technology. In comparison, a cancelled feature freeze that had a high impact on
199

the entire team, is remembered by four out of five members. The onset of pre-study
coding is not mentioned as important by anyone at this point. The onset of ‘coding’ is
however mentioned by Matthew and Justin. Matthew refers to a couple of events
involving the choice of technology in the project before midterm, but says nothing
explicitly about coding in this period. He places ‘coding startup’ right after midterm.
Justin places ‘code start’ on his timeline just before midterm. “Before that we had
only been working with report, and then we actually started writing code.” When
accounting for the impact of a necessary change of framework some time after
midterm, Justin explains that most of the coding of the final product took place during
a short period towards the end of the project.
5.3 Pre-study Coding Emerging through Examination of Trac
With the help of Trac in Step 2, all team members identify new events. In addition,
there are some modifications of names of existing ones and changes of their timeline
position. Justin and Matthew are the team members for which the examination of the
timeline brings up most new events. As they chronologically examine the timeline,
Justin and Matthew both examine the changesets [81] and [82] made by Matthew on
13 February. The comment ‘Initial import’ associated with both changesets indicates
that they are about the inclusion of files needed to start producing and testing Java
code. Fig. 2 shows the sequence of screenshots as Justin investigates the timeline item
of changeset [82]. Starting from the timeline view (a), he clicks [82]. The window (b)
now displays what has been changed; a list of files. As a programmer Justin knows
that the coding done by Matthew is in FindPerAnton.java, the rest of the files being
files necessary to set up a running program and included for the first time in the initial
import. He clicks the filename, and the window changes to show the file contents (c).
Examining the code, Justin exclaims: “Oh yes, it was only, kind of ‘Hello world’ on
the mobile” and returns to the timeline (a). ‘Pre-study coding’ is then added to the
whiteboard timeline (see Fig. 4).
Fig. 4. Justin’s and Matthew’s timelines. Frames/emphasis of arrows show
additions/modifications after examination of Trac.
Matthew, on examining the same item during his timeline construction, says:
“Right there, we have started coding. It was probably pre-study coding.” He suggests
that the event be placed in the first half of the pre-midterm project period (see Fig. 4).
200

As a consequence, ‘startup coding’, already on Matthew’s timeline just after midterm,
is changed to ‘startup coding according to requirements spec (start iterations)’.
5.4 Pre-study Coding in the Co-constructed Account
In the collective session of Step 3, most events from the individual timelines were
included in the shared timeline. Occasionally, the team on their own initiative looked
into Trac to check details, typically the timing of events. The issue of pre-study
coding was explicitly addressed. Justin early on repeated his statement that coding did
not start until midterm, and this was not questioned by anyone. This can be
interpreted as a shared understanding in the team that ‘coding’ (as a standalone term)
meant ‘coding in accordance with requirements specification’. As part of the roundrobin
provision of events, ‘pre-study coding’ was mentioned by Justin and added to
the timeline. Eric suggested another name for the same event: “Unofficial coding”.
This received no further comments. In the later step of drawing satisfaction curves
along the whiteboard, Justin explicitly referred to pre-study coding as he made an
upward curve bend early in the project, explaining that he enjoyed the pre-study
coding because they got things working then.
5.4 Pre-study Coding in Reflections on Lessons Learned
The topic of pre-study coding was recurrent in the discussion in Step 4, particularly
when the team addressed lessons learned. Justin suggested that they might have
overrated their own capabilities by starting coding as late as midterm. Matthew said
that more pre-study coding might have helped the team avoid the extra work of
learning a new framework, because they would have known better the limitations of
the technology. Overall in the discussion, pre-study coding was given a central role in
the account of what happened in the project and in conceptions of how to ideally run a
similar project. The considerations on pre-study coding were made by Justin and
Matthew. There seemed to be full agreement in the team that pre-study coding is
necessary to reach the technical competence needed for producing design and code
when a development task requires use of technology new to the team.
5.5 Insights from the Follow-up Interviews
In the follow-up interviews with the team members, they were presented with the
researcher’s interpretation that the workshop had lead to a shift in the team’s attention
to pre-study coding and conception of its significance to the project. All team
members expressed that this finding is in line with their own interpretation of what
happened in the workshop. The interviews with Eric, Travis and Kyle however
revealed some viewpoints that had not been brought into the team’s discussion. When
Eric during the collective reconstruction of the timeline had used the term ‘unofficial
coding’ about the pre-study coding, he did not refer to the fact that the team deemphasized
this activity but to the fact that pre-study coding had been conducted by
201

some team members only. In Eric’s opinion, the project would have gained from a
more distributed effort, making the entire team more competent with the technology
at an earlier point. Travis and Kyle expressed similar views. While these viewpoints
might have matured in the period after the reflection workshop, the workshop
recordings indicate that the views were there during the workshop but had never
properly entered the discussion.
6 Discussion and Implications
The example in the Findings illustrates how collaboration technology supporting dayto-day
work practice in a SD project contained data that could be utilized by team
members in a systematic and collaborative effort to reconstruct and reflect upon the
project process.
The event called ‘pre-study coding’ was identified only through the aid of
collaboration tool history and only by two of the five team members during the
individual timeline construction. The event was later brought into the team’s shared
timeline, was referred to by a team member as impacting on his personal experience
of the project, and was central in the team’s reflections on lessons learned. Comparing
the team members’ original accounts of the project at the outset of the workshop with
the final discussion, there was a clear difference: the significance of pre-study coding,
in this project in particular and in SD projects in general, had become clearer. We see
this as an example of knowledge relevant to the SD practice being created through
retrospective reflection.
In this section we address why the tool-aided retrospective reflection was
successful. We do this by considering the work practice, including the retrospective
reflection effort itself, as a case of distributed cognition [18], and by looking into the
type of historical data available in the tool, the tool features used to retrieve the data
in the reflection workshop, and the organization of the workshop.
6.1 Historical Data in the Collaboration Tool
Cognition involved in a work practice is distributed over participants and the artifacts
involved. Accordingly, it will typically be necessary to examine different data sources
to make sense of an issue or event associated with that practice. In our case, what
made Justin recognize that coding had happened in the project, was an item in the
Trac timeline (see Fig. 2). The type of timeline item (a change to a file in the
underlying file versioning system) and the comment following the change (i.e. the
comment explicitly added by the user making the change, for purposes of day-to-day
coordination) made it possible for him to quickly infer what this was about. Following
the link, he recognized the element of interest in the list of files, and selecting the file
in its there-and-then version, he could see what the piece of code was actually doing.
The combination of the data and Justin’s background (as a programmer, Trac user,
and co-creator of the data in front of him), enabled him to reconstruct the state of the
project and identify the event.
202

The type of data involved included 1) data used for day-to-day coordination of
project work, reflecting ongoing tasks (e.g timeline items with comments, made by
different team members; the list of artifacts to which the selected timeline update
applies) 2) data describing the state of an artifact in the project workspace (e.g. the
there-and-then version of the source code file). Different aspects of the development
work (e.g. project management, coding) being distributed in accordance with team
roles, the data used to identify pre-study coding reflects a social distribution of the
cognitive processes involved in the onset of pre-study coding.
The combination of data originating in coordination and data from the
development workspace is useful for identifying events associated with development.
A type of data less relevant for the identification of pre-study coding, but that
frequently evoked team members’ recall and reaction in the workshop of our study,
are timeline comments addressing the social/emotional side of work and thus
providing social awareness in day-to-day work.
Comparing the data examined in the workshop sequence illustrated in Fig. 2 with
the aggregate data on the project process shown in Fig. 3, we note that the aggregate
data could not have been used to provide context for single events. The diagram,
though a useful resource, provides only a partial overview of the process. In our case,
the aggregate diagram shows that all team members participated. What cannot be seen
from the diagram is the exact type and amount of work done by each team member.
For instance, we knew from observation of the team that Matthew tended to make
updates in large increments, uploading new code to the shared server after working on
it locally on his own machine for long, sometimes for days. While the diagram thus
seems to indicate that Matthew and Eric did a similar amount of coding in the project,
Matthew in reality did more coding than Eric. Another example is that when two team
members were working in pairs, one of them tended to be in charge of the keyboard.
In sum, the aggregate timeline data provides an additional overview. To get
adequately detailed and contextualized historical data from Trac for the recall of
specific project events, however, an examination of individual timeline items is
necessary. It should also be noticed that the fact that project work was sometimes
conducted in pairs is not reflected anywhere in Trac. The day-to-day usage of the tool
provides it with potential to be used in shedding light on some aspects of the project
process - but not all.
6.2 Navigating the Historical Data in the Collaboration Tool
It was through chronological traversal of the timeline events that Justin identified the
first uploading of source code to the shared workspace. Also, the timeline provided a
condensed overview, including the possibility – unexploited in this case - to filter the
type of events displayed. From the timeline item of interest, it took just a click to get
more detailed, but still overview level information about the change, e.g. which files
had been changed. The usefulness of this feature fits with the point made in [45] that
missing links from bug records to source code is highly problematic for the
reconstruction of the bug fixing process. Going up and down between different levels
of detail thus proved important. From the overview, there was direct access to the
state of the artifact (e.g. the selected file) at the time of the change. Switching between
203

the modes of traversal, overview, and exploring the state of the artifact was easily
achieved. The ease with which several months’ work can thus be navigated in Trac by
participants contrasts with the effort needed to examine less structured data on
collaborative work, e.g. video recorded meetings [49].
While the data in the tool reflects socially distributed cognition, the timeline
reflects the temporal distribution of cognition and is a starting point for identifying
significant events. Another aspect of the temporarily distributed cognition is that
when the tool is used retrospectively to make sense of data, familiarity with the tool
provides context for the user. Justin had been using the Trac timeline daily throughout
the project to get an overview of current project status. Justin’s experience and skills
in how to make sense of the process through the collaboration tool mediated his
retrospective reflection.
6.3 Appropriately Organizing the Process of Retrospective Reflection
Essential to the success of the reflection workshop was its organization. The example
in Fig. 2 shows individual use of Trac to reconstruct a timeline, but as a whole, our
study shows how historical data successfully plays a role in supporting cooperative
work. A core idea in the approach [8-9] on which the workshop was based is that
individual perspectives should be utilized in the construction of a shared
understanding. In the study, individual timelines mediated individual recall and
reflection as well as collective timeline construction.
From a distributed cognition perspective, the individual and shared timelines can
be seen as external representations of the project trajectory, and the steps of the
workshop as a process of coordinating the cognitive processing between internal and
external representational states. In our study, we looked at a collaboration tool as a
resource for the cognitive processing between internal and external representational
states. This happens in three ways, as illustrated in Fig. 5: The day-to-day work, seen
as a process trajectory along which there are multiple points of action and interaction,
leaves a trace of external representations of the distributed cognition involved, in the
form of historical data in the tool. This is a result of Trac’s role in supporting the dayto-day
cooperative project work (1). In the process of retrospective reflection, Trac is
used as an aid to individuals’ recall of project events (2), resulting in the creation of
the individual timelines, which are also external representations of the project
trajectory. The individual timelines are next used as a resource when the team
collaboratively creates another external representation of the project trajectory: the
shared timeline. To aid the alignment of the individual timelines, Trac is used
collaboratively by the team as an aid to clarify details in the construction of a shared
representation of the project (3), e.g. exact dates and times of specific events. Fig. 5 is
an instantiation of the more generic model of retrospective reflection outlined in [18].
Used in retrospective reflection on day-to-day development work the collaboration
tool serves as a boundary object between that practice and retrospective reflection
[50]. Developers’ awareness that the tool will be used retrospectively may affect their
day-to-day work practice [24]. For instance, software developers may start providing
more informative comments about their updates to make it easier to understand them
retrospectively. This is likely to benefit day-to-day coordination. A change with
204

potentially negative effects is if developers, to make a good retrospective impression
of their work, start hiding information. For instance, mistakes in intermediate stages
of work can be hidden by making less frequent updates or by avoiding commenting
about error corrections (a type of comment common in the Trac timeline in our
study). This could adversely affect day-to-day workspace awareness as well as the
possibility to get a realistic picture of the development process from the Trac timeline.
Fig. 5: Trac in a dual role as a collaboration tool in a project: supporting day-to-day
work and retrospective reflection on that work. Adapted from [18].
Our findings indicated that the team in our study through the tool-aided
retrospective reflection succeeded in creating new knowledge about their software
development practice. However, the individual differences in respect of how helpful
Trac was in the recall of events, point to challenges of uneven power distribution in
the team. The benefit of using a development-oriented collaboration tool to aid
reflection appeared greater for those dominating the development work. Strong
involvement in the activity from which the data originated provided them with
context for recognizing the data as connected to project events. Having identified the
pre-study coding event, the two lead programmers seemed to retain ownership of the
event in the team’s discussion, which can explain why alternative viewpoints on prestudy
coding among some team members were never properly disclosed.
205

7 The Potential of Collaboration Tools to Support Reflection in SD
Work: A Set of Guiding Questions
The previous sections outlined how certain tool features as well as the type of data
stored in the tools and the organization of the reflection activity were important when
historical data in Trac was used to aid the reconstruction of the project trajectory in
retrospective reflection. In this chapter we ask: in what way are these findings useful
from a perspective of supporting reflection on SD work more generally?
Our answer is twofold. First, we believe that the results hold promise for the use of
tools like Trac to assume a dual role of supporting daily work as well as retrospective
reflection in SD projects. It was relatively unproblematic in our study to include steps
of investigating collaboration tool history into an existing approach to organizing
retrospective reflection. To make the tool-aided version of the approach integral to SD
practice, the workshop organization must be adjusted to feasible schedule, i.e. one
that can be followed under resource constraints in real projects.
Second, we believe that the findings can be generalized to the use of collaboration
tools in retrospective reflection in SD work, answering the intent to provide
empirically grounded insights to project teams and tool designers about what makes
collaboration tools useful for such a purpose. In Section 6, we structured the findings
into some issues that seem crucial to the success of the selected collaborative tool to
aid retrospective reflection. Based on this understanding, we outline questions that
point to essential aspects of how a collaboration tool and its historical data can
support retrospective reflection. The objective is not to find the ‘best’ tool for the
support of reflection but to help users and designers identify, or modify, strengths and
weaknesses of the tools for such use. To illustrate how the answers may differ for
various lightweight collaborative tools, we refer to the research literature on some
tools often used in project work: project wikis, instant messaging, and email.
What aspects of the project process are reflected in the historical data in the
tool? Which SD challenges worth reflecting upon have been addressed by use of this
tool? An issue tracker contains data that may be used to aid reconstruction of the
development process, with a focus on its technical and formal aspects. The informal
communication is likely to be reflected in other tools, e.g. instant messaging tools [28,
51]. Other lightweight project management tools, e.g. project wikis [52], might reflect
more of the informal collaboration in the team than what is documented in the
historical data of an issue tracker. Stakeholder collaboration, a frequently challenging
SD issue [36], may partially be reflected in email logs.
Does the tool provide features for getting a chronological overview of aspects
of the project process? As seen in our study, the traversal of a chronological
overview can be used to structure the process of examining historical data.
Chronological overviews are essential in project management and thus provided by
project management tools such as Trac. A project wiki, by being a wiki, contains
functionality for getting an overview of the revisions of each page [52]. The
disadvantage of browsing through numerous revisions of a wiki page is that changes
may be small and uninteresting. Conversely, increments reflecting the process as it
206

unfolded may shed light on the process in a way complementing more formal
overviews. An email application allows for the filtering of email messages to be
displayed in a mailbox, enabling for instance rapid overview of all email sent to the
project customer or all messages with a particular keyword in its title (e.g. ‘status
report’). IM tools generally provide poor overview information, but the log contents
are chronological and time-stamped.
Does the tool provide features for accessing the project artifacts in their thereand-then
versions? One of the strengths of Trac as a SD tool is the way task
organization links to project artifacts, stressing the close connection between process
and product in SD work, the complexity of the process depending on the complexity
of the product [1]. A project wiki may provide direct access to some project artifacts
[17], but is not likely to provide read access to the entire development-related
workspace with its artifacts in their there-and-then versions, as does Trac. Email
frequently includes relevant project artifacts as attachments. Instant messaging logs
might contain elements of project artifacts (e.g. source code excerpts) [51].
Does the tool provide features for easy navigation between overview and
detail? The possibility to go into detail on a specific project event helped the
participants in our study recognize it as an event worth including in their project
timeline. Looking at project wikis, they provide reasonably good support for this e.g.
through the page history overviews with links to each specific page [52]. Navigation
in an email reader can be more cumbersome, but typically allows e.g. the use of
different windows for the mailboxes and the contents of individual messages. Instant
messaging logs mainly allow navigation on a detailed level of project interaction only.
With respect to overviews and navigation, limited features in the collaboration
tools used to create the data may be amended by the use of other lightweight tools
(e.g. search tools) to navigate the data. However, this comes at the loss of the
contextualization of the data in the work (tool) environment from which it originated,
a loss which may negatively affect participants’ sense-making of the data.
Are the historical data subject to privacy issues? While the study reported in
this paper considered data that were regarded as shared within the team, these
characteristics do not apply to all data from project collaboration. IM logs, for
instance, may be considered by their owners as too private to be shared [18]. Their
potential to support individual reflection on the project process may be considered.
How will the use of historical data from the tool be integrated into a
structured reflection activity with individual and/or collaborative sense making
enabling participants to return to experience, reconstruct the stories [45] and draw
lessons learned from the data? The organization of the activity can for instance be
based on SD best practice for project retrospectives [8-9]. An issue which is not the
main focus in this paper, but which is nevertheless essential and should be considered
as part of the organization of retrospective reflection, is how to subsequently draw
upon the lessons learned in the work practice, within and/or across projects and teams.
For a project team considering the potential of one or more of their collaboration
tools to support their reflection on, and learning from, their project process, it may be
207

a good start to consider what aspects and challenges of their project work they would
like to shed light on, and next consider what are the candidate tools to contain
relevant data.
Finally, it should be stressed that when considering the potential of using a certain
tool to aid retrospective reflection in a specific case, a general framework is only a
useful starting point. The functionality of the specific tool will have to be considered
along with the usage of the tool in the specific project, which heavily impacts on what
historical data is being stored through everyday use of the tool.
8 Conclusion
By investigating a case of retrospective reflection in a software development
project aided by historical data in an issue tracker (Trac), we have shown in detail
how a collaboration tool in daily use in a project can be used to help participants learn
from the project experience. We have demonstrated how certain types of data and
certain features for retrieving them proved particularly valuable to aid participants’
recall and reflection, aiding the reconstruction of the project trajectory.
Based on the findings we identified a set of questions that may be asked by a
project team or a collaboration tool designer when considering the potential to use
specific collaboration tools in retrospective reflection. The questions can serve as a
starting point for the development of a more general framework outlining issues that
should be addressed. To gain insight on the potential of different types of
collaboration tools to aid retrospective reflection, more empirical studies are however
needed. By unveiling project teams’ use of specific tools in retrospective reflection
we may identify ways of utilizing the tools that significantly differ from the way Trac
was used in our study. This is an area of further research.
Another issue to be addressed in further research is how a project team’s
knowledge of the future use of a collaboration tool in retrospective reflection affects
their daily use of the tool.
Finally, we will examine how the retrospective use of Trac can be integrated into
retrospective reflection workshops within a time schedule feasible for real SD
projects. We will do this by trying out the approach in SD projects on a larger scale.
References
1. Carstensen, P.H. and K. Schmidt, Computer Supported Cooperative Work: New Challenges
to Systems Design, in Handbook of Human Factors/Ergonomics, K. Itoh, Editor. 2002
(1999), Asakura Publishing: Tokyo.
2. Grinter, R. Doing Software Development: Occasions for Automation and Formalisation. in
ECSCW'97. 1997. Lancaster, United Kingdom: Springer.
3. Herbsleb, J.D., et al. Distance, dependencies, and delay in a global collaboration. in
CSCW'00. 2000. Philadelphia, PA: ACM
4. Keil, M., J. Mann, and A. Rai, Why Software Projects Escalate: An Empirical Analysis and
Test of Four Theoretical Models. MIS Quarterly, 2000. 24(4).
208

5. Lyytinen, K. and D. Robey, Learning failure in information systems development.
Information Systems Journal, 1999. 9: p. 17.
6. Basili, V.R. and G. Caldiera, Improving Software Quality by Reusing Knowledge and
Experience. Sloan Management Review, 1995. Fall 1995.
7. Dybå, T. and T. Dingsøyr, Empirical studies of agile software development: A systematic
review. Information and Software Technology, 2008. 2008(50): p. 833-859.
8. Derby, E., D. Larsen, and K. Schwaber, Agile Retrospectives. 2006: Pragmatic Bookshelf.
9. Kerth, N., Project Retrospectives: A Handbook for Team Reviews 2001: Dorset House.
10. Schindler, M. and M.J. Eppler, Harvesting project knowledge: a review of project learning
methods and success factors. International Journal of Project Management, 2003. 21: p. 10.
11. Bjørnson, F.O., A.I. Wang, and E. Arisholm, Improving the effectiveness of root cause
analysis in post mortem analysis: A controlled experiment. Information and Software
Technology, 2009. 51: p. 150-161.
12. Schön, D., The Reflective Practitioner. 1983: Basic Books, Inc.
13. Kasi, V., et al., The post mortem paradox: a Delphi study of IT specialist perceptions.
European Journal of Information Systems, 2008. 17: p. 62-78.
14. Churchill, E.F. and S. Bly. It's all in the words: Supporting work activities with lightweight
tools. in GROUP '99. 1999. Phoenix, Arizona, USA: ACM.
15. Gutwin, C., R. Penner, and K. Schneider. Knowledge sharing in software engineering:
Group awareness in distributed software development in CSCW'04. 2004. Chicago, Illinois,
USA.: ACM Press.
16. Handel, M. and J.D. Herbsleb. What is Chat Doing in the Workplace? in CSCW'02. 2002.
New Orleans, Louisiana, USA: ACM.
17. Krogstie, B.R. The wiki as an integrative tool in project work. in COOP. 2008. Carry-le-
Rouet, Provence, France: Institut d’Etudes Politiques d’Aix-en-Provence.
18. Krogstie, B.R. A model of retrospective reflection in project based learning utilizing
historical data in collaborative tools. in EC-TEL 2009. 2009. Nice, France: Springer.
19. Hutchins, E., Cognition in the Wild. 1995, Cambridge, Massachusetts: The MIT Press.
20. Rogers, Y. and J. Ellis, Distributed Cognition: an alternative framework for analyzing and
explaining collaborative working. Journal of Information Technology, 1994. 9: p. 119-128.
21. Daradoumis, T. and M. Marques, Distributed Cognition in the Context of Virtual
Collaborative Learning. Journal of Interactive Learning Research, 2002. 13(1/2): p. 135-
148.
22. Flor, N.V. and E.L. Hutchins. Analyzing Distributed Cognition in Software Teams: A Case
Study of Team Programming during Perfective Software Maintenance. in Empirical Studies
of Programmers: Fourth Workshop. 1991: Ablex Publishing.
23. Sharp, H. and H. Robinson, Collaboration and co-ordination in mature eXtreme
programming teams. International Journal of Human-Computer Studies, 2008. 66(7): p.
506-518.
24. Ackerman, M.S. and C. Halverson, Organizational Memory as Objects, Process, and
Trajectories: An Examination of Organizational Memory in Use. Computer Supported
Cooperative Work, 2004. 13(2): p. 155-189.
25. Salomon, G., No distribution without individuals' cognition, in Distributed Cognitions.
Psychological and educational considerations, G. Salomon, Editor. 1993, Cambridge
University Press.
26. Strauss, A., Continual permutations of action. 1993, New York: Aldine de Gruyter.
27. Grudin, J. and T. Lovejoy. Messaging and Formality: Will IM Follow in the Footsteps of
Email. in INTERACT 2003. 2003. Zurich: IOS Press.
28. Isaacs, E., et al. The Character, Functions, and Styles of Instant Messaging in the
Workplace. in CSCW'02. 2002. New Orleans, Louisiana, USA: ACM.
29. Dourish, P. and V. Bellotti. Awareness and coordination in shared workspaces. in ACM
CSCW. 1992. Toronto, Ontario, Canada ACM Press.
209

30. Gutwin, C. and S. Greenberg, A Descriptive Framework of Workspace Awareness for
Real-Time Groupware. Computer Supported Cooperative Work 2002. 11(3-4): p. 411-446.
31. Gutwin, C., R. Penner, and K. Schneider. Group Awareness in Distributed Software
Development. in CSCW'04. 2004. Chicago, Illinois, USA: ACM.
32. Bødker, S. and E. Christiansen, Computer Support for Social Awareness in Flexible Work.
Computer Supported Cooperative Work, 2006. 15: p. 1-28.
33. Schmidt, K. and C. Simone, Coordination Mechanisms: Towards a Conceptual Foundation
of CSCW Systems Design. Computer Supported Cooperative Work, 1996. 5: p. 155-200.
34. Johnson, J.N. and P.F. Dubois, Issue Tracking. Computing in Science and Engineering,
2003(November/December ): p. 71-77.
35. Prause, C.R., et al. Managing the Iterative Requirements Process in a Multi-National
Project using an Issue Tracker. in IEEE International Conference on Global Software
Engineering. 2008. Bangalore, India: IEEE.
36. Poole, W.G. The Softer Side of Custom Software Development: Working with the Other
Players. in Conference on Software Engineering Education and Training. 2003. Madrid,
Spain: IEEE.
37. Kraut, R.E. and L.A. Streeter, Coordination in Software Development. Communications of
the ACM, 1995. 38(3).
38. Whittaker, S., D. Frohlich, and D.-J. Owen. Informal Workplace Communication: What Is
It Like and How Might We Support It? in Human Factors in Computing Systems. 1994.
Boston, Massachusetts, USA: ACM Press.
39. Dekel, U. and J.D. Herbsleb. Pushing Relevant Artifact Annotationsin Collaborative
Software Development. in CSCW'08. 2008. San Diego, USA: ACM.
40. Storey, M.-A., et al. Shared waypoints and social tagging to support collaboration in
software development. in CSCW'06. 2006. Banff: ACM.
41. Fitzpatrick, G., P. Marshall, and A. Phillips. CVS integration with notification and chat:
lightweight software team collaboration. in CSCW'06. 2006. Banff, Alberta, Canada: ACM.
42. Cubranic, D., et al. Learning from project history: a case study for software development. in
CSCW '04. 2004. Chicago, USA: ACM.
43. Burge, J. and D.C. Brown. SEURAT: integrated rationale management. in ICSE'08. 2008.
Leipzig, Germany: ACM.
44. Burge, J. and J. Kiper. Capturing Collaborative Design Decisions and Rationale. in Design,
Computing, and Cognition. 2008. Atlanta, Georgia, USA.
45. Aranda, J. and G. Venolia. The Secret Life Of Bugs: Going Past the Errors and Omssions in
Software Repositories in International Conference on Software Engineering (ICSE'09).
2009. Vancouver, Canada: IEEE.
46. Trac home page, http://trac.edgewall.org/, Last accessed 10 September 2009.
47. Krogstie, B.R. and M. Divitini. Shared timeline and individual experience: Supporting
retrospective reflection in student software engineering teams. in CSEE&T 2009. 2009.
Hyderabad: IEEE Computer Society.
48. Klein, H.K. and M.M. Myers, A Set of Principles for Conducting and Evaluating
Interpretive Field Studies in Information Systems. MIS Quarterly, 1999. 23(1): p. 67-94.
49. Wolf, C.G., J.R. Rhyne, and L.K. Briggs. Communication and Information Retrieval with a
Pen-based Meeting Support Tool. in CSCW 1992.
50. Star, S.L., The Structure of Ill-Structured Solutions: Boundary Objects and Heterogeneous
Distributed Problem Solving, in Distributed Artificial Intelligence, M. Huhns and L.
Gasser, Editors. 1990, Morgan Kaufmann Publishers. p. 37-54.
51. Krogstie, B.R. Do's and dont's of instant messaging in students' project work. in NOKOBIT
2009. 2009. Trondheim, Norway: Tapir.
52. Krogstie, B.R. Using Project Wiki History to Reflect on the Project Process in 42nd Hawaii
International Conference on System Sciences. 2009. Big Island, Hawaii: IEEE Computer
Society.
210

Appendix B: Other research publications
The following two research papers were published during the period of the PhD work.
They address student projects, but with a focus outside the scope of the thesis.
Practice-Based Learning as Mobile Learning: The Role of
Boundary Objects
Authors:
Krogstie, Birgit and Divitini, Monica
Published in:
IADIS Mobile Learning 2007 , Lisbon, Portugal, 5-7 July.
Abstract:
Practice-based learning is an important element in study programs educating professional
practitioners. It is characterized by a focus on students’ transition from university to professional
work life. The students are exposed to real work challenges within the pedagogical scaffolding of the
university program. An essential aspect of the learning process is students’ reflection on the
relationship between school and real life, theory and practice.
As we strive to provide the best possible pedagogical support for students’ learning in these
educational settings, we look for the elements most likely to have a positive or negative impact on
learning results. In this reflection paper, we will argue that a focus on interactional mobility is helpful
in developing adequate support for practice-based learning, both in terms of course design and, more
specifically, in terms of identifying collaboration technologies that may be helpful to the students. In
particular, we focus on the role played by boundary objects and how scaffolding can be related to
these objects.
211

The work-reflection-learning cycle in SE student projects: Use of collaboration tools
Learning from achievement: scaffolding student projects in
software engineering
Authors:
Bygstad, Bendik; Krogstie, Birgit and Grønli, Tor-Morten
Published in:
International Journal of Networked and Virtual Organizations, 6:2, 2009.
Abstract:
It has become almost a truism that students learn more from working on projects than from lectures.
This is reflected in pedagogical approaches such as Problem-based Learning, Project based learning
(PBL) and Work-based Learning. A problem in PBL, underrated in the literature, is that while trivial
tasks hold limited potential for learning, students may not succeed in solving nontrivial ones. We
suggest that the solution lies in appropriate scaffolding: providing support for the learner to gradually
master what is needed to complete a task. The empirical background for the study is a two-semester
Software Engineering (SE) course at the Norwegian School of IT, with data collected over five years.
We conclude that PBL in this setting may be successfully scaffolded by a formal, iterative and
incremental SE method. As our main contribution we point to six types of scaffolding addressing
essential aspects of SE project work.
212