learning - Academic Conferences Limited

Proceedings
of the
10 th European Conference
on e-Learning
Brighton Business School
University of Brighton
UK
10-11 November 2011
Volume One
Edited by
Sue Greener and Asher Rospigliosi
University of Brighton
UK

Copyright The Authors, 2011. All Rights Reserved.
No reproduction, copy or transmission may be made without written permission from the individual authors.
Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper
abstracts were read and selected by the conference panel for submission as possible papers for the
conference.
Many thanks to the reviewers who helped ensure the quality of the full papers.
These Conference Proceeding have been submitted to the Thomson ISI for indexing.
Further copies of this book can be purchased from
http://www.academic-bookshop.com
ISBN: 978-1-908272-22-5 Book
Published by Academic Publishing Limited
Reading
UK
44-118-972-4148
www.academic-publishing.org

Contents
Paper Title Author(s) Page
No.
Preface xii
Biographies of Conference Chairs, Programme
Chair, Keynote Speaker and Mini-track Chairs
Biographies of Contributing authors xv
Revisiting the Personal Transferable Skills
Debate - an eLearning Pedagogical Perspective
Survey of Teachers’ use of Computer/Internet in
Secondary Schools in South West Nigeria
Issues and Challenges in Implementing
eLearning Projects in Higher Education: The
Case of Jordan
The use of Open Educational Resources in Intra-
Organisational eLearning and Continuing
Education
Constructing a Survey Instrument for Assessing
Characteristics of Effective Online Teachers
i
Samuel Adu Gyamfi, Lene Tolstrup
Sorenson and Thomas Ryberg
Babatunde Alabi Alege and Stephen
Olufemi Afolabi
Hussein Al-Yaseen, Saheer Al-Jaghoub
and Nidal Al-Salhi
Antonios Andreatos
xiii
Jonathan Barkand 34
When Agents Make Suggestions About Readings Orlando Belo 41
Some Reflections on the Evaluation of Virtual
Learning Environments
Designing A New Curriculum: Finding The Right
Blend
Critical Success Factors for the Adoption of
eLearning in the Kingdom of Saudi Arabia
Educational Institutions
Challenges in Developing e-Submission Policy
and Practice
Enhancement of e-Testing Possibilities With the
Elements of Interactivity Reflecting the Students’
Attitude to Electronic Testing
e-Assessment Using Digital Pens – a Pilot Study
to Improve Feedback and Assessment Processes
Digital Educational Resources Repositories in
Lower and Middle Education in Portugal: Quality
Criteria in the International Context
Nabil Ben Abdallah and Françoise Poyet 48
Andrea Benn 56
Latefa Bin Fryan and Lampros Stergioulas 63
Alice Bird 73
Martin Cápay, Martin Magdin and
Miroslava Mesárošová
Tim Cappelli 91
Cornélia Castro, Sérgio André Ferreira and
António Andrade
eLearning: Roles in Distance Tertiary Education Ivana Cechova, Dana Zerzanova and Jana
Berankova
Independent Learning in Need or in Crisis?
Independent Learning Under the new Four-Year
Undergraduate Curriculum in Hong Kong
The Development and Application of a web
Based Metacognitive Mapping Tool
An Exploratory Comparative Study of Distance-
Learning Programmes
The Optimal Teaching Style Based on Variability
of Study Materials
Yin Ha Vivian Chan, Delian Dawn Gaskell,
Mei Ah Tan and Lip Yan Felix Chao
1
8
16
23
82
100
109
117
Serdar Çiftci and Mehmet Akif Ocak 124
Marija Cubric, Karen Clark and Mariana
Lilley
Blanka Czeczotková, Kateřina
Kostolányová and Jana Šarmanová
134
145

Paper Title Author(s) Page
No.
Changing Academics, Changing Curriculum: How
Technology Enhanced Curriculum Design can
Deliver Strategic Change
Web Conferencing for us, by us and About us –
the Leeds Met Elluminate User Group
Tools for Evaluating Students’ Work in an
Interactive (Open) Virtual Space: Case Study of
an eLearning Course in an International Network
of Universities
Putting Things in Context - Designing Social
Media for Education
ii
Christine Davies 152
Mark de Groot, Gill Harrison and Rob Shaw 156
Jana Dlouhá, Martin Zahradník, Jiří Dlouhý
and Andrew Barton
Jon Dron, Terry Anderson and George
Siemens
Experimental Assessment of Virtual Students Michaela Drozdová, Ondřej Takács and
Jana Šarmanová
Priming for Modules: A Case Study Evaluation of
‘Pre-Workshop’ Online Resources for an
Executive MBA Course
Computer-Mediated Reading and its Impact on
Learners’ Reading Comprehension Skills
Do you see What I see? - Understanding the
Challenges of Colour-Blindness in Online
Learning
Researching in the Open: How a Networked
Learning Instance can Challenge Ethical
Decision-Making
Making Constraints and Decisions Explicit to
Support Project-Based Collaborative Learning
A Strategy for the Inductive Generation of
Learning Objects in Low-Tech Contexts
Cognitive Communication 2.0 in the Classroom –
Resonance of an Experience in Higher Education
To What Extent Does a Digital Audio Feedback
Strategy Support Large Cohorts?
Messages of Support: Using Mobile Technologies
to Support the Transition of Students on
Articulation Routes From Higher National Level to
Degree
Blended Learning at the Alpen-Adria-Universität
Klagenfurt
Evaluating the use of Social Networking Sites as
a Tool for Knowledge Sharing for Developing
Higher Education in Developing Countries: An
Exploratory Study of Egypt and Iraq
The Relationship Between Mindful Learning
Processes and Course Outcomes in Web-Based
Learning
Researching and Sharing – Business School
Students Creating a Wiki Glossary
Glenn Duckworth
Francisco Perlas Dumanig, Maya Khemlani
David and Rodney Jubilado
Colin Egan, Amanda Jefferies, Edmund
Dipple and David Smith
166
177
186
195
203
210
Antonella Esposito 218
Gert Faustmann 225
Ana Mª Fernández-Pampillón, Elena
Domínguez, José Mª Lahoz, Dolores
Romero, Isabel de Armas, Susana Palmaz
and Jorge Arús
Sérgio André Ferreira, Cornélia Castro and
António Andrade
235
246
Rachel Fitzgerald 256
Julia Fotheringham and Emily Alder
266
Gabriele Frankl and Sofie Bitter 274
Elaine Garcia, Ibrahim Elbeltagi, Sawasn
Al-Husseini and Ahmed Abdelkader
284
Danny Glick and Roni Aviram 295
Andrea Gorra and Ollie Jones 303

Paper Title Author(s) Page
No.
A Qualitative Evaluation of Academic Staff’s
Perceptions of Second Life as a Teaching Tool
Introducing and Using Electronic Voting Systems
in a Large Scale Project With Undergraduate
Students: Reflecting on the Challenges and
Successes
A Methodology for Incorporating Usability and
Accessibility Evaluations in Higher Education
The Virtual Learning Environment - Directions for
Development in Secondary Education
Mutlimodal Teaching Through ICT Education: An
e-Twinning Program as a Case Study of
Intercultural Exchange
Effectiveness and Learners’ Evaluation of
Combining Audio and Written Online Formative
Feedback for Language Learning
iii
Rose Heaney and Megan Anne Arroll 311
Amanda Jefferies 319
Anne Jelfs and Chetz Colwell 326
John Jessel 332
Paraskevi Kanari and Georgios Potamias
340
Rosario Kane-Iturrioz 345
Model of eLearning Project Evaluation Jana Kapounova, Jana Sarmanova and
Marketa Dvorackova
Bridging the Gap – From Teacher to eTeacher Andrea Kelz 363
Open Courses: The Next big Thing in eLearning? Kaido Kikkas, Mart Laanpere and Hans
Põldoja
Using a Social Networking Environment to
Facilitate Transition Into Higher Education
Evaluation of the Quality of Learning Scenarios
and Their Suitability to Particular Learners’
Profiles
Models of eLearning: The Development of a
Learner-Directed Adaptive eLearning System
Can eLearning Enhance Practice-Based Design
Courses?
Sophisticated Usability Evaluation of Digital
Libraries
Social Networks, eLearning and Internet Safety:
Analysing the Stories of Students
Learning Management Versus Classroom
Management in Technology-Supported Blended
Learning
How to Represent a Frog That can be Dissected
in a Virtual World
Learning by Wandering: Towards a Framework
for Transformative eLearning
Online Student Engagement: Unfulfilled Promises
or Promises Unfulfilled?
355
370
John Knight and Rebecca Rochon 377
Eugenijus Kurilovas, Inga Zilinskiene and
Natalija Ignatova
380
Stella Lee, Trevor Barker, and Vive Kumar 390
Jake Leith, Joanna Zara and Malcolm
McInnes
399
Stephanie Linek and 408
Birgy Lorenz, Kaido Kikkas and Mart
Laanpere
415
Arno Louw 423
Robert Lucas 434
Marie Martin and Michaela Noakes 442
Linda Martin, Gary Spolander, Imran Ali
and Beulah Maas
Personalized e-Feedback and ICT Maria-Jesus Martinez-Argüelles , Josep-
Maria Batalla-Busquets, Patricia Noguera-
Guerra and Ernest Pons-Fanals
449
456

Paper Title Author(s) Page
No.
Evaluation of Multimedia Tools and e-Feedback
in Virtual Learning Environments
iv
Maria-Jesús Martínez-Argüelles, Marc
Badia-Miro, Carolina Hintzmann and Dolors
Plana-Erta
Cyberbullying: A Workplace Virus David Mathew 473
Learning in Smart Environments – From Here to
There
Using Courseware for More Than Courses: You
May Already Hold the Lease on a Versatile Virtual
Meeting Space
An Analysis of Collaborative Learning as a
Prevalent Instructional Strategy of South Africa
Government eLearning Practices
Ideas for Using Critical Incidents in Oral
Debriefing From a Business Strategy Simulation
Game
Volume Two
eNOSHA and Moodle – the Integration of two
eLearning Systems
CASE Learning to Structure and Analyze a Legal
Decision
A Framework for Decision Support for Learning
Management Systems
Learning for Life - Building Blocks to Holistic
Education
Student's Characteristics for Note Taking Activity
in a Fully Online Course
Freeing Education Within and Beyond Academic
Development
Why Recording Lectures Requires a new
Approach
465
Peter Mikulecky 479
Karen Hughes Miller and Linda Leake 485
Peter Mkhize, Magda Huisman and Sam
Lubbe
492
Jonathan Moizer and Jonathan Lean 502
Peter Mozelius, Isuru Balasooriya and
Enosha Hettiarachchi
509
Antoinette Muntjewerff 517
Phelim Murnion and Markus Helfert 526
Shekhar Murthy and Devi Murthy 535
Minoru Nakayama, Kouichi Mutsuura and
Hiroh Yamamoto
550
Chrissi Nerantzi 558
Paul Newbury, Phil Watten, Patrick Holroyd
and Clare Hardman
eSubmission – UK Policies, Practice and Support Barbara Newland, Lindsay Martin and Andy
Ramsden
Harnessing the Internet for Authentic Learning:
Towards a new Higher Education Paradigm for
the 21st Century
Motivational Predictors of Academics’ Electronic:
Publishing in Nigerian Colleges of Education
Psycho-Social Predictors of Students With
Disabilities’ eLearning: Usage at the Federal
College of Education (Special), Nigeria
An Integrated Environment for Providing Learning
Style Information in a Unified Manner
Using Lifeworld-led Multimedia to Enhance
Learning
567
578
Abel Nyamapfene 586
Maruff Akinwale Oladejo and Adelua
Olajide Olawole
Adelua Olajide Olawole and Maruff
Akinwale Oladejo
Fatemeh Orooji, Fattaneh Taghiyareh and
Zahra Rahimi
Andy Pulman, Kathleen Galvin, Maggie
Hutchings, Les Todres, Anne Quinney,
Caroline Ellis-Hill and Peter Atkins
593
601
609
620

The Project Mobile Game Based Learning Thomas Putz 628
Using the Common Cartridge Profile to Enhance
Learning Content Interoperability
The Design and Development of an eLearning
System Based on Social Networking
Kansei Design Model for eLearning: A
Preliminary Finding
Changing Teacher Beliefs Through ICT:
Comparing a Blended and Online Teacher
Training Program
Moodle and Affective Computing: Knowing who´s
on the Other Side
Using Google Applications to Facilitate an
Effective Students’ Collaboration in the Teaching
of Informatics to Students of Secondary
Education
Training Methods and Tools: Could eLearning be
a Viable Solution to Solve SMEs Training
Problems?
Using Blended Learning to Develop Critical
Reading Skills
A Mobile aid Tool for Crafting Active Learning
Experiences
King-Sized eLearning - how Effective can an
Online Approach be for Large Module Groups?
v
Ricardo Queirós and José Paulo Leal
637
Andrik Rampun and Trevor Barker 646
Fauziah Redzuan, Anitawati Mohd
Lokman, Zulaiha Ali Othman and Salha
Abdullah
Bart Rienties, Simon Lygo-Baker, Natasa
Brouwer and Danielle Townsend
Manuel Rodrigues, Florentino Fdez-
Riverola and Paulo Novais
658
670
678
Eleni Rossiou and Erasmia Papadopoulou 686
Andrée Roy 697
Zuzana Šaffková 705
Ahmed Salem 716
Marie Sams, Mary Crossan and Kate
Mottram
Designing Effective Online Group Discussions Rowena Santiago, Amy Leh, and Minoru
Nakayama
The Game and the Alternating Roles of
Learner/Teacher as Facilitators of the Learning
Process in Organizations
Implementing and Evaluating Problem-Based
Virtual Learning Scenarios
The Evolution of eLearning Platform TESYS User
Preferences During the Training Processes
Teachers’ Skills set for Personal Learning
Environments
Bridging the Feedback Divide Utilising Inclusive
Technologies
Post-Academic Masters Course in Management
of Transfusion Medicine: Why the Difference in
Access to the eLearning Between Countries?
Engagement With Students in ‘Middle Ground’: A
Flexible Learning Environment Allowing
Simultaneous Access to Social Networking Sites
and Formal Academic Space
The Learning Management System as a Social
Mediator: A Story With a Happy Ending
Can the Medium Extend the Message? Using
Technology to Support and Enhance Feedback
Practices
724
731
Vitor Santos and Luis Amaral 739
Maggi Savin-Baden, Cathy Tombs and
Katherine Wimpenny
Adriana Schiopoiu Burlea, Amelia Badica
and Carmen Radu
Zaffar Ahmed Shaikh and Shakeel Ahmed
Khoja
746
754
762
Angela Shapiro and Aidan Johnston 770
Cees Th. Smit SibingaI 776
Anne Smith and Sonya Campbell 780
Dina Soeiro, António Dias de Figueiredo
and Joaquim Armando Gomes Ferreira
788
Mekala Soosay 794

Implementation and Analysis of an Online,
Student Centred Learning Environment to
Support Personalised Study
The Danger of the Downward Spiral: Teachers
and Digital Literacy
PeerWise - The Marmite of Veterinary Student
Learning
iSELF: An Internet-Tool for Self-Evaluation and
Learner Feedback
Using a Learning Management System for
Executing Role Play Simulations
The Effects of Self-Directed Learning Readiness
on Learning Motivation in Web 2.0 Environments
Usage Cases: A Useful way to Improve
Effectiveness of eLearning web Based Platforms
The Virtual Path to Academic Transition: Enabling
International Students to Begin Their Transition to
University Study Before They Arrive
Identifying and Locating Frames of Reference to
Inform the Design of Virtual Worlds in Higher
Education
Reaction Lecture: Text Messaging to Increase
Student Engagement in Large-Scale Lectures
A Holistic Approach to Instructional Design for
Blended Learning Environments
vi
Iain Stewart, William McKee and Kevin
Porteous
802
Caroline Stockman and Fred Truyen 811
Amanda Sykes, Paul Denny and Lesley
Nicolson
820
Nicolet Theunissen and Hester Stubbé 831
Tone Vold 841
Chien-hwa Wang and Cheng-ping Chen 846
Cristina Wanzeller and Orlando Belo 854
Julie Watson 862
Katherine Wimpenny, Maggi Savin-Baden,
Matt Mawer, Nicole Steils and Gemma
Tombs
Koos Winnips, Joost Heutink, and Hans
Beldhuis
870
878
Li Zhong Zhang 886
PhD Papers 895
Evaluating the Impact of an Arabic Version of an
Adaptive Learning System Based on the Felder-
Silverman’s Learning Style Instrument
Negotiating Doctoral Practices and Academic
Identities Through the Adoption and Use of Social
and Participative Media
Enabling Disruptive Technologies for Higher
Education Learning and Teaching
Exploring the Potential of a Mobile Computer lab
in a Developmental Context: The Teacher’s
Perspective
Collaborative eLearning in a Developing Country:
A University Case Study in Uganda
Applying the Multimedia Learning Theory in the
Primary School: An Experimental Study About
Learning Settings Using Digital Science Contents
Designing a U-Learning Course Platform for the
Identified Teacher Training Needs
Nahla Aljojo, Carl Adams, Huda Saifuddin
and Zainab Alsehaimi
897
Andy Coverdale 909
Michael Flavin 917
Fortunate Gunzo and Lorenzo Dalvit 925
Evelyn Kigozi Kahiigi, Henrik Hansson,
Mats Danielson, F.F Tusubira and Mikko
Vesisenaho
Fabio Serenelli, Enrico Ruggeri, Andrea
Mangiatordi and Paolo Ferri
932
943
Nazime Tuncay and Hüseyin Uzunboylu, 953
Work In Progress Papers 971
Someone to Talk to – Using Automated
Characters to Support Simulated Learning
Activities
Liz Falconer and Manuel Frutos-Perez 973

Extreme Scaffolding in the Teaching and
Learning of Programming Languages
Benefits and Barriers: Applying eLearning in the
Context of Organisational Change to Improve the
Learning Experience for Mature, Part-Time
Students
Posters with Papers
Investigating Student Engagement With an
Electronically Delivered Simulation of
Professional Practice
Instrumental Distance Learning in Higher Music
Education
Reflections on Academic Blogging as a Vehicle
for Professional Development
A Framework for Understanding Online Learning
Communities
Trust in Distributed Personal Learning
Environments: The Case Study of LePress PLE
Breaking Down Barriers: Development of a Wiki
Based Module to Enhance the International
Learning Experience
eLearning in German Higher Education:
Technology Implementation as a Challenge for
Organizational Change
vii
Dan-Adrian German 978
Simon McGinnes
982
Olivia Billingham 989
Karin Levinsen, Rikke Orngreen, Mie Buhl,
Marianne Løkke Jakobsen and Jesper
Andersen 1
Peps Mccrea
Sónia Sousa, David Ribeiro Lamas, José
Braga de Vasconcelos and Ilya Shmorgun 1
Sónia Sousa, David Ribeiro Lamas and
Vladimir Tomberg
Karen Strickland, Liz Adamson, Carolyn
Blight and Wendy McInally
Novita Yulianti, Michael Lund and Georg
Müller-Christ
993
997
1000
1006
1012
1015

Conference Committee
Conference Executive
Sue Greener, Brighton Business School, University of Brighton, UK
Asher Rospigliosi, Brighton Business School, University of Brighton, UK
Andrea Benn, Brighton Business School, University of Brighton, UK
Laurence Olver, Brighton Business School, University of Brighton, UK
Stephanos Avakian, Brighton Business School, University of Brighton, UK
Barbara Newland, Centre for Learning and Teaching, University of Brighton, UK
Steve Kilgallon, School Of Environment and Technology, University of Brighton, UK
Avril Loveless, School of Education, University of Brighton, UK
Lyn Pemberton, School of Computing, Engineering and Maths., University of Brighton, UK
Committee members
The conference programme committee consists of key people in the e-learning community around the world.
The following people have confirmed their participation:
Ariffin Abdul Mutalib (Universiti Utara Malaysia, Malaysia); Siti aishah Abdullah (University Technology Mara,
Kelantan, Malaysia); Tofan Cezarina Adina (Spiru Haret University, Romania); Wilfried Admiraal (Universiteit
van Amsterdam, Netherlands); Shafqat Ali (University of Western Sydney, Australia); Abdallah Al-Zoubi
(Princess Sumaya University for Technology, Jordan); Margarida Amaral (University of Porto, Portugal);
Antonios Andreatos (Hellenic Air Force Academy, Greece); Alla Anohina (Riga Technical University, Latvia);
Jane Ardus (Stevenson College,Edinburgh, UK); Mohamed Arteimi (7th of April University, Tripoli, Libya);
William Ashraf (University of Sussex, UK); Bunyamin Atici (Firat University, Turkey); Anders Avdic (Orebro
University, Sweden); Simon Bachelor (Gamos, Reading, UK); Joan Ballantine (University of Ulster, UK);
Trevor Barker (University of Hertfordshire, UK); Josep-Maria Batalla (Universitat Oberta de Catalunya,
Spain); Orlando Belo (University of Minho Campus de Gualtar, Portugal); David Benito (Public University of
Navarre, Pamplona, Spain); Yongmei Bentley (University of Luton, UK); Daniel Biella (University of Duisburg-
Essen, Germany); Radu Bilba (George Bacovia University,, Romania); Eric Bodger (University of
Winchester, UK,); Stephen Bowman (Ravensbourne College of Design and Communication, UK); Willem-
Paul Brinkman (Delft University of Technology, Netherlands); Ann Brown (CASS Business School, London,
UK); Mark Brown (Massey University,Palmerston North, New Zealand); Giuseppe Cannavina (University of
Sheffield, UK); Sven Carlsson (School of Economics and Management, Lund University, Sweden); James
Carr (University of Newcastle, UK); Maggie Carson (Edinburgh University, UK); Antonio Cartelli (University of
Cassino,, Italy); Rommert Casimir (Tilburg University, The Netherlands); Ivana Cechova (University of
Defence, Czech Republic,); Maria Celentano (University of Lecce, Italy); Athina Chatzigavriil (LSE, London,);
Satyadhyan Chickerur (M.S.Ramaiah Institute of Technology, Bagalore, India); Burhan China (PDSA,
Somalia); Barbara Class (University of Geneva, Switzerland); Lynn Clouder (Coventry University, UK);
Thomas Connolly (University of West of Scotland, UK); Ken Currie (Edinburgh University, UK); Valentina
Dagiene (Institute of Mathematics and Informatics, Vilnius, Lithuania); Mark De Groot (Leeds Metropolitian
University, UK); Antonio De Nicola (ENEA, Italy); Carmen De Pablos Heredero (Rey juan Carlos University,
Spain); Rajiv Dharaskar (GH Raisoni College of Engineering, Nagpur, India); Vicenzo Di Lecce (Politecnico
di Bari, Italy); Martina Doolan (University of Hertfordshire, UK); Christopher Douce (Institute of Educational
Technology, Walton Hall, UK); Yanqing Duan (University of Luton, UK); Jane Eberle (Emporia State
University, USA); Colin Egan (University of Hertfordshire, Hatfield, UK); Bulent Gursel Emiroglu (Eskisehir
Yolu Baglica Mevkii, Turkey); Chew Esyin (University of Glamorgan, UK,); Ariwa Ezendu (London
Metropolitan University, Uk); Bekim Fetaji (South East European University, Tetovo, Macedonia); Andrea
Floros (Ionian University, Greece); Duncan Folley (Leeds Metropolitian University, England); Katie Goeman
(Free University of Brussels (VUB), Belgium); Colin Gray (Edinburgh Napier University, Scotland); Susan
Greener (University of Brighton, UK); David Guralnick (Kaleidoscope Learning, New York, USA); Richard
Hall (De Monfort University, Leicester, UK); Patricia Harvey (Greenwich University, London, UK); Thanos
Hatziapostolou (International faculty of the university of sheffield, Greece); Rose Heaney (University of East
London, UK,); Alan Hilliard (University of Hertfordshire, Hatfield, UK); Uwe Hoppe (Bildungswerk der
Sächsischen Wirtschaft gGmbH, Germany); Md. Fokhray Hossain (Daffodil International University,
Bangladesh); Stefan Hrastinski (Uppsala University, Sweden); BALDE IDIATOU (NOBLE GROUP
ORGANISED SOLUTIONS, GUINEA); Antonin Jancarik (Faculty of education, Charles University, Czech
Republic); Amanda Jefferies (University of Hertfordshire, Hatfield, UK); Runa Jesmin (Global Heart Forum,
UK); Aidan Johnston (Glasgow Caledonian University, UK); Paul Jones (University of Glamorgan, UK);
Geraldine Jones (University of Bath, UK,); Jowati Juhary (National Defence University of Malaysia,
viii

Malaysia); Tuomo Kakkonen (University of eastern Finland, Finland); Michail Kalogiannakis (School of
Pegadogical and Technicological Education, ASPETE, Crete); Clifton Kandler (University of Greenwich,
UK,); Jana Kapounova (University of Ostrava , Czech Republic); Andrea Kelz (University of Applied Sciences
Burgenland,Campus Pinkafeld, Austria); Saba Khalil (Virtual University of Pakistan, Lahore, Pakistan);
Jasna Kuljis (Brunel University, UK); Sunaina Kumar (Indira Gandhi National Open University, New Delhi,
INDIA); Swapna Kumar (University of Florida, USA); venkata Durga kumar (Sunway University College,
Malaysia,); Blair Kuntz (University of Toronto, Canada,); Eugenijus Kurilovas (Vilnius Gediminas technical
university / institute of mathmatics and informatics of Vinius University, Lithuania); Eleni Kyza (Cyprus
University of Technology, Lemesos, Cyprus); Maria Lambrou (University of the Aegean Business School,
Greece); Andy Lapham (Thames Valley University, UK); Mona Laroussi (Institut National des Sciences
Appliquées et de la Technologie, Tnis and Lille, Tunisia); Deepak Laxmi Narasimha (University of Malaya,
Malaysia); Fotis Lazarinis (Applied Informatics in Management and Finance, Greece); Denise Leahy (Trinity
College, Dublin, Ireland); Kate Lennon (Glasgow Caledonian University, UK); Mariana Lilley (University of
Hertfordshire, UK); Jorgen Lindh (Jonkoping International Business School, Sweden); Gi-Zen Liu (National
Cheng Kung University, Taiwan); Ying Liu (Cambridge University, UK); Jenny Lorimer (University of
Hertfortshire, UK); Sam Lubbe (University of South Africa, South Africa); Nick Lund (Manchester
Metropolitan University, England,); Alejandra Magana (Purdue University, United States of America, United
States of America); Adnan Mahmood (University of Jinan, P.R.China,); Francis Maietta (Real Thinking
Company, UK); Christina Mainka (Heidelberg University , Germany); Chittaranjan Mandal (School of IT,IIT
Kharagpur, India); Augostino Marengo (University of Bari, Italy); Maria J Martinez-Arguelles (Universitat
Oberta de Catalunya, Spain); Sephanos Mavromoustakos (Cyprus College, Cyprus); Erika Mechlova
(University of Ostrava, Czech Republic); Cherifa Mehadji (University of Strasbourg, France); Rosina Merry
(the school of Education The University of Waikatio, New Zealand); Linda Joy Mesh (Universita degli Studi di
Siena, Italy); Jaroslava Mikulecka (University of Hradec Kralove, Czech Republic); Peter Mikulecky
(University of Hradec Kralove, Czech Republic); Mike Mimirinis (Middlesex University, London, UK); Julia
Mingullon (Universitat oberta de catalunya, Spain); Ali Moeini (University of Tehran, Iran); Johann Moller
(University of South Africa (UNISA), South Africa,); Peter Monthienvichienchai (Insitute of Education,
London, UK); Pam Moule (University of the West of England, Bristol, UK); Radouane Mrabet (ENSIA,
Morocco); Minoru Nakayama (Tokoyo Institute of Technology, Japan); Julian Newman (Glasgow Caledonian
University , UK); Chetsada Noknoi (Thaksin University, Songkhla, Thailand); Abel Nyamapfene (University of
Exeter, United Kingdom); Sinead O’Neill (Waterford Institute of Technology, Ireland); Kamila Olsevicova
(Univeristy of Hradec Kralove, Czech Republic); Rikke Orngreen (Aarhus University, Denmark); Jalil
Othman, (University of Malaya ,Malaysia); Kutluk Ozguven (Dogus University, Turkey); Ecaterina Pacurar
Giacomini (Louis Pasteur University, FRANCE); Alessandro Pagano (University of Bari, Italy); Vasileios
Paliktzoglou (University of eastern Finland, Finland); Stefanie Panke (University of Ulm, Germany); George
Papadopoulos (University of Cyprus, Cyprus); Iraklis Paraskakis (South East European Research Centre
(SEERC) Research Centre of the University of Sheffiled, Thessaloniki, Greece); Vivien Paraskevi (TECFA -
FPSE, Educational Technology Unit, University of Geneva, Switzerland, Switzerland); Angie Parker (Anthem
College Online, USA,); Paul Peachey (University of Glamorgan, Treforest, UK); Arna Peretz (Ben Gurion
Univeristy of the Negev, Omer, Israel); Christine Perry (University of the West of England, Bristol, UK);
Donatella Persico (Istituto Tecnologie Didattiochje-Consiglio Nazionale Ricerche, Genova, Italy); Pit
Pichappan (Annamalai University, India); Selwyn Piramuthu (University of Florida, Gainesville, USA); Michel
Plaisent (University of Quebec in Montreal, Canada); Lubomir Popelinsky (Masaryk University, Czech
Republic); Andy Pulman (Bournemouth University, UK); Muhammad Abdul Qadir (Mohammad Ali Jinnah
University, Islamabad, Pakistan); Ricardo Queirós (ESEIG/KMILT & CRACS/INESC, Portugal); Susannah
Quinsee (City University, London, UK); Abdul Rafay (Asia Pacific University College of Technology &
Innovation, Malaysia); Liana Razmerita (Copenhagen Business School, Denmark); Christopher Reade
(Kingston University, UK); Hugo Ribeiro (University of Porto, Portugal,); Vivien Rolfe (De Monfort University,
Leicester, UK); Asher Rospigliosis (University of Brighton, UK); Florin Salajan (North Dakota State University
, Canada); David Sammon (Univesity College Cork, Ireland); Gustavo Santos (University of Porto, Portugal);
Venkat Sastry (Defence College of Management and Technology, Cranfield University, UK); Guy Saward
(University of Hertfordshire, UK,); Brian Sayer (University of London, UK); Jeanne Schreurs (Hasselt
University, Diepenbeek, belgium); Jane Secker (London School of Economics, UK); Angela Shapiro
(Glasgow Caledonian University, Scotland); Aileen Sibbald (Napier University, Scotland, UK); Petia Sice
(University of Northumbria, Newcastle-upon-Tyne, UK); Gurmeet Singh (The University of The South Pacific,
Suva , Fiji, Fiji); Cees Th. Smit Sibinga (Academic insitute for the international development of transfusion
medicine, The Neverlands); Alisdair Smithies (Manchester Medical School, UK); Keith Smyth (Napier
University, Edinburgh, UK); Bent Soelberg (Copenhagen Business School, Denmark); Or Kan Soh
(University Tunku Abdul Rahman (UTAR), Malaysia); Yeong-Tae Song (Towson University, Maryland, USA);
Michael Sonntag (FIM, Johannes Kepler University, Linz, Austria); Rumen Stainov (University of Applied
Sciences, Fulda, Germany); John Stav (Sor-Trondelag University College, Norway); Roxana Taddei
(Université Clermont Ferrand 2, Montpellier, France); Yana Tainsh (University of Greenwich,, UK); Heiman
Tali (The Open University, Israel); Bénédicte Talon (Université du Littoral, France); Marian Theron (False
ix

Bay College, Tokai, South Africa); John Thompson (Buffalo State College, USA); Claudine Toffolon
(Université du Mans - IUT de Laval, France); Eulalia Torras-Virgili (Open University of catalonia, Spain);
Kathryn Trinder (Glasgow Caledonian University, UK); Christopher Turner (University of Winchester , UK);
Karin Tweddell Levinsen (Danish University of Education, Denmark); Aimilia Tzanavari (University of Nicosia,
Cyprus); Huseyin Uzunboylu (Near East University, CYPRUS); Linda Van Ryneveld (Tshwane University of
Technology, Pretoria, South Africa); Carlos Vaz de Carvalho (Porto Polytechnic, Portugal); Andreas Veglis
(Aristotle University of Thessaloniki, Greece); Bruno Warin (Université du Littoral, Calais, France); Fahad
Waseem (University of Northumbria, Middlesbrough, UK); Garry Watkins (University of Central Lancashire,
UK); Anne Wheeler (Aston University, UK); Steve Wheeler (Faculty of Education, University of Plymouth,
UK); Nicola Whitton (Manchester Metropolitan University, UK); Roy Williams (University of Portsmouth, UK);
Shirley Williams (University of Reading, UK); Rowena Yeats (University of Birmingham, UK); Panagiotis
Zaharias (University of the Aegean, Greece); Mingming Zhou (Nanyang Technological University,
Singapore); Chris Zielinski (External relations and Governing Bodies, World Health Organization, Geneva,
Switzerland); Anna Zoakou (Ellinogermaniki Agogi, Greece);
x

Preface
These Proceedings represent the work of contributors to the 10th European Conference on e-Learning,
ECEL 2011, hosted this year by Brighton Business School, University of Brighton, UK. The Conference Chair
is Sue Greener, and the Programme Chair is Asher Rospigliosi, both from Brighton Business School, UK.
The conference will open with a keynote address by Don Clark, former CEO of EPIC software and winner of
the ‘Outstanding Achievement in e-learning Award’. Expect to be enlivened by his challenge: “Don't lecture
me!, ..and why technology is only scalable solution”. Also on the first day we have Anne Boddington from the
University of Brighton speaking on the topic of “Designing Education and Reshaping Learning”. The second
day will be opened by Professor Gráinne Conole, Director of the Beyond Distance Research Alliance at the
University of Leicester in the UK with a talk entitled “Trajectories of learning - new approaches and
directions".
With an initial submission of 239 abstracts, after the double blind, peer review process there are 108
academic papers, 7 Phd Papers, 3 Work in Progress papers and 3 non academic papers in these
Conference Proceedings. These papers reflect the truly global nature of research in the area with
contributions from Australia, Austria, Belgium, Canada, China, Czech Republic, Denmark, Estonia, France,
Germany, Greece, India, Iran, Ireland, Israel, Italy, Japan, Jordan, Lithuania, Malaysia, Nigeria, North
Cyprus, Norway, Pakistan, Portugal, Romania, Saudi Arabia, Singapore, Slovakia, South Africa, Spain,
Sweden, Taiwan, The Netherlands, Turkey, United Kingdom and the United States.
A selection of papers – those agreed by a panel of reviewers and the editor will be published in a special
conference edition of the EJEL (Electronic Journal of e-Learning www.ejel.org ).
We know you will enjoy and be stimulated by this conference. We hope you will also revel in the uniqueness
of Brighton.
Sue Greener and Asher Rospigliosi
November 2011
xi

Biographies of Conference Chairs, Programme Chairs and
Keynote Speakers
Conference Chairs
Dr Sue Greener is a University teacher: HRM, Business Context, Research Methods
and Learning & Development and has received a Teaching Excellence award from the
University of Brighton and is Programme Leader for the Foundation Degree in
Business. Sue is also the Course Director: online final year undergraduate course with
students in diverse world regions, her researcher interests are focused on e-learning
strategy, teacher development and reflective learning. Sue is the co-founder of the
Business e-Learning Research Group and a member of the CROME research group
on employment issues at Brighton Business School. Her Doctoral research focused
on exploring students’ readiness for online learning. Sue holds a BA, MBA, EdD, FHEA and is a Chartered
Fellow of CIPD.
Programme Chair
Asher Rospigliosi lecturers on e-commerce, management information systems, IS
strategy, public sector IS and digital marketing at the University of Brighton. His
research interests extend to e-learning and innovation in SMEs. Asher is a cofounder
of the Business e-Learning Research Group and a member of the CROME
research group on employment issues at Brighton Business School.
Keynote Speakers
Gráinne Conole is Professor of e-Learning at the Open University, with research
interests in the use, integration and evaluation of Information and Communication
Technologies and e-learning and impact on organisational change. She was
previously chair of educational innovation at Southampton University and before that
Director of the Institute for Learning and Research Technology at the University of
Bristol. She has extensive research, development and project management
experience across the educational and technical domains She serves on and chairs a
number of national and international advisory boards, steering groups, committees
and international conference programmes.
Donald Clarke was CEO and one of the original founders of Epic Group plc, which
established itself as the leading company in the UK e-learning market, floated on the
Stock Market in 1996 and sold in 2005. Describing himself as ‘free from the tyranny of
employment’, he is now a board member of Ufi LearnDirect (Government agency
delivered e-learning to 2.8 million learners), Caspian Learning (learning games tool
provider), LearningPool (content provider), Brighton Arts Festival, and a school
governor. Donald has won many awards for the design and implementation of elearning,
notably the ‘Outstanding Achievement in e-learning Award’.
Anne Boddington is Dean of the Faculty of Arts at the University of Brighton.
Educated and qualified as an Architect and subsequently as a Cultural Geographer,
she currently leads a Faculty of .3,900 students and a portfolio that includes the Visual
and Performing Arts, Architecture, Design, Media Studies, Literature, Languages and
Humanities. She is also Co-director of the ADM HEA Subject Centre, which we are
proud to host at the University. Anne is a Fellow of the Royal Society of Arts (RSA)
and an affiliate member of the Royal Institute of British Architects (RIBA) and is an
elected member of the executive for the Council for Higher Education in Art & Design
(CHEAD). Her initial research interests were rooted in the design and development of
the urban and cultural landscape and identity but have expanded to include the strategic design and
development of learning and research space and its relationships to pedagogic practice and to educational
strategies and governance.
xii

Mini Track Chairs
Dr Antonios Andreatos is a Professor at the Computer Engineering Division of the
Hellenic Air Force Academy. He received the Diploma in Electrical Engineering from
the Univ. of Patras, the M.S. in Computer Engineering from the Univ. of
Massachusetts, the M.Ed in Adult Learning from the Hellenic Open Univ. and the
Ph.D. in Computer Engineering from the National Technical Univ. of Athens.
Research interests include e-Assessment, Active Learning methods, ICT & Web 2.0
& Open Resources in Education, Didactics of Computer Engineering, etc. He has
published over 60 papers in journals and conference proceedings and a book. He is
also involved in the scientific committees of many conferences in his fields of interest.
Orlando Belo is an associate professor in the Department of Informatics at Minho
University, Portugal. His main research topics are related with data warehouse design,
implementation and tuning, ETL services, database preferences, and distributed
multidimensional structures processing. During the last few years he was involved with
several projects in the decision support systems area designing and implementing
computational platforms for specific applications like fraud detection and control in
telecommunication systems, data quality evaluation, and ETL systems for industrial
data warehousing systems. More recently, he was developing some research work
establishing OLAP usage profiles and optimizing OLAP selection methods, applying some of the techniques
studied on these areas over typical e-learning scenarios.
Dr Colin Egan is a senior lecturer in the School of Computer Science, at the University
of Hertfordshire. Colin has had an interest in accessibility and accessibility issues for a
number of years and has presented his work to a range of International Conferences.
Dr Amanda Jefferies is a Reader in Technology Enhanced
Learning at the University of Hertfordshire. She is passionate about
promoting a positive learning experience for students in HE through careful choice of
technologies, to enhance their engagement with studies. She is well-known for
innovative research into understanding the student experience through using student
constructed reflective video and audio diaries, a technique she refined through her JISCsupported
‘Learner Journeys’ STROLL project during 2007-2009. She has presented
her research to international audiences across the UK, Europe and in North America.
Dr Eugenijus Kurilovas is a Research Scientist in Vilnius University Institute of
Mathematics and Informatics, an Associate Professor in Vilnius Gediminas Technical
University, and a Head of International Networks Department in the Centre of
Information Technologies in Education of the Ministry of Education and Science of
Lithuania. He has published over 50 scientific papers, 2 books chapters and 2
monographs, participated in many large scale international scientific projects. He is a
member of 12 scientific committees of the international journals and conferences. His
papers presented on ECEL-2008, 2009 and 2010 have been selected for publication in
Electronic Journal of e-Learning.
Dr Barbara Newland is a National Teaching Fellow with over 17 years experience of
leading educational developments in Higher Education. She is a Principal Lecturer in the
Centre for Learning and Teaching at the University of Brighton. Previously, Barbara was
Blended Learning Co-ordinator at Glasgow Caledonian University, Manager of the
Educational Development Services at Bournemouth University and the Learning
Technology Team at Durham University.
Dr Keith Smyth is a Senior Teaching Fellow and Senior Lecturer in Higher Education at
Edinburgh Napier University, where he leads the MSc Blended and Online Education
(www.napier.ac.uk/ed/boe). Keith is interested in how simple uses of technology can
enhance learning, and where blended and online approaches can underpin community
and work-based learning, broader skills development, and more inclusive educational
practices.
xiii

Biographies of contributing authors (in alphabetical
order)
Samuel Adu Gyamfi is currently a PhD Fellow at the centre for Communication, Media and Information
technologies (CMI) at Aalborg University, Copenhagen. He is currently researching on developing the
Personal Transferable Skills (PTS) of the university graduate through E-learning.
Imran Ali is a Senior Learning technologist at Coventry University. He is passionate about the potential of
technology in education. His background is in multimedia production and education. He currently supports
staff across the Faculty of Health and Life Sciences in making effective use of technology for teaching,
learning, assessment and research purposes.
Nahla Aljojo worked as an IT manager in the economics and administration department at King Abdul Aziz
University in Saudi Arabia. She has a master’s degree in computer systems and Information Technology
from the Washington International University (USA) and is also a Ph.D. doctoral student at the School of
Computing at the University of Portsmouth (UK).
Hussein Al-Yaseen holds a PhD degree in Information Technology from the University of Brunel, UK. His
areas of interest are in information systems evaluation, project management and the implementation of
information systems in developing countries. Hussein is currently an associate professor at the Management
Information Systems Department at Al-Ahliyya Amman University, Jordan.
António Andrade PhD in Technologies and Information Systems MSc in Information and Management
Director of the MSc in Information and Documentation.
Roni Aviram is Chair of the Center for Futurism in Education, Department of Education, Ben-Gurion
University. Aviram is interested in the impact of ICT on education and society, and in structuring theoretical
and practical change processes in education. He has led R&D projects dedicated to designing virtual
lifelong learning environments for enhancing human development and flourishing.
Jonathan Barkand has a B.S. in Technology Education and an M.S. in Multimedia Technology from
California University of Pennsylvania. He is currently completing his dissertation work at Duquesne
University for a doctorate in Instructional Technology. His area of focus is online education and its effects on
teachers and students.
Trevor Barker is Principal Lecturer and University Fellow in Teaching and Learning in the Department of
Computer Science at the University of Hertfordshire. He obtained a PhD for research into developing and
evaluating adaptive multimedia computer systems in education. His recent research relates to the design
and evaluation of adaptive educational systems and the affordances of virtual worlds such as Second Life
for study and work.
Josep Batalla has a Degree and a Phd in Economics from the University of Barcelona. He is a Professor of
Economics and Business Studies at the Universitat Oberta de Catalunya (UOC), where he teaches courses
in the field of applied economics. Resident scholar at the Internet Interdisciplinary Institute
Olivia Billingham is a research fellow in the e-learning development unit at the University of the West of
England. Her research interests include student perceptions and experiences of technology enhanced
learning.
Latefa Bin Fryan (BSc, MSc) is a Researcher in the School of Information Systems, Computing and
Mathematics at Brunel University, UK, where she is pursuing her PhD in the area of e-Learning systems.
She received M.Sc. (2009-2010) in Information Systems Management from Brunel University, UK. She
has published papers in several international conferences.
Sonya Campbell As senior service manager within Library and Information Services, Sonya supports the
delivery of flexible learning environments and the development of services designed to maximise learner
productivity and success. Current research includes the opportunities social media may bring to enable
student engagement, learning and collaboration. Teaching themes are corporate strategy, effective
management and customer service.
xiv

Cornélia Castro is a PhD student in Educational Sciences: Educational Computing Master in Quality
Control: Environment . Graduation in Pharmaceutical Sciences High School Chemistry and Physics teacher
. Research interests: ICT integration in the classroom, Web 2.0 tools in Education, Open Educational
Resources, Learning Objects, Repositories and Social Media in Education
Martin Cápay works as a professor assistant at the Department of Informatics. He deals with the theory of
teaching informatics subjects, mainly programming. He participates in the projects aimed at the usage of new
competencies in teaching and also in the projects dealing with learning in virtual environment using elearning
courses.
Tim Cappelli is an experienced Project Manager who has successfully managed and contributed to a range
of Technology Enhanced Learning and change management projects. Tim has developed a rich
understanding of the application of informal and collaborative learning through the use of social networking
tools that reinforce the links between individual and organisational development.
Ivana Cechova graduated from the Faculty of Arts at Masaryk University, Brno, with specialisations in
pedagogy, English and Russian language and literature. She has worked as Head of Research and Deputy
Head of the Language Department at the Faculty of Economics and Management. Currently she is a senior
lecturer at the Language Training Centre of the University of Defence.
Vivian Chan Yin Ha is a Senior Instructor of the Independent Learning Centre at the Chinese University of
Hong Kong. Her research interests include language teaching, learning autonomy, modern and
contemporary Chinese literature, and gender studies.
Serdar Ciftci is a doctoral candidate studying in Department of Educational Technology, Institute of
Educational Studies of Gazi University. He is also an expert working in Directorate of Computer Center of
Gazi University. His researcher interests are focused on e-learning, metacognition, web-based learning
strategies.
Chetz Colwell is a Project Officer in the Learning and Teaching Development team of the Institute of
Educational Technology at the Open University. Her role is to support the university in making online
teaching materials accessible to disabled students by conducting technical accessibility testing, and
conducting student evaluations.
David Comiskey lectures in Architectural Technology at the University of Ulster. He is a Fellow of the Higher
Education Academy and his research interests include the use of video and screencasting technology to
improve the student learning and feedback experience. He was recently funded by the Higher Education
Academy to continue researching this area.
Mary Crossan has worked at Coventry University for six years and has industry experience of over 16
years. Her experience has been mainly in the financial area but with a high emphasis on managing people
and resources for project delivery. Research interests are in the areas of improving student experience and
work-based learning approaches.
Marija Cubric is a Principal Lecturer at the UH Business School, where she teaches information systems,
and project management related subjects. She is also a member of the UH Learning and Teaching Institute
that promotes the use of blended learning practices across the University. Her research interests include
educational technologies, and their influence on learning and teaching processes.
Maya Khemlani David (Faculty of Languages and Lingusitics, University of Malaya) received the Linguapax
Award in 2007. She is an Honorary Fellow of the Chartered Institute of Linguists, United Kingdom and an
Honarary Member of the Foundation of Endangered Languages. Her areas of specialisation are crosscultural
communication, discourse analysis and language maintenance and language shift.
Christine Davies is a Senior Lecturer in Technology-Enhanced Learning at the University of Glamorgan.
Originally a Biology teacher, she has taught in schools and FE, and continues to teach on OU science
courses and on Wales-based post-graduate education courses. She has also worked for JISC RSC Wales
as an e-Learning Advisor.
Mark de Groot works for the Centre for Teaching and Learning team at Leeds Metropolitan University. He
has more than 15 years’ experience initiating and supporting learning technology related staff development
activities across and beyond the University.
xv

Jiří Dlouhý is a specialist on cybernetics and works at the Environmental Center of the Charles University as
head of the Environmental Education department. He is also a member of executive board of the European
Environmental Bureau, chairman of the Society for Sustainable Living; and a founding member of the
International Society of Information Specialists
Elena Dominguez-Romero holds a Master’s degree in Innovative Teaching in Tertiary Education (Huelva,
2008). She has participated in several Research Projects concerning Innovative Teaching (Universidad de
Huelva: 2005, 2006a, 2006b; UCM: 2009, 2010). The results of this ECEL paper are derived from her
participation in a current project funded by UCM (268-2010 UCM).
Jon Dron is a member of the Technology Enhanced Knowledge Research Institute (TEKRI) at Athabasca
University, Canada, an Honorary Faculty Fellow in the Faculty of Education & Sport, University of Brighton,
UK, and a National Teaching Fellow of the UK Higher Education Academy.
Michaela Drozdova graduated in 2009 with a Masters degree in Secondary Teaching Mathematics and
Informatics approbation. In September 2009 she began teaching high school Teleinformatics, while
commencing doctoral studies in the Department of Information and Communication Technology in Education
at the University of Ostrava.
Glenn Duckworth currently works at the University of the West of England in the role of e-Learning
Development Officer. He is employed by the Faculty of Business and Law and is a member of the e-Learning
Development Unit. He previously worked at the University of Huddersfield as Senior Lecturer, Researcher
and Chief Psychology Technical Officer.
Ibrahim Elbeltagi is a lecturer in information and knowledge management at the School of Management,
University of Plymouth. His publications are related to electronic commerce, adoption of ICT, information
systems in developing countries, social networking and knowledge management. He has more than 30
journal and conferences papers published in national and international journals and conferences.
Antonella Esposito. Elearning practitioner since 1996, Antonella led the CTU (E-learning center) of the
University of Milan for seven years. Currently she is a PhD candidate in the E-learning program, Open
University of Catalonia and is completing a dissertation on digital scholarship for the MRes in Educational
and Social Research, Institute of Education, University of London.
Liz Falconer completed her MPhil at Manchester Polytechnic in 1992 and her PhD at Salford University in
2001. In 2001 she moved to the University of Bath as Director of the Centre for Distance Education. She is
now Professor of Technology Enhanced Learning at the University of the West of England.
Gert Faustmann studied Information Technology at the Technical University Berlin. From 1992 to 2001 he
was a software developer (Siemens AG), researcher (Fraunhofer Institute for Software and Systems
Engineering) and consultant (debis - later T-Systems). He is now professor and course director for the
division of computer science at Berlin School of Economics and Law.
Sérgio André Ferreira has a Degree in Geography, specialization in Education Sciences. Master in
Sciences of the Education, specialization in Education Computer Sciences. PhD student in Sciences of the
Education, specialization in Education Computer Sciences (Grad Student). Research interests in Technology
Enhanced Learning Environments. Geography teacher and Teacher Trainer in the areas of ICT and
Geography.
Rachel Fitzgerald, MSc is a Senior Lecturer (Information Science) at Northampton Business School,
University of Northampton and a PhD student at the University of Lancaster. A former learning technologist,
she holds CMALT certification and a Teaching Fellowship Award from the University of Northampton.
Current research interests include Networked Learning and Distance Education Innovation.
Michael Flavin is a research student at the Open University’s Institute for Educational Technology. He
previously gained a PhD in nineteenth-century literature (Kent, 1999). He has taught at King’s College
London and the Open University, and is currently Head of Learning and Teaching at the IFS School of
Finance, a degree-awarding business school in London.
Gabriele Frankl is head of the eLearning Service department at the Alpen-Adria-Universität Klagenfurt since
2008. Her main research interests are eLearning and Blended Learning, Knowledge Management and winwin
constellations as well as self-organisation processes. She has successfully implemented eLearning and
Knowledge Management systems in the production industry.
xvi

Manuel Frutos-Perez is the Leader of the E-learning Development Unit at the University of the West of
England, Bristol, UK. He has worked for several universities in the UK, Spain and Austria. He has also
written and contributed to numerous textbooks and e-learning interactive materials.
Elaine Garcia is undertaking a part time PhD at the University of Plymouth focusing on the use of blogs
within education. This is alongside employment as Head of Operations and Resources at Plymouth College
of Art. Research interests include: Social Networking Sites, Social Media, Blogs, Social Learning and
Knowledge Management.
Danny Glick is an international e-learning consultant and an expert in the implementation of large-scale
computer-based solutions. He has worked as a consultant for both public and private organizations. Glick is
a research fellow at the Center for Futurism in Education at Ben-Gurion University, where he conducts
research on mindful learning processes in Web-based learning environments.
Andrea Gorra is a Senior Lecturer in the Business School at Leeds Metropolitan University (UK). Prior to
this she has worked as a researcher in the area of learning technologies. Further research interests include
grounded theory methodology, and the use of social software and mobile learning devices for assessment,
learning and teaching.
Rose Heaney is a learning technology advisor and Teaching Fellow at the University of East London with
responsibility for facilitating the use of learning technologies in the schools of Health, Sport & Bioscience and
Psychology. One of her research interests is the use of Second Life® in clinical education as well as for
more general purposes.
Carolina Hintzmann has a Degree in Economics from the Universitat de Barcelona, Spain. She is a teacher
in the Economics and Business Sciences Department, Universitat Oberta de Catalunya (UOC), Barcelona,
Spain
Karen Hughes Miller, PhD, is Director for Graduate Medical Education (GME) Curriculum Design,
Evaluation and Research, University of Louisville School of Medicine. Her work in adult education includes
teacher education, military education, and currently medical education. She is interested in curriculum design
and analysis and education research, and enjoys the challenges of teaching advanced content to very bright
adult learners.
Anne Jelfs is Manager of the Learning and Teaching Development team of the Institute of Educational
Technology at the Open University. She manages a team that conducts evaluations of OU materials
including usability testing. She also leads on a university-wide initiative to embed accessibility within the
university's production and delivery systems.
John Jessel is head of the PhD Programme in the Department of Educational Studies at Goldsmiths,
University of London. His research activities focus on the social and cognitive processes that underlie
learning and development both inside and outside the school setting and within the context of the use of
digital technologies.
Aidan Johnston, BSc. (Hons), PGDip, AHEA, is a Blended Learning Advisor with many years’ experience
designing and embedding video and audio resources to enhance the teaching and learning experience of the
learner. He has collaborated on projects involving the JISC, BBC Information & Archives, The National
Science Foundation, The Higher Education Academy and REAP.
Rosario Kane-Iturrioz is a Senior Lecturer in Spanish at Coventry University and also a member of the
Chartered Institute of Linguists and the Higher Education Academy (UK). Her research interests are the
integration of Information and Communication Technology (ICT) into language learning and assessment and
the use of VLEs as an effective learning tool.
Jana Kapounova is an associate professor at the Department of ICT, University of Ostrava in the Czech
Republic. She teaches subjects as Educational Technology, ICT in Education, eLearning. She guarantees
studies of ICT in Education in bachelor, master and Ph.D. degrees. Her research field is eLearning and
evaluation of its quality.
Shakeel Khoja is a Commonwealth Academic Fellow. He received his Ph.D. from the University of
Southampton, UK, in 2001, and is working as a Professor at IBA, Karachi. Shakeel’s research interests
xvii

include Learning Technologies, Web Technologies, and Internet programming. He has a professional career
of over 15 years and has fifty research publications to his credit.
Kaido Kikkas is an Associate Professor at Estonian Information Technology College and an Associate
Professor of Social and Free Software at Tallinn University. His main focus of research and teaching has
been a wide range of ethical and social issues of IT (information society, licensing, hacker culture etc).
John Kleeman is the Founder and Chairman of Questionmark. John wrote the first version of the
Questionmark assessment software system and founded Questionmark in 1988 to market, develop and
support it. John has been heavily involved in assessment software development for over 20 years and has
also participated in several standards initiatives including IMS QTI.
John Knight started his career teaching EAP at universities in Turkey and Qatar, but developed an interest
in computers and education in the late 90s and has worked in the field ever since. He now works in the
Learning Development Unit, Bucks New University, where he has a particular interest in technologyenhanced
learning and teaching.
Katerina Kostolanyova has worked in the Faculty of Education, Institute of Information and Communication
Technologies in Ostrava since 1999. She specializes in eLearning technology. Her further professional
growth focuses on students’ learning styles in the e-Learning environment. She is an author and co-author of
almost thirty professional articles and ten e-contents.
Vivek Kumar is an Associate Professor in the School of Computing and Information Systems at Athabasca
University researching in online learning technologies. Vivek's research centres around Technology-
Enhanced Teaching, Learning, and Research that extends to mixed-initiative human-computer interaction,
causal modelling, model tracing, automated instructional design, lifelong learning, cognitive modelling of selfregulated
and co-regulated learning, semantics of online learning interactions, and competency modelling in
portfolios.
Eugenijus Kurilovas is Research Scientist in Vilnius University Institute of Mathematics and Informatics and
Associate Professor in Vilnius Gediminas Technical University. He is a member of 12 scientific committees,
has published more than 50 scientific papers, and participated in more than 10 EU-funded projects. His
papers for ECEL 2008, 2009 and 2010 were published in Electronic Journal of e-Learning.
Linda Leake, M. Ed., serves as a computer specialist, University of Louisville Blackboard technical
support/trainer and instructional designer for the Delphi Center for Teaching and Learning. She received
both her Bachelor of Science in Business Administration degree in 1994 and her Master of Education in
Instructional Technology in 2003 from the University of Louisville.
Jake Leith leads a Business and Professional Practice in Fashion & Textiles and Design & Craft at the
University of Brighton, and has over 20 years’ experience in successfully running his own design practice
and 13 years as a teacher-practitioner. Leith is currently President Elect of the Chartered Society of
Designers.
Birgy Lorenz is an eSafety trainer in Estonia. Her activities include being part of developing National
Curricula ICT syllabus, writing articles about e-safety, project management in TurvaLan project what has
recently been awarded by: Microsoft (2009): Innovative Teacher Award and European Schoolnet (2010): 1 st
eLearning Award, in ‘Internet Safety’ category.
Beulah Maas is a lecturer in social and health care leadership and management in the Faculty of Health and
Life Sciences at Coventry University. She is an occupational psychologist and her key areas of teaching and
research are developing and managing people and social entrepreneurship.
Linda Martin is a senior lecturer and qualified social worker working in the Faculty of Health and Life
Sciences at Coventry University. She teaches leadership and management in health and social care,
including programmes which are entirely on line and has a particular interest in developing approaches to
student engagement.
Marie Martin is an education consultant in Northern Ireland specialising in e-learning, and is adjunct faculty
of Duquesne University, Pittsburgh, USA, where she earned her doctorate in education in 2007. She has
presented internationally and authored several publications on aspects of e-learning. In 2010 she was
Rooney International Visiting Scholar in Robert Morris University, Pittsburgh.
xviii

David Mathew works at the Centre for Learning Excellence at the University of Bedfordshire. In addition to
his work as a Learning Technologist, working cross-faculty on online learning programmes, he co-edits The
Journal of Pedagogic Development and teaches writing/publishing. He is also interested in psychoanalysis
and creative writing, and published a novel in 2011.
Peps Mccrea is a Senior Lecturer in Initial Teacher Education at the University of Brighton. He is interested
in pedagogy and the role of the teacher in our increasingly digital age.
Peter Mikulecky is a professor of Managerial Informatics at the Faculty of Informatics and Management at
the University of Hradec Kralove, Czech Republic, since 1993. He has been the head of the Department of
Information Technologies since 1994; recently he acts also as Director for Research and Director of
Postgraduate Studies.
Jonathan Moizer is a lecturer in business operations and strategy. His research interests include the use of
computerised simulation for both decision-making and learning. He has particular interest in the use of
system dynamics modeling as a simulation approach. In recent years Jonathan has organized three national
workshops on using simulation games in teaching and learning for the Higher Education Academy.
Kate Mottram works at Coventry University as a Graduate Teaching Assistant. Her research interests
include general management and the theories surrounding quality in business organisational settings along
with management in engineering.
Peter Mozelius is working as a researcher and IT-Pedagogue at the Department of Computer and Systems
Sciences, Stockholm University, Sweden. His research interests are in the fields of ICT4D, e-learning,
software engineering and software visualization. During the last six years he has been working in aid projects
in developing countries in southern Asia.
Antoinette Muntjewerff is an Assistant Professor in General Legal Theory at the University of Amsterdam.
Studied Educational Science and Law. PhD research involved theoretical and empirical studies into legal
case solving and evaluation of an instructional environment for learning to solve legal cases PROSA. Her
research is on modelling legal knowledge and legal reasoning for developing electronic materials for learning
the law.
Phelim Murnion is a Senior Lecturer in Information Systems in the School of Business at Galway-Mayo
Institute of Technology; lecturing on business information systems, data warehousing, and knowledge
management. His research interests include Data Warehousing, Business Intelligence and Educational
Technology. Phelim is a member of the Business Informatics (research) Group at Dublin City University.
Chrissi Nerantzi is an Academic Developer at the University of Salford. She is involved in the development
and delivery of the Postgraduate Certificate in Academic Practice (PGCAP) and CPD across the university.
Chrissi’s current professional interests are social media for learning, creative teaching and learning in HE,
online Problem-Based Learning, Open Educational Practice and mobile learning.
Michaela Noakes is a Doctoral candidate and Research Assistant in Instructional Technology in the School
of Education at Duquesne University. She received her MS-ISM, MBA and M. A. from Duquesne. She also
is an Adjunct Faculty member at Duquesne University, Point Park University and Butler County Community
College. She was recently inducted into Pi Lambda Theta, the International Honor Society for Educators.
Abel Nyamapfene received his Ph.D. in Computing from the University of Surrey, Guildford, UK in 2006,
and his MSc. in Communication Engineering and B.Sc. (Hon) in Electrical Engineering from the University of
Zimbabwe in 1990 and 1997 respectively. He is currently the Assistant Director for Education (Engineering)
at the University of Exeter, UK.
Maruff Akinwale Oladejo is a faculty in the Department of Educational Foundations, Federal College of
Education (Special), Oyo State, Nigeria. He is a member of the Editorial Advisory Board, JournalsBank
Publishing Inc. (UK, Pakistan, India and West Africa).
Fatemeh Orooji is a Ph.D. Candidate in the Department of Computer Engineering, Software Engineering
group, at the University of Tehran, Iran. She is interested in web- based educational systems (WBES) and
technology-enhanced learning. Her research activities focuses on Collaborative and Networked Learning,
Educational Data Mining, Adaptive and Personalized Learning.
xix

Kanari Paraskevi is a five year experienced school teacher of French Language in Greece with a master’s
degree in French Literature and is a PhD student in the same field. She has carried out several cultural
programs promoting education through art and theatrical research. She speaks four languages and has
published various articles concerning French Literature.
Lyn Pemberton is Reader in Human Computer Interaction at the University of Brighton. Lyn has been
involved in numerous learning technology projects, mostly concerned with aspects of communication, writing
and language learning. Her most recent projects have involved interactive television for learning, augmented
reality and in particular mobile language learning.
Katie Piatt has been working in the field of Learning Technologies for 10 years, with experience in
development, implementation and evaluation. Her research focuses on methods of student engagement.
Current projects include investigating the integration of social media with traditional environments and the
introduction of game-based learning principles as part of the ALT special interest group.
Andy Pulman works for the School of Health and Social Care (HSC) at Bournemouth University, where
he manages and creates web resources and e-learning initiatives. He began studying for a part-time PhD in
2008, concerning the role of Web 2.0 and Mobile technology to support people living with chronic illness and
enhance quality of life.
Thomas Putz is responsible for the initiation, the application and the coordination of national and
international projects and acts as the contact point with funding organizations in the evolaris. He has been
involved as coordinator and as specialist in eLearning and mLearning in many projects funded under Adapt,
Grundtvig, Socrates, FP6, eContent+ and the LLP.
Ricardo Queirós is an assistant professor at ESEIG/IPP which is responsible for Programming Languages
courses. He is a Ph.D. student in the Faculty of Sciences of the University of Porto. His scientific activity is
mostly related with e-Learning standards and Interoperability. He is member of CRACS/ INESC-Porto and a
founding member of KMILT research group.
Zahra Rahimi is a Master of Science Student in the Philosophy of Science group, at Amir Kabir University of
Technology. She received a Bachelor degree in Computer Engineering- Software, from University of Tehran.
She is interested in interdisciplinary topics, subjects such as use of software in education and philosophy of
intellectual intelligence.
Yambu Andrik Rampun recently completed his M.Sc Software Engineering (Department of Computer
Science) at the University of Hertfordshire. He previously received his Diploma in Computer Science and
B.Sc (Hons) degree Software Engineering from University Technology Mara (Department of Computer
Science and Mathematics). He is interested in designing, development and evaluation of e-learning
applications as well as human computer interactions.
Fauziah Redzuan is a junior lecturer at the University of Technology MARA, Malaysia. She holds a Master
of Science in Information Technology from the University of Science Malaysia in the area of multimedia
learning. Currently she is pursuing her study in the area of emotion and online learning using the Kansei
Engineering technique.
Bart Rienties is lecturer at the Centre for Educational and Academic Development at University of Surrey.
His primary research interests are focussed on computer-supported collaborative learning and the role of the
teacher to design effective blended and online learning courses.
Manuel Rodrigues teaches Computer Science at Secondary School Martins Sarmento, Portugal, where he
is responsible for the IT infrastructure of the school. He has a Computer Science degree and an MsC, both
from Minho University, Portugal. Currently he’s a PhD Student at Vigo University, Spain, and working on
recommendation and personalization systems for E-learning platforms such as Moodle.
Andrée Roy is an engineer and professor of Management Information Systems at the Université de
Moncton. Ms Roy is also an e-learning consultant and the owner of a business specialising in the field of elearning
called Web Training Solutions. Previously, Ms Roy worked as an industrial engineer and in the
information technology and economic development sectors.
Zuzana Šaffková is a senior lecturer at the Department of English, Technical University of Liberec, Czech
Republic, where she specializes in the teaching of academic reading and writing to teacher trainees.
xx

Research interests include the investigation of the effectiveness of e-learning in developing student
academic capability, especially critical reading competence.
Ahmed Salem is an Assistant Professor at the faculty of Engineering, King Abdul Aziz University at KAU. He
graduated from CMU, PA, USA and he is interested in Active Learning, E-Learning and Excellence in
engineering teaching. His contributions include a Design Helping Mechanism for Active Learning Courses
that aid in designing learning experiences.
Rowena Santiago is a tenured professor (emeritus) in Instructional Technology and founding/former director
of the Teaching Resource Center, California State University, San Bernardino. Her research activities
include learner characteristics in e-learning (US and Japan), the use of faculty learning communities for
faculty development, and the use of student response systems for active learning.
Vitor Santos is an invited Professor at Trás os Montes e Alto Douro University (UTAD), Minho University
(UM) and Lusofona University (ULHT), teaching Information Systems, Compilers, and Artificial Intelligence
courses in Computer Science and Informatics Engineering Degrees. Before that, he was the Microsoft
Portugal Academic Computer Science Program Manager for 8 years.
Marie Sams has worked in higher education and the NHS in a number of roles over the past fifteen years.
This has included areas predominantly in marketing management and project management. Her research
interests are in team creativity and peer learning.
Maggi Savin-Baden is Professor of Higher Education Research at Coventry University, UK, and is director
of the Learning Innovation Research Group. She has been researching learning for over 20 years and has
just completed her 11th book. In her spare time she runs and tries out extreme sports
Adriana Schiopoiu Burlea is Professor PhD at the University of Craiova and visiting professor at
universities in France and Poland. She is Romanian ambassador for AGRH. She has written more than 20
books and 130 articles in the field of Management. She is a member of the editorial boards of many
prestigious journals and conferences.
Fabio Serenelli is about to conclude the PhD course in "Information Communication Technology applied to
knowledge society and learning processes" at the Milano Bicocca University where he collaborates as e-
Learning Manager and Instructional Designer at the CPM (Multimedia Production Center). He adopts an
interdisciplinary approach and his research interests are related to instructional formats for multimedia
learning, infographics and User Experience Design, One to One Computing, PLE - Personal Learning
Environment.
Angela Shapiro MSc, Dip Ad Ed, TQFE has been a Lecturer at Glasgow Caledonian University for ten
years. She teaches across all academic schools applying an advanced academic literacies (AAL) approach.
This includes workshops across the university within the context of particular programmes, as well as
distance and blended learning support to students.
Cees Smit Sibinga is professor of International Development of Transfusion Medicine, University of
Groningen. He founded the Academic Institute for International Development of Transfusion Medicine
(IDTM), focusing on higher education in developing countries. Objective - to support leadership development.
The first post-graduate course designed and implemented is a Masters in Management of Transfusion
Medicine (MMTM).
Dina Soeiro. Besides being a trainer of college teachers on E-learning, Dina supports and studies the use of
Moodle at the College of Education of the Polytechnic Institute of Coimbra where she teaches. She is
finalizing her PhD on ICT in Education at the University of Coimbra and she is a researcher at the CISUC.
Mekala Soosay is a university teaching fellow and Technology-Enhanced Learning (TEL) team member with
a passion for preparing and disseminating creative methods for assessment, learning and teaching. She
also trains colleagues in applying TEL for their teaching practice. She has worked on a number of JISC and
EU-funded projects, the current ones being PC3 and EuroPlot.
Sonia Sousa holds a PhD in Education from Sheffield Hallam University, UK and an honors degree in
Communication Engineering from Universidade Fernando Pessoa, Portugal. She is currently researching the
influence of trust in online communities. R&D work includes Michigan State University's MIND Lab, USA,
Universidade Fernando Pessoa Multimedia Research Center, Portugal, as well as Cape Verde and
Mozambique.
xxi

Iain Stewart has worked at Glasgow Caledonian University for more years than he cares to remember. He is
primarily interested in the ways technology can be used to improve the learning experience of students. His
other activities include working to improve FE to HE articulation and critical testing of the elasticity of
deadlines.
Caroline Stockman works at the Catholic University of Leuven as a lecturing assistant for the course of
'Online Publishing', a key subject within the Master's degree of Cultural Studies. Her time is divided between
Belgium and the UK, where she trains teaching staff in secondary and higher education in the use of
educational technology.
Amanda Sykes (PhD, DipAcPrac, FHEA) works with students of Science, Engineering and Veterinary
Medicine at The University of Glasgow to enhance their learning. She is particularly interested in using
emerging technologies to shift the focus from didactic lecturing towards scholarship that enables students to
understand their own learning process as well as academic content.
Fattaneh Taghiyareh is an Assistant Professor of Computer Engineering- Software & Information
Technology, at the University of Tehran, where she has served since 2001. She received a Ph.D. in
Computer Engineering from the Tokyo Institute of Technology in 2000. Her research interests include
Human-Centered Computing applied to Learning Management Systems and based on Multi-Agent Systems.
Nicolet Theunissen is senior research scientist ‘Training Innovations’ at TNO, The Netherlands. Her
research is dedicated to the development and testing of training solutions for professionals. These learning
solutions imply both innovative learning technology and didactics for adult learning. Formerly, she worked at
Utrecht University, Leiden University and at the Netherlands Institute of Health Services Research.
Vladimir Tomberg works as a researcher in the Centre for Educational Technology, Tallinn University,
Estonia. His main research topics are interoperability of online assessment tools and course coordination
within blog-based Personal Learning Environments. He has stayed recently as a visiting researcher in Simon
Fraser University at Vancouver, Canada and in RWTH at Aachen, Germany.
Panos Vlachopoulos is Lecturer in the Centre for Learning Innovation and Professional Practice at Aston
University. He leads the Postgraduate Certificate in Professional Practice in Higher Education and advises
Aston University staff on the development of fully online distance learning programmes. His research
interests include the online facilitation of student-centred learning and reflective practice.
Chien-hwa Wang is a professor in the Department of Graphic Arts and Communications at National Taiwan
Normal University. His recent research emphasis is Web 2.0 applications in Education. He also directed a
government supported digital archiving project to preserve Taiwanese puppetry master Tien-lu Li’s historic
works and collections .
Cristina Wanzeller is professor at the Informatics Department, Technology and Management School, Viseu
Polytechnic Institute (IPV), Portugal. Her teaching activities focus on Information Systems and Databases
areas. She is a member of the IPV’s research centre CI&DETS. Her main scientific areas of research are
Web Usage Mining, Case-Based Reasoning, e-Learning and Information Systems.
Julie Watson is Principal Teaching Fellow in eLearning in Modern Languages at the University of
Southampton. She manages eLanguages which produces and licenses the EAP and Study Skills Toolkits.
She also led the development of the Prepare for Success website, works on projects involving emergent
technologies, and develops online courses and resources.
Phil Watten is from the University of Sussex, has been researching and developing learning-based
technology and production methods since 2005. Using the Sussex-based Media Technology Lab, he has
produced studio-based and portable production systems and is now working on fully automated approaches.
Katherine Wimpenny, PhD, MSc, DipCOT, Cert Ed, is a Research Fellow at Coventry University within the
Learning Innovation Research Group. She is involved in researching the impact of virtual worlds on teaching
and learning. She has recently gained funding to conduct two qualitative research syntheses, enjoys
supervising PhD students and professional background is occupational therapy.
Koos Winnips is advisor in ICT and Education, with a particular interest in digital pedagogies: sorting out
learning arrangements with the help of the electronic learning environment. He has worked at the University
of Twente in the Netherlands and Glasgow Caledonian University as researcher, lecturer and advisor of ICT
in Education.
xxii

Novita Yulianti is a PhD student at the Faculty of Business Studies and Economics, University of Bremen,
Germany. She received a Master of Science degree on International Technology Transfer Management. Her
current research interests include change management, organizational management, information and
technology management, innovation management.
Li-Zhang Zhang received BA from Huazhong University of Science and Technology, China, MSc from the
University of London, UK, and PhD from the University of Sydney, Australia. He currently teaches
International Business and Economics at La Trobe University, Australia. His research interests include
Asian-Australian Business and Trade, E-Commerce, Theory and Practice in English-Chinese Translation,
and Online Education and Training.
xxiii

Revisiting the Personal Transferable Skills Debate - an
eLearning Pedagogical Perspective
Samuel Adu Gyamfi 1 , Lene Tolstrup Sorenson 1 and Thomas Ryberg 2
1
CMI, Aalborg University, Copenhagen
2
Faculty of Humanities, Aalborg University
sagy@plan.aau.dk
Abstract: Personal Transferable Skills (PTS) are essential work skills which are not specific to any subject or
profession, and which, though learned in one context may be successfully transferred to and applied in many
other contexts. They are skills that enable people to acquire, structure, interpret and put to efficient use, their
subject knowledge. There is a growing concern among educational providers and employers’ organisations on
the lack of PTS of university graduates. This phenomenon has been amplified by the need of university
graduates of the twenty-first century to possess skills and knowledge that can be effectively used in new domains
and in different situations since there is also an increasing tendency for graduates to take jobs outside their
academic field of study. There have been several attempts to promote PTS by university institutions in the
traditional classroom settings where three broad approaches for developing PTS within the curriculum have been
experimented with. To date, progress so far has been patchy. The paper is in two theoretical parts. The first part
seeks to advance the theoretical framework of REALs as the better approach to teaching and learning in our
universities. The second part of the paper argues that theoretically, communication theory (which draws on
contemporary rhetorical theory) and social informatics theory provide important perspective for the application of
eLearning based on REALs in the development of PTS for university graduates. The paper would contribute
significantly to theoretical underpinnings of eLearning based on REALs for the development of PTS of university
graduates.
Keywords: personal transferable skills, eLearning, Rich Environment for Active Learning, intentional learning,
co-operative learning
1. Introduction
The main aims of university education are to inspire and enable individuals to develop their
capabilities to the highest potential levels throughout life, so that they grow intellectually, are wellequipped
for work, can contribute effectively to society and achieve personal fulfilment; and to serve
the needs of an adaptable sustainable, knowledge-based economy at local regional and national
levels (Dearing, 1997). Consequently, the graduates of our universities who would constitute the
bedrock of the workers in this knowledge-based economy are expected to have more applied skills in
terms of teamwork, communication skills, problem solving and lifelong learning (OECD, 2006). These
skills have been variously referred to as core skills, personal skills, personal competences and
personal transferable skills (Justice, Rice, & Warry, 2009; Bridges, 1993; Drummond, Nixon, &
Wiltshire, 1998 & Nabi & Bagley, 1998). In this article, it has been referred to as personal transferable
skills (PTS). Personal transferable skills are essential work skills which are not specific to any subject
or profession and which, though learned in one context may be successfully transferred to and
applied in many other contexts. They are skills that enable people to acquire, structure, interpret and
put to efficient use, their subject knowledge (Gash & Reardon, 1988).
There is growing concern among educational providers and employers’ organisations on the lack of
PTS of university graduates (Drummond, Nixon, & Wiltshire, 1998 & Kemp & Seagraves, 1995). This
phenomenon has been amplified by the need of university graduates of the twenty-first century to
possess skills and knowledge that can be effectively used in new domains and in different situations,
since there is also an increasing tendency for graduates to take jobs outside their academic field of
study (Nabi & Bagley, 1998). There have been several attempts to promote PTS in university
institutions in the traditional classroom settings where three broad approaches for developing PTS
within the curriculum have been experimented with (Drummond, Nixon, & Wiltshire, 1998; Kemp &
Seagraves, 1995 & Chadha, 2006). These approaches include the embedded or integrated
development, parallel (or stand-alone) development and work placements or work-based projects.
Kemp & Seagraves (1995) report on findings of an empirical study of transferable skills provision on
five courses at Glasgow Caledonian University, which was carried out as part of a larger investigation
into a novel form of work-based learning experience – Structured Industrial Practice Studies (SIPS),
where the three approaches were used. The authors found out that a fairly high percentage of
students at the end of the course did not feel they had received formal instruction or additional help in
1

Samuel Adu Gyamfi et al.
the development of these skills and consequently, concluded that there should be a radical rethinking
of course structuring and its delivery if these skills are to be addressed seriously in higher education.
Curry & Sherry (2004) report on a survey of 919 Irish graduate employers, students, recent graduates
and academic staff of Trinity College of Dublin, Dublin City University and Waterford Institute of
Technology to determine their attitudes to the importance of transferable skills and their perception of
the current level of development of such skills in higher education programmes. Curry & Sherry (2004)
found out that the two most important transferable skills for a graduate’s career were oral
communication and time management. Again they found out that oral communication, research skills
and written communication were being developed through academic programmes of the universities.
Curry & Sherry (2004) concluded that transferable skills are highly valued by all stakeholders in higher
education and considered an important factor in shaping graduates’ employability and in enhancing
their personal development. Many transferable skills are and could be developed through academic
programmes and that awareness must be created in higher education of the development of such
skills.
Justice, Rice & Warry (2009) report on a quasi-experimental study at McMaster University as part of a
larger McMaster Innovative Learning Study which compared skill levels and academic performance
over time of students who had taken an Inquiry course when they began university with students who
had not taken the course. The Inquiry course was meant to prepare students to be more successful
learners in their subsequent university courses and beyond. The authors found evidence that the
Inquiry learning process was transferred from the course. Additionally, the students who had taken
the course were able to use the skills acquired in new situations. The authors therefore, concluded
that the transfer of core skills occurs under particular learning conditions that can be fostered through
course design and enhanced through specific pedagogical objectives.
It can therefore, be deduced from the literature that PTS could be developed in the traditional
classroom. However, as Justice, Rice & Warry (2009) put it, “there are critical components to curricula
design that enhance knowledge and skills transfer to other contexts – a process integral to learning to
learn. Transfer does not just happen – it is not a natural bi-product of education; rather it is a process
that needs to be consciously fostered through course design, and nurtured in the classroom”. Again
the literature establishes the fact that for PTS development, the established models of good practice
depends on opportunities to practice these skills with support and guidance which encourages and
informs constructive reflection and the definition of strategies for improvement (Drummond, Nixon, &
Wiltshire, 1998). Consequently, a new teaching and learning environment has to be created for
effective PTS development of the graduate of the twenty-first century.
2. Theoretical considerations
Two possible answers that readily come to mind when the question of what constitutes learning is put
are ‘the change in behaviour’ and ‘acquisition of knowledge’. However, in this write-up, we share
Visser (2008) definition of learning as the “disposition of human beings, and of the social entities to
which they pertain, to engage in continuous dialogue with the human, social, biological and physical
environment, so as to generate intelligent behaviour to interact constructively with change”. For if
learning should apply to the most comprehensive level of human endeavours then it should involve
the development and maintenance of a lifelong disposition to dialogue with one’s environment for the
purpose of constructively interacting with change in that environment (Visser, 2008).
Research shows that at the university, what and how students learn should be the main responsibility
of university academics ( (Laurillard, 2002). According to Laurillard (2002) the familiar methods that
support learning by which academics offer tuition in our universities are through:
Acquisition – and therefore academics offer lectures and reading;
Practice – and therefore academics set exercises and problems;
Discussion – and therefore academics conduct seminars and tutorials;
Discovery – and therefore academics arrange fieldtrips and practicals.
However, due to the impacts of ‘information society’, both the nature of knowledge and how
knowledge is acquired should change (Nunes & McPherson, 2003). Indeed, learning in our higher
education system has often focused on producing inert knowledge and abstract concepts which are of
no use if the learner does not have the understanding to apply them in appropriate settings
2

Samuel Adu Gyamfi et al.
(Grabinger & Dunlap,1995 and Nunes & McPherson, 2003). CTGV, (1993) and Grabinger & Dunlap
(1995) outline the following flaws in approaches to teaching that lead to the development of inert
knowledge in higher education:
Academics tend to fill students with facts, and leave no time for dealing with topics in depth.
Academics expect students to remember dates, formulae, algorithms, quotations and whole
poems, yet show little practical use for that knowledge despite the fact that the students have
difficulty in transferring the knowledge.
Academics rely on decontextualized instructional strategies where the focus is on abstract basic
skills, concepts and technical definitions. Academics believe that decontextualized skills have
broad applicability and are unaffected by the activities or environments in which they are acquired
and used. When this happens students do not learn when to apply those skills or within what
kinds of contexts they work.
Students are given arbitrary, uninteresting, and unrealistic problems to solve when it comes to
providing them with practice. This emphasises the point that academics mistakenly believe that
they must emphasise decontextualized skills that are applicable everywhere.
Additionally, students are treated passively, rarely giving the opportunity to take responsibility for
their own learning, to explore ideas of their own choosing, to collaborate with one another or with
academics, or to make valuable contributions to the learning of others. Hence, the students do not
learn the skills necessary to become life-long learners and daily problem solvers.
Furthermore, students are not evaluated in authentic ways. After teaching in decontextualized
ways academics tend to use paper-pencil tests to measure the quantity of knowledge learned
without examining the quality of students’ thinking and problem-solving performance.
3. New learning environment for the development of PTS
To be able to overcome these flaws in our current teaching and learning in higher education which
produce inert knowledge, our students should operate in learning environments which prepare them
for the complexities of the professional world. These learning environments should utilize instructional
activities that reflect the problem-solving and challenge-meeting processes that professionals use on
their jobs (Grabinger, Dunlap, & Duffield, 1997). And as Grabinger, Dunlap, & Duffield (1997) put it
our higher education students need a learning environment “that places the students in the driver’s
seat of the learning process – involving them in the planning, controlling and directing of learning
activities and the application and assessment of learning processes and outcomes. Such a learning
environment is what Grabinger & Dunlap (1995) refer to as rich environments for active learning
(REALs).
Grabinger & Dunlap (1995) and Grabinger, Dunlap, & Duffield (1997) have outlined five key attributes
of REALs as follows:
Student responsibility and initiative – this attribute explains that REALs are “student-centred
learning environments which place a major emphasis on developing intentional learning and lifelong
learning skills. These skills include the abilities to construct higher-order questions to guide
learning, reflect on consequences and implications of actions, and monitor and modify their
personal cognitive activity. This attribute is critical because students cannot actively construct and
evolve their knowledge structures without taking responsibility and initiative for their learning”.
According to Grabinger & Dunlap (1995) and Scardamalia, et al. (1991) research points to
passive and immature learners possessing certain characteristics that prevent them from
becoming skillful problem-solvers. These characteristics which include their inability to organize
their mental activities around topics ultimately result in their inability to transfer their knowledge to
new problems because they have learned content and strategies in a decontextualised context.
To be able to develop intentional learning and thereby promote transferability of knowledge to
other context, REALs suggest reciprocal teaching strategies. According to Grabinger & Dunlap
(1995), reciprocal teaching promotes social interactions, scaffolding and contextualized, holistic
activities that have relevance for learners.
Generative learning activities – According to Grabinger & Dunlap (1995) generative learning
activities require that students engage in argumentation and reflection as they try to use and then
refine their existing knowledge as they attempt to make sense of alternate points of view.
Students, in a generative learning situation apply the information they learn. Students are
therefore, involved in creating solutions to authentic problems through the development and
3

Samuel Adu Gyamfi et al.
completion of projects. Generative learning activities call for the use of cognitive apprenticeship
form of instruction (Grabinger & Dunlap (1995). According to Grabinger & Dunlap (1995) in
cognitive apprenticeship the goal of the teacher is to help students generalize and transfer their
learning through conditionalised and indexed knowledge. This is done through the presentation of
a range of tasks that encourage reflection and identification of common transferable elements
across tasks.
Authentic learning contexts – according to Grabinger & Dunlap (1995) authenticity is important to
REALs for three reasons. First realistic problems hold more relevance to students’ needs and
experiences because they can relate what they are learning to problems and goals they see every
day. Secondly, because the situations students encounter during learning are authentic, they
develop deeper and richer knowledge structures, leading to a higher likelihood of transfer to novel
situations. Finally, because complex problems require a team approach that provides natural
opportunities for learners to test and refine their ideas and to help each other understand the
content, it encourages collaboration and negotiation. REALs therefore, propose anchored
instructional strategies to achieve authenticity. According to Grabinger & Dunlap (1995) because
anchored instructional strategies encourage students to work together to solve complex problems
they are exposed to multiple perspectives in an environment that gives them the opportunity to
test out their ideas, solutions and processes and consequently overcome the problem of inert
knowledge.
Authentic assessment strategies – Authentic assessment strategies refer to the measurement of
intellectual accomplishments that are worthwhile, significant and meaningful as compared to
standardized tests inherent in conventional schooling (Grabinger & Dunlap (1995). Accordingly
Grabinger & Dunlap (1995), quotes Wiggins (1989) as explaining that authentic tests show the
intellectual design features that emphasize complexity, stress depth more than breath, include illstructured
tasks or problems, and require students to contextualize content knowledge. When
applying authentic assessment to student learning and achievement, a teacher applies criteria
related to “construction of knowledge, disciplined inquiry, and the value of achievement beyond
the school.
Co-operative support – according to Grabinger & Dunlap (1995), “working in groups helps
students refine their knowledge through argumentation, structured controversy and the sharing
and testing of ideas and perspectives”. “Co-operative learning and problem-solving groups also
address students’ needs for scaffolding during unfamiliar learning and problem-solving activities;
therefore, with the support of others in the group, students are more likely to achieve goals they
may not have been able to meet on their own (Dunlap, 1999). Grabinger & Dunlap (1995) suggest
problem-based learning as a REAL strategy for achieving co-operative learning. In problem-based
learning students work with problems in a manner that fosters reasoning and knowledge
application appropriate to their levels of learning. The skills and knowledge acquired by this study
are applied back to the problem to evaluate the effectiveness of learning and to reinforce learning
Grabinger & Dunlap (1995).
The five key principles of REALs amply demonstrate their achievement in promoting the development
of PTS in higher education institutions as outlined in the literature. Indeed Grabinger (1999) has
looked at some research conducted with various implementations of REALs which showed positive
results.
Paradoxically, research has shown that lecturers in our higher educational institutions are challenged
and de-motivated on a day-to-day basis with the practicalities of teaching increasing numbers of
students with a decreasing unit of resources in institutions, especially where competition for funding is
fierce and where there is pressure from external reviews of research and teaching performance, Kitts
& Hancock (1999). Consequently, students often find it difficult to get seating spaces in the lecture
halls let alone hear what the lecturer is saying. Libraries are overcrowded, books are out of date and
journal holdings lag years behind due to budgetary constraints. Therefore, the acquisition function in
higher educational institutions is unduly hampered in most developing economies.
Excessive student numbers have also resulted in a situation whereby academics could hardly set
meaningful exercises let alone mark and present feedback to students. There is also the pressure on
the academics to meet research targets in order to qualify for funding which has compounded the
problem of meeting students’ needs. Therefore, students hardly had time to practice what they learn.
Additionally, small group tutorials and seminars necessary for discussion exercises are hardly feasible
4

Samuel Adu Gyamfi et al.
in many departments. Furthermore, because of the large student numbers with its attendant budget
constraints, hardly do departments organise fieldtrips and practical sessions which are key ingredients
for discovery learning. This situation rather promotes teacher-centred methods of practice instead of
student-centred learning approach which facilitates effective PTS development (Drummond et al,
1998).
4. The argument for eLearning strategies
Grabinger (1999) has outlined some issues that need investigation in REALs such as individual
differences, learner control, scaffolding and support, learning and assessment. These are issues
which it is argued theoretically, could be supported by eLearning and consequently promote the
development of PTS. In order to justify the use of eLearning to facilitate the acquisition of PTS in
university education, it is prudent to look at the theories that are useful for examining and
understanding eLearning. In this article, much of the work on the theoretical framework of eLearning
relies heavily on the work of Andrews & Haythornthwaite (2007)
For the electronic aspect of eLearning Andrews & Haythornthwaite (2007) argue for the
communication theory (which draws on the contemporary rhetorical theory) and social informatics as
providing important perspectives. Contemporary rhetoric, according to Andrews & Haythornthwaite
(2007) is ‘concerned with the relationship among three key elements namely, the speaker/writer, the
audience and the subject matter. This communicative triangle enables exploration and definition of
the purpose of the communicative act, as well as the possibility of investigation of the means by which
the communication takes place’. Consequently, they argue that the rhetorical theory can be used to
analyse communication and predict the patterns and means of communication that might be
necessary in a particular situation. This theory is therefore in alignment with eLearning since
eLearning takes place in a particular situation because it is contextualised.
This rhetoric theory has been depicted in the adopted model in figure 1. Here learning is considered
as a “dialogic and dialectical exchange, not only between the learner and the teacher, but also
between the learner and the body of knowledge that is being explored” (Andrews & Haythornthwaite,
2007) . The model therefore places the learner in a more powerful, active position in relation to the
social dynamics of learning who can even critique the teacher’s mediation of existing knowledge
(Andrews & Haythornthwaite, 2007). This supports REALs co-operative learning strategies where
students work together to learn and are responsible for one another’s learning as well as their own.
Working together in groups helps students refine their knowledge through argumentation, structured
controversy and reciprocal teaching (Grabinger, 1999). Accordingly, co-operative learning and
problem-solving activities encourage generative learning (Grabinger, 1999) which eventually leads to
the development of students’ personal transferable skills.
Figure 1: Model of communication (source: Andrew and Haythornthwaite, 2007)
For social informatics theory, Andrews & Haythornthwaite (2007) quotes the definition of Kling,
Rosenbaum, & Sawyer (2005) to buttress their point of view. “Social informatics refers to the
interdisciplinary study of the design, uses, and consequences of ICTs that takes into account their
5

Samuel Adu Gyamfi et al.
interaction with institutional and cultural contexts” (Kling, Rosenbaum, & Sawyer, 2005) Quoting
extensively from many researchers and fields that have shaped social informatics including studies
about adoption and diffusion of innovations, social construction and social shaping of technology,
activity theory, social networks and communities of practice, Andrews & Haythornthwaite (2007)
assert that “social informatics provides another theoretical foundation for addressing eLearning
particularly where it intersects with computing use by groups, organisations, communities and
societies”.
Additionally, Andrews & Haythornthwaite, (2007) have extended the principles of social informatics
into the learning situation where they have adopted the term ‘educational informatics’. Quoting Levy,
et al. (2003) they have defined educational informatics as “the study of the application of digital
technologies and techniques to the use and communication of information in learning and education”.
It has therefore created the opportunity for an isolated student of old to be able to carry on authentic
learning with other students through class discussion boards, and emails, whispering, moulding and
forming the communication dialogue they prefer” (Andrews & Haythornthwaite, 2007). Consequently,
not only does the model advance the position of a student in a better relation to a community of
learners and body of knowledge, but it also enables reflection to become an integrated part of the
actual dialogic interaction between the participants while in the learning process (Andrews &
Haythornthwaite, 2007). Provision of digital technology tools in the model can therefore be used to
support cognitive processes, share the cognitive load by providing support for lower-level cognitive
skills, allow learners to engage in learning activities normally out of reach and allow learners to
generate test hypotheses Lajoie (1993) and consequently promote the development of PTS.
5. Conclusion
It has been established that for personal transferable skills development in higher education to be
achieved in the traditional classrooms a new learning environment that encourages knowledge and
skills transfer to other contexts must be encouraged. This paper has argued for the use of REALs. It
has been established that REALs promote intentional learning by encouraging the growth of students’
responsibility, initiative, decision-making and intentional learning, a constructivist learning theory that
promotes the development of PTS. REALs apply dynamic, generative learning activities that promote
high level thinking process (that is, analysis, synthesis, problem-solving, experimentation, creativity
and examination of topics from multiple perspectives to help students integrate new and old
knowledge and thereby create rich and complex knowledge structures, including creating and
providing access to resources. REALs also utilise authentic learning contexts to promote study and
investigation, including contextualising learning and thereby promote PTS development. Again,
REALs encourage collaboration to cultivate an atmosphere supportive of knowledge building
communities and reinforce reflection by embedding opportunities to reflect on the learning process as
well as on the content acquired to promote both learning and meta-cognitive skills development
necessary for the development of PTS. However, challenges in our current higher educational
institutions demand new learning environments. The paper has consequently made a case for the use
of communicative and social informatics theories in an eLearning environment to facilitate the
development of PTS in higher education.
References
Andrews, R., & Haythornthwaite, C. (2007). Introduction to eLearning Research. In R. Andrews, & C.
Haythornthwaite (Eds.), Sage Handbook of eLearning Research (pp. 1-52). London: Sage Publications.
Bostock, S. J. (1998). Constructivism in Mass Higher Education: A Case Study. British Journal of Educational
Technology , 29 (3), 225-240.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How People Learn: Brain, Mind, Experience and School.
Washington, D.C.: National Academy Press.
Bridges, D. (1995). Transferable Skills: A Philosophical Perspective. Studies in Higher Education , 20 (3), 43-51.
Chadha, D. (2006). A Curriculum Model for Transferable Skills Development. Journal of the Higher Education
Academy Engineering Subject Centre , 1 (1), 1-10.
CTGV, (. a. (1993). Integrated Media: Toward a Theoretical Framework for Utilizing their Potential. Journal of
Special Education Technology , 12 (2), 76-89.
Cullen, J., Hadjivassiliou, K., Hamilton, E., Kelleher, J., Sommerlad, E., & Stern, E. (2002). Review of Current
Pedagogic Research and Practice in the Fields of Post-compulsory Education and Lifelong Learning.
London: The Tavistock Institute.
Cunningham, D. J., Duffy, T. M., & Knuth, R. A. (2000). The Textbook of the Future. Bloomington: Centre for
Research on Learning and Technology.
6

Samuel Adu Gyamfi et al.
Curry, P., & Sherry, R. (2004). The Hidden Value of Higher Education Learning: Transferable Skills and their
Importance for Graduates of Modern Language Programmes. A Survey of Students, Academic Staff,
Graduates and Employers. Dublin: Transferable Skills Project.
Dearing, R. (1997). Higher Education in the Learning Society. London: HCIHE Publications.
Drummond, I., Nixon, I., & Wiltshire, J. (1998). Personal Transferable Skills in Higher Education: The Problems of
Implementing Good Practice. Quality Assurance in Education , 6 (1), 19-27.
Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the Design and Delivery of Instruction.
In D. H. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology. NY:
Macmillan Library Reference USA.
Dunlap, J. C. (1999). Rich Environments for Active Learning on the Web; Guidelines and Examples. WebNet 99:
World Conferene on the WWW and Internet. Honolulu, Hawaii: EDRS.
Gash, S., & Reardon, D. F. (1988). Personal transferable skills for the modern information professional: a
discussion paper. Journal of Information Science , 15, 285-292.
Grabinger, S. R. (1999). Rich Environments for Active Learning. In D. H. Jonassen (Ed.), Handbook of Research
for Educational Communications and Technology (pp. 665-692). New York: Macmillan.
Grabinger, S. R., & Dunlap, J. C. (1995). Rich Environments for Active Learning: A Definition. Association for
Learning Technology Journal , 3 (2), 5-34.
Grabinger, S. R., Dunlap, J. C., & Duffield, J. A. (1997). Rich Environments for Active Learning in Action:
Problem-based Learning. ALT-J , 5 (2), 5-17.
Jonassen, D. H. (1994). Thinking Technology: Toward a Constructivist Design Model. Educational Technology ,
34 (3), 34-37.
Justice, C., Rice, J., & Warry, W. (2009). Developing Useful and Transferable Skills: Course Design to Prepare
Students for a Life of Learning. Georgia Southern University.
Kemp, I. J., & Seagraves, L. (1995). Transferable Skills - Can Higher Education Deliver? Studies in Higher
Education , 20 (3), 315-328.
Kitts, S. A., & Hancock, J. T. (1999). Putting Theory into Practice: The Creation of REALs in the Context of
Today's Universities. Association of Learning Technology Journal (ALT-J) , 7 (2), 4-14.
Kling, R., Rosenbaum, H., & Sawyer, S. (2005). Understanding and Communicating Social Informatics. Medford,
NJ: Infomation Today.
Lajoie, S. P. (1993). Computer Environments as Cognitive Tools for Enhancing Learning. In S. Lajoie, & S. Derry
(Eds.), Computers as Cognitive Tools (pp. 261-188). Hilldfslr, NJ: Erlbaum.
Laurillard, D. (2002). Rethinking University Teaching: A Framework for the Effective Use of Learning
Technologies (2nd ed.). Abingdon: RoutledgeFalmer.
Lebow, D. (1993). Constructivist Values for Instructional Systems Design: Five Principles Toward a New Mindset.
Educational Technology Research and Development , 41 (3), 4-16.
Levy, P., Ford, N., Foster, J., Madden, A., Miller, D., Nunes, M. B., et al. (2003). Educational Informatics: An
Emerging Research Agenda. Journal of Information Science , 29 (4), 298-310.
Nabi, G. R., & Bagley, D. (1998). Graduates' Perceptions of Transferable Personal Skills and Future Career
Preparation in the UK. Career Development International , 3 (1), 31-39.
Nunes, M. B., & McPherson, M. (2003). Constructivism vs. Objectivism: Where is Difference for Designers of
eLearning Environments? The 3rd IEEE International Conference on Advanced Learning Technologies
(ICALT'03). IEEE COMPUTER SOCIETY.
OECD. (2006). Year Report. Organisation of Economic Co-operation and Development.
Scardamalia, M., Bereiter, C., McLean, R. S., Swallow, J., & Woodruff, E. (1991). Higher levels of Agency for
Children in Knowledge building: A Challenge for the Design of New Knowledge Media. The Journal of the
Learning Sciences , 1 (1), 37-68.
Visser, J. (2008). Constructive Interaction with Change: Implications for Learners and the Environment in which
They Learn. In J. Visser, & M. Visser-Valfrey (Eds.), Learners in a Changing Learning Landscape:
Reflections from a Dialogue on New Roles and Expections (Vol. 12, pp. 11-35). USA: Springer.
Wiggins, G. (1989). A True Test: Toward more Authentic and Equitable Assessment. Phi Delta Kappan (70), 703
713.
7

Survey of Teachers’ use of Computer/Internet in Secondary
Schools in South West Nigeria
Babatunde Alabi Alege and Stephen Olufemi Afolabi
Kwara State University, Malete-Nigeria
babatunde.alege@kwasu.edu.ng
stephen.afolabi@kwasu.edu.ng
Abstract: The Federal Government of Nigeria developed a National Policy for Information Technology at the
beginning of the new millennium in order to keep pace with the rest of the world and to meet the challenges of a
rapidly changing global environment. Nine years later the Technical Working Committee Group on Information
and Communication Technology submitted its report. A major propelling philosophy is to have a modern ICTdriven
educational system for the effective delivery of educational services at all levels. To what extent this has
been achieved remains to be investigated. The study was a descriptive survey carried out to investigate the
teachers’ literacy profiles, attitudes towards computers, general integration of Information and Communication
Technology and also determine the hindrances experienced by them on the integration of ICT in their teaching.
Five hundred and sixty two teachers from private and public schools in southwest Nigeria were randomly
selected. Their basic statistics include 58% females and 42% males with varying educational qualifications and
working experiences. The questionnaire used consisted of five parts based on teachers’ general information,
computer literacy, attitude, integration of ICT in their teaching and hindrances. Research questions were
generated. Data collected were analysed using simple percentage. The results indicated among others that 87%
of the teachers irrespective of their gender, school proprietorship and experience do not use computers in their
teaching. The teachers’ tendency not to use computers was determined by their lack of expertise in using ICT,
non-availability of infrastructure, equipment and software, laziness, technophobia, lack of incentives and support,
and their general attitude. It was recommended among others: that computer training programmes are organized
for all the teachers in Nigeria, adequate infrastructure and equipment should also be provided in each school,
teachers should be well remunerated to serve as incentives and Nigeria should address ICT issue with all
seriousness it deserves within the context of her education policy and implementation.
Keywords: Information and Communication Technologies, Technology Integration, Computer/Internet,
Secondary schools, Southwest Nigeria, Technophobia
1. Introduction
The end of the last millennium witnessed rapid technological advances and deep changes in several
aspects of human activity which, are taken together as indicative of a shift into knowledge era. Such
discussion according to Sringer, (2008) has stimulated much discussion about the role and processes
of education and about the role of Information and communication technology (ICT) in teaching and
learning in this new era. Presently the world is experiencing an ever-increasing use of computers in
diverse areas of human endeavour. (Chika, 2004). Numerous blueprints on education reforms and on
ICT in education and other sectors have also been set out by various governments since the mid-
1990s. In Nigeria, Information and Communication Technology (ICT) had been widely used in the
commercial and Business sectors for quite some time, particularly when Banks and other business
organizations began to automate their offices with desktop computers and other accessories for
financial networking transactions (Wodi,2009). The rapid rate at which ICTs have evolved since the
mid 20 th century, their convergence and pervasiveness have given them an active central role in
development and globalization (Nwagu, 2006; Adomi, 2010). It has been argued that ICT is the
bedrock for National survival and development in a rapidly changing environment.
As the world is witnessing rapid development in science and technology with the capability of
changing our life for better, there is an overwhelming awareness that there are great potentials that
the computer can offer when properly used in our schools. Bradly, 1996 noted that the adoption and
integration of ICT in the education sector is on the increase. A major consensus among educators is
that teachers should keep abreast of modern technology to integrate ICT in their teaching (Tahir and
Okebukola, 1995). The teacher of the 21 st century is expected to rise up to the challenges occasioned
by the new technology. Over the years, Nigeria had made several attempts to improve her education
system and make it more accessible to Nigerians and compete favourably in a competitive global
economic and social environment. Many reforms and strategic plans were introduced to address vital
educational issues for the achievement of set goals.
8

2. Statement of the Problem
Babatunde Alabi Alege and Stephen Olufemi Afolabi
Some fifty years after independence, the education sector in Nigeria is still grappling with many
challenges. These include acute underfunding, poor facilities, shortage of qualified teachers
especially in core subject areas, poor teaching and learning environment, decayed infrastructure, and
very low research activities in higher institutions. There is also the problem of access, mass failure in
senior secondary school certificate examination, examination malpractices and cultism. Recent
studies revealed that about 10 million children who are supposed to be in school were not there while
current national enrolment ratio is about 57% in public schools. Between 2003 and now, there have
been several new national and global initiatives. The Jomtien World Conference on Education for all
(EFA) in 1990 was sponsored by the World Bank, the United Nations Educational, Scientific and
Cultural Organisation (UNESCO), the United Nations Development Programme (UNDP) and the
United Nations Children Emergency Fund (UNICEF) to bring education to everyone irrespective of
race, colour, creed, physical disability or socio-economic status. Similarly the Millennium Development
Goals (MDGs), drawn from the actions and targets contained in the Millennium Declaration is to be
achieved by the end of 2015. In the past, Nigeria had made several attempts at making education
accessible to Nigerians. The old western region started free education during Chief Obafemi
Awolowo’s regime. In 1976, the Universal Primary Education Scheme was launched. Among the
reasons adduced to its near collapse are poor management, poor infrastructure, high rate of illiteracy
as a result of limited access to, poor retention in, and poor quality of basic education. As a result of
this, the Federal Government re-launched the UBE programme in September 1999 as a radical
intervention strategy for ensuring access to quality basic education (UBEC, 2008). A major challenge
of the reform for the education sector therefore is the need to reform the curriculum so as to bring it in
line with the strategic objectives of NEEDS and MDGs. To this end, the country developed a national
Policy for Information Technology that would among others, have a modern ICT-driven educational
system for the effective delivery of educational services at all levels. UNESCO (2004) identified a
number of frameworks for setting ICT for education programmes. These include: policy and vision of
ICT use in schools, technology and infrastructure, curriculum, pedagogy and content development,
professional development, monitoring and supervision. (Adeosun, 2010). To what extent Nigeria has
been able to meet these criteria remains to be investigated. This study therefore determines teachers’
use of Computer/Internet in teaching and learning in secondary schools and examines the
implementation of education policy in terms of the availability of ICT tools in schools and their general
integration for the improvement of teaching and learning.
2.1 Purpose of the study
The study was undertaken to investigate teachers, literacy profiles, attitudes towards computers,
general integration of Information and Communication Technology and also determine the hindrances
experienced by them on the integration of ICT in their teaching.
The study would specifically:
Find out the attitude profile of the teachers in Southwest Nigeria towards the integration of ICT in
their teaching practice.
Find out whether teachers use Computer/Internet in their teaching.
Find out the competency profile of teachers with respect to the use of various computer and
communication environments.
Determine the nature of hindrances experienced by teachers on the pedagogical integration of
ICT.
2.2 Scope of the study
The focus of this research is on computer/Internet use by teachers in Southwest Nigeria. The
study surveyed teachers’ use of Computer/Internet in Southwest Nigeria comprising of Ekiti, Ogun,
Ondo, Osun, Oyo and Lagos states.
2.3 Significance of the study
This study is significant in the sense that it would evaluate teachers’ use of Computer/Internet in
Nigerian secondary schools.
9

Babatunde Alabi Alege and Stephen Olufemi Afolabi
The present crop of teachers would benefit from the study as it would serve as eye opener for
them on the need to teach using information technology and also help them gain appreciable
insight into the power of these technologies worldwide.
The study hopes to be of benefit to curriculum developers and those charged with responsibility of
implementing various educational policies in Nigeria.
The authors may also benefit from the study as it may have shed light on some aspects that
needed to be reviewed.
The study would have implications for the improvement of the quality of education in Nigeria as a
nation.
3. Research questions
The following questions shall guide the study:
What are the attitudes of teachers toward the use of Computer/Internet as teaching tools in
Southwest Nigeria?
What are the competency levels of teachers on the use of Computer/Internet as teaching tools in
Southwest Nigeria?
To what extent do teachers in Southwest Nigeria use Computer/Internet in the instructional
process?
What are the barriers to ICT integration in teaching-learning process in Southwest Nigeria?
3.1 Current Situation
The presence and potentials of information and communication technologies in almost every human
activity is overwhelming. Globalization and advances in ICT have brought about phenomenal
improvements and great opportunities for developing countries to participate meaningfully in the
global digital economy (Nigeria Vision 2020, 2009). The use of ICT promotes development and
improves services in any organisation. It brings changes in today's business environment. In
academic environment, it speeds up information delivery, facilitates teaching, learning and research.
In spite of the above observation about the potentials, and benefits of using ICT, the level of
awareness and use in Nigeria appears to be very minimal. (Haliso, 2011). The literature dealing with
technology and pedagogy attests to the powerful impact ICT can have on the teaching and learning
process (John, 2010). Research indicates that level of collaboration and communication are
enhanced by the use of computers for knowledge building and thinking skills (Howe et al., 1997;
McFarlane, 1997).
Nigeria started implementing its ICT policy in April 2001 with the establishment of National Information
Technology Development Agency (NITDA) with a vision ‘to make Nigeria IT capable country in Africa
and a key player in the information, using IT as the engine for sustainable development, and global
competitiveness (Federal Ministry of Science & Technology, 2001). The strategic Action Plan provides
for a period of 5 years for the key sectors- health, education infrastructure, human resource
development, agriculture, legal/regulations, private sector/industry, media/community- as part of an
integrated approach to achieving national development within the context of the Millennium
Development Goals (MDGs), the Federal Government 7-point Agenda, the National Economic
Empowerment Development Programme (NEEDS) and a host of other socio-economic development
programmes and initiatives. According to Adeosun, (2010) the mission statement of NITDA
recognized the need to use IT for education (p.iii). In addition, the general objectives (3 out of 31)
focused on integrating ICT into the mainstream education and training, with a strategy to “restructure
educational systems at all levels to respond effectively to the challenges and imagined impact of the
information. Yusuf (2005) noted that although the mission, general objectives and strategies
recognized the importance of ICT in education, the document has no sectored application to
education and issues relating to education are subsumed under human resources development. ).
Adeosun (2010) also pointed out that Nigeria has no particularly articulated policy for ICT in education
in spite of her strong commitment to the promotion of ICT in her educational and economic goals. The
National Information Technology Development Agency (NITDA) developed some standards for
computer uses in schools, including students- computer ratio, in which schools are required to
10

Babatunde Alabi Alege and Stephen Olufemi Afolabi
establish computer laboratories and classrooms equipped with interactive white boards and other
teacher-aided learning tools.
The education sector has been affected by ICTs worldwide. Consequently, the processes of teaching,
learning and research have also been affected (Yusuf, 2005). The Federal Government of Nigeria, in
the National Policy on Education (2004), recognizes the prominent role of ICTs in the modern world,
and has integrated ICTs into education in Nigeria. To actualize this goal, the document states that
government will provide basic infrastructure and training at the primary school. In the new 9-year
Basic Education curriculum, Computer Studies/ICT has been made compulsory at the lower, middle
and upper levels of education. The government also intended to provide necessary infrastructure and
training for the integration of ICTs in the secondary school system. Adomi and Kpangban (2010)
noted that this was not the first attempt the Nigerian government made to introduce computer
education in schools. In 1988, the Nigerian government enacted a policy on computer education. The
plan was to establish pilot schools and diffuse computer education innovation first to all secondary
schools, and then to primary schools. Unfortunately, the project did not really take off beyond the
distribution and installation of personal computers (Okebukola, 1997; cited by Aduwa-Ogiegbaen and
Iyamu, 2005). Nkerenwem (1996) identified recent effort made by the Government towards the ICT
Development which includes the following:
Launching of the National Telecommunications policy (September, 2000)
Development of a comprehensive science and Technology policy. (2001)
Development and launching of the National Information Technology Policy (2001)
Establishment of the National Information Development Agency (NITDA) (2001)
Launching of the Nigerian Satellite Systems Programme by the National space Research and
Development Agency (NASRDA) (2001)
Development of the National Infrastructure Backbone (NIIB)
Development and launching of the Mobile Internet Units, these are buses equipped with computer
systems and other accessories with a VSAT installed in the buses for Internet access. Others are:
SchoolNet Nigeria
Computer-in-School project
One-Laptop-Per-Child (OLPC) project
Interactive Radio program; and
NEPAD e-school Initiative
In terms of ICT and teacher development, the National Policy on Education (2004) stipulates that
teacher education shall continue to take cognizance of changes in methodology and the curriculum
and teachers shall be regularly exposed to innovations in their profession. Similarly the National
Policy on Teacher Education (FME, 2007) developed a vision “to produce quality, highly skilled,
knowledgeable and creative teachers based on explicit performance standards through pre-service
and in-service programs to raise a generation of students who can compete globally’ (p.6). The goal is
to ‘ensure teachers are trained and recruited to teach world-class standards and continue to develop
their competence over their entire career’ (p.6). ICT was identified as one of the conditions for the
achievement of the goal, as ‘the training of teachers on strategies of collaboration, reflection on
enforcement of ICT practices and action research’ (p.5). According to Hennessy et al (2010), bringing
ICT into the classroom can have a considerable impact on the practice of teachers, in particular when
ICT is conceptualised as a tool that supports a real change in the pedagogical approach. Not only do
the teachers need to change their roles and class organisation, they also need to invest energy in
themselves and their students in preparing, introducing and managing new learning arrangements.
Okebukola (1997) however pointed out that the chalkboard and textbook still continue to dominate
classroom activities as computer is still not part of classroom technology in more than 90 percent of
Nigerian public schools. Many barriers have been attributed to lack of use of technology in the
teaching and learning process. In a survey conducted by Adomi and Kpangban (2010) on the
application of ICT in Nigerian schools, it was reported that limited/poor information infrastructure;
lack/inadequate ICT facilities in schools; poor ICT policy/project implementation strategies and
frequent electricity interruptions ranked very high as causes of low ICT application.
11

Babatunde Alabi Alege and Stephen Olufemi Afolabi
Goshit, (2006) predicted that ICT application and use will prove beneficial in improving Nigeria's
educational system and giving students a better education. A technologically-advanced workforce will
lead to ICT growth in Nigeria, with the potential to improve military technology and
telecommunications, media communications, and skilled ICT professionals who will be well-equipped
to solve IT problems in Nigeria and other parts of the world.
4. Methodology
The research design used for the study is a descriptive survey design. Five hundred and sixty two
(562) Secondary school teachers from public and private schools were selected by using
proportionate random sampling from the six states that comprised the Southwest zone of Nigeria. Five
hundred and fifty (550) teachers eventually participated in the study. Their basic statistics include 58%
females and 42% males with varying educational qualifications and working experiences. The
instrument used was questionnaire. Two types were used. The first one was adopted from Computer
Attitude Questionnaire developed by Knezek & Miyashita, 1994, consisting of twenty two items which
was used to produce a measure of the teachers’ attitude to computer/Internet. The second was
adapted from a survey developed by Damiraslan and Usluel (2005) which had been used in earlier
research studies and had an established content validity. It consisted of four sections to collect
information on demography, computer application, frequency of use and barriers to integration of
computer applications as an instructional tool. The instrument was administered on the respondents
by the researchers during teacher training sessions and workshops. This enabled them to collect
nearly all the questionnaires.
5. Data analysis
Collected surveys were analyzed using simple percentages.
Table 1: Demographic analysis of Teachers’ Experience
Computer
Experience
1-5 years 6-10 years 11-15 years 16-20 years >20 years
42.6% 25.7% 23.3% 10.8.% 7.6%
Background information of the teachers included computer experiences of participants. (58%).
percent of the teachers responding to the survey were females. Their teaching experience varied from
1-5 years to over 15 years. (Table 1.) Only 7.6% of the teachers indicated that they had experience
with computers more than 20 years, while 42.6% of teachers indicated 1-5years of computer
experience.
5.1 Research Question 1
What are the attitudes of teachers toward the use of Computer/Internet as teaching tools in Southwest
Nigeria?
Table 2: Teachers’ Attitude towards the use of Computer/Internet as teaching tools
Computer/Internet Attitude YES% NO%
1 I enjoy doing things on a computer. 21 79
2 I am tired of using a computer. 23.5 76.5
3 I will be able to get a good job if I learn how to use a computer. 45.3 54.7
4 I concentrate on a computer when I use one. 46.2 53.8
5 I enjoy computer games very much. 26 74
6 I would work harder if I could use computers more often. 45.1 54.9
7 I think that it takes a long time to finish when I use a computer. 34.9 65.1
8 I know that computers can give me opportunities to learn many new things. 87.4 12.6
9 I can learn many things when I use a computer. 46.8 53.2
10 I enjoy lessons on the computer. 67.5 32.5
11 I believe that the more often I use computers, the more I will enjoy school. 78 22
12 I believe that it is very important for me to learn how to use a computer. 84 16
12

Babatunde Alabi Alege and Stephen Olufemi Afolabi
13 I think that computers are very easy to use. 27.8 72.2
14 I would like to study in the library rather than using a computer. 45 55
15 I feel comfortable working with a computer. 38.6 61.4
16 I get a sinking feeling when I think of trying to use a computer. 80.8 19.2
17 Working with a computer makes me nervous. 74.3 25.7
18 Using a computer is very frustrating. 76 24
19 I will do as little work with computers as possible. 68.8 31.2
20 Computers are difficult to use. 68 32
21 Computers do not scare me at all. 69.4 30.6
22 I can learn more from books than from a computer. 80.8 19.2
Adapted from Part 1
Adapted from Computer Attitude Questionnaire, Knezek & Miyashita, 1994
Table 2 revealed that the teachers display favourable dispositions toward computers when 87.4% of
the respondents admitted that computer can give opportunities to learn many new things.
Interestingly, 80.8% of the respondents who reported that they get a sinking feeling when they think of
trying to use a computer also believe they could learn more from books than from a computer. The
outcome of this study corresponds with the finding of Namlu and Ceyhan (2002) that individuals who
perceive themselves as computer introverts do have a higher level of computer anxiety and thus tend
to exhibit unwelcoming attitude towards computer use. Meanwhile,
5.2 Research Question 2
What are the competency levels of teachers on the use of Computer/Internet as teaching tools in
Southwest Nigeria?
Table 3: Competency levels of teachers on the use of Computer/Internet as teaching tools
Very high High Average Low Very low
Proficiency in: % % % % %
Word processing 3.6 5.8 43.6 38 9
Spread sheets 2.8 3.5 20.3 38.8 34.6
Database programs 0.8 3.6 26 69.6 45.8
Graphics and drawing programs 6,9 11.2 34.8 23 24.1
Desktop publishing 8.4 12.6 40 20.2 18.8
Presentation programs 0.6 1.5 12.9 57 28
Educational CDs 18 24.5 30.6 15.5 11.4
Email 9.6 12.2 32.8 36 9.4
Internet 9.8 12.8 40 24.8 12.6
Table 3 shows the teachers’ competency levels on the use of Computer/Internet. The table indicates
that teachers’ proficiency in the use of ICT tools is generally low. Only 9.4% of the teachers are highly
proficient in word processing, 43.6% are average while 47% are lower.
5.3 Research Question 3
To what extent do teachers in Southwest Nigeria use Computer/Internet in the instructional process?
Table 4: Computer/Internet utilization
I use computer-based technology in my class for: Very Often Rarely Not at all
Teaching 33 (6%) 38 (6.9%) 479 (87.1%)
Assignment 52 (9.5%) 165 (30%) 333 (60.5%)
Word processing 35(6.4%) 158 (28.7%) 357 (64.9%)
Tutorials 22 (4%) 26 (4.7%) 502 (91.3%)
Keyboarding 37 (6.7%) 133 (24.2%) 380 (69.1%)
Information access 76 (13.8%) 220 (40%) 254 (46.2%)
Games 43 (7.8%) 69 (12.5%) 438 (79.7%)
Multimedia applications 36 (6.5%) 115 (20.9%) 439 (79.8%)
Graphics 87(15.8%) 154 (28%) 309 (56.2%)
13

Babatunde Alabi Alege and Stephen Olufemi Afolabi
I use Internet for: Very often Rarely Not at all
e-mail 148 (26.9%) 114 (20.7%) 288(52.4%)
Chat 36 (6.5%) 87 (15.8%) 429 (78%)
Downloading instructional materials 58 (10.1. %) 165 (30%) 328 (59.9%)
Giving assignments 69 (12.5%) 182 (33.1%) 299 (54.4%)
News and bulletin 76 (13.8%) 86 (15.6%) 388 (70.6%)
Information access via Web browsers (e.g. Google) 102 (18.5%) 69 (12.5%) 379 (68.9%)
Games 43 (7.8%) 136 (24.7%) 371 (67.5%)
File transfer 25 (4.5%) 200 (36.4%) 325 (59.1%)
In Table 4, the results show that only 6% of the teachers use computer-based technology for their
teaching very often while 87.1% do not use it at all. There is a clear indication that lack of access
have accounted for the low utilization of computers in many Nigerian secondary schools.
5.4 Research Question 4
What are the barriers to ICT integration in teaching-learning process in Southwest Nigeria?
Table 5
Causes of low level/barriers to ICT integration No. Percent
Limited/poor information infrastructure 440 80
Lack of/inadequate inadequate ICT facilities in schools 460 85
Frequent electricity interruption 275 50
Non integration into the school curriculum 143 26
Poor ICT policy/project implementation strategy 400 73
Inadequate ICT manpower in the schools 505 92
High cost of ICT facilities/components 380 69
Limited school budget 275 50
Lack of/limited ICT skills among teachers 500 91
Lack of/poor perception of ICTs among teachers and administrators 330 60
Inadequate educational software 375 68
Technophobia 405 74
Laziness 330 60
Poor management on the parts of school administrators and government 50 9
Lack of maintenance culture 35 7
Lack of interest in ICT application/use on the part of the school 220 40
N=550
Adapted from Adomi & Kpangban Causes of low level of ICT application in Nigerian high schools
Table 5 shows the teachers’ responses to the barriers experienced by them for not integrating ICT in
the teaching-learning process. Top on the list is inadequate ICT manpower in the schools as reported
by 92% of the teachers. Lack of/limited ICT skills among teachers came second with 91%. The
facilities in schools are not adequate as reported by 85 of the teachers. Only 9% reported that poor
management on the parts of school administrators and government is responsible for the barriers.
6. Discussion
As Chika (2004) pointed that the world is experiencing an ever-increasing use of computers in diverse
area of human endeavour. There is evidence of this in Nigeria too, as governments at both the
Federal and State levels vote much money to supply computer equipments into secondary schools.
Yet, the proficiency of the teachers has been revealed by the study to be at its low ebb. However, it is
evident, that computer is now in Nigerian schools to affirm the position of Nwagu (2006) and Adomi
(2010) that the rate at which the countries of the world accept the use of computer is a major factor for
development.
The findings of this survey have shown that there is still a lot to be done in terms of human
development, provision of facilities and monitoring of schools to ensure strict compliance with
educational policies and practices.
Nigeria realised the positive impact that Information and Communication technology can have on
teaching and learning and have made several efforts to integrate it into the primary and secondary
14

Babatunde Alabi Alege and Stephen Olufemi Afolabi
school systems. There remains a concerted effort to remove all barriers to effective use of these
technologies in Nigerian schools.
Recommendations
All the schools should be provided with computers and adequate infrastructure to facilitate the good
intention of the government.
Teachers should be adequately trained and retrained to improve on their ICT capabilities while
teachers that are computer literate should be posted to every school to impart ICT skills to students.
All other staff should be encouraged and given adequate incentives to improve on their ICT literacy
levels.
Nigeria should ensure that teachers are well remunerated like their counterparts in other sectors. The
government should remove all forms of discriminations in form federal, state and local government
workers/teachers so that all teachers are placed on equal pedestal...
References
Abolade A.O. and Yusuf M.O. (2005) Information and Communication Technologies (ICTs) and the Nigerian
Teacher Education Programme 1
African Journal of Educational Studies, Vol.3 No.1, p. 1
Adebowale, O.F, Adediwura, A.A & Bada, T. A. (2009) Correlates of Computer Attitude among Secondary
School Students in Lagos State, Nigeria International Journal of Computing and ICT Research, Vol. 3, No.
2, pp. 20 - 30. http://www.ijcir.org/volume3-number2/article3.pdf
Adeosun, O. (2010) Quality Basic Education Development in Nigeria: Imperative for Use of ICT
http://digitalcommons.unl.edu/libphilprac/345
Adomi, E.E. & Anie, S.O. (2006) An Assessment of Computer Literacy Skills of Professionals in Nigerian
University Libraries. Library Hi Tech News Vol.23 No 2.pp.10-14
Anderson, N. & Baskin, C. (2002). Can we leave it to chance? New learning technologies and the problem of
professional competence. International Education Journal, 3 (3), 126 – 137 (Electronic Version), retrieved
January 24 th , 2004, from http://iej.cjb.net/
Esharenana E., Adomi and Kpangbany, E. (2010) Application of ICTs in Nigerian
Secondary Schools Library Philosophy and Practice (e-journal) CICE Hiroshima University, Journal of
International Cooperation in Education, Vol.13 No.2 (2010) pp.193 ~ 211
Federal Republic of Nigeria (2001). Nigerian national policy for information technology. Abuja:
Federal Republic of Nigeria (2004). National Policy on Education (4 th ed.). Lagos: NERDC.
Goshit, T. (2006). Nigeria's need for ICT: SP. 259 technology and policy in Africa. Available:
http://ocw.mit.edu/NR/rdonlyres/Special-Programs/SP-259Spring-2006/891209EE-E63B-4617-
BA9D7635A63C754B/0/goshit.pdf
Hennessy, S, Harrison, D & Wamakote, L. (2010) Teacher Factors Influencing Classroom Use of ICT in Sub-
Saharan Africa Itupale Online Journal of African Studies, 2
Knezek, G & Miyashita (1994) Survey of Teachers’ Attitude Toward Computers: Texas Center for Educational
Technology
National Information Development Technology (2008) ICT 4D Strategic plan for Economic Sector of Nigeria
NITDA (2003). Use IT: National Information Technology Development Agency, Abuja Nigeria. Available:
http://www.nitda.gov.ng/use_it.htm.
NITDA. Retrieved 23 rd of January, 2004, from http://www.nitda.gov.ng/docs/policy/ngitpolicy.pdf
Nigeria Vision 2020 (2009) Report of the Vision 2020 National Technical Working Group on Information and
Communication Technology.
Nwagwu, W.E. (2006). Integrating ICTs into the globalization of the poor developing countries. Information
Development 22 (3): 167-179.
Ocak, M.A., and Akdemir O. (2008) An Investigation Of Primary School Science Teachers’ Use Of
Computer Applications The Turkish Online Journal of Educational Technology – TOJET October 2008 ISSN:
1303-6521 volume 7 Issue 4 Article 6
Oduroye, A.P. (n.d.) Challenges of learning and teaching with computers. Available:
http://www.itnetwork.org.uk/56.htm
Okebukola, P. (1997). Old, new, and current technology in education. UNESCO Africa 14 (15): 7-18.
Okebukola, P. (2004). E-learning in varsities, others underway, NUC boss lists strategies. The Guardian (12
October): 35, 39.
Osei Tutu Agyeman2007) ICT for Education in Nigeria
Sam E. O. Aduwa-Ogiegbaen, and Uwameiye,R. (2006) Internet Usage Among University
Lecturers in Southern Nigeria: International Journal of Information and Communication Technology Education,
Vol 2, Issue 1 edited by Lawrence Tomei
15

Issues and Challenges in Implementing eLearning Projects
in Higher Education: The Case of Jordan
Hussein Al-Yaseen, Saheer Al-Jaghoub and Nidal Al-Salhi
Al-Ahliyya Amman University, Amman, Jordan
hyaseen@ammanu.edu.jo
saljaghoub@ammanu.edu.jo
nalsalhi@ammanu.edu.jo
Abstract: Jordanian universities have started to adopt eLearning projects aiming to improve the effectiveness
and efficiency of the educational process for both instructors and learners. This paper presents the results of a
preliminary study that aims to identify the main issues and challenges facing these universities in implementing
their eLearning projects. The data were collected using a survey to investigate the main factors affecting
eLearning projects in Jordanian universities. The results of this study showed that the most important issues and
challenges faced by Jordanian universities are human, cultural, regulatory, support, technical and financial
respectively. These findings are hoped to be useful for researchers and policy makers as they provide some
insights to universities in Jordan and other developing countries about the main issues and challenges that may
be faced when planning for and developing their own eLearning systems.
Keywords: eLearning, challenges, higher education, developing countries, Jordan
1. Introduction
eLearning (electronic learning/ distance learning) concept is argued to be one of the most important
recent developments in information systems (Wang, 2003). It is a learning trend that challenges the
traditional concepts of education. eLearning has been viewed as identical with web-based learning,
Internet-based training, advanced distributed learning, web-based instruction, online learning and
open learning (Khan, 2001). It is about the delivery of course content through electronic media, such
as Internet, Intranets, Extranets, satellite broadcast, audio/video tape, interactive TV, and CD-ROM
(Urdan & Weggen, 2000). eLearning system (models) can be asynchronous (real time) or
asynchronous (anytime and/or anywhere). Synchronous learning can supports live; oral; visual
communications between instructors and students. Synchronous learning can be attained by using
the audio systems to support oral communications or by using of interactive keypad devices to
support the exchange of data and voice or by using of the videoconferencing technologies. In
asynchronous learning the communication between instructors and students occurs at a different time
and can be achieved using text material; recorded video and recorded audio (Holden and Westfall,
2010).
eLearning is argued to have many benefits that can be delivered, such as: cost savings and resource
efficiency, flexibility and accessibility, ease of renew and update, joint/collaborative learning,
scalability, student achievement, inclusion, and participation (Alexander, 2001; Marengo and
Marengo, 2005; Brady et al., 2010; Paechter et al., 2010). These benefits have caused many
universities in both developed and developing countries to move towards implementing eLearning
projects. For example, the UK market for eLearning remains the largest European market (Patterson
et al., 2010). The US market for self-paced eLearning products and services reached $18.2 billion in
2010; the revenues generated by eLearning are expected to double every year in the United States,
while in the Asian Pacific region, it is worth US$233 million (Ambient Insight Report, 2010).
A number of developing countries have started implementing a number of eLearning projects
(Omidinia et al., 2010), but the level of these projects vary among countries because of the
differences in a number of factors such as accessibility, income, and ICT infrastructure (Dadzie,
2009). However, the ease of access to education provided by ICTs makes it a viable option to provide
better education to people who may have been otherwise deprived from such opportunities (Casal,
2007). For developing countries, better access to education provided by eLearning constructed an
important means to achieve socio-economic development and fight poverty (Khan and Williams,
2006).
For Jordan in particular, there is a strong interest in education, given the country’s lack of natural
resources and its dependence on human resources. As John Chambers, CEO of Cisco Systems said:
“No country in the world has done what Jordan did in such a short period of time... Jordan is
16

Hussein Al-Yaseen et al.
improving by using technology to implement strategies and to achieve dreams”. Because of the
promised benefits of eLearning and the increasing need in Jordan for new opportunities to provide
higher education for students, a number of Jordanian universities have started implementing
eLearning projects. However, investing in such projects requires resources, is faced by many
challenges and is influenced by a number of factors. Therefore, this paper aims to identify the main
challenges facing universities undergoing such projects.
The paper is structured as follows: the introduction is followed by a brief review of the issues and
challenges in implementing eLearning projects in higher education. In Section 3 The higher education
system in Jordan is briefly presented. Section 4 discusses the research approach, followed by a
discussion of the main findings in Section 5. Finally, in Section 6 the main conclusions and
recommendations of the paper are discussed.
2. Issues and challenges in implementing eLearning projects in higher
education
eLearning may offer many benefits, yet its adoption by universities is faced by a number challenges
and obstacles that need to be identified and tackled in order to realise the promised benefits. Previous
studies have identified six main challenges in implementing such projects, which are briefly presented
below.
The first challenge is Legislation of eLearning in higher education: which refers to the extent by which
the legislations set by the ministry of higher education support the adoption of eLearning because
rules and regulations can act as enablers or inhibitors for eLearning (Rezaei Mood, 2006). Second,
awareness of using eLearning content and its efficiency and effectiveness amongst learner groups
(students): this is argued to be an important factor that will affect learners’ rate of adoption (Tham and
Werner, 2002; Tyan, 2003 and Rezaei Mood, 2006). eLearning is argued to be a student-centred way
of delivering education and therefore, if the main parties involved, especially students are not fully
aware of the benefits of online learning the success of such programmes becomes questionable.
Third, management support of developing and adopting eLearning in higher education: the
acceptance of eLearning methodology by decision makers in higher education increases and
supports the success of eLearning projects (Murphy and Terry, 1998; Omidi Najafabadi et al., 2008).
Generally speaking, for any project to be successful, management support is essential and eLearning
projects are no exception especially that implementing such programmes requires many changes in
the traditional methods of teaching and learning, training for educators and students and resources.
Therefore, management support is necessary for providing such requirements. Fourth, connectivity
and speed of downloading eLearning content: having a successful eLearning programme requires
efficient internet connections and high rates of internet diffusion to enable students to realise the
benefits of online education, which will contribute to a high rate of adoption (Zhang et al., 2002;
Cantoni et al., 2004). Fifth, quality of e-content and availability of expertise that are responsible for
developing a good e-content combined with sufficient financial resources will lead to low or high
quality eLearning content which in effect will impact the learner’s rate of adoption (Andersson, 2008).
Sixth, language barrier: the widespread use of English language in eLearning content is one of the
major issues that could affect the rate of eLearning adoption especially in non-English speaking
countries (Rezaei, 2006; Andersson, 2008). Therefore, universities in such countries have to create
their own content for students.
3. Higher education in Jordan
Before moving on to discuss eLearning in Jordan, it may be useful to briefly present the country’s
higher education system. The Ministry of Education (MoE) is responsible for the pre-primary, primary
and secondary levels of education, whilst, the post-secondary education is the responsibility of the
Ministry of Higher Education and Scientific Research (MoHESR). This Ministry includes the Higher
Education Council and the Accreditation Council. The MoHESR has outlined a National Strategy for
Higher Education for the years 2007-2012 (MoHE, 2011).
Access to higher education is open to holders of the General Secondary Education Certificate (or its
equivalent) who can then choose between private community colleges, public community colleges or
universities (public and private). The credit-hour system, which entitles students to select courses
according to a study plan, is implemented at universities. The higher education system of the country
has evolved considerably in the past years, but still a lot needs to be done to keep up with a rapidly
growing knowledge based economy that Jordan aims to be transformed into. In the years between
17

Hussein Al-Yaseen et al.
2000/2001 and 2006/2007, Jordan has seen an increased demand for higher education with
enrolments growing at an annual rate of 14 percent from 77,841 to 218,900 students (MoHE, 2011).
The first Jordanian university, the University of Jordan, was established in 1962. Today there are ten
public universities. In the past, higher education was the exclusive duty of government, but the role of
the state in this sector decreased with the establishment of the first private university, Amman Al-
Ahliyya University, in 1990. The emergence of private universities was a necessary development to
face the problem of the lack of university seats and to solve the problem of increasing demand.
Privatization of higher education has increased gradually and today there are seventeen private
universities. Foreign universities also became common in higher education. (e.g., German Jordanian
University, established in 2005, Arab Open University established in 2002 to provide continuous
education for all ages, and New York Institute of Technology).
The use of ICTs as a catalyst for development coupled with the importance of education for
Jordanians has led to the start of a number of eLearning initiatives both at school and university
levels. The benefits of eLearning in addition to the need to meet the increasing demand for higher
education in Jordan has motivated most Jordanian universities to invest in some sort of an eLearning
system in order to support “blended learning” or pure “on-line learning” modules. However, these
projects have faced a number of challenges. Our aim in this paper is to identify some of the issues
and challenges facing Jordanian universities which have adopted eLearning projects.
4. Research approach
This research follows the quantitative (survey) approach at this preliminary stage of work. A
questionnaire was developed based on interviews with some experts and previous literature. The
questionnaire was then revised with the help of experts with significant experience in eLearning
including academics from Jordanian universities. The questionnaire included fixed-choice questions
about the main issues and challenges of eLearning in higher education. A 5 point Likert scale ranging
from 1 as strongly disagree to 5 as strongly agree was used for the measurement; and 0 as not
applicable. Data collection for this research took place in May 2011.The questionnaire was sent by
email to the people who are directly involved in developing and implementing eLearning projects in all
the public and private universities in Jordan.
The data was analyzed using a combination of the parametric statistical methods: Descriptive
Analysis and Factor Analysis (Pett et al., 2003).
A factor analysis technique was employed in order to identify possible categories. Factor analysis was
performed in three steps (following Berthold and Hand, 2003):
A matrix of correlation coefficients for all possible pairings of the variables was generated.
Factors were then extracted from the correlation matrix using principal factors analysis.
The factors were rotated to maximize the relationships between the variables and some of the
factors and minimize association with others using Varimax Kaiser Normalization, which
maintained independence among the mathematical factors. The Eigenvalues determined which
factors remained in the analysis. Following Kaiser’s criterion, factors with an Eigenvalue of less
than 1 were excluded. A Screen plot provides a graphic image of the Eigenvalue for each
component extracted.
5. Discussion of findings
The questionnaire was distributed by e-mail to all public and private universities in Jordan (10 public
and 17 private universities). From 10 questionnaires sent to the public universities, 6 useable
questionnaires have been returned with a response rate of 60%, and from 17 questionnaires sent to
the private universities, 12 useable questionnaires have been returned with a response rate of 70.5%.
This makes the total response rate from all Jordanian universities 66.7%; which is considered to be
above expectation given that the generally accepted average responses to non-incentive based
questionnaires are around 23%. This response rate indicated that the respondents in both
public/private universities found the topic interesting and relevant, and more interesting in the private
universities where the competition rate is higher.
The results of descriptive statistics showed that most universities in Jordan have used an open-source
learning management system such as (Moodle or Blackboard). 40% of the public universities have an
18

Hussein Al-Yaseen et al.
independent eLearning centre that is responsible for developing and managing eLearning projects;
while 70% of the private universities have their eLearning centre.
Based on factor analysis of the questionnaire, the most important issues and challenges in
implementing eLearning projects in higher education in the Jordanian universities are discussed
below: (see Appendix 1 for a detailed description of these factors referred to as ICIe-LHE).
Human: the findings showed that the human factor was the most important issue and challenge that
hinder the success of implementing eLearning projects at private and public universities. Human
factor is highly correlated with seven variables as factor analysis showed. The most two important
variables were the perceived threat of eLearning to instructors’ career and the lack of sufficient
experts in eLearning. The less important variables for this factor were lack of training for both
educator and learner and lack of relationship between instructors and students. This may be due to
the fact that some of the instructors do not have sufficient computer skills and/or see the use of
technology as a way by which they will be replaced.
Cultural: this factor is highly correlated with five variables. The most two important variables were not
taking the eLearning project seriously form both developers and learners and the problem related to
the users’ culture toward eLearning, while the least two variables were the problems related to group
working and the negative preconceptions of users towards eLearning. Such factor could be related to
the traditional face to face methods of teaching used at universities. The soft skills in general are a
problem facing many graduates that need further training on such skills. For some faculty members, it
may be difficult to change the way they were teaching for many years, as eLearning is a new trend
that requires new teaching cultures (Uhomoibhi, 2006) and for some students changing the way they
were taught was also difficult, which has also been the case for students elsewhere (O'Donoghue et
al., 2003)
Regulations and management: this factor is highly correlated with five variables. The most two
important variables were lack of strategic vision in developing eLearning projects and the unfamiliarity
of top management with eLearning applications. However, the least two important variables for this
factor were lack of intellectual property right and lack of network security regulations. The lack of
vision could be related to the fact that some universities actually started an eLearning project because
it is a new trend rather than being actually related to the organisations’ vision. In terms of regulations,
intellectual property right laws have been enforced in Jordan since 2002. However, the major obstacle
is that according to the laws and regulations of the Ministry of Higher Education (MoHE) offering
online degrees is still not permitted as a certain percentage of any course has to be offered face to
face.
Support: this factor is highly correlated with four variables. Lack of appropriate support services for
users and limited of lack of incentives for using eLearning system were found the most important
variables for this factor. Lack of appropriate support services for maintaining eLearning equipments
and problem related to the course starting were found the least important variable to this factor. These
challenges underline the importance of having an incentive system to encourage instructors and
students to adopt eLearning, especially that it requires change in the teaching methods which is time
consuming and requires effort especially at the start-up phase
Technical: was found to be one of the least important challenges for in implementing eLearning
projects at the universities. The technical factor is highly correlated with eight variables. The most two
important variables were lack or limited access to appropriate contents and the low bandwidth, while
lack of hardware and lack of appropriate infrastructure were found to be the least important variables.
Appropriate content may be related to the language barrier, which makes it necessary for universities
to create their own content in subject areas that are taught in Arabic. Infrastructure is not a major
issue given the general enhancements in technological infrastructure in Jordan since the start of the
ICT related initiatives in the country in 1999.
Financial: is also one of the least important challenges. This factor is highly correlated to seven
variables. The most two important variables were lack of budget for setting up eLearning and the cost
of upgrading the e-content, while, the least two important variables were cost of upgrading the system
and cost of maintaining the system. Given that some learning management systems are open source
(such as Moodle) contributes to the fact that the financial factor is not a major issue. The start-up
19

Hussein Al-Yaseen et al.
phase involves investments considering that many universities established an eLearning centre as
part of their projects. However, this may not be a major obstacle especially in private universities.
6. Conclusion
In Jordan, eLearning adoption level is still at its early stages. In order to achieve eLearning benefits in
higher education it is important for universities and government to work together to resolve the issues
and challenges discussed above. By analyzing the findings of this paper, we can propose some
recommendations that may be useful to universities and decision makers in implementing eLearning
projects.
eLearning approach to use: Jordanian universities are still at the early stages of transition between
the traditional and eLearning modes. It is proposed that the blended approach is more suitable (faceto-face
and online interactions). Using blended approach may be a successful delivery mode and
could contribute to overcoming many of the issues and challenges such as improving the relationship
between learners and students, increasing of the understanding level of the advantages of eLearning,
increasing level of awareness about knowledge and skills needed for using eLearning, and change
management. Furthermore, blended learning is allowed by current legislations of the MoHE while fully
online courses are not.
Awareness: Generally there is still a lack of awareness amongst the main stakeholders (instructors
and students) of the effectiveness of eLearning. Many instructors feel that the traditional learning
mode is better. Workshops and training courses need to be set up to increase the awareness level of
top management, instructors and students of eLearning. Such awareness will overcome some of the
obstacles such as feeling that eLearning will threat instructors’ role, problems related to the culture of
eLearning users, not taking eLearning seriously by users, the negative preconceptions of users and
universities towards eLearning.
Top management support: Decreasing the unfamiliarity of decision makers in higher education
institutions about eLearning advantages will contribute to increasing the rate of adoption between
users and decrease the effect of some problems related to culture of developers, educators and
learners towards eLearning systems.
Legislations of eLearning in higher education: Legislations of eLearning in higher education in
Jordan restrict adoption level of eLearning systems in Jordanian universities as the fully online
certificates are still unrecognized by the Ministry of Higher Education (MoHE) regardless of the quality
and source of these certificates. It is proposed to establish a national committee, which represents the
various stakeholders such as representatives from the ministry of higher education, ministry of
education, ministry of information and communication technology, and representatives from both
private and public universities. The role of this committee is to review the legislation of distance
learning in developed and developing countries; develop regulations and standards for eLearning
systems; establish accreditation policies for online certificates; increase the awareness level of
eLearning in higher education; encourage and support eLearning in higher education; monitor and
control eLearning practices in higher education; train and develop eLearning content experts;
establish of standards of quality e-content; encourage eLearning knowledge sharing among higher
education institutes in Jordan.
Finally, eLearning is hoped to play an important role in improving the quality of education and provide
learning opportunities to people who may otherwise be deprived from such opportunities. In order to
improve the effectiveness and efficiency of eLearning practices in Jordan, ministries related to
education need to move towards a higher level of eLearning development which will require
establishing a strategic vision in developing eLearning and more effort especially technically and
financially. Issues like human, legislation, culture appear to be the major challenges of adopting
eLearning projects in higher education institutes in Jordan and need attention in the deployment of
distance learning. Furthermore, in order to realize the full benefits of eLearning, the decision makers
in the government need to play an active role in supporting and promoting eLearning practices and
initiatives adopted by higher education institutes in Jordan. The research findings may be useful as
they provide a clear insight to other developing countries about the main issues and challenges that
face the adoption of eLearning projects in Jordanian universities when planning for and developing
their eLearning systems.
20

Hussein Al-Yaseen et al.
7. Appendix 1: Issues and challenges of implementing eLearning projects in
higher education in Jordan
Factor Code/Variable Code Description Eigenvalue
ICIe-LHE1 Lack of budget for setting up eLearning
0.967
ICIe-LHE2 High cost of buying hardware 0.982
Financial
ICIe-LHE3
ICIe-LHE4
High cost of buying software
High cost of access to internet
0.991
0.986
ICIe-LHE5 Cost of maintaining the system 0.950
Technical
Support
Regulations &
Management
Culture
Human
References
ICIe-LHE6 Expense of upgrading the system 0.942
ICIe-LHE7 Expense of upgrading the content 0.972
ICIe-LHE8 Lack of appropriate infrastructure 0.870
ICIe-LHE9 Lack of hardware 0.855
ICIe-LHE10 Lack of software 0.886
ICIe-LHE11 Low bandwidth 0.898
ICIe-LHE12 Limited access to the educational
software
0.889
ICIe-LHE13 Inequality in access to equipments 0.884
ICIe-LHE14 Speed of changes in computer
technology
0.842
ICIe-LHE15 Lack or limited access to appropriate
contents
0.899
ICIe-LHE16 Lack of appropriate support services for
user
0.880
ICIe-LHE17 Limited or lack of incentives (for using
eLearning system)
0.902
ICIe-LHE18 Lack of appropriate support services for
maintains equipments
0.932
ICIe-LHE19 Problem related to the establishing
courses
0.926
ICIe-LHE20 Lack of strategic vision in development
eLearning
0.936
ICIe-LHE21 Unfamiliarity of top manager with
eLearning applications
0.951
ICIe-LHE22 Lack of intellectual property right 0.951
ICIe-LHE23 Lack of appropriate rules for using
eLearning
0.941
ICIe-LHE24 Lack of network security 0.912
ICIe-LHE25 Negative preconceptions of user towards
eLearning
0.977
ICIe-LHE26 Negative preconceptions of
organizations towards eLearning
0.978
ICIe-LHE27 Problem related to group working 0.982
ICIe-LHE28 Not taking eLearning seriously by people 0.946
ICIe-LHE29 Problem related to the culture of users 0.919
ICIe-LHE30 Threat to instructor 0.922
ICIe-LHE31 Require new knowledge and skills 0.973
ICIe-LHE32 Lack of adequate experts in eLearning 0.869
ICIe-LHE33 Not understanding about advantages
and disadvantages of eLearning
0.894
ICIe-LHE34 Lack of preparatory training for educator 0.865
ICIe-LHE35 Lack of preparatory training for learner 0.973
ICIe-LHE36 Lack of relationship between instructors
and students
0.974
Alexander, S. (2001) eLearning developments and experiences, Journal of Education and Training, Vol. 43, No.
4, Pp. 240-8.
Ambient Insight Report, 2010 [On-line], available: http://www.ambientinsight.com/Reports/eLearning.aspx
Andersson, A. (2008) Seven major challenges for eLearning in developing countries: Case study eBIT, Sri Lanka,
International Journal of Education and Development using ICT, Vol. 4, No. 3. Available online at:
http://ijedict.dec.uwi.edu/viewarticle.php?id=472.
21

Hussein Al-Yaseen et al.
Berthold, M. and Hand, D. J. (2003) Intelligent Data Analysis, 2nd Ed., Springer-Verlag, Berlin.
Brady, K., Holcomb, L., and Smith, B. (2010) The Use of Alternative Social Networking Sites in Higher
Educational Settings: A Case Study of the ELearning Benefits of Ning in Education, Journal of Interactive
Online Learning, Vol. 9, No. 2, Pp. 151-170.
Cantoni, V., Cellarer, M. and Porta, M. (2004) Perspective and challenges in eLearning: towards natural
interaction paradigms, Journal of visual languages and computing, Vol. 15, Pp. 335-345.
Casal, C. (2007) ICT for education and development, Info, Vol. 9, No. 4, Pp. 3-9.
Dadzie, P. (2009) ELearning and E-Library Services at the University of Ghana: prospects and challenges,
Information Development, Vol. 25, No. 3, Pp. 207-217
Holden, J. and Westfall, P. (2010), An Instructional Media Selection Guide for Distance Learning – Implications
for Blended Learning Featuring an Introduction to Virtual Worlds, United States Distance Learning
Association, 2nd edition.
Khan, B. H. (2001). A framework for web-based learning, Engelwood CliVs, NJ: Educational Technology
Publications.
Khan, H., Williams, J. (2006) Poverty Alleviation through Access to Education: Can ELearning Deliver.
U21Global. Graduate School for Global Leaders.
[Online].http://www.u21global.com/PartnerAdmin/ViewContent?module
Marengo, A. and Marengo, V. (2005) Measuring the Economic Benefits of ELearning: A Proposal for a New Index
for Academic Environments, Journal of Information Technology Education, Vol. 4, Pp. 329-346.
Murphy, T.H. and Terry, H.R. Jr. (1998) Opportunities and obstacles for distance education in agricultural
education, J. Agric. Edu., Vol. 39, No. 1, [online], http://pubs.aged.tamu.edu/jae/pdf/vol39/39-01- 28.pdf.
Najafabadi, M., Hosseini, J. and Mirdamadi, M. (2009) A Case Study of Requirements and Challenges of
Information and Communication Technology System to Train Private Agricultural Insurance Brokers,
American-Eurasian J. Agric. & Environ. Sci., Vol. 6, No. 2, Pp. 152-159.
O'Donoghue, J., Singh, G. and Handy, D. (2003) Higher education – IT as a catalyst for change, On the Horizon,
Vol. 11, No. 3, Pp. 23-28.
Omidi Najafabadi, M., Farajollah Hosseini, J., Mirdamadi, M. and Moghadasi, R. (2008) Designing an Efficient
Information and Communication Technology (ICT) System to Train Private Agricultural Insurance Brokers in
Iran, Australian Journal Basic and Application Science, Vol. 2, No. 41, Pp. 1041-1051.
Omidinia, S., Masrom, M. and Selamat, H. (2010) Review of ELearning and ICT Infrastructure in Developing
Countries (Case Study of Iran), American Journal of Economics and Business Administration, Vol. 3, No. 1,
Pp. 120-125.
Patterson, D., Jung, G., Broadhead, G., and Halton, R. (2010) The UK eLearning market, [On-line], Available:
http://www.eLearningcentre.co.uk/resource.
Paechter, M., Maier, B., and Machera, D. (2010) Students’ expectations of, and experiences in eLearning: Their
relation to learning achievements and course satisfaction, Computers and Education, Vol. 54, No. 1, Pp.
222-229.
Pett, M. A., Lackey, N. R. and Sullivan, J. J. (2003) Making sense of factor analysis: the use of factor analysis for
instrument development in health care research, Sage Publications, London
Rezaei Mood, F. (2006) Challenges Assessment and ways of implementing eLearning in Iran, Master Degree
Thesis, College of Technology, Sharif University of Technology, Iran.
Tham, C. and J. Werner (2002) Designing and Evaluating ELearning in Higher Education: a Review and
Recommendations, Proceedings of the 2002 annual Conference of the Midwest Academy of Management,
Indianapolis, [online]:http://cobacourses.creighton.edu/MAM/2002/papers/Tham.doc.
Tyan, K.J. (2003) Diffusion barriers to eLearning in corporate Taiwan: a factor analysis of practitioners’
perspective. Phd dissertation, Indian University.
Urdan, T. A., & Weggen, C. C. (2000). Corporate eLearning: Exploring a new frontier WR Hambrecht + Co. [Online].
Available:http://www.wrhambrecht.com/ research/coverage/elearning/ir/ ir_explore.pdf .
Wang, Y. (2003). Assessment of learner satisfaction with asynchronous electronic learning systems, Information
& Management, Vol. 41, No. 1, Pp. 75–86.
World Bank (2009) Higher Education Reform for the Knowledge Economy Project, Project Appraisal Document,
World Bank. Washington, DC".
Zhang, W., J. Niu and G. Jiang (2002) Web-Based Education at Conventional Universities in China: A Case
Study, International Review of Research in Open and Distance Learning, Vol. 2, No. 2, [online],
www.irrodl.org/index. php/ irrodl/article/download/63/130.
22

The use of Open Educational Resources in Intra-
Organisational eLearning and Continuing Education
Antonios Andreatos
Div. of Computer Engineering & Information Science, Hellenic Air Force
Academy, Dekeleia Air Force Base, Dekeleia, Greece
aandreatos@gmail.com
Abstract: Sustained learning is a crucial factor of an organisation’s ability to survive and effectively compete in
the globalisation era worldwide. Continually shrinking half-life periods of knowledge and fast changing globalised
workplaces, force employees working in knowledge-intensive companies to acquire new knowledge and skills for
life. The evolution of Web 2.0, the advantages and boom of New Media, the massive production of open
educational resources and globalisation, are bringing revolutionary changes to all forms of education: formal,
non-formal and informal. Thus, the need for continuing education and lifelong learning is obvious. Various
surveys performed worldwide show that university graduates are not well prepared for today's working
environment. Most important skills missing are related to the use of ICT and adoption of emerging technologies,
teamwork/ collaboration, creativity, innovation and change management, diversity and Leadership and
fundamentals of enterpreneurship. Thus, there is also a need for education of newcomers. The purpose of this
work is to examine the use of open educational resources in intra-organisational learning and to examine the
properties of Learning Objects such as their metadata, that will facilitate this task. More precisely, this paper
initially argues about the advantages of using open educational resources in eLearning and continuing education.
Prerequisites for selecting and combining external Learning Objects will be examined. Ways for externalising the
organisation's social capital in reusable Learning Objects appropriate for intra-organisational learning will be
proposed. The issue of certification of informally acquired knowledge and skills will be mentioned. The use of
open educational resources may enrich existing social capital and facilitate intra-organisational eLearning and
continuing education in a cost-effective way. For better results, a formal characterisation scheme of LOs, in the
form of a posteriori metadata, will greatly facilitate their organisation, search and retrieval process.
Keywords: open educational resources, learning objects, continuing education, metadata, intra-organisational
learning, social capital
1. Introduction
With financial conditions changing constantly and affecting every aspect of business activities,
declining resources and growing competition, modern businesses are faced with multiple problems on
all fronts. On the one hand they should be able to do more with less resources, and the other, they
have to plan and achieve goals faster, more efficiently and of higher quality than in the past. As a
consequence, work environments and conditions are continually changing.
Continually shrinking half-life periods of knowledge and ever changing workplaces, force employees
working in knowledge-intensive companies to acquire new knowledge and skills for life. These
professionals have to continually update their experiences and knowledge profile (Fonstad & Lanvin,
2010). Thus, sustained learning is a crucial factor of an organisation’s ability to survive and effectively
compete in the globalisation era (Burke et al., 2006).
In their effort to survive, organisations turn to ICT. Technologists invent and propose new solutions
promising to facilitate enterprises to reduce their working cost and increase their revenues, such as
Cloud Computing and Virtualization. The problem is that all these developments are ... new! Who will
design the plans, who will implement them and who will manage them? The answer is of course “the
business' people”, who should be trained not only effectively but also very quickly in their new tasks.
1.1 The missing skills - what surveys show
Various studies (Fonstad & Lanvin, 2010; Azami et al., 2009; Workforce Readiness Report, 2006;)
concerning three different regions of the World (Europe, Malaysia and USA respectively), state that
graduates are not well prepared for today's working environment.
According to Fonstad & Lanvin (2010), there is a shortage of key ICT skills in Europe. Most missing
skills are basic and professional skills as well as cross-sectoral skills (such as principles of economy
and entrepreneurship). It is anticipated that by 2015 the unfilled vacancies of ICT professionals will be
between 1.7% and 13%.
23

Antonios Andreatos
Figure 1: Estimated skills shortage in Europe for the next years (source: Fonstad & Lanvin (2010))
Azami et al. (2009, p. 27, 29) have found out that the skills considered most important by employers
were related to teamwork, communication and problem-solving ability. Other missing skills were those
related to lifelong learning, application of basic knowledge, as well as, understanding professional,
social and ethical responsibilities.
The American Workforce Readiness report was prepared collaboratively by the Conference Board,
Corporate Voices for Working Families, the Partnership for the 21st Century Skills and the Society for
Human Resource Management. They conducted an in-depth study on the readiness of new entrants
into the United States workforce by level of educational attainment from the corporate perspective.
The four organisations surveyed over 400 employers across the U.S. These employers articulated the
skill sets that new entrants need in order to succeed in the workplace. Among the most important
skills cited by employers were: professionalism/ work ethic, oral and written communications,
teamwork/ collaboration and critical thinking/ problem solving.
Employer respondents determined as the top six skills expected to increase in importance over the
next five years the following (Workforce Readiness Report, 2006, p. 49) are related to: Critical
Thinking, Problem Solving, Information Technology adoption, Teamwork/Collaboration, Creativity,
Innovation management, Diversity and Leadership.
Employer respondents identified as the “most critical” for future graduates entering the U.S. workforce
in the next five years the following:
Use entrepreneurial skills to enhance workplace productivity and career options
Understand economic issues and the role of business in the U.S. and global economy
Demonstrate understanding of global markets and cultural effects of globalisation
Use non-English languages as a tool for understanding other nations, markets and cultures.
The findings of the three surveys agree that, apart from knowledge related to their profession,
graduating engineers and IT professionals lack: a) communication, cooperation and team work skills;
b) skills related to lifelong learning and continuing education; c) basic knowledge and a general
understanding of organisational operations, d) media literacy and adoption of current and emerging
ICT technologies in their job and e) social and ethical responsibilities.
24

Antonios Andreatos
Hence, enterprises should strengthen personal development of their employees in non-formal
learning courses developed especially for their staff (Andreatos, 2011). Fonstad & Lanvin (2010) have
found that organisations invest in at the following alternatives for developing the competences of their
employees:
In-house training provided by a university;
In-house training provided by a consulting firm;
In-house training provided by employees;
External business degree programs;
External ICT degree programs;
External university courses;
A combination of the above options.
Professional development goes hand in hand with organisational development; therefore,
organisations ought to foster employees' continuing education in various ways. Lifelong learning is
needed, with courses focusing on the skills most needed by the organisation, as well as, specific
inter-disciplinary modules related to entrepreneurship, e-business, lifelong learning, face to face and
distance cooperation, communicative and ICT -related skills. The certification of informally acquired
knowledge and skills by official formal education degrees such as master's or doctoral titles or even
by informal titles awarded by the company (such as “technical expert”), constitute ways to promote
and foster creativity.
2. Learning at the workplace
Several schemes for the continuing education of employees have been proposed (Ley et al., 2008;
Fonstad & Lanvin, 2010; Andreatos, 2011) with varying cost (Workforce Readiness Report, 2006, p.
46). Courses could be offered either face-to-face or from distance by universities or specialised
continuing education organisations, either public or private, or either by the enterprises themselves
(e.g. the Human Resources Dept). However, such programmes are often difficult or even impossible
for small enterprises to run, especially in crisis periods; hence, other alternatives should also be
examined, such as in-house training provided by experienced or specialised employees (Fonstad &
Lanvin, 2010, p. 21) and self-directed learning.
Other reasons calling for learning at the workplace are: the direct application of knowledge to current,
real job activities hence maximisation of the learning transfer, the maintainance of knowledge work,
and hence, the increase of productivity (Haskell, 2001; Ley et al., 2008). Aligning learning to
organisational goals and task requirements is an important factor posing challenges for traditional
personnel development instruments and trainings. How this alignment can be addressed within
knowledge work remains an open issue (Elkjaer, 2000).
2.1 Cost reduction of lifelong learning
In-house continuing education saves direct costs such as fees as well as indirect costs related to
employees' absence. However, in-house continuing education programmes still have a cost related to
tutor compensation, cost of educational materials, learning need analysis etc.
In order to reduce cost, organisations may take advantage of their asset such as social capital and
communities of practice.
Social capital refers to knowledge artifacts and resources are stored within organisational databases.
A more formal definition given by Nahapiet and Ghoshal (1998: p. 243) is the following: social capital
is “the sum of the actual and potential resources embedded within, available through and derived from
the network of relationships possessed by an individual or social unit”.
A community of practice (abbreviated as CoP) is an important intra-organisational structure that has
been proven to effectively promote informal learning (Wenger, 1998; Andreatos, 2009a). A CoP is a
human network enabling professionals to communicate either face-to-face or virtually (vCoPs) and
25

Antonios Andreatos
exchange applied knowledge related to their profession. People engaged in CoPs share some
commonly accepted background of beliefs and a common pool of resources (social capital of the
CoP); therefore, they are continually educated in their profession informally. Hence CoPs are very
important since they not only keep their members up-to-date, but they also produce applied
knowledge and professional skills. For this reason, it is essential for their members to externalise this
knowledge and diffuse it within the organisation, as well as, to incorporate it to the organisation's
social capital databases. A modern as well as convenient way to externalise knowledge is to use web
2.0 platforms and technologies, especially New Media (Andreatos, 2011).
An important issue is to capture, formalise and externalise the knowledge generated within the
organisation for intra-organisational learning and on-the-job training. Perhaps this is the best way to
cultivate and proliferate the skills most needed by the organisation to its personnel. One way to do
this is to have technical experts and other employees mastering specific techniques or processes, to
externalise their experience in Learning Objects and New Media products. These will constitute or will
contribute to the organisation's social capital (Andreatos, 2009a; Nahapiet and Ghoshal, 1998; Lesser
& Prusak, 1999; Andreatos, 2011). In fact, technical experts are talented workers incorporating a
great part of the organisational social capital. The encoded and written social capital in digital
products can be used as the primary source of educational materials on which intra-organisational
learning may be based. Social capital can (and should) be used to train newcomers (Andreatos,
2009b).
The cost of educational materials such as collectively authored books by experts, dealing with modern
hot topics (like those related to ICT) or books especially written for distance education, which is an
attractive formal continuing education option for working adults (Holmberg et al., 1982; Race, 1989;
West, 1996; Mena) is not negligible. An alternative is to use open educational resources, but will they
fit the needs? Several questions arise, concerning their quality, interoperability, coverage of learning
needs, etc.
Another scheme for intra-organisational learning on issues on which the organisation does not
produce knowledge but they are considered important is the formal or informal continuing education.
Informal continuing education may heavily rely on LO's available through repositories or New Media
channels (such as YouTube, slideshow, podcast sources, Internet educational TV channels, etc.)
3. In-flow of new knowledge
Just as a CoP can externalise its own knowledge, it can as well internalise new knowledge produced
by other sources such as universities or individuals or even other CoPs, in the form of New Media.
Free digital products which have an educational nature (also called Learning objects) are known as
OER, standing for Open Educational Resources. Traditionally, OER are reusable materials; i.e., they
may be used in various educational contexts, for various purposes (Andreatos& Katsoulis, 2011).
3.1 Selecting OER to fulfill specific needs
Although this is an open issue, we shall attempt to specify a set of criteria for selecting OER to fulfill
specific needs.
Quality;
Relevance to programme targets;
Matching with existing social capital, as well as, organisational culture;
Matching with other materials already approved;
Conformance to the programme orientation (e.g. theoretical vs. practical, etc.).
Organisations offering continuing education courses to enterprises (such as universities, firms or
consultants) may preserve a pool of LOs covering a wide range of needs and learning styles, and
then select specific LOs to build programmes tailored to specific needs.
The persons responsible for selecting OER for a specific programme have several options:
One is to adopt an existing open course, for instance, from a recognised university. Several
universities have opened their lectures as OER, for instance Harvard, MIT (“MIT OpenCourseWare”),
26

Antonios Andreatos
Stanford (“Stanford Engineering Everywhere”) and Yale (“Open Yale Courses”) - see
http://www.jimmyr.com/blog/ 1_Top_10_Universities_With_Free_Courses_Online.php
There are several advantages associated with this option: guaranteed quality, leading to known
expected results; saving of time, possibility of certification by the specific university or its affiliates,
prestige, etc. If a specific course (e.g. communication, cooperation and team work skills or Innovation
management) makes part of a specific master's programme (e.g., MBA), then there is an additional
motivation for employees to attend the course: enrolling in the programme and taking additional
courses, even from distance, will allow them to earn a formal degree, possibly leading to a better
position. Professional development goes hand in hand with organisational development; therefore,
organisations ought to offer continuing education motivation to their people for mutual advantage
(Fonstad & Lanvin, 2010). Table 1 below summarises the most important options with their pros and
cons.
Table 1: The most important options in designing a continuing education in-house programme
No Option Pros Cons
1 Adopt an existing
course, for instance,
from a recognised
university
2 Make your own
course
3 Adopt an existing
course and enhance it
with selected LOs
Assured quality and specific level (grad., postgrad,
etc.); Ready; time saving option;
Guaranteed acceptance;
No certification problems; May easily lead to
certification by external universities or firms
Fully adapted to organisation's needs; also may
adapted to various learning styles; economic
under conditions; certification is questionable
May by adapted to learning styles, personalised
needs & organisation social capital; possible
certification by external universities or firms
Cannot be adapted to specific
personalised needs & learning
styles; may not fully comply
with organisation social capital
or needs; cost in case of
external certification
Time consuming; unscertain
recognition
Moderate time consuming;
Moderate difficulty in
assessment;
The most important reasons for adopting an existing course from a recognised university are:
ascertained quality, ready-to-use, easier way to certification (such as a Mastrer's degree, perhapswith
additional study).
In case the organisation selects to make its own course, the person responsible to select the LOs has
two options: either carefully examine (i.e., read, watch or listen to) the OER exhaustively; or to select
LOs by examining their metadata.
3.2 LO metadata
Learning Objects are described by metadata, i.e., information required to fully or adequately describe
their content. Typical metadata information may be author's name, institution, file size, description,
location, time of creation, language, culture etc. This information is important for the recall of learning
objects, their appropriateness regarding specific uses tasks and their quality (Andreatos & Katsoulis,
2011).
From another perspective, metadata can be either a priori or a posteriori. A priori metadata are
created in advance by the authors of learning objects or professional indexers. A posteriori
metadata, in contrast, are created after usage by the users themselves or by automatic means
(Andreatos & Katsoulis, 2011).
However, professional LOs are described by high quality metadata information related to their content
or even the specific learning needs they intend to cover. Such a scheme greatly facilitates search and
data mining processes. As an example, consider the articles embedded in the “Help” of office suites.
When a user needs help, he/she types a specific keyword in the help textbox and immediately the
system finds, recalls and presents a set of articles, as well as, related topics (Figure 2).
Ley et al. (2008) describe a smart technology-enhanced work-integrated platform facilitating in-house,
just-ιn-time learning, developed in the APOSDLE project. They model the typical IT-based workplace
as a set of three conceptual inter-related spaces: the work space, the learning space and the
27

Antonios Andreatos
knowledge space, called the Workplace Learning Context. LOs stored in the knowledge space,
encoding the organisation's knowledge, are assigned a number of concepts from a domain-specific
ontology in the form of metadata. The system keeps a historical learning profile for each employee
based on tasks performed in the past; then it uses this profile to find and propose LOs related to
competencies needed in order to successfully carry out future assigned tasks (Ley et al., 2008: p.
2103).
Figure 2: Search results from the “Help” of OpenOffice suite are based on metadata
From the above examples it comes out that a proper metadata assignment and process scheme can
greatly facilitate the search of proper LOs, as well as, the construction of proper courses tailored to
specific or personalised needs.
But when it comes to OER from various sources, by various authors, made to cover different needs,
with minimum and incompatible metadata, there is a problem. Τhere is a great difficulty to classify,
search and recover LOs according to specific criteria; moreover, there is no uniformity, no quality
guarantee, nothing. The following three figures show metadata concerning LOs about Stakeholder
Analysis from three different repositories.
As Andreatos & Katsoulis (2011) have noticed, there is also a lot of good quality educational LOs in
New Media formats (flash content, presentations, webcasts, podcasts etc.), stored in general-purpose
New Media sites such as YouTube. All resources stored in YouTube are accompanied by metadata
information, as well as, statistics such as number of views, “likes” and “dislikes” and users' comments.
Another important feature of YouTube is the projection of related videos. It is noticeable that famous
universities and university professors maintain their own New Media channels (Figure 6).
28

Figure 3: LO metadata from EDNA repository
Figure 4: LO metadata from Ideas repository
Antonios Andreatos
29

Figure 5: Video metadata from YouTube
Antonios Andreatos
Figure 6: Yale maintains educational channels in YouTube and iTunes
From the above figures it comes out that there is high incompatibility of metadata, making their
automatic process difficult. This means that in most cases the selection of LOs from various sources
will have to be done by a human: somebody will have to exhaustively examine the LOs and manually
assign the missing metadata according to a formal scheme, to make them machine-usable.
Of course, other options exist, such as examining part of the LOs exhaustively, while choose some
others (for instance, those coming from trusted sources) only by checking their metadata. Another
solution is to rate sources or authors by quality and then easily accept LOs from these sources or
authors.
One form of metadata added by users is social tags or folksonomies. Various schemes for automatic
metadata generation using combinations of author indexing, expert indexing, peer review, automatic
metadata generation and/or collaborative social tagging have been proposed (Andreatos & Katsoulis,
2011).
As far as quality is concerned, many repositories use a peer process to assess their LOs. New Media
channels on the other hand use statistics, “likes” and “dislikes” and users' comments, which provide
some quality indication.
Recently some experimental tools facilitating the search of OER have been proposed (Andreatos &
Katsoulis, 2011). However, the author believes that the research in this area is still in its infancy.
30

4. Propositions
Antonios Andreatos
It has been ascertained that:
CoPs promote informal learning and work as a mechanism of professional continuing education.
Organisations are interested in promoting employees' personal development (Andreatos, 2011).
There is a high diversity in employees' personal educational needs, contrary to that of students.
CoPs produce social capital which may be encoded in New Media products.
A huge plethora of OER covering all areas of knowledge is available in the Internet.
There are a lot of university open courses available.
There are a lot of New Media LOs, appropriate for education.
Continuing education is needed in order to make organisations antagonistic in today's globalised
environment (Fonstad and Lanvin, 2010; Andreatos 2011).
It is of highest importance for an organisation to know its knowledge status. Questions like the
following have to be answered:
Is the organisation producing any knowledge? Of what kind ?
What percentage of the produced knowledge is currently stored within the organisational databases?
In what format?
Does the organisation need to assimilate new knowledge? and of what kind ?
What is the existing knowledge gap at various levels (organisational, departmental, CoP, individual)?
Which are the missing skills?
Is there a learning needs analysis? Learning needs analyses can also identify inter- and intradepartmental
needs, common missed skills etc. Can uniform groups be formed?
What are the various continuing education programme options and what is their cost?
Who will design them and how will they assure their quality?
Will there be a certification?
How shall we assess the quality of the certification?
Is this certification widely accepted?
Can our staff be certified by an external, universally accepted authority such as a University? At what
cost?
Thus, organisations have to adopt sustained learning policies exploiting their staff (e.g., technical
experts, CoPs), their social capital, as well as, OER freely available in the Internet.
5. Conclusion
In this paper the following points were discussed:
Sustained learning is a crucial factor of an organisation’s ability to survive and effectively compete
in the globalisation era worldwide.
Organisations are learning entities producing their own knowledge encoded in their social capital,
part of which is stored in their databases.
This social capital is a very important asset because it summarises the organisation's experience.
31

Antonios Andreatos
The organisational social capital can be encoded in LOs stored in its databases.
It has to be continually enhanced and enriched, reflecting the company's effort to assimilate new
ICT and new knowledge.
The organisational social capital plus selected external LOs may form the basis of the educational
materials used to train newcomers and continually educate knowledge workers.
After proper learning needs analysis, an organisation investing in knowledge and ICT needs to design
and adopt a sustained continuing education plan. Several options appear (Fonstad & Lanvin, 2010),
ranging from formal post-graduate university education to informal in-house training designed and
offered by the organisation itself. In most cases, cost reduction policies make the use of proper OER
an attractive choice.
In order for an organisation to design its own in-house continuing education programme, a set of
scientific, pedagogical and quality and cost criteria have to be met. Fundamental prerequisites
include: the quality credentials of the course designers; the quality of the educational materials;
coverage of specific learning needs; correctness and efficiency. Additional issues regard the
effectiveness, efficiency and certification of such programmes.
After collecting a set of appropriate LOs meeting predefined criteria, selected LOs may be stored in
the organisation's databases to enhance the existing social capital. The use of a formal
characterisation of LOs in the form of a posteriori metadata will greatly facilitate their organisation,
search and retrieval process.
In conclusion, the use of open educational resources may enrich existing social capital and facilitate
intra-organisational eLearning and continuing education in a cost-effective way.
References
Andreatos, A. (2009a) “On the Definition and Impact of Virtual Communities of Practice”, International Journal of
Virtual Communities and Social Networking, 1(4), pp 73-88.
Andreatos, A. (2009b) “Informal learning in FOSS communities; the Greek case”. In A. Lionarakis (Ed.),
Proceedings of the 5th International Conference on Open and Distance Learning. 27 -29 Nov. 2009,
Athens, Greece.
Andreatos, A. & Katsoulis, S. (2011) “Enriching a course syllabus with open educational resources”. In
Proceedings of ICICTE 2011, International Conference on Information Communication Technologies in
Education. Rhodes, Greece, July 7 - 9, 2011. http://www.icicte.org/.
Andreatos, A. (2011) “Educating the 21st century's Engineers and IT Professionals” Chapter in Book “New Media
Communication Skills for Engineers and IT Professionals: Trans-National and Trans-Cultural Demands”, IGI
Global (forthcoming).
Azami, Z., Yuzainee, Y., Mohd Zaidi, O., Azah, M. and Norhamidi, M. (2009) “Employers’ Perceptions and
Expectation toward Engineering Graduates: A Study Case”. Proceedings of the 6th WSEAS International
Conference on Engineering Education, pp 23-29.
Burke, S. C. et al., (2006) ”What types of leadership behaviors are functional in teams? A meta-analysis”,
Leadership Quarterly, Vol. 17, 2006, pp 288-307.
Elkjaer, B. (2000) "Learning and getting to know: the case of knowledge workers", Human Resource
Development International, Vol. 3 No. 3, pp 343-359.
Fonstad, N. O. & Lanvin, B. (2010) Strengthening e-Skills for Innovation in Europe - Building on partnerships
between academia, industry and government for better e-competences curricula. European Commission
INSEAD eLab, [online], http://www.insead.edu/facultyresearch/centres/ecompetences/library/documents/
INSEADECNewSkillsforInnovationsReport.pdf.
Haskell, R.E. (2001) Transfer of Learning: Cognition and Instruction, Elsevier Academic Press.
Holmberg B. et al., (1982) Zur Effizienz des gelenkten didaktischen Gespräches, Fernuniversität (ZIFF), Hagen.
Lesser E. and Prusak, L. (1999) Communities of practice, social capital and organizational knowledge. White
paper, [online], http://www.providersedge.com/docs/km_articles/Cop_-_Social_Capital_-_Org_K.pdf.
Mena M., “New Pedagogical Approaches to Improve Production of Materials in Distance Education” [online],
http://cade.icaap.org/vol8.3/10b_mena-english.html.
Nahapiet, J. and Ghoshal, S. (1998) “Social Capital, Intellectual Capital and the organizational Advantage”,
Academy of Management Review, 23(2), 242-266.
Nonaka, I. (2008) The knowledge-creating company. Boston: Harvard Business Press.
Race, Ph. (1989) The Open Learning Handbook – Selecting, designing and supporting Open Learning materials.
London: Kogan Page.
Wenger, E. (1998) Communities of Practice: Learning, Meaning and Identity. Cambridge University Press.
32

Antonios Andreatos
West R., (1996) “Concepts of text in distance education”, In Motteram G., Walsh G., West R. (eds), Proceedings
of 2nd symposium on Distance Education for Language Teachers, University of Manchester, Manchester
1996, pp 62-72.
Workforce Readiness Report (2006) “Are they really ready to work? Employers’ perspectives on the basic
knowledge and applied skills of new entrants to the 21st century U.S. workforce”. Publisher: The
Conference Board, Corporate Voices for Working Families, Partnership for 21st Century Skills and the
Society for Human Resource Management. [online], http://www.p21.org/documents/
FINAL_REPORT_PDF09-29-06.pdf.
33

Constructing a Survey Instrument for Assessing
Characteristics of Effective Online Teachers
Jonathan Barkand
Duquesne University, Pittsburgh, USA
Jbarkand@gmail.com
Abstract: This paper will report on a proposed research project, which seeks to create and validate an
instrument that can be used to determine the effectiveness of online teachers. This study is the first step in a
comprehensive research project that is intended to establish the reliability and validity of a survey for online
teachers. A significant body of research has been conducted in identifying and quantifying characteristics of
effective online teachers. However, to date, a standardized survey instrument has not been formally accepted or
approved to make a qualitative assessment of which characteristics are considered to be most essential for
success in an online environment. As education continues to make the shift to a completely online environment,
there will be a need to design an assessment system to determine effective and ineffective online teachers.
Paralleling this transition is the need for the training and education of teachers to be more effective in an online
environment. Certificates in Online Teaching and Learning (OTL) are being offered at universities at the graduate
level. These programs are designed to prepare teachers for teaching and learning online. The quality of OTL
programs would be greatly increased with a survey instrument to be used with current and graduated students.
The survey results will be categorized based on the frameworks used by the National Council for Accreditation of
Teacher Education (NCATE). Data will be sorted into the three NCATE categories of Knowledge, Skills, and
Professional Dispositions. This categorization will create an easy to use characteristic breakdown for effective
online teachers. The use of these results is not limited to self-improvement, but could be used by supervisors and
school administration. Based on the literature and results, an assessment matrix will be developed that will allow
teachers from any discipline to see a measurement of their effectiveness as an online teacher.
Keywords: online teaching, eLearning teachers, teacher characteristics, effective teaching survey instrument,
quality online teaching
1. Introduction
Teachers are a key component of students learning in the education environment. There are many
state and national standards for effective teachers in a traditional classroom, though there are very
few, if any, standards that relate directly to effective online teachers. An online teacher will be
accessible to hundreds of students, and the students should be learning from a highly qualified and
effective online teacher. The transition for students from a traditional education setting to a fully online
setting has been increasing each year. An instrument that can reliably measure teacher self
perceptions and also student perceptions of effective online teachers will be necessary if we are to
continue to improve the quality of online instruction.
The expanding enrollments in online courses are occurring in both higher education and K-12
institutions. In fall 2009, the enrollment of higher education students who take at least one online class
increased 21% over fall 2008 (Allen & Seaman, 2010). Since 2002, there has been a compounded
annual growth rate of 19%, bringing the total enrollment to 5.6 million students in 2009 (Allen &
Seaman, 2010). A growth rate of this level will eventually peak, but until then, more students are
taking online courses each year. The enrollments in basic education K-12 institutions are more difficult
to predict, although certain estimates can be extrapolated from previous data. For example, there
were an estimated 1,030,000 students engaged in online courses in 2008 according to a survey of
school district administrators (Picciano & Seaman, 2009). The number of K-12 students is lower than
higher education students, which is likely due to the fact that K-12 education is years behind higher
educations’ shift to an online environment. An interesting fact is that the 2008 survey figure
represented a 47% growth over two years since 2006. Both K-12 and higher education trends appear
to be following similar growth patterns. Given these growth patterns, online education should already
be a part of both K-12 and higher education institutions education plan ( Kim & Bonk, 2006).
A survey instrument which can be used to determine the effectiveness of online teachers in this ever
increasing online learning environment will be necessary for improving the quality of online teaching
and learning. Based on literature and published standards, a survey instrument will be developed
which can be used for K-12 or higher education online environments. The National Council for
Accreditation of Teacher Education (NCATE) framework was used to categorize each of the
standards into the three categories of knowledge, skills, and professional dispositions. Each question
34

Jonathan Barkand
on the survey was aligned with a specific standard. A conceptual framework was necessary to tie
different standards, suggestions, and criteria together into a standardized survey instrument. The
NCATE framework was chosen because of its wide-spread use for developing high quality teacher
education programs on a national level within the United States.
Requiring candidates to meet state standards for teacher certification is the most common way to
obtain licensure. Current teacher licensure testing is based on preparing candidates for a traditional
education setting. Most states require teacher candidates to meet a combination of passing scores on
the Praxis examinations and having successfully completed the requirements in an approved program
of study. Most online schools hire candidates who have met the traditional state guidelines for teacher
certification. The certification process is still trying to catch up to the specific requirements for
successful online education in the K-12 environment. In the absence of specific state standards,
several colleges and universities have started offering certificates in Online Teaching and Learning
(OTL) in an effort to fill the gap. Professional organizations such as the North American Council for
Online Learning (NACOL) in 2006 have promulgated prototype standards in order to provide
consistent standards which can be used on a national level.
Teacher preparation will be a critical component in preparing teachers for an online environment.
Without modeling effective online teaching strategies and methods, most of the 86,000 new teachers
who enter the online profession each year, begin without having specific online teaching skills in their
professional repertoire (NEA, n.d.). Understanding the characteristics that make teachers effective in
an online environment will be an important component in shaping teacher preparation programs.
2. Standards
After a comprehensive review of the literature, NACOL standards and indicators were used as the
basis for the proposed survey instrument (NACOL 2006). The NACOL standards are based on the
Southern Regional Education Board (SREB) Standards for Quality Online Teaching. The standards
were developed by knowledgeable and experienced resource persons from K-12 and postsecondary
education from national and regional organizations, SREB state departments of education, colleges,
and universities (SREB, 2006).
NACOL adopted the work of SREB. NACOL added two additional standards from the Ohio
Department of Education’s Ohio Standards for the Teaching Profession and the Electronic Classroom
of Tomorrow’s Teacher Evaluation Rubric based on the results of the review (NACOL, 2008). The
indicators for each standard were also modified slightly.
SREB Standards:
The teacher meets the professional teaching standards established by a state licensing agency or
the teacher has academic credentials in the field in which he or she is teaching.
The teacher has the prerequisite technology skills to teach online.
The teacher plans, designs and incorporates strategies to encourage active learning, interaction,
participation and collaboration in the online environment.
The teacher provides online leadership in a manner that promotes student success through
regular feedback, prompt response and clear expectations.
The teacher models, guides and encourages legal, ethical, safe, and healthy behavior related to
technology use.
The teacher has experienced online learning from the perspective of a student.
The teacher understands and is responsive to students with special needs in the online
classroom.
The teacher demonstrates competencies in creating and implementing assessments in online
learning environments in ways that assure validity and reliability of instruments and procedures.
The teacher develops and delivers assessments, projects, and assignments that meet standardsbased
learning goals and assesses learning progress by measuring student achievement of
learning goals.
The teacher demonstrates competencies in using data and findings from assessments and other
data sources to modify instructional methods and content and to guide student learning.
35

Jonathan Barkand
The teacher demonstrates frequent and effective strategies that enable both teacher and students
to complete self-assessments and pre-assessments.
NACOL Additional Standards:
The teacher collaborates with colleagues.
The teacher arranges media and content to help students and teachers transfer knowledge most
effectively in the online environment. (Optional)
3. NCATE categorization
The NCATE Professional Standards for the Accreditation of Teacher Preparation Institutions was
used to categorize the online teacher standards collected. NCATE is officially recognized by the U.S.
Department of Education as an accrediting body for institutions that prepare teachers and other
professional persons for work in preschool, elementary, and secondary schools (NCATE, 2008).
NCATE Standard 1 related to Candidate Knowledge, Skills, and Professional Dispositions was
specifically used for categorization. The optional NACOL additional standard was related more to
instruction design rather than teacher characteristics and was not used. The SREB/NACOL standards
have had a naming structure applied for alignment purposes for the survey.
1. Content and Pedagogical Knowledge for Teacher Candidates:
1.1. The teacher meets the professional teaching standards established by a state licensing agency or
the teacher has academic credentials in the field in which he or she is teaching.
1.2. The teacher has the prerequisite technology skills to teach online.
2. Pedagogical and Professional Skills for Teacher Candidates:
2.1. The teacher plans, designs and incorporates strategies to encourage active learning, interaction,
participation, and collaboration in the online environment.
2.2. The teacher provides online leadership in a manner that promotes student success through
regular feedback, prompt response, and clear expectations.
2.3. The teacher has experienced online learning from the perspective of a student.
2.4. The teacher understands and is responsive to students with special needs in the online
classroom.
2.5. The teacher demonstrates competencies in creating and implementing assessments in online
learning environments in ways that assure validity and reliability of instruments and procedures.
2.6. The teacher develops and delivers assessments, projects, and assignments that meet standardsbased
learning goals and assesses learning progress by measuring student achievement of learning
goals.
2.7. The teacher demonstrates competencies in using data and findings from assessments and other
data sources to modify instructional methods and content and to guide student learning.
2.8. The teacher demonstrates frequent and effective strategies that enable both teacher and
students to complete self-assessments and pre-assessments.
3. Professional Dispositions for Teacher Candidates:
3.1. The teacher models, guides and encourages legal, ethical, safe, and healthy behavior related to
technology use.
3.2. The teacher collaborates with colleagues.
36

4. Indicators and survey questions
Jonathan Barkand
The following Indicators were part of NACOL Standards for Quality Online teaching. Not all indicators
were used to create the assessments. An indicator for each standard was chosen based on the ability
to accurately measure the self-assessment. Some questions cover more than one indicator. Some
standards contained multiple methods of assessment and required more than one question to
accurately cover the scope of the standard. The actual question was created by the researcher and
the question type is added to show the form of measurement.
Indicator 1.1: Meets the state’s professional teaching standards or academic credentials in the field.
Question 1.1: Are you a state certified teacher in your content area?
Type: Dichotomous (Yes or No)
Indicator 1.2: Demonstrates the ability to effectively use word-processing, spreadsheet, and
presentation software.
Question 1.2: How would you rate your technology skills in Microsoft Office and Internet systems
(browsing, emailing)?
Type: Five-Point Likert Scale
Indicator 1.2: Utilizes synchronous and asynchronous tools (e.g., discussion boards, chat tools,
electronic whiteboards) effectively.
Question 1.2: How would you rate your technology skills with teaching synchronous and
asynchronous tools (discussion boards, chat/tutor tools, electronic whiteboard)?
Type: Five-Point Likert Scale
Indicator 2.1: Facilitates and monitors appropriate interaction among students.
Indicator 2.1: Promotes learning through group interaction.
Question 2.1: How many hours per week are spent on discussion boards and facilitating group
interaction?
Type: Nominal with ranges (Less than an hour, 1-2 hours, 2-4 hours, 4-8 hours, 8+ hours)
Indicator 2.2: Models effective communication skills and maintains records of applicable
communications with students.
Question 2.2: What is your average response time on student communications
(emails/discussions/phone)?
Type: Nominal (less than an hour, 1-3 hours, 3-6 hours, within 24 hours, 24+ hours)
Indicator 2.2: Encourages interaction and cooperation among students, encourages active learning,
provides prompt feedback, communicates high expectations, and respects diverse talents and
learning styles.
Indicator 2.2: Provides timely, constructive feedback to students about assignments and questions.
Question 2.2: How would you rate the quality of your timely, personalized, and constructive
feedback for assignments/questions?
Type: Five-Point Likert Scale
37

Jonathan Barkand
Indicator 2.3: Has taken an online course and applies experiences as an online student to develop
and implement successful strategies for online teaching.
Question 2.3: How many online courses have you taken?
Type: Nominal (0, 1-2, 3-4, 5-6, 6+)
Indicator 2.3: Demonstrates an understanding of the perspective of the online student through
appropriate responsiveness and a supportive attitude toward students.
Question 2.3: How much has taking an online course changed your perspective of teaching online?
Type: Five-Point Likert Scale (must include a Not Applicable choice)
Indicator 2.4: Adapts and adjusts instruction to create multiple paths to learning objectives.
Question 2.4: How would you rate your ability to adapt and adjust lessons for students with special
needs?
Type: Five-Point Likert Scale
Indicator 2.5: Creates or selects fair, adequate and appropriate assessment instruments to
measure online learning that reflect sufficient content validity (i.e., that adequately cover the content
they are designed to measure), reliability and consistency over time.
Question 2.5: How would you rate your assessments ability to measure the content covered?
Type: Five-Point Likert Scale
Indicator 2.6: Includes authentic assessment (i.e., the opportunity to demonstrate understanding of
acquired knowledge and skills as opposed to testing isolated skills or retained facts) as part of the
evaluation process; assesses student knowledge in a forum beyond multiple guess.
Question 2.6: What estimated percentage of assessments in your course are not computer graded?
Type: Nominal (1-20%, 21-40%, 41-60%, 61-80%, 81-100%)
Indicator 2.7: Reviews student responses to test items to identify issues related to test validity or
instructional effectiveness.
Indicator 2.7: Creates opportunities for self-reflection or assessment of teaching effectiveness
within the online environment (e.g., classroom assessment techniques, teacher evaluations, teacher
peer reviews).
Question 2.7: How many hours per week do you devote to self-reflection and improvement in your
teaching style and content?
Type: Nominal (Less than an hour, 1 – 2 hours, 2 - 4 hours, 4-8 hours, 8+ hours)
Indicator 2.8: Understands that student success (e.g., grade, level of participation, mastery of
content, completion percentage) is an important measure of teaching and course success.
Question 2.8: How much do you feel student success (e.g., grade, level of participation, mastery of
content, completion percentage) measures effective teaching and course success?
Type: Five-Point Likert Scale
Indicator 3.1: Identifies the risks of academic dishonesty for students.
38

Jonathan Barkand
Indicator 3.1: Establishes standards for student behavior that are designed to ensure academic
integrity and appropriate uses of the internet and written communication.
Question 3.1: How important is academic integrity, academic honesty, and acceptable use policies
to you?
Type: Five-Point Likert Scale
Indicator 3.2: Networks with others involved in online education.
Indicator 3.2: Leads collaborative efforts to create common assessments among grade-level and/or
content-area teachers and share assessment results with colleagues to collaboratively plan instruction
that will best meet individual student needs.
Question 3.2: How often do you collaborate with others in the online education field to create,
improve, and/or share instruction strategies?
Type: Nominal (2 or more times daily, daily, weekly, monthly, less than monthly)
5. Research methodology and conclusions
The survey instrument will contain 15 questions following the structure and sequence listed above.
The survey will be provided in electronic form through email to 80 teachers in an online asynchronous
environment. Participants will remain anonymous and personal information will not be collected. The
data will be collected through the survey system and will be entered into SPSS for statistical analysis.
Reliability will be determined using the single-administration method. The internal consistency will be
measured using Cronbach’s alpha. Descriptive statistics including the mean, median, mode, range,
variance, and standard deviation will be calculated. Cross-tabulation will be used to show the
relationship among answers and specifically used for the two questions for Standard 2.3. Bar charts
will be used as the visual representation for many of the distributions.
In summary, the survey instrument design is grounded in specific national online teaching standards
and questions are based on indicators for each standard. Categorizing into NCATE standards will
allow the researcher to show qualifications based on Knowledge, Skills, and Professional
Dispositions. It is important to note that there are not many standards that are associated with
knowledge and professional dispositions. Knowledge is usually proven through other methods of
teacher training, certification, and testing. Professional dispositions usually require a specific survey
for teachers. A standardized teacher disposition instrument would be necessary in order to properly
assess this section.
Teacher assessment and student assessment have similar qualities, but only student assessment is
required by law. If students are performing poorly, a similar measure of the effectiveness of the
teacher should be conducted. But the lack of a consensus on what constitutes good teaching has
made it difficult to create or enforce teacher assessment laws (Rebell, 1991). Student assessment
has become a very important aspect in K-12 education in the United States since the passing of the
No Child Left Behind (NCLB) law in 2002 (U.S DoE, 2009). The assessment was done through tests
developed by each state and determined if the schools were meeting Adequate Yearly Progress
(AYP). The NCLB law also provides funding to help teachers learn to be better teachers, but doesn’t
include any assessment of current teacher quality (U.S. DoE, 2009). The concept of holding students
accountable for low test scores without holding teachers accountable doesn’t appear to be a
sustainable solution. An instrument that can assess what makes an effective teacher, along with the
criteria for effective teaching will be extremely relevant in future education discussions.
Overall, the reliability of the survey instrument for effective online teachers will be examined as an
initial step in the development of the instrument. The self-perception aspect for teachers will allow
organizations to have an anonymous collection of information that assess the online teacher group as
a whole. While it is important in most cases to keep this information anonymous, it is possible to use
this survey as a self-evaluation and collect personal data. With such a rapidly expanding online
education environment the collection of data that can influence standards, practices, certification, and
university programs will be a critical component for the quality of online teachers in the future.
39

References
Jonathan Barkand
Allen, E. I., & Seaman, J. (2010), Class differences: online education in the United States, Sloan Consortium,
Needham, MA, [Online], http://www.sloanconsortium.org/publications/survey/ pdf/class_differences.pdf [10
February 2011]
Kim, K-J, & Bonk, C. J. (2006), The future of online teaching and learning in higher education: the survey says...,
Educause Quarterly, [Online], http://faculty.weber.edu/eamsel/ Research%20Groups/Online%20Learning/Bonk%20%282006%29.pdf[12
February 2011]
National Council for Accreditation of Teacher Education (2008), Standards for the Accreditation of Teacher
Preparation Institutions, [Online], http://www.ncate.org/documents/standards/
NCATE%20Standards%202008.pdf [8 January 2011]
National Education Association (n.d), Guide to Teaching Online Courses, [Online],
www.nea.org/assets/docs/onlineteachguide.pdf [8 February 2011]
North American Council for Online Learning (2008), National Standards for Quality Online Teaching, [Online],
www.inacol.org/resources/nationalstandards/NACOL%20Standards%20Quality%20Online%20Teaching.pdf
[8 February 2011]
Picciano, A. G., & Seaman, J. (2009), K-12 online learning: A 2008 follow-up of the survey of U.S. school district
administrators, Sloan Consortium, Needham, MA, [Online], http://sloanconsortium.org/sites/default/files/k-
12_online_learning_2008.pdf [10 February 2011]
Rebell, M.A.(1991), Teacher Performance Assessment: The Changing State of the Law, [Online],
http://www.pearsonassessments.com/hai/images/NES_Publications/1993_04Rebell_584_1.pdf [15
February, 2011]
Smith, R., Clark, T., and Blomeyer, R. L. (2005), A synthesis of new research on K–12 online learning, Naperville,
IL: Learning Point Associates, 2005.
Southern Regional Education Board (2006), Standards for Quality Online courses, Educational Technology
Cooperative, [Online], http://publications.sreb.org/2006/06T05_Standards_quality_online_courses.pdf [8
February 2011]
U.S. Department of Education (2009), Facts and Terms Every Parents Should Know About NCLB, [Online],
http://www2.ed.gov/nclb/overview/intro/parents/parentfacts.pdf [15 February, 2011]
40

When Agents Make Suggestions About Readings
Orlando Belo
Algoritmi R&D Centre, University of Minho, Portugal
obelo@di.uminho.pt
Abstract: Significant efforts have been made during the last few years in the design and implementation of
pedagogical agents for a wide range of application domains. One of the most common target area is the
assistance to students in cases of regular subject studying, promoting means that help them to improve their
performance and expertise in some specific subject areas. Frequently students ask their teachers about the
“best” and more effective bibliographic resources that they could use to study and validate knowledge for some
working topics. In this paper we will discuss the basic characteristics of pedagogical agents, approaching their
typical functional architecture, and services, reinforcing the discussion on a specific class of pedagogical agents
that are responsible to support students during their studying sessions, helping them in the validation of their
knowledge, suggesting bibliographic resources information whenever requested.
Keywords: eLearning platforms, agent based computing, intelligent tutoring systems, software agents, artificial
intelligent tutors, bibliographic resources suggestion
1. Introduction
Agent based applications are very appellative. Software agents have been used to support a lot of
tasks in real world applications (Van der Hoek & Wooldridge 2008). Ranging from telecommunications
to retail, or doing monitoring services on hydroelectric power plants, agent based computing has been
always a very good asset in a lot of problem solving arenas. eLearning is no exception to this
attractive paradigm and to all of its characteristics and potentialities (Agarwal et al. 2004) (Leung & Li
2001). One of the most relevant agent applications on this field was in the design and development of
pedagogical agents, normally designed by artificial tutors or intelligent assistants. Basically, these
entities are conceived to ensure more effective tutoring services, in some very specialized areas of
studying, having clearly pedagogical purposes, giving assistance to students in cases of regular
subject studying or even doing evaluation tasks. Additionally, they are also able to perform
administrative and optimization services inside eLearning platforms, supervising what users are doing
and suggesting better ways to do it or recommending particular resources that can help them in
current tasks.
The use of software agents as sophisticated autonomous means helping students on bibliographic
resource selection seems to be very useful and appellative. The ability that agents have to adapt to
new scenarios and to communicate with other means of learning makes possible a very dynamic
eLearning environment, where students needs can be satisfied easily. An agent can also adapt in real
time different plans of readings for current user needs, finding the best solution for a given studying
plan, and personalizing processes and exploitation scenarios (Schiaffino et al. 2008). To do that, they
simply need to act as search engines over their databases looking for a list of references that satisfies
student current, his preferences, and a previous plan of readings prepared by his teachers. This
means that any assistant agent (Okamoto et al. 2009) must establish usage profiles and accordingly
prepare its plan of action an user satisfaction. It is very desirable that the processes of readings
suggestion is versatile and proactive, providing the references that students require and, at the same
time, giving viable alternatives that follow other suggestion indicators (e.g. readings usage ranking,
recommended references, teacher’s preferences, or external sources identification).
In this paper we will focus our attention on a specific class of pedagogical agents that are responsible
to support students during their studying sessions, helping them in the validation of their knowledge in
a particular domain, and (as their priority goal) suggesting bibliographic resources information
whenever requested or inferred as necessary during an evaluation process, accordingly current
studying status of the students. We will discuss the basic characteristics of these pedagogical agents
(section 2), reinforcing the discussion presenting an application domain for bibliographic resource
suggestion in a conventional eLearning scenario (section 3), and present the basic characteristics and
functional architecture of a specific software agent for personalised assistance in bibliographic
resource suggestion (section 4). Finally, in section 5, we will present some conclusions and future
work.
41

2. Pedagogical agents
Orlando Belo
Agent-based computing is not a recent area of research and technological development (Wooldridge
1998) (Jennings 1999). A lot of work and applications were made for a large diversity of domains
(Jennings & Wooldridge 1998). The agent paradigm is by nature quite attractive inspiring researchers
from many distinct application arenas. During the last few years, agents have been introduced in a
large range of applications, ranging from financial markets to medical services, passing by
telecommunications, retail, or transports. Today, we can find agents doing a lot of tasks, 24 hours a
day, 365 day a year, without interruptions.
In a few set of links, a simple Web search reveals us that eLearning is no exception in the adoption of
agent-based technology (Nedev & Nedeva 2008) (Gregg 2007) (Kazar & Bahi 2009). Intelligent or
not, agents have been developing important roles in the eLearning arena, improving effectiveness of
educational games (Conati & Zhao 2004), done procedural training in virtual environments (Jonhson
& Rickel 1997), supporting distance education (Silveira & Vicari 2002), animating educational
sessions (Nunes et al. 2002) (Lester et al. 1997), doing virtual tutorship (Remondino 2007), retrieving
selective information from the web or other information sources (Pankratius et al 2004), or managing
processes in eLearning environments (Lai et al. 2008). This is only a brief panorama of what we can
find about agents in eLearning environments.
Pedagogical agents could be seen today as good alternatives to some parts of the educational
process, being very useful in the formative process of students. Some of the most experienced
eLearning solutions integrate now in some way “expert” software components, especially in areas
where attendance services must be ensured continuously, 24 hours per day. Pedagogical agents are
designed and implemented with the goal to support students in their daily work, providing information
about their classes, supporting studying processes, accessing, or retrieving pertinent information
about some topics. Basically, they intend to improve students learning outcomes. Thus, we can see
that this particular kind of agents have the ability to attenuate several usage restrictions and user
services limitations that we can find normally in eLearning platforms (Brusilovsky 2000) (Zhang et al.
2004).
One of the most sophisticated aspects that we are particularly interested to explore is customization
(namely frequently by personalization) of eLearning processes and services (Allison et al. 2005).
Trying to go towards the needs of students, providing the learning materials they need and
complement them with other resources in anticipation it’s a clear goal for us. The use of profiling
techniques (Marques & Belo 2010) to establish a standard modus operandi of the student and predict
what students will need and use are quite important for eLearning services personalization (Paneva
and Zhelev 2007). This allows to catch individual studying styles and resources options used,
especially the ones concerning about bibliographic references support – the goal of this work –,
personalizing them and creating more student oriented learning objectives and material support. In
the following sections we will see how we used agent technology to design and develop a
bibliographic resource assistant, with the ability to follow studying sessions, receiving reference
requests about some particular studying topic, and suggesting on the fly a set of references that could
help students and contribute to the improvement of knowledge acquisition and, as referred before, of
learning outcomes.
3. A bibliographic resource assistant
3.1 Overview
Students are very critical and demanding about bibliographic resources. They never are satisfied with
teachers’ proposals and suggestions concerning with what they must or need to read about some
course’s programme topic. Today, we have strong reasons to believe that a significant part of the
students’ failures is due to their inability to select appropriated bibliographic resources and use it.
Teachers use to indicate in every beginning of a scholar year at least one bibliographic list covering
the topics they approach in classes. Some of them even try to explain for each entry presented in the
lists its coverage and importance. But it seems that such efforts are useless so many times.
In order to attenuate such problem we design a small software component especially oriented to
assist and help students during their working sessions. It wasn’t our intention to conceive a generic
assistant. However all the pointed development lines followed here could be used in such direction.
42

Orlando Belo
Data Warehousing Systems (Golfarelli & Rizzi 2009) were our main scientific and technological target,
which means that we intended to provide an auxiliary mean for studying in the area of decisionsupport
systems, business intelligence, and their related technologies. The target community selected
- university students - is also very specific. But, in our point of view, is sufficient relevant (and critical)
to apply and test such mean of studying support. So, please keep on mind the referred area and
student community during the rest of the paper.
3.2 Readings suggestion life cycle
Our main goal was to design and specified all services and functionalities of specific tool with the
ability to provide studying maps, locating for each course’s programme topic a list of recommended
readings, and promoting, when requested, cross-reference recommendation scenarios for different
working areas. This tool, a specific subject-oriented software agent, was idealized as a typical
assistant, which can be seen as a conventional pedagogical agent specialized in bibliographic
resource suggestion. The way we though this studying assistant, its functional structure and
behaviour, demands some preliminary work, usually done by the lecturing team of a university’s
course, they will be responsible for preparing the bibliographic references according the programme
that they establish for a course.
Easily we see that to provide valid and effective bibliographic suggestions, it is necessary (at least) to
maintain up-to-date a bibliographic catalogue, including a detail reference list, their correspondent
index terms for searching and cross reference, as well a categorization and an importance level
associated with each references (or a set of references) for each specific studying topic referred in the
course’s programme. Having this information we own the minimum to establish a first set of
suggestion. The rest, the most sophisticated part of this project, which consider inclusive student
preferences definition, will be achieved with special techniques of profiling and preferences
establishment.
Figure 1: Readings suggestion process life cycle
The assistant agent’s behaviour follows a very common life cycle for cases related with suggesting
and index based searches. As we can see in Figure 1, according such life cycle, the first task that we
must perform is to prepare a detail (readings list definition) list containing the bibliographic references
(Table 1) recommended by the lecturing team. It is one of the most important pieces of information
that the readings assistant agent has to support its job. This list includes some elementary data about
bibliographic references, complemented with some other attributes that will be used by the agent to
establish suggestion priorities and preferences about readings. The basic structure of the list,
excluding control and monitoring attributes, includes the following fields:
Identification (Id), which identifies uniquely the bibliographic reference in the entire system.
CiteRef, the way reference could be referred.
Reference, the complete description of the reference.
Year, the publication year of the bibliographic reference.
43

Orlando Belo
Type (Tp), the reference type (B-Book, P-Proceedings, J-Journal, etc.).
Area, the knowledge area of the reference.
Importance level (I), the value of importance that the lecturing team established for it, based on
the programme defined; we assumed 1(min) .. 5(max) as the importance scale to associate with
each reference included in the list.
Preference status (P), the preference value that is calculated accordingly the use given by
students; it follows the same scale of the importance level and it’s calculated during the studying
sessions.
Keywords, some priority terms for searching and cross-referencing.
Link, that is an eventual identification of a source where the document could be found.
Figure 2: An excerpt of the semantic network for bibliographic references
Latter, the reference list will be complemented with some additional terms that will establish the
relationships among the references themselves and all the topics presented in the course’s
programme. This will allow the agent in the suggestion process, giving it the basis to present, for a
particular topic (introductory concepts, conceptual modelling, development life cycle, data warehouse
monitoring, optimizations, etc.), the most appropriated references, taking into consideration their
importance and preference levels (if requested).
Table 1: Part of a bibliographic reference list preliminary description
T
Id CiteRef Reference Year p Area I P Keywords Link
1 Kimball Kimball, R., Reeves, 2008 B DW 4 - Data Warehousing, --et
al. L., Ross, M.,
Data Warehouses,
2008 Thornthwait, W.
Dimensional Modelling,
(2008), The Data
System Development
Warehouse Life
cycle Toolkit –
Piratical Techniques
for Building Data
Warehouse and
Business
Intelligence
Systems, John
Wiley & Sons, 2ª
Edição.
Life Cycle, …
2 Golfarell
i &
Rizzii
2009
M. Golfarelli, S.
Rizzi (2009), “Data
Warehouse Design:
Modern Principles
and Methodologies”,
McGraw-Hill.
2009 B DW 5 - Data Warehousing,
Data Warehouses,
System Life Cycle,
Analysis and
Conciliation of Data
Sources, User
Requirements,
44
---

3 Vassilia
dis et al.
2001
P. Vassiliadis, C.
Quix, Y. Vassiliou,
M. Jarke (2001),
“Data Warehouse
Process
Management”, Inf.
Syst. 26, 3 (May
2001), 205-236.
Orlando Belo
2001 J DW 3 -
Conceptual Modelling
and Design, …
DataWarehousingProc
essModellingEvolution
Quality, …
http://citeseerx
.ist.psu.edu/vie
wdoc/downloa
d?doi=10.1.1.2
8.7227&rep=re
p1&type=pdf
Id CiteRef Reference Year
T
p Area I P Keywords Link
4 Inmon Inmon, W.H., 1996 B DW 3 - Data Warehousing, ---
1996 Building the Data
Design and Build a
Warehouse , John
Data Warehouse, Data
Wiley & Sons, 1996.
Warehouse Monitoring,
Data Models, …
5 Romero O. Romero, A. 2009 C DW 5 - Multidimensional http://citeseerx
& Abelló Abelló: A Survey of
Modelling Survey, .ist.psu.edu/vie
2009 Multidimensional
Comparison, Data wdoc/summar
Modeling
Warehousing,
y?doi=10.1.1.1
Methodologies.
Multidimensional 57.2644&rank
IJDWM 5(2): 1-23
(2009)
Design, OLAP, … =1
(.) (.) (.) (.) (.) (.) (.) (.) (.)
After, we will gather the text sources (if available) and full-index them (index terms generation), in
order to provide data to refined searching and topic context definition. If due to some circumstances
we cannot do full indexing, we substitute it as much as possible with a list of searchable terms
collected in reviews or web pages where the text could be referred or commented. With the reference
list data and the generated indexes we build a concept map (suggestion knowledge acquisition),
representing the knowledge (Figure 2) that will support the suggestion work. With the execution of
these three initials tasks execution we finish the first stage of the readings suggestion life cycle – the
data preparation.
The second stage – studying session evaluation – is concerned exclusively in following what students
request to the agent, and what kid of suggestion that accept and preferred. During this process, the
agent stores all the requests done – studying session profile -, per student and per studying session,
and the readings selection selected. Finishing a studying session, the agent evaluates the preference
level for each reference that was selected – student preferences evaluation -, and rewrites its own
knowledge base, updating the student profile, and registering its preferences. As we can predict,
process after process, the agent’s knowledge base will be richer, and will provide better readings
suggestions and cross-referring paths. This second task group of the agent’s life cycle will be
executed so many times as the number of studying sessions performed under its assistance. While
the first one is executed only when the lecturing team decide to restructured the course’s programme
and the list of bibliographic references.
Figure 3: Agent’s conceptual architecture components
45

3.3 The agent’s architecture
Orlando Belo
To be able to support all the most common services required by a typical process for bibliographic
references suggestion, we designed the agent’s architecture (Figure 3) organized into three functional
layers with the following components:
Layer 1 – Interface
User Interface, which provides sensors and actuators to establish and maintain communication
between the agent and the students.
Information Retrieval, with the special task to retrieve document’s sources from the Web and
make their full indexation.
Layer 2 – Reasoning
Reasoning, which receives the user requests, evaluating them and generating suggestion about
some bibliographic references, taking into consideration the student’s preferences (if already
defined) and the importance level attributed by the lecturing team.
Acquaintance, a component that keeps information about other assistant agents (application area,
expertise and knowledge, references lists and application domains, etc.) working in the same
platform.
Monitoring and Control, which observes and records all user requests and their reference lists
selection for later user profiling and preferences evaluation.
Layer 3 – Knowledge and Data Access
Working Memory, which keeps intermediate results and control parameters.
Knowledge Base, a knowledge repository that keeps all the data objects and their relation ships
about bibliographic references and their relation with the topics of the course’s programme – see
a brief example in Figure 3.
This agent software was conceived to work as a complementary tutoring component that can be
integrated in an eLearning platform, acting accordingly its directives and communicate with other
services. The model proposed in this paper will allow the implementation of an assistant agent that
would reduce lecturing teams efforts and time wasted in the explanation and presentation of
bibliographic resources. At the same time will provide an entity permanently available and that can be
access almost everywhere if we have access to it. It was also considered the integration of several
assistant agents in the same platform, in order to ensure access to different bibliographic resources
and promote agent cooperation in cases of bibliographic cross-referencing.
4. Conclusions and future work
Today, eLearning platforms are introduced in almost teaching institutions. From a partial fulfilling of
some specific oriented services to a global integration in all aspects of teaching, eLearning tools
perform very important roles in learning processes, providing tutoring and supporting services
permanently, and with easy access for the entire student (and teaching) community. In this work we
were concerned with only a specific aspect of tutoring: bibliographic suggestion; with the simple
objective to help and support university students improving their learning outcomes. The option by an
agent application was related to the intrinsic characteristics of any agent-based applications:
autonomy, adaptation and cooperation. The proposed model followed the characteristics of the most
common agent-based systems, differing essentially in how the agent will acquire and maintain its
knowledge, and extend its application context. The rest of the components are quite regular, as well
as the global behaviour of the agent.
According to the most basic characteristics of the architecture proposed for the implementation of a
personal assistant for bibliographic resources suggestions, we intend to implement the assistant
agent as soon as possible, following the FIPA (The Foundation for Intelligent Physical Agents -
http://www.fipa.org/) specifications, which provides us a set of well know standards for agent-based
services and inter-agent communication. Additionally, to ensure a well-structured and robust platform
for agents we will use the JADE platform (Bellifemine et al. 2008), a generic software for agent-based
applications, which provides the most basic functionalities required by software agents (asynchronous
messages, agent life cycle management, mechanisms for agent subscription, ontology support, etc.).
Jade supports all the major FIPA’s specifications and will allow us to develop rapidly the software for
the bibliographic assistant.
46

References
Orlando Belo
Agarwal, R., Deo, A., Das, S. (2004), “Intelligent agents in ELearning”, SIGSOFT Softw. Eng. Notes 29, 2 (March
2004), 1-1.
Allison, C., Cerri, S.A., Ritrivato, P., Gaeta, A. Gaeta, M. (2005), “Services, semantics and standards: elements of
a learning grid infrastructure”, Applied Artificial Intelligence, Vol. 19, pp. 861-79.Bellifemine, F., Caire, G.,
Greenwood, D: (2008). “Developing multi-agent systems with Jade.” New York, John Wiley & Sons, Ltd.
Brusilovsky, P. (2000), “Adaptive hypermedia: from intelligent tutoring systems to web-based education (invited
talk)”, in Gauthier, G., Frasson, C. and VanLehn, K. (Eds), Intelligent Tutoring Systems, Lecture Notes in
Computer Science, Vol. 1839, Springer Verlag, Berlin, pp. 1-7.
Conati, C., Zhao, X. (2004), “Building and Evaluating an Intelligent Pedagogical Agent to Improve the
Effectiveness of an Educational Game”, IUI-CADUI ’04, January 13-16, 2004, Island of Madeira, Portugal.
Golfarelli, M., Rizzi, S. (2009), “Data Warehouse Design: Modern Principles and Methodologies”, McGraw-Hill.
Gregg, D. (2007), “ELearning agents”, The Learning Organization Vol. 14 No. 4, 2007 pp. 300-312.
Van der Hoek, W., Wooldridge, M. (2008), “Multi-Agent Systems”, In F. van Harmelen, V. Lifschitz, and B. Porter,
editors, Handbook of Knowledge Representation, pages 887--928. Elsevier.
Jennings, N. (1999), "Agent-based Computing: Promise and Perils" Proc. 16th Int. Joint Conf. on Artificial
Intelligence (IJCAI), Stockholm, Sweden. (Computers and Thought award invited paper) 1429-1436.
Jennings, N., Wooldridge, M. (1998), "Applications of Intelligent Agents" in Agent Technology: Foundations,
Applications, and Markets (eds. N. R. Jennings and M. Wooldridge) 3-28.
Johnson, W., Rickel, J. (1997), “Steve: an animated pedagogical agent for procedural training in virtual
environments”, SIGART Bull. 8, 1-4 (December 1997), 16-21.
Kazar, O., Bahi, N. (2009), "Agent based approach for ELearning", Page(s): 126-131, MASAUM Journal of
Computing (MJC) (ISSN 2076-0833), Volume: 1 Issue: 2 Month: September 2009.
Lai, H., Wang, M., He, J., Wang, H. (2008), "An Agent-Based Approach to Process Management in ELearning
Environments", IJIIT, pp.18-30.
Lester, J., Converse, S., Stone, B., Kahler, S., Barlow, S. (1997), “Animated pedagogical agents and problemsolving
effectiveness: A large-scale empirical evaluation”, In Proceedings of Eighth World Conference on
Artificial Intelligence in Education, pp 23-30.
Leung, E., Li, Q. (2001), “Agent-Based Approach to eLearning: An Architectural Framework”, In Proceedings of
the First International Conference on The Human Society and the Internet - Internet Related Socio-
Economic Issues, Won Kim, Tok Wang Ling, Yoon-Joon Lee, and Seung-Soo Park (Eds.). Springer-Verlag,
London, UK, 341-353.
Marques, A., Belo, O. (2010), “Discovering Usage Profiles on Web Based eLearning Platforms Using Markov
Chains”, In Proceedings of 9th European Conference on eLearning (ECEL’2010), Instituto Superior de
Engenharia do Porto, Porto, Portugal, 4-5 November.
Nedev, D., Nedeva, V. (2008), “Aspects of multi-agent systems application in eLearning”, Computer
Science’2008, Kavala, Greece, 18-19 September 2008, p. 1022-1028.
Nunes, M., Dihl, L., Fraga, L., Woszezenki, C., Oliveira, L., Francisco, D., Machado, G., Nogueira, C.,
Notargiacomo, M. (2002), “Animated Pedagogical Agent in the Intelligent Virtual Teaching Environment”,
Interactive Educational Multimedia, No. 4, April.
Okamoto, S., Sycara, K., Scerri, P. (2009), Personal Assistants for Human Organizations, in Multi-Agent Systems
- Semantics and Dynamics of Organizational Models, Ed. V. Dignum, IGI-Global, Hershey, Pennsylvania,
U.S.A.
Paneva, D., Zhelev, Y. (2007), “Models, Techniques and Applications of eLearning Personalization”, International
Journal "Information Technologies and Knowledge" Vol.1.
Pankratius, V., Olivier, S., Stucky, W. (2004), “Retrieving content with agents in web service eLearning systems”,
Symposium on Professional Practice in AI, IFIP WG12.5 – in Proceedings of the First IFIP Conference on
Artificial Intelligence Applications and Innovations (AIAI), 91-100.
Remondino, M. (2007), “Agent Based Virtual Tutorship and ELearning Techniques Applied to a Business Game
Built on System Dynamics”, GAMEON 2007 Proceedings,157.
Schiaffino, S., Amandi, A., Gasparini, I., Pimenta, M. (2008), “Personalization in eLearning: the adaptive system
vs. the intelligent agent approaches”, In Proceedings of the VIII Brazilian Symposium on Human Factors in
Computing Systems (IHC '08). Sociedade Brasileira de Computação, Porto Alegre, Brazil, Brazil, 186-195.
Silveira, R., Vicari, R. (2002), “Developing Distributed Intelligent Learning Environment with JADE - Java Agents
for Distance Education Framework”, In Proceedings of the 6th International Conference on Intelligent
Tutoring Systems (ITS '02), S. A. Cerri, G Gouardères, and F. Paraguacu (Eds.). Springer-Verlag, London,
UK, UK, 105-118.
Wooldridge, M. (1998), “Agent-based computing”, In Interoperable Communication Networks. 1(1), pages 71-97.
January 1998.
Zhang, D., Zhao, J.L., Zhou, L. and Nunamaker, J.F. (2004), “Can eLearning replace classroom learning?”,
Communications of the ACM, Vol. 47 No. 5, pp. 75-9.
47

Some Reflections on the Evaluation of Virtual Learning
Environments
Nabil Ben Abdallah and Françoise Poyet
ELICO (Équipe de recherche de Lyon en sciences de l’Information et
COmmunication), Lyon, France
nabil.ben-abdallah@iut-dijon.u-bourgogne.fr
francoise.poyet@free.fr
Abstract: This paper aims at pondering over the evaluation question of the VLE. We have carried out an
overview of concepts and methods commonly used in Human-Computer Interaction (HCI) in the evaluation of
information systems and then we have discussed the limits of their application on the VLE. We are proposing to
evaluate the latters according to an activity-oriented approach. It will allow us to expand our field of study to
learning situations in which a VLE can be integrated. The learning activity is thus observed not only through the
interactions of motivated learners and teachers with VLE but also with the objects and objectives of learning
implemented via VLE. This observation is based on the theoretical framework proposed by activity theory. The
Engeström model as a form of conceptualization of the structure of activity is essentially explored to understand
the elements interacting in a learning situation integrating VLE. We conclude our contribution by the application of
ideas developed by observing the use of a VLE in a learning situation of information seeking and involving small
groups of students.
Keywords: VLE evaluation; activity theory; usability; utility; practical acceptability; social acceptability
1. Introduction
The increasingly generalized use of Virtual Learning Environments (VLE) in different learning
situations places the issue of the evaluation of their impacts on learning as well as on the
organizations at the core of designers and managers’ concerns regarding the systems of learning.
VLE is not only a technical solution integrating heterogeneous technologies, it is also a space of
information around which a social space that organizes and structures the various interactions
necessary for any learning activity can be formed and developed. Several studies and research have
been carried out (Britain 2004; Colace 2006; Keller 2005; Dobson 2001) to help the actors of learning
to choose the appropriate VLE so as to cope with specific situations of learning, to assess the
consequences of the use of these VLEs on learners and on learning itself, and to identify interaction
problems with these VLEs.
In this work, the learning activity is observed not only through the interactions of motivated learners
and teachers with VLE but also with the objects and objectives of learning implemented via VLE. This
observation is based on the theoretical framework proposed by activity theory. The Engeström model
(Engeström 1987) as a form of conceptualization of the structure of activity is essentially explored to
understand the elements interacting in a learning situation integrating VLE.
The data resulting from the analysis of the activity system of learning based on the model of
Engeström enable us to get, on the one hand, data on the degree of practical and social acceptability
of VLE, and on the other hand, data that can be used to improve the methods of usability and utility
evaluation of VLE. We conclude our contribution by the application of ideas developed above by
observing the use of a VLE in a learning situation of information seeking and involving small groups of
students.
2. What is a VLE?
According to Weller (Weller 2007, p.5) a VLE is: "a software system that combines a number of
different tools that are used to systematically deliver content online and to facilitate the learning
experience around that content". Other authors such as Colace and al. (Colace 2006, p.2) admit that
a VLE or a eLearning platform is organized in three macro-components: "In our opinion the most part
of contemporary eLearning platform can be viewed as organized into three fundamental macro
components: a Learning Management System (LMS), a Learning Content Management System
(LCMS) and a Set of Tools for distributing training contents and for providing interaction". In the light
of these definitions and some others, we are considering that a VLE is a particular computer system
allowing to create learning situations with specific learning objectives (e.g. learning such a type of
knowledge). A VLE is usually conceived with the aim to train the learning actors into achieving these
48

Nabil Ben Abdallah and Françoise Poyet
objectives through a favorable activity. VLE offers a set of integrated functionalities to complete the
requirements of various learning situations. Four groups of functionalities are usually mentioned:
course management (design, maintenance, and diffusion), tracking learning, technical administration
(set-up, security, integration, maintenance) and learner interface (offering communication and
collective work tools, and a customizable space). Depending on the learning objectives, a VLE must
also propose expanded parameterization facilities to create more or less elaborate learning situations.
eLearning platforms such as Blackboard, Moodle, Claroline, and others are examples of VLE (either
proprietary or free software).
From a technical point of some view, a VLE integrates various types of software tools to support the
various functionalities mentioned in the preceding paragraph. For examples we have: a database
management server, a web server, a mail server, a file server. These servers are developed with
technologies of various origins, mostly meeting with the requirements of various standards. Among
these technologies, those of the Internet have had considerable repercussions on the design of VLEs.
In addition to the technical possibility of having applications interconnected and making use of
resources coming from different sites, collaborative or co-operative activities and as well as human
interaction seem to be favored thanks to tools enabling communication, sharing and the production of
documents.
The use of a VLE encourages written communication and, except with the synchronous
communication tools, makes the learner’s participation less improvised because he has a certain
amount of time for consideration and easy access to resources enabling him to enrich his ideas and
to pursue his thinking. In short, a learning situation built around a VLE bears little resemblance to a
conventional class where student is constantly under a "social pressure" emerging from the activity of
the other students and the requirements of the teacher. The metaphors (class, group, office, etc.)
used in the design of VLEs (Falconer 2008) should not allow us to forget that this is above all a
computer system having features which are not those of a learning device built around a
conventional class.
In order to understand these characteristics and thus expand our specific acception of VLEs beyond
the principle integrating functional and technological aspects, we quote here Dillenbourg and his
colleagues (Dillenbourg 2002) who think that VLEs can be identified through six characteristics : 1) A
virtual learning environment is a designed information space 2) A virtual learning environment is a
social space 3) The virtual space is explicitly represented 4) Students are not only active, but also
actors 5) Virtual learning environments integrate heterogeneous technologies and multiple
pedagogical approaches 6) Most virtual environments overlap with physical environments.
It is obvious that a given VLE may not verify all these characteristics, the objective here is to have a
comprehensive understanding of roles that a VLE can play in learning activities. The information
space being composed, among other things, of the courses and of all the information resources that
enrich and complement them must be sufficiently structured and organized so that it can be suitably
exploited in learning activities. Beyond this information space, a social environment which organizes
and structures the various interactions (learner to learner, learner to teacher, etc.) which are essential
to any form of learning activity, is built up. The learning environment can be based on more or less
sophisticated interfaces going from simple text interfaces to graphical user interfaces in 3D. Whatever
the interface used, it must be sufficiently explicit so that learners can achieve the proposed learning.
Although VLEs are often associated with distance learning because they are essentially Web-based
applications, they are also widely used as a support for classroom teaching. Integration within a VLE
of several software tools makes the adaptation of various approaches possible. For example,
teachers can request individual or collective work from the students and decide to intervene or not in
the various phases of the completion of this work. The package of software tools (exerciser, wiki, blog,
chat ...) offered by a VLE makes it possible to adopt one or the other approach.
All these characteristics are important to set up a learning situation with precise objectives. Beyond
the impact which these VLE can have on the learning outcome, it is the learner's statute which
changes: from a passive learner who merely memorizes some patterns or knowledges which are
passed on to him, we will end up with a learner who is not only active but who is also an actor of his
own learning. He will use and produce information, and he will continuous interaction with learning
environment. According to Dillenbourg (Dillenbourg 2002, p.7) "… the notion of a learning activity in
virtual learning environments refers to something richer than in individual courseware, closer to the
49

Nabil Ben Abdallah and Françoise Poyet
notion of project. The difference between other constructivist environments and what virtual
environments potentially offer can be described as making students not only active, but also actors,
i.e. members and contributors of the social and information space". For example, if we consider the
learning user, these theoretical possibilities provided by a VLE should not make us forget that there is
always a gap between the use of VLE as it is defined by the designers according to the representation
that they have of learning activities, and the use of VLE as actually practiced by the learner as
regards what he must do to achieve his objectives. The fact that the use of the VLE by the learner is
different from that defined by the designer is not in itself a design problem or a dysfunction in learning,
it is inherent to what Perriault (Perriault 2008) called the logic of use which is construction by the
individual of a choice of instrument and a kind of employment he uses to achieve a project.
This description of the various facets which a VLE can have is necessary to even better understand
the difficulties that we have to evaluate it. Indeed, for this evaluation to be reasonably reliable, it must
include both the product VLE (interface and functionalities) and the learnings it supports. We believe
the facility with which a VLE can be used and the variety of its functionalities alone does not explain
the success or the failure of its design. Other factors must be taken into account to understand and
perhaps predict the adoption or not of a given VLE by the learning actors.
This specific acception of the evaluation takes again the distinction made by Nielsen (Nielsen, 1994)
between practical acceptability and social acceptability of a computer system. Nielsen does not
describe the components of social acceptability in detail, but it clearly emphasizes its significance in
the overall system acceptability. We are indeed dealing with social acceptability which refers to the
way in which the learning actors perceive the different stakes related to the use of a VLE in a learning
situation. These actors react favorably or unfavorably to the use of VLE according to opportunity,
risks, and benefits that may arise from this use. Practical acceptability refers to a kind of "diagnostic of
use" in which the designers as well as the learning actors are invited to measure the utility, for
learning tasks, of functionalities suggested by a VLE and the way in which they can be used. Here
they are two principal categories of a computer system evaluation: usability and utility (we will deal
with that idea later). Practical acceptability can also take into account the factors (such as the
compatibility with the existing systems, the cost, the reliability, etc.) which support the integration of a
VLE in a given learning situation.
3. Evaluation of computer systems and VLEs
The concept of usability, introduced by Eason (Eason, 1984) and developed by other authors (Nielsen
1994; Bastien 2001; Rosson 2001, Shneiderman 2005) generally refers to the quality of the interface.
Bastien suggest that the usability is defined in a way much broader designating the related aspects as
much in the use of the computer systems that those related to the utility of these system. The two
dimensions (usability and utility) are therefore not completely independent: an interface with good
ergonomic qualities (optimal usability) cannot alone satisfy the user’s expectations if essential
functions for carrying out the task (real utility) are not implemented in the application. Standard ISO
9241-11 defines the usability of a computer system as follows: "the extent to which a product can be
used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a
specified context of use".
Two approaches are generally used to evaluate the computer systems: analytical and empirical; these
approaches are not necessarily opposite but complementary. The data collected from empirical
methods are primarily behavioral; they are used to locate the problems encountered by users. As for
data collected from analytical methods, these provide information on the ergonomic quality of the
interface without the user directly interacting with the system.
The evaluation methods can be complementary according to the objective of the evaluation, to the
means used and the moment of the life cycle of the application. Dix (Dix, 1998) defined eight criteria
to help the appraiser in the choice of a suitable evaluation technique. They are: the time of the
evaluation (in relation to the life cycle of the application), the type of data to be collected, human and
material resources which are available, constraints imposed by the evaluation, mode of evaluation
(laboratory or field), the degree of objectivity or subjectivity sought, information available and data
acquisition mode (online/offline). One or several methods may be used to collect the maximum
amount of information on the system being observed in order to bring the evaluation operation to a
successful conclusion. For example, interviews and questionnaires may supplement cognitive
walkthrough (Lewis 1992; Mahatody 2010) by providing information on the characteristics of the users
50

Nabil Ben Abdallah and Françoise Poyet
and the tasks to be performed. In an article devoted to studying methods by inspection (“heuristic
evaluation, cognitive walkthrough, formal usability inspections, and pluralistic usability walkthrough”),
Hollingsed and Novick (Hollingsed 2007) assert that heuristic evaluation and cognitive walkthrough
are methods that are still actively used and they enables to obtain the best possible evaluation in
terms of usability.
A VLE, as a particular computer system, offers a set of functionalities to carry out various activities
directly or indirectly related to learning processes. The utility of these suggested functionalities as well
as the way in which they are used by the users may be evaluated by HCI methods based on an
analytical or/and empirical approach (Huart 2004, Hvannberg 2007). Let us examine here if the
cognitive walkthrough method allows us to evaluate the usability and utility of a VLE. This method
requires a specific description of the tasks to be carried out using the computer system, a description
of the actions which the user must carry out in order to perform these tasks and, finally, a general
description of the users and context of use. The appraiser explores the interface in four stages of the
human-computer interaction, to evaluate the facility with which actions will be carried out. The tests
make it possible to determine the parts of the interface which are likely to hinder optimal use of the
application. For a VLE, the description of the tasks to be performed and the sequence of the actions
to be carried out pose a problem for the appraiser. The tasks carried out, for example, by a userlearner
are indeed complex and are closely connected. The appraiser must be able to describe the
reality of a learning activity involving different actions which are not naturally carried out in a
predefined sequence; within this learning activity, the commitment of the learner is not always
predictable. If the learning situation had been broken down into scenarios the appraiser could use the
scenario which seems the most relevant according to the elements to be evaluated. He must be able
to operationalize and implement the chosen scenario on the VLE. This method will be of the greatest
interest if it is focused on "simple tasks" for which the series of actions required to carry them out is
easy to describe. For example, evaluating the ease with which a VLE user-administrator will choose
the actions to be carried out is perfectly possible using the cognitive walkthrough method. These
principles are fairly general and consequently they can be used when evaluating different types of
computer systems, including VLEs. Similar to a cognitive walkthrough, a heuristic evaluation can be
applied to VLEs, as the principles, such as flexibility and coherency, which are at the root of
heuristics, are fairly general. However, for this evaluation to be relevant, it must be adapted to these
specific environments.
We can therefore acknowledge that all the methods presented in the preceding paragraphs are
potentially valid to measure the usability of VLEs and provide, in certain conditions, information on
their practical utility. Nevertheless, as a VLE must make it possible to carry out various tasks that are
directly or indirectly related to the complex process of learning, it is necessary to ponder over specific
criteria referring to the teaching uses of VLE.
A VLE, as we have already mentioned above, should be seen as a social space where the learner
can be an active participant in learning. It is thus necessary that the evaluation is comprehensive
relating at the same time practical acceptability (utility and usability) and the social acceptability of a
VLE. These two dimensions (practical and social) are not independent, they are articulated and
complement each other. In view of this complexity posed by the evaluation of the VLEs, other
theoretical approaches are worth questioning: this is the case with activity theory.
4. Origins and nature of activity theory
Activity theory comes from the writings of Leontiev, a disciple of Vygotsky in the 1930s. It finds its
roots in the theory of social development, according to which social interactions play a considerable
part in the development of cognition, and it suggests that the social dimension is at the heart of
human activity. Engeström (Engeström 1987) completed the triad (subject, tool, object) initially worked
out by Vygotsky, adding the element “community” and two mediating elements: rules and the division
of labor. The activity is observed thereby like a system with its own structure, its own internal
transformations, and its own development. The diagram below shows a system of human activities
according to Engeström.
The subject-community-object triangle represents a process connecting the object to a community of
work which plays a part, with the subject, in producing or transforming the object. The relationship
between the subject and the object is mediated by the tool; between the community and the subject is
mediated by explicit or tacit rules; and finally, the relationship between the community and the object
51

Nabil Ben Abdallah and Françoise Poyet
itself are mediated by a certain form of division of labor. The whole system provides a set of dynamic
relations which adjust themselves and adapt continuously to maintain a certain productive stability
(transformation of the object into outcome).
Figure 1: Structure of a system of human activities (Engeström1987)
The principles commonly described of the activity theory are: object orientation, mediation, the
hierarchical structure of activity, internalization/externalization, and development. According to the
principle object orientation of the activity theory, all the human activities are directed towards objects.
They make it possible to transform an object considered as "a raw material'' to a significant object
which is potentially shared by various system activities. These interactions with the object are,
according to the principle of mediation, inevitably mediated by material or intellectual tools. These
tools are able to adapt to environments composed of other tools and / or other human actors. The
principle of the hierarchical structure of activity is one of the principal contributions of Leontiev
(Leontiev 1981), it supposes the existence of three hierarchical levels of activities to carry out a given
task: activities in close relation with the motivations and intentions, actions subordinate and related to
specific goals, and operations determined by the real conditions of the activity. For Leontiev the
human activity exists only in the form of an action or a chain of actions. The principle of
internalization/externalization informs us about our mental processes which are the consequences of
our internalized external activities. These processes are connected to their social and cultural
environment. Lastly, the principle of development requires that the human activity is analyzed in the
context of development. It is a question of understanding how an activity develops, in the course of
time, in an historical and cultural context, and how the actions undertaken on the object of the activity
affect such a development. According to Kaptelinin and Bonnie (Kaptelinin 2006, p.72): "principles of
activity theory should be considered as an integrated system because they are associated with
various aspects of the whole activity. That is, systematic application of any of the principles makes it
eventually necessary to engage all the others".
The activity theory is currently used in various studies related directly or indirectly to human activity. In
the field, for instance, the human-computer interaction, activity theory is used as a relevant theoretical
framework for the design and analysis of computer systems by focusing on the objective, constraints
and specificities of the human activity (Engeström 2010; Larry 2009; Kaptelinin 2006; Mwanza 2002;
Kuutti 1996; Nardi 1996).
From the definition of an activity system and principles of activity theory described above, we try to
sketch the outlines of an approach that allows us to conduct a comprehensive evaluation connecting
both practical acceptability (utility and usability) is the social acceptability of a VLE.
5. Towards a comprehensive evaluation of VLE
In the activity theory, the principle of mediation is a key principle to understand the system of activities
in its globality. It informs us about the mediating role of the Tool that essentially mediates subject-
52

Nabil Ben Abdallah and Françoise Poyet
object interactions, as well as community-object interactions. According to the nature of the analyzed
activity, the two other mediators of the activities system (rules and division of work) can influence the
mediation around of the activity tool.
To analyze and understand the role of these mediators in the achievement of an activity, we initially
decompose the global activity system in subsystems represented by triads consisting of three
elements including a mediator. Each subsystem is composed of the following elements: the subject or
community, the object and a mediating element (tool or rules or division of labor). There are therefore
six triads: (subject, tool, object), (subject, rules, object), (subject, division of labor, object),
(community, division of labor, object), (community, tool, object) and (community, rules, object).
The theoretical framework of activity theory (primarily its concepts and its principles) makes it possible
to identify various evaluation questions which an appraiser must ask to determine the various factors
which can influence the global acceptability of the VLE. This is essentially the contextual factors of the
interaction with the VLE whose determination helps to identify and to understand social and practical
acceptability. Here we are suggesting a few series of questions which can be formulated around each
the activity system triads. The data analysis collected from these questions must take into account the
global nature of the activity, and the fact that the application of a given principle inevitably triggers the
application of the other principles. For example, analysis of the impact of the VLE on learning must
take into account the hierarchical structure of activity, object-oriented nature of the activity, the
acquisition of knowledge through the process internalization / externalization, and development of
system activities.
Table 1: Examples of questions
(subject, tool, object)
(subject, rules, object)
(subject, division of
labour, object),
(community, tool, object)
(community, rules,
object)
(community, division of
labour, object)
Aspects of individual activity
- Did you find the VLE useful to perform the work?
- Did you meet with obstacles during the use of the VLE?
- Did the rules of the work organization pose problems to you?
- is the use of the VLE compatible with the rules of the work organization?
- Did you agree with the allocation (whether agreed or implicit) of tasks?
- Does the division of labour imposed by the use of the VLE facilitate the
performance of work?
Aspects of collective activity
- Do you wish to use a VLE to complete a collective work?
- Are the functionalities of the VLE related to collective work easy to use?
- Did the rules of the work organization pose problems to you?
- Is the use of the VLE compatible with the rules of the work organization?
- Was there a collective definition of the different roles in the team?
- Was there was an evolution of roles of the team members according to operations
carried out (query writing, information filtering,) and the intermediate results?
- Does the division of labour imposed by the use of the VLE facilitate the
performance of work?
To demonstrate this process, we would like to provide an example. In this example, groups of
students carry out information searches on a given topic. Each group must produce a report gathering
and synthesizing retrieved information. The Representation of the activity according to Engeström
model enables us to understand the work of a student (subject) engaged in the production of a report
(object) in collaboration with the members of its group (community). The relation which links a student
with a group is determined by explicit rules (duration of work, mandatory work participation, collective
presentation of work, concurrent teams, etc) imposed by the learning device which has been set up.
The mediation between the subject and the object is carried out, among other things, by tools offered
by a VLE. The division of work is based on an internal organization for each group, the lecturers do
not operate in the allocation of tasks among group members.
Among other things, collected data show us that the students consider the VLE as a communications
tool, as well as a production of intermediate documents tool, and especially as an information sharing
tool among the members of the same group. The tested functionalities appeared useful to the
53

Nabil Ben Abdallah and Françoise Poyet
students, they are almost all of them have agreed on the VLE utility for a collective work: less space
and time constraints ; a better organization of collective work; facilitates the interactions between the
group members and it supports, consequently, the "collaborative work". The sharing of documents
favors the reactivity of the members and stimulates their production. In a nutshell, we have only
favorable opinions as regards of the VLE in this kind of exercise.
The data of this example consolidates our idea of evaluating the VLE compared to the learning
activities (context of use), while into taking account the evaluation data which can be provided by HCI
methods. Thus, in our example of evaluation, if the students are satisfied with the VLE they use does
not mean that it is powerful and that it meets the various expectations that this kind of activity
generates. If we consider the results of our analysis, we are quite convinced that the functionalities
suggested by the VLE are not completely adapted to the nature of the exercise in question. VLE
would gain social acceptability (way in which the users perceive the various stakes related to the use
of a VLE) and also practical acceptability if the constraints and contradictions between the various
components of the activity system revealed by the analysis are best supported by the VLE.
6. Conclusion
The question of evaluating VLEs is a huge one and is far from being closed. The attempt of this
contribution is a comprehensive approach in order to shed light on this question. Initially, we tried to
understand what differentiates a VLE from other computer systems. It is at the same time a technical
solution integrating heterogeneous technologies, and an information space around which a social
space which organizes and structures the various interactions necessary to any learning activity can
be formed and developed. The different facets that can have a VLE make its evaluation difficult. This
evaluation must relate both the VLE product (interface and functionalities) and the learning it
supports. We have acknowledged that several evaluation methods of computer systems are
potentially valid to measure the usability of a VLE and provide, in certain conditions, information on
their practical utility. However, as a VLE should allow to perform various tasks related directly or
indirectly with the complex processes of learning, it is necessary for the evaluation to be
comprehensive regarding both the practical acceptability (utility and usability) and the social
acceptability of VLE. These two dimensions (practical and social) are not independent; they are
articulated and complement each other. Facing this complexity posed by the evaluation of a VLE, we
have attempted to explore the theoretical framework proposed by the activity theory to reflect on an
approach which enables us to carry a comprehensive evaluation regarding both the practical and
social acceptability of a VLE.
If the idea that the measure of practical acceptability of a VLE must be completed by the measure of
its social acceptability is sufficiently argued in this contribution, the data processing which can be
collected according to our evaluation approach requires to be explained more thoroughly; it is what
we will strive to do in the coming days.
References
Bastien, J. M. C. & Scapin, D. L., (2001). Évaluation des systèmes d'information et Critères Ergonomiques. In C.
Kolski (Ed.), Systèmes d'information et interactions homme-machine. Environnements évolués et évaluation
de l'IHM. Paris, Hermès, vol. 2, 53-79.
Britain, S. & Liber, O. (2004). A Framework for the Pedagogical Evaluation of eLearning Environment, JISCcommissioned
report. http://www.jisc.ac.uk/index.cfm?name=elearning_pedagogy
Colace, F., De Santo, M. & Pietrosanto, A. (2006). Evaluation Models for ELearning Platform: an AHP approach.
36th ASEE/IEEE Frontiers in Education Conference, San Diego, CA. http://fieconference.org/fie2006/papers/1168.pdf
Dillenbourg, P., Schneider, D. K., & Synteta, P. (2002). Virtual Learning Environments. In A. Dimitracopoulou
(Ed.), 3rd Hellenic Conference "Information & Communication Technologie in Education", 3-18. Greece:
Kastaniotis Editions.
Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (1998). Human-Computer Interaction. London, Prentice Hall Europe.
Dobson, M., McCracken, J., & Hunter, W. (2001). Evaluating Technology-Supported Teaching Learning: A
Catalyst to Organizational Change. Interactive Learning Environments, vol.9 n° 2. 143-170.
Eason, K. D. (1984). Towards the Experimental Study of Usability. Behaviour and Information Technology, vol. 3,
n° 2, 133-143.
Engeström, Y. (1987). Learning by Expanding : An Activity-Theoretical Approach to Developmental Research.
Helsinki, Finland, Orienta-Kosultit Oy.
Engeström, Y. (2001). Expansive Learning at Work: toward an activity theoretical reconceptualisation. Journal of
Education and Work, vol. 14, n° 1, 133 – 156.
54

Nabil Ben Abdallah and Françoise Poyet
Engeström Y. et al. (2010). Grand challenges for future HCI research. In Proceedings of the 6th Nordic
Conference on Human-Computer Interaction: Extending Boundaries. Reykjavik, Iceland.
Falconer, L. (2008). Evaluating the use of metaphor in online learning environments. Interactive Learning
Environments, vol. 16, n° 2, 117-129.
Hollingsed, T. & Novick, D. (2007). Usability Inspection Methods after Fifteen Years of Research and Practice.
Proceedings of the 25th annual ACM international conference on Design of communication. El Paso, Texas,
USA, 249-255.
Hvannberg, Ebba Thora, Effie Lai-Chong Law, & Marta Kristín Lárusdóttir. (2007). "Heuristic evaluation:
Comparing ways of finding and reporting usability problems". Interacting with Computers 19(2):225-240.
Huart, J., Kolski, C., & Sagar, M. (2004). Evaluation of multimedia applications using inspection methods: the
Cognitive Walkthrough case. Interacting with Computers, vol.16, n°2, 183-215.
Kaptelinin, V., & Nardi, B. A. (2006). Acting with technology: Activity Theory And Interaction Design. Cambridge:
MIT Press.
Keller, C. (2005). Virtual learning environments: three implementation perspectives. Learning, Media and
Technology, 30(3), 299.
Kuutti, K. (1996). Activity Theory as a Potential Framework for Human-Computer Interaction Research. In Nardi
B. (Ed.): Context and Consciousness: Activity Theory and Human Computer Interaction. MIT Press, 17-44.
Larry, L. Constantine (2009). Human Activity Modeling: Toward A Pragmatic Integration of Activity Theory and
Usage-Centered Design. In Seffah A., Vanderdonckt J., and Desmarais M. C. (Ed.), Human-centered
software engineering: software engineering models, patterns and architectures for HCI. Springer London.
27-51.
Leontiev, A. (1981). Problems of the Development of the Mind. Moscow, Progress.
Lewis, C. and al. (1992). Cognitive walkthroughs: a method for theory-based evaluation of user interfaces.
International Journal of ManMachine Studies, Vol 36, no. 5, 741-773.
Mahatody T., Sagar M., and Christophe Kolski (2010). State of the Art on the Cognitive Walkthrough Method, Its
Variants and Evolutions. International Journal of Human-Computer Interaction 26, no. 8 : 741-785.
http://www.univ-valenciennes.fr/LAMIH-intra/site/specifique/publications/204316.pdf
Mwanza, D. (2002). Towards an activity-oriented design method for HCI research and practice. PhD thesis, The
Open University, United Kingdom.
Nardi, B. A. (1996). Some Reflections on the Application of Activity Theory . In Nardi B. A. (Ed.), Context and
Consciousness: Activity theory and human-computer interaction. MIT Press, 235-246.
Nielsen, J. (1994). Usability Engineering. San Francisco, Morgan Kaufmann.
Perriault J. (2008). La logique de l’usage: essai sur les machines à communiquer. Paris, L’Harmattan.
Rosson, M. B. and Carroll, J. M. (2001). Usability Engineering: Scenario-Based Development of Human
Computer Interaction". Morgan Kaufmann.
Shneiderman, Ben and Plaisant, Catherine (2005). Designing the User Interface: Strategies for Effective Human-
Computer Interaction" (4th ed.). Addison-Wesley
Weller, Martin. (2007). Virtual Learning Environments: Using, Choosing and Developong Your VLE. Routledge.
55

Designing A New Curriculum: Finding The Right Blend
Andrea Benn
Brighton Business School, University of Brighton, UK
a.benn@brighton.ac.uk
Abstract: This paper discusses the development process for a new Undergraduate (UG) Business Programme
to be introduced at the University of Brighton (UoB) for the academic year 2012/13. It is the research and
preparation which forms the basis of this paper. The focus for this paper was about finding the right balance of elearning
to support learners during their UG experience, with a view to progressing them from passive to active
and independent learners. The principal pedagogical strategy is to be collaborative enquiry-based learning and
this research was to identify appropriate blended strategies and technologies which would enhance and
complement this approach. The concept of collaborative learning indicates that consideration must be given to
the formation and development of groups for working in both physical and virtual environments. Both will bring
their own challenges but consideration is given to how these will be addressed throughout the lifetime of the
programme. The plethora of research into E and blended learning advises that learning with technology needs
to be driven by pedagogical principles, including the specific skills that these new learners will require in order to
cope with and enjoy the new approach of enquiry-based learning, not normally associated with UG business
students. The development required a review of the current technologies available at UoB, alternative
technologies and how they are being utilised and incorporated by colleagues in other institutions, together with
ways of introducing them to the learners. The research also explored whether or not there should be an
overlapping of educational and recreational technologies and is it right to mix. Final attention was given to the
assessment of the students, how frequently this should occur and how much should be formative or summative,
what format it would take and how much of it would be the responsibility of the tutors. Some of the technologies
and collaborative learning were already integrated into the current business programme at UoB and provided
useful evaluation from observations of the students together with their feedback. This provided some useful
understanding of what their expectations were as well as identifying specific requirements prior to starting any
activities.
Keywords: blended; collaborative; problem-based learning; curriculum design, action learning
1. Introduction
The recent Higher Education Funding Review, which will see an increase in tuition fees for the next
academic year, as well as a timely review of the undergraduate (UG) programmes within the Business
School (BBS) at UoB provided the opportunity to explore alternative learning and teaching practices.
The focus of this paper is a new UG business course based entirely on an enquiry-based learning
(EBL) model.
As described in the Abstract, the research and preparation for designing a new curriculum must not
be underestimated, particularly when the idea is to radically redesign the whole curriculum and the
pedagogical strategy underpinning it. This became very apparent at the start of the research into
what enquiry-based learning actually meant. The literature refers to problem-based learning (PBL),
action learning and work-based learning rather than enquiry-based learning but all offer similar
definitions which would apply equally well to an enquiry-based strategy: all agree that they are models
which merge theory with practice and knowledge with experience.
The EBL model will require the students to work in groups to solve a problem by managing their own
learning process and deciding what information they will need as well as the skills they will have to
develop, ascertained from earlier research, (Benn, 2011).
However, Duch et al (2001) advise that PBL is not generally available to many undergraduate (UG)
courses for a variety of reasons among them being class sizes; intellectual maturity; motivation levels;
and epistemological beliefs.
These are entirely valid reasons and were considered continually during the design process, including
the reason to continue with the research in this area. The preparation for redesigning the curriculum
came also from a desire to offer the students an alternative learning experience: one which would
challenge them, encourage and instil confidence in them to become active learners; illustrate to them
that more often than not there is no one right answer, and to provide the support; guidance; resources
and feedback opportunities at times when they required it.
56

Andrea Benn
It was also clear that this was going to be as much a challenge for the lecturers as it was hoped it
would be for the students.
Research Questions
The main objectives for this research were:
To identify the specific skills new learners would require in order to cope with and enjoy, enquiry
based learning (EBL)
To identify appropriate blended strategies and technologies which would complement EBL within
a business programme
To identify ways to introduce EBL to colleagues, students and the local business community
2. Research Methods
The primary method for this report was desk research reviewing the available literature for problembased;
work-based and action learning and in particular the findings of some of the experts in this
field as Barrows & Tamblyn (1980); Savin-Baden (2006), Boud & Feletti (2001) and Duch et al (2001).
More recent journal articles discuss the experiences of implementing PBL but only one example was
found for an UG programme in the UK, Parrott (2009)
Students studying on the existing course at UoB, the FdA Business, were asked for feedback on
completion of a fully online module ascertaining what they liked or disliked most about the activities;
the time spent on each activity and whether they considered they had sufficient IT skills to cope with
the module or how these had developed during the module. They were also asked how they would
feel about using social networking sites ie Facebook for University work and interacting with lecturers.
Results from earlier action research by the author, Moon (2010), which looked at the development of
her practice and skills in order to bring about change in the module delivery to encourage the students
to rise to their new challenge, were also incorporated into this report.
2.1 The Pedagogical Strategies
The concept of EBL was determined as the preferred principal strategy as it requires the programme
to be student-centred, therefore the curriculum design must also have a student-led approach,
allowing students to “take care of their needs... particularly pushing them into reflecting on a certain
number of aspects they are already acquainted with but which they tend to forget when working on
their own”, Brudermann (2010).
The underpinning skills required can be identified as being those suggested by Bloom (1956) as
knowledge; comprehension; application; analysis; synthesis and evaluation as well as collaborative
group working and co-operative learning. Developments of these skills is already expected of
graduates upon completion of each stage of existing UG programmes, and are now translated to the
QAA level descriptors for UG programmes. What differs is the expectation that students from the
outset of Level 4, will take control of their learning for themselves, ie they adopt an active approach,
Savin-Baden & Wilkie (2006) and put these skills to use immediately. It is recognised that for Level 4
UG this may be a tall ask and the design of the first year needs to be more formative, demonstrating
to the students what is expected of them.
The next stage is to understand how, through the design of the curriculum, the students will
recognise these skills and map their development over the course of their study and whether using
blended strategies and incorporating technology would complement or complicate and how far should
it go.
Earlier research emphasised that the first consideration needs to be for the technology to support and
be driven by the pedagogical principles. From a business perspective, Laudon & Traver (2009, 4-4)
advise that business decisions drive the technology and not the other way around, to ensure that its
technology platform will be aligned with the business and not the other way around. This view is
reiterated by Cox et al (2010) in an educational setting, which is to find the best technology that suits
the subject being offered, the context in which it is offered and which can be utilised for the greatest
advantage for that purpose.
57

Andrea Benn
For this new course, it is proposed that a blended-PBL framework (Savin-Baden, 2006) be considered
as it will combine “an approach for building an online learning community” (Yeh, 2010) alongside
traditional group-working enabling students to develop their own working boundaries and task
allocation in both environments. However, blending the curriculum needs to be planned within the
Course Team just as the overall assessment strategy is planned to ensure that the students meet with
a variety of learning, teaching and assessment methods and not just one or two. The blend therefore
will come from discussions with colleagues also acknowledging their own areas of expertise and
preferences, as well as the technology available within our own institution at this time.
It is acknowledged too that the students will have different levels of expertise and preferences for how
they wish to participate in the learning activities. The current Level 5 students this year who had
participated in a fully online module, fed back via a questionnaire for this report that they liked the new
way of learning; the chance to voice their own opinion; being able to compare and discuss and to
share and receive ideas as well as peer support. What they did not like was the volume of posts;
what they considered to be a complicated discussion board; the feeling that they either should not
post too much information as it would not give the others a chance or finding that too much
information had already been posted and for some the dislike of having to get on and do it on their
own with what they perceived to be little support.
With respect to their levels of expertise they advised that the majority of them had some but limited IT
skills, they could navigate their way around the Internet and could use basic office applications but
only half of the group agreed that their IT skills had developed in some way.
Both the L4 and L5 students were asked how they felt about using social networking sites (SNS) ie
Facebook for University work and the results were totally opposite. Not one reply received from the
L4 cohort was in favour of using SNS in this way, yet all of the L5 cohort thought it was a good idea as
most students used it and would check it regularly, a couple were surprised that it was not in use
already.
Previous research and observations from conferences indicates that there is also a conflict of opinion
among academics as to whether the use of SNS for teaching or interacting with students is productive
or whether it is just encroaching “on their space” (Smyth, 2010).
2.2 The Challenges
There are a number identified at this time, not only in relation to the students and how they will
manage or cope with this strategy but also in relation to colleagues who will also be required to
undergo a process of change.
For colleagues it will be about participating in action learning which requires us as a set (team) to
collaboratively work on a real issue: the development of the new curriculum; understand a new
strategy: EBL; while reflecting on what we as individuals have previously experienced and can share
with this set, in order to drive this initiative forward (McGill & Brockbank, 2006).
For the Faculty (BBS) it will be to offer support maybe from colleagues already familiar with this
strategy or with time for research and maybe changes to the learning environment.
A specific challenge for the Course Team will be the accrual, selection and design of appropriate
problems and materials to enable the students to achieve a more in-depth knowledge of the concepts
and principles of the course, while bringing about a change in students from passive to active
learners.
Otting et al (2010) reviewed the relationship between students’ epistemological beliefs and
conceptions of teaching and learning and found that students do recognize the importance of selfdirected
learning, but observations of our current students found that they are constantly seeking
reassurance that they are “doing it right” because they expect there to be a prescriptive criteria for
getting good marks; as well as an unwillingness to do more than is required on the assessment brief.
Earlier statements in this report refer to the levels of motivation and intellectual maturity but the
Course Team need to reduce the level of dependency and instead find ways to encourage, instil
confidence in them to try it for themselves to see what happens and thereby persuade the students to
understand and accept the concept that there is no one right answer.
58

Andrea Benn
Duch et al (2001) suggests that it may be the way that we teach that reinforces a view of learning
which is that the teacher is responsible for delivering content and the students are the passive
receivers of knowledge. Duch continues to argue that the way we test, the expectations we set and
the learning materials we use may also contribute to the level of student motivation and intellectual
maturity.
By making the assessment brief so prescriptive are we restricting the students’ natural inquisitiveness
as well as telling them what we expect the answer to be? These views are seen as a very good
reason to radically review and change what we do in order to bring about the change we are looking
for/expecting from UG students.
2.3 The Resources
Continuing with the literature review, Duch et al (2001) offers some practical advice which has greatly
influenced the design suggestions for writing the problems which will be forwarded to the Course
Team for discussion:
The first question to ask in this situation is: what should the students know; value and be able to do by
the end of the course? Followed by what evidence will indicate that they have reached these goals.
The advice continues that the problems need to be written as open-ended, real-world problems which
will require a certain amount of creativity. At this point, links with the local business community can be
created as professionals in the field could be consulted for ideas of a more realistic nature for
application of the concepts of the course. Alternatively, news articles, journals, current events can be
used and adapted for the purpose.
The problem also needs to provide opportunities for the students to be able to demonstrate the
underpinning skills previously identified. Duch et al (2001) identify five characteristics considered to
be important for a good PBL problem but recognises also that they may vary between disciplines.
Those most appropriate to this course are:
It must engage students’ interest and motivate them to probe for deeper understanding of the
concepts introduced;
Ideally requires the students to make decisions or judgements based on facts, information, logic
and/or rationalization;
It should be complex enough to necessitate cooperation from all members of the group to
effectively work toward a solution;
The initial questions should be open-ended based on previously learned knowledge and/or be
controversial to draw all students into a discussion;
The content objectives need to be incorporated into the problems, connecting previous knowledge
to new concepts.
The intention at BBS is for the whole course to be delivered as EBL therefore it could provide more
time for the students to adapt to this process as they progress. It is of course the freshers’ year that
will require more support and guidance, therefore Duch et al (2001) suggest that the problems be
delivered in stages, allowing the students to cooperate, to make assumptions based on the
information they have and decide what more they need to request. On the problem brief the learning
objectives stated should include those beyond the content objectives and should be more complex
and involve process skills objectives too. Eventually, as they become more familiar with this process
and their knowledge and experience grows, this could also help the students to identify the direction
they need to work towards.
Linking this again to QAA level descriptors, the authors need to ensure that there is sufficient
distinction between the stages particularly as students progress from level 4 to level 5 and level 6.
Reviewing how technology can assist with this process, Savin-Badin and Wilkie (2006) explore the
concept of a fully online PBL strategy stating that it is an approach to learning that is both varied and
flexible. A fully online strategy is not the main objective here; but to find a way that technology can
support the principal strategy of EBL, their findings provide a good basis for the decisions for this
purpose.
59

Andrea Benn
Technology is already integrated into the existing course to varying degrees but is entirely subject to
the module tutor’s experience, innovation and aspiration to try new approaches. The technology
needs to complement the principle for the new course to be student-centred. The students will work
in teams and will have to work collaboratively to achieve their shared goal. Communication will
therefore form a major part of this collaboration but may not necessarily take place face to face.
Students too, will be required to leave the University premises for periods of time, and if not timetabled
to be together may find that their commitments to the activity in terms of time and hours spent
may not coincide with everybody else. Using the technology as an asynchronous tool may assist the
students here but does this tool need to be supplied by the University? It begs the question do we
want to or need to see the learning process exactly as it unfolds or are we happy to see the end result
in whatever format that is determined via the course assessment strategy? The issue of whether to
join the students in the environment of their social network sites need not arise, particularly if we are
timetabled to meet with the students on a face to face basis during the scheduled tutorial sessions
which would be required as part of the EBL strategy. It also puts the responsibility for this part of the
learning process onto the students, and in particular forming the virtual and actual group dynamics,
resolving conflicts and division of labour, just as they would in a social environment and without the
interference from the tutors.
It is here that the issue of assessment and in particular formative assessment may arise. The
students will be required to reflect as part of this learning strategy and one way this can be achieved
through IT is via the ‘journal’ facility currently available in the UoB’s virtual learning environment
(VLE). It can offer the students the opportunity to evaluate their own learning and because the tool
can be restricted to collaboration between one individual student and the tutor, it can offer 1-1
feedback which would be entirely relevant to each student and timely at the point that it is required.
Either online or with pre-arranged 1-1 tutorials the student can discuss specific points of concern. At
a recent UoB symposium relating to feedback and assessments, students advised that they needed
instant feedback and that waiting for 2 weeks or longer is of no use – the moment has gone.
The VLE that UoB has currently invested in is primarily used as more of a repository of information
provided by the tutors. Savin-Baden and Wilkie (2006) warn that some online environments can be
over managed by the tutors but there is scope on the VLE to permit the students to upload information
via the group forums/discussions.
Tuition to demonstrate exactly how this learning tool can be used should be offered during the early
stages of the course, and maybe now it is time to look to the Web 2.0 technologies that students will
experience in the field to see how businesses share documents and information and work across time
and geographical zones and prepare for the limited occasions when they will meet face to face. The
inference on this latter point is that the time spent face to face is to be regarded as precious and
should therefore be planned for to ensure that the time is maximised. This is another expectation that
the tutors need to advise students of from the outset.
Some information will still be disseminated by tutors or professional guest speakers via lectures or
electronic methods but the drive for information needs to come from the students, discovered and
shared between them but also allowing for peer feedback, amendments and additions.
2.4 Transition
Before looking at the transition period for the students it is as well to consider the transition period for
the tutors. Savin-Badin & Wilkie (2006) warn that for many there will have to be a recognition of a
loss of power and control because it will be about letting go of decisions about what students should
learn and instead trusting the students to acquire knowledge for themselves and accepting that
students will learn even if they have not been supplied with a lecture or a handout from their tutor.
Given the observations and statements mentioned earlier in this report, the transition period needs to
be carefully planned and supported allowing colleagues time to discuss and share ideas and to reflect
and evaluate.
Discussions with colleagues over the years have found that for many of us, the best teaching
experiences are those when we are presenting information for the first time, ie at the cutting edge of
our knowledge. Perhaps this is because we are yet to become comfortable with the new information
and so we take more time in preparing how we disseminate it. So, this may also be true with this new
60

Andrea Benn
pedagogical strategy, in order to fully appreciate how we want the students to react so we have to
experience the same process for ourselves, to understand the concerns, fears and high points that
the students will go through.
The literature review finds that careful planning and consideration of how the problem is introduced to
the students will enable them to identify the learning issues and therefore which theory concepts need
to be researched. As a Team the planning needs to include the structure of the problem, how many
stages it should be broken down into, over how long a period of time it will take to complete, the
resources that need to be made available as and when the students request them and what the end
product is going to look like before moving onto discuss the format of the tutorial sessions with the
students to facilitate discussion and progression of the task. For this a suggested model as an initial
guide has been provided by a UoB colleague based on an example illustrated by Barrows & Tamblyn,
1980 and is shown below:
Table 1: Facilitating Initial Discussions:
1
INFORMATION
What facts do we have?
What do we already know?
What is our experience of this situation?
Move to: what do we still need to find out?
3
QUESTIONS
What do we need to learn/understand before we
can deal with/improve the situation?
2
IDEAS/HYPOTHESES
Define the situation/the “problem” strengths,
negatives
Move to: how to manage the situation in light of
the present understanding?
4
ACTION PLAN
How/where will we learn these things?
What resources will we need/use?
This model can be adapted to support the task within our discipline and help to create an environment
where discussion and brain storming can be promoted and deeper exploration of the topic developed.
Turning to the student experience, Parrott (2009) warns that the first year at UG level represents a
major challenge for students, acknowledging that students can benefit from greater levels of support
during the early stages of the course. It is essential to offer the students a carefully planned
transition period. The Course Team will need to consider what our expectations of the students will
be for this new strategy which will have to be explained, demonstrated, reflected on and reiterated
through an active transition period. The expectations will also include ‘course rules’ explicitly informing
students of their tutor’s availability, whether online or face-to-face and the preparation required from
them for all tutorials.
The ideas already under discussion for the new course include students from the higher levels
offering peer support to the newer students through mentor schemes already in operation with UoB
but also during the course of their academic year by presenting research that they have undertaken.
Demonstration of the IT tools that will be used through the course should be provided at this stage ie
any discussion or group forums and in particular the journal facility where the concept of reflection
should be introduced.
3. Conclusion
The reason for exploring alternative learning and teaching strategies is to bring about a change in the
students in their approach to their learning which will last beyond their academic careers. The ultimate
aim is to change the students’ behaviour from being passive to active learners.
The literature review argues that the reason for the way the students currently behave is because of
the way they are taught and the expectations we as tutors are portraying. It would therefore suggest
that the change initially needs to come from us and our approach to our teaching and the resources
we wish to utilise in doing so.
This change however, will not be without some substantial challenges that the teaching team will
encounter not only from the students’ perspective but their own too. It will require the teaching team
to let go of the responsibility for providing the information while relying on the co-operation of the
students to find out for themselves, albeit with guidance and support from the teachers.
61

Andrea Benn
To bring about the co-operation of the students, they are going to have to want to engage with the
programme, be interested in the topic and see some value to them in it. It will also be about
managing the students’ expectations in relation to their new learning environment as most will
probably have experienced a more teacher-led approach so time and patience must be invested in
explaining what a student-led framework will look like. Demonstrating the skills to cope with this will be
a priority and a carefully planned transition period will be essential for a group of level four UG
students.
Co-operation and collaboration across the Course Team will be essential to ensure that no learning
outcomes are left uncovered or are doubled-up particularly as this is now an holistic approach to the
programme. This experience will help the Course Team to reflect on the course progression and to
give the opportunity to anticipate the needs and possible fears of the students.
Finding the right blend is not just about finding the right mix of IT to include but choosing the right IT to
complement the learning strategies and support the students in their progression through the course,
particularly through collaboration with each other and with the Course Team. The Course Team will
need to continue to be innovative with their application of any technology to enhance their teaching or
dissemination of information, but care needs to be taken not to over manage the learning environment
when implementing IT. Some aspects of the course may be offered fully online other aspects may
provide an opportunity for personal reflection and tutorial feedback. The collaborative elements for
this pedagogical strategy perhaps need to be managed by the students themselves with an invitation
to the tutors to join them if they desire it. This approach may give the control to the students to
develop a group working relationship between them, leaving the time allocated to tutorial or seminars
to focus completely on progressing the task, presenting to or receiving feeding from the tutor.
It will without doubt be a challenge but one that should prove to be exciting and rewarding for all
stakeholders. The timing for the change is also right.
References
Benn, A., (2011) Changes Facing Higher Education – Opportunity or Challenge? Exploring Alternative Learning &
Teaching Practices for Undergraduates ICICTE 2011 Annual Conference, Rhodes, 7-9 July
Boud, D., Feletti, G., (2001) The Challenge of Problem-Based Learning,2 nd Edition, London: Kogan Page
Brudermann, C., (2010) From Action Research to the Implementation of ICT Pedagogical Tools: Taking Into
Account Students’ Needs to Propose Adjusted Online Tutorial Practice, European Association for Computer
Assisted Language Learning, 22(2): 172-190
Cox, M. J., San Diego, J. P., Quinn B., Hindmarsh, J., Dunne S., Newton T., Woolford, M., Reynolds, P., Green,
T-A., Shahriari-Rad, A., Hyland, L., Banerjee A., (2010), University Teachers’ Pedagogical Practices in
Using Technology-Enhanced Learning: strategies to integrate haptic technologies into undergraduate
teaching, p1-6, BERA 2010 Annual Conference, University of Warwick, 1-4 September
Duch, B.J., Groh, S.E., Allen, D.E., (2001) The Power of Problem-Based Learning – A Practical “How To” For
Teaching Undergraduate Courses In Any Discipline, USA: Stylus Publishing LLC
Laudon, K. C. and Traver, C. G. (2009) E-Commerce Business Technology Society 5 th Edition, London: Pearson
McGill, I., Brockbank, A., (2006) The Action Learning Handbook – powerful techniques for education,
professional development & training, Oxon: Routledge Falmer
Moon (nee Benn), A., (2010) Exploration and Experimentation: applying new technologies to learning – 2 nd stage
action research, BERA 2010 Annual Conference, University of Warwick, 1-4 September
Otting, H., Zwaal, W., Tempelaar, D. and Gijselaers, W. (2010) The structural relationship between students’
epistemological beliefs and conceptions of teaching and learning, Studies in Higher Education, 35(7), 741-
760, First published on 7 October 2010
Parrott, G., (2009) Redesigning the first year business curriculum at the University of Bedfordshire” International
Journal of Management Education 8(2): 13-21
Sadlo, G., Piper, D. W., Agnew, P. (1994), Problem-Based Learning in the Development of an Occupational
Therapy Curriculum, Part 1: The Process of PBL British Journal of Occupational Therapy (0308-0226),
57(2), 49-54
Savin-Baden, M., Wilkie, K., (2006) Problem-based Learning Online, Maidenhead: OU Press
Smyth, K. (2010), Our Space or Theirs? ICT Skills Briefing, Issue 14
Yeh, Yu-Che., (2010) Integrating Collaborative PBL with Blended Learning to Explore Preservice Teachers’
Development of Online Learning Communities, Teaching and Teacher Education, 26: 1630-1640
62

Critical Success Factors for the Adoption of eLearning in
the Kingdom of Saudi Arabia Educational Institutions
Latefa Bin Fryan and Lampros Stergioulas
Department of Information Systems and Computing, Brunel University,
London, UK
Latefa.Fryan@brunel.ac.uk;
Lampros.Stergioulas@brunel.ac.uk
Abstract: In recent years, Electronic Learning (eLearning) has gained interest and has been widely adopted and
acknowledged; it is integrated in many institutions' educational systems. Owing to the major transformation of
traditional educational system in the Kingdom of Saudi Arabia (KSA), most of higher educational institutions are
currently switching to eLearning system and the rest are expected to switch in the near future. Nowadays there is
a universal need to investigate such Critical Success Factors (CSFs) that have significant impact on the adoption
of eLearning. The aim of this paper is to identify and prioritise, in order of their importance, the Critical Success
Factors (CSFs) for the adoption of eLearning in educational institutions of the Kingdom of Saudi Arabia (KSA).
Therefore, the chosen area of research for this paper holds importance in terms of considering the CSFs before
and during the adoption of eLearning process especially at this time when KSA is ready to prepare for this
educational reform. Although the significance of eLearning cannot be denied and its impact on the improvement
of the traditional educational system is well-recognised, there are still gaps reported in the literature for the
adoption of eLearning. In order to fully realise the potential of eLearning and understand the challenges in its
adoption, it is imperative to identify the CSFs that affect the adoption of eLearning in KSA educational institutions.
In order to do so, this paper identifies key success factors that affect the adoption of eLearning through extensive
analyses of case studies and the existing literature. For the case study analysis, five KSA educational institutions
are investigated through interviews and questionnaires; for the purpose of literature review, data is collected from
different resources such as journals, books, conferences papers, newspaper articles, magazines, etc. The
research gap is identified so as to highlight the importance of this research. Additionally, the features and benefits
of eLearning are discussed and the CSFs for eLearning adoption are identified and presented in a table with
respective categorisations. The interpretive, qualitative data analysis methods are used, and findings are presented
in tables and figures. Findings show that of the total 52 CSFs identified, 25% were highlighted only through
interviews and survey questionnaires and were not found in the existing literature.
Keywords: eLearning, adoption of eLearning, distance learning, critical success factors, higher educational
institutions, education in Kingdom of Saudi Arabia, educational reform
1. Introduction
Technology has played a crucial role for the enhancement of learning. Nowadays, online learning
programs have significant value in the educational process due to the growing trend of globalization,
technical innovation, increasing competition, and rising student expectations (Rovai and Downey,
2010). Recent developments in Information System (IS) and Information Communication
Technologies (ICT) have led to the facilitation of learning and teaching process. eLearning, also
known as online learning, refers to the use of ICTs to enhance and/or support learning in the
educational process (Al-Jaghoub et al, 2009; NZCER, 2004).
The use of technology in the educational process is recognised globally. Availability of eLearning
provides equal opportunities for learners who may be deprived from such opportunities of learning (Al-
Jaghoub et al, 2009); and use of digital technology has positive impact on the learning process
(Underwood, 2009). The global demand for seats in higher education will double to 200 million by
2020 which will increase the number of universities providing distance learning (Rovai and Downey,
2010); this illustrates a positive trend in the uptake of eLearning. Sun et al. (2008) reported a growth
rate of 35.6% for the eLearning market. Due to increasing competition among various education
providers (specifically higher education), eLearning is being adopted in order to increase their number
of students both nationally and globally (Rovai and Downey, 2010).
There are several benefits of eLearning including location and time independence; students prefer
eLearning as compared to the traditional face-to-face learning methods (Kelly et al, 2007). It facilitates
the provision of education to large communities at a very low cost by reducing (or eliminating) the
number of classrooms, staff, or hard copies of teaching materials needed (Papp, 2000). eLearning
supports both individualised and collaborative learning (Suddaby and Milne, 2008; Bell, 2007; Volery
and lord,2000), and compensates any deficiencies in the traditional learning system (Hsbollah and
Idris, 2009).
63

Latefa Bin Fryan and Lampros Stergioulas
When comparing eLearning with the traditional classroom learning, one can find pros and cons for
either case. Zhang et al. (2004) assert that: eLearning offers a learner-centred approach, while the
conventional classroom learning is less centred on the tutor and puts more emphasis on the
instructor. Zhang et al. (2004) also demonstrate that the eLearning offers more advantages than the
traditional classroom learning, which implies that the new technology is changing the way of learning
and constitutes a real alternative to conventional classroom learning. However, Russell (2001)
indicates that there is no significant difference between eLearning and traditional learning and offers
considerable evidence that technology does not depreciate teaching.
Many countries are adopting eLearning for the purpose of improving educational standards,
processes, and to overcome time, space or resource limitations. Abouchedid and Eid (2004)
conducted a survey about the challenges for eLearning adoption in the Arab world and commented on
the need for change in the attitudes of instructors towards the adoption of eLearning, however he also
argues that it is imperative for the Arab world to adopt eLearning in order to compete with the
educational systems of the developed countries of the world. Pursuing this further, this research
focuses on Kingdom of Saudi Arabia (KSA) for investigating and analysing the adoption of eLearning
in their educational system.
2. Background: Concept and adoption of eLearning in KSA educational
institutions
Kingdom of Saudi Arabia (KSA) has population expanding at 3.7% to 4.0% per year (Field, 1994;
Long, 1997), with 16.9 million in early 1900 and expected to reach 25 million by 2011 (Saudi Arabian
Information Resource, 2007). This rapid growth in population is expected to exert a huge pressure on
the country’s educational systems creating increasing demands for technological solutions (Albalawi,
2007). Since majority of KSA’s population is young requiring more jobs and good education (Al-Faisal
et al., 2010). The educational system in KSA is struggling to cope with increasing demands of growing
population and diversifying economy with student to teacher ratio of 15:1, which is one of the lowest in
the world (ameinfo.com, 2009). Since 1998, the number of universities in KSA has increase from 7
public universities to 17 public universities and 24 private colleges, containing almost one million
students (Al-Faisal et al., 2010). In order to enable KSA educational system to cope with the 21 st
Century developments, it is essential to improve the system by using ICT in learning and
management and by increasing cooperation with local and international partners (Al-Faisal et al.,
2010). KSA Government is undertaking initiatives for educational system reform to improve quality
and efficiency of learning at all levels by allocating SR137 billion ($36.5 billion) for education in
country’s 2010 budget. (Al-Faisal et al., 2010); examples of these reforms are curriculum
development, ICT and eLearning strategy, and partnering with foreign schools. Universities and
colleges in KSA vary in their levels of technology advancements and developments; for instance,
some universities are less developed in terms of using technology or ICT (i.e. limited to a few
numbers of computers which can only be used by faculty members), while others are highly equipped
with technology such as King Abdullah University of Science and Technology (KAUST) which
operated in 2009, with cost of $10B at estimated area of 36 million meters square (KAUST, 2010).
Owing to the rapid population growth, lack in the number and quality of teachers and instructors, and
to adhere with KSA Government’s initiative for the utilization of ICT, the National Centre of eLearning
and Distance Learning (NCEL) was established in 2006, with an aim to facilitate the next generation
eLearning across higher education sectors in KSA and to organise the change and prepare eLearning
material (NCEL, 2010). The goals of NCEL (NCEL, 2010) are: extending the use of eLearning
applications in higher educational institutions; participating in increasing the capacity for institutions by
using eLearning system; improving awareness of the benefits and importance of eLearning; holding
seminars, workshops and conferences; building educational software applications; and supporting
researches in the field of eLearning and distance learning. NCEL provides training to various
university staff through workshops. NCEL encourages its trainees to use eLearning system as some
of them are resistant to use it.
Resulting from the major transformation of traditional education in KSA, most educational institutions
are switching to eLearning system; some universities have already started implementation of
eLearning system while others are considering this now.
64

3. Motivation of research - The gap
Latefa Bin Fryan and Lampros Stergioulas
Although the significance of eLearning cannot be denied and its impact on the improvement of the
traditional educational system is well-recognised, there are still gaps in the adoption of eLearning.
Reasons for this include: lack of understanding about the success factors of eLearning, and
neglecting the factors influencing lecturers to adopt the internet as a teaching tool (Hsbollah and Idris,
2009). Existing literature reports that considerable amount of work has been done on the benefits or
features of eLearning and about eLearning implementation, and several studies have been conducted
for the adoption of eLearning listing many CSFs; however, the lists are not comprehensive; for some
CSFs student’s perspectives alone are taken into consideration while the others merely consider the
education provider’s perspectives. Due to the lack of research into this area, further research is
needed for investigating and identifying such CSFs that would lead to a successful adoption of
eLearning. In order to fully realise the potential of eLearning and understand the challenges in the
adoption of eLearning, it is imperative to identify the Critical Success Factors (CSFs) that affect the
adoption of eLearning in KSA educational institutions.
The purpose of the planned research is to address this gap and present a comprehensive list of such
CSFs that influence the adoption of eLearning in KSA educational institutions.
4. eLearning features and benefits
Electronic learning (eLearning) can be defined as a way of learning that is facilitated and supported
through the use of information and communications technology (NZCER, 2004; Al-Jaghoub et al,
2009). eLearning is a web-based educational system that uses IT and computer networks; it widely
utilises modern technology, internet, electronic media, or web-based applications to deliver the
ultimate learning experience (Selim, 2007; Hsbollah and Idris, 2009). It is one of the most significant
recent developments in the information system industry (Wang, 2003) and is considered as an
appropriate means of providing education for universities which lack enough staff, study material, or
classes. With the adoption of eLearning there is no need to minimise the number of student in each
course. eLearning is a major transformation of traditional education provision to more modern,
effective and efficient alternative educational methods (Freire, 1994 Selim, 2007). It has been
reported in the literature that eLearning outranges the traditional learning methods in terms of courses
and student performance (Beyth-Marom et al., 2003). Moreover, students prefer to use eLearning
because it makes their learning more effective, efficient, and flexible; i.e. they can study anytime,
anywhere, and in their own way (Papp, 2000). eLearning supports both individualised and
collaborative learning (Suddaby and Milne, 2008; Bell, 2007; Volery and lord,2000), and compensates
any deficiencies in the traditional learning system (Hsbollah and Idris, 2009).
The effectiveness-level increases as we shift from distance learning to eLearning (as shown in Figure
1). eLearning has also been used interchangeably or with partly overlapping meanings as computer
mediated learning, advanced distributed learning (ADL), online learning (OL), networked learning,
virtual learning, web-based learning (WBL), internet-based training (IBT), web-based instruction
(WBI), open/flexible learning (OFL), and digital learning (Khan, 2001; NZCER, 2004; Selim, 2007).
While eLearning can be used as part of distance learning programs, it does not necessarily have to
involve distance learning (NZCER, 2004).
Several benefits of eLearning have been reported in the literature, the key benefits are locationindependent
and time-independent learning (Fayter, 1998; Homan and Macpherson, 2005; Bell,
2007). Using IT in education not only facilitates the access to education and training but also
improves teaching and learning quality (Newton, 2003). Through eLearning, the delivery of
educational programs to more students can be realised at a much lower cost (Peled, 2000; Volery
and Lord, 2000). Moreover, using IT in education has added value to the existing teaching
methodology (Broad et al., 2003). eLearning makes it possible to continue studies in parallel with job
or work due to it is flexible nature in term of time and location independence. The quality for eLearning
needs to be maintained and the importance of quality cannot be compromised (Inglis, 2008).
Existing literature highlights gaps in the adoption of eLearning and reports that despite considerable
amount of work done in this area, there is still lack of understanding about the success factors of
eLearning (Hsbollah and Idris, 2009). Most studies present the Critical Success Factors (CSFs)
listings for the adoption of eLearning; however, the lists are not comprehensive. In some studies
student’s perspectives alone are taken into consideration (Selim, 2007; Sun et al., 2008; Underwood,
65

Latefa Bin Fryan and Lampros Stergioulas
2009) while the others merely consider the education provider’s perspectives (Volery and Lord, 2000;
Bell, 2007; Hsbollah and Idris, 2009).
Distance
Learning
eLearning
LEARNING
More effective
LEARNING
eLearning
Figure 1: Effectiveness of learning
Due to the lack of research into this area, further research is needed for investigating and identifying
such CSFs that would lead to a successful adoption of eLearning. In order to fully realise the potential
of eLearning and understand the challenges in the adoption of eLearning, it is imperative to identify
the Critical Success Factors (CSFs) that affect the adoption of eLearning in KSA educational
institutions.
5. Critical success factors for eLearning adoption
Literature suggests that there has been enormous research conducted on the adoption of eLearning
and several CSFs have been identified. There are two perspectives to the adoption of eLearning, one
is the students’ perspective and the other is the education providers’ (or teaching staff) perspective.
Since, it is important to consider both these perspectives while adopting and implementing eLearning,
therefore this research considers both perspectives.
Bell (2007) highlighted cost effectiveness and time-independent as the CSFs for the adoption of
eLearning, and Homan and Macpherson (2005) reported flexibility in term of the time of learning, the
way of learning, where you learn as the CSFs for eLearning adoption. Johnson et al. (2009) argued
that technology can play a key role in the effectiveness of eLearning. Volery and Lord (2000) defined
four categories of the CSFs for the uptake of eLearning by education providers:
Expanding access
Alleviating capacity constraints
Capitalising on emerging market opportunities
Serving as a catalyst for institutional transformation
In another study, Hsbollah and Idris (2009) conducted a survey from 244 lecturers of University Utara
Malaysia and identified the following five CSFs for eLearning adoption:
Relative advantage
Compatibility
Complexity
Trialability
Observability
However, the CSFs for all of the above mentioned studies do not cover the student’s perspective in
terms of what makes eLearning adoption a success from student’s point of view. Selim (2007)
conducted a survey of 538 university students and highlighted 8 categories of eLearning CSFs; each
66

Latefa Bin Fryan and Lampros Stergioulas
of the categories contained several critical eLearning acceptance and success measures which are
as follows:
Instructor’s attitude towards and control of the technology
Instructor’s teaching style
Student motivation and technical competency
Student interactive collaboration
eLearning course content and structure
Ease of on-campus internet access
Effectiveness of information technology infrastructure
University support of eLearning activities
However, these CSFs are taken only from the student’s perspective and are lacking the education
provider’s perspective which is equally essential to be considered in order to realise the potential for
eLearning adoption. Underwood (2009) discussed the following CSFs for education with digital
technology (i.e. eLearning):
Increased student effectiveness or performance
Increased student efficiency
Increased student engagement or satisfaction
More positive student attitudes twards learning (student’s motivation)
However, these four factors focused only on the student’s performance irrespective to the general
CSFs (i.e. it lacks several CSFs that in general are significant to the adoption of eLearning such as
time and location independence).
Another study (Papp, 2000) presented CSFs for distance learning using electronic means (i.e. via
eLearning), these factors include:
Security of Intellectual Property
Suitability for distance learning environment
Familiarity with the courses
Course contents
Mechanisms for troubleshooting
Sun et al. (2008) conducted a survey and presented seven CSFs for the satisfaction of an eLearning
user (or student):
Student computer anxiety
Instructor attitude toward eLearning
eLearning course flexibility
eLearning course quality
Perceived usefulness
Perceived ease of use
Diversity in assessments
However, these lists are not comprehensive and lack several important aspects such as effectiveness
(or impact) of eLearning in terms of performance. Rovai and Downey (2010) discussed the major
seven factors causing the failure of many distance education programs:
Planning
Marketing and recruitment
Financial management
Quality assurance
Student retention
67

Faculty development
Online course design and pedagogy
Latefa Bin Fryan and Lampros Stergioulas
However, these factors are not sufficient as they neglect other important learning aspects such as
time and location independence focusing mostly on the management-type of CSFs; on the other hand
Park and Wentling (2007) presents another set of CSFs that focuses on the CSFs from the individual
or personal perspective, suggesting that the popularity of eLearning is due to the following factors:
Space flexibility
Cost savings
Timely access to educational resources
Workplace flexibility
Increased students interest
Quick response to the student
6. Findings and analysis
In order to: “Identify and prioritise the Critical Success Factors (CSFs) for the adoption of eLearning in
the educational institutions of the Kingdom of Saudi Arabia (KSA)”. The data was collected through
primary and secondary data sources and then analysed using the interpretative qualitative analysis.
Findings from the secondary data collection are already presented in the Literature Review.
The primary data were collected through conducting semi-structured interviews and a questionnairebased
survey in order to extract the CSFs for the adoption of eLearning in KSA. For this purpose,
FIVE different KSA educational institutions including universities and eLearning training centres were
considered. The participants were chosen according to their related jobs in the area of eLearning in
KSA to ensure high quality and consistent findings. The questions for both the interview and the
survey were designed keeping into consideration the literature surveyed about the adoption of
eLearning. Findings from the interviews and the survey were prioritised according to their significance
in terms of their average or cumulative percentage importance from five educational institutions (the
repetition of each CSFs).
The data collected from both sources was interpreted and qualitatively analysed. The CSFs found
from the primary and secondary data collection were listed in individual tables and then mapped to
each other in another table in order to show the relationships amongst various CSFs; finally the
findings were analysed using Miles and Huberman (1994) scale, and the CSFs were ranked in order
of significance. The resulting CSFs are presented in table 1.
Findings from the interviews and survey questionnaires confirm the anecdotal evidence reflected in
the literature. Most of the CSFs found through primary data collection are common to the ones that
are identified through secondary data collection (literature review), hence confirming the significance
of CSFs identified. However, there are some CSFs that were only identified through primary data
collection and were not found in the literature (presented in Table 2). These CSFs that are identified
only through interviews and survey questionnaires are a value-add provided by this paper as they will
form basis for future research in the area of eLearning adoption.
Findings from primary data and secondary data are shown in Table 1, which presents a list of CSFs
for eLearning adoption, elicited from the existing literature as well as interview and questionnaire
conducted in KSA. The CSFs are listed in four categorisations: Individual; Social; Economic; and
Governmental or Organisational. The individual aspect encompasses both learner and instructor.
These four categorisations help in understanding the different aspects of eLearning adoption
From secondary data 39 CSFs were identified; from primary data 46 CSFs were identified. Most of
the CSFs identified are common in both sources; 25% identified through primary data only; and 12%
identified through secondary data only (presented in Table 3). Finally, a total of 52 CSFs split into four
categories presented in the table 1.
68

Latefa Bin Fryan and Lampros Stergioulas
Table 1: Findings from primary data and secondary data
No. Categorisations CSFs for eLearning Adoption in KSA Educational Institutions
1.
2.
3. Individual
Accuracy
Attendance not compulsory
Attractiveness
4. Compatibility (the degree to which the technology is easy to understand and
use)
5.
Ease of use (user-friendly interfaces)
6.
Effectiveness
7.
Efficiency
8.
eLearning course content and structure
9.
Encourages personalised experience (independent learning or self-study)
10.
Enhances learner's achievements
11.
Enriches learner's experiences
12.
Flexibility in term of time, location, and ways of learning
13.
Increased learner’s engagement or satisfaction
14.
Instructor’s attitude toward eLearning
15. Instructor’s attitude towards the adoption of technology (acceptance for the
technology)
16.
Instructor’s teaching style
17.
Instructors are able to ensure more practice for learners in different ways
18.
Instructors can share good practice (teaching methodology)
19.
Interactive way of learning
20.
Learner’s motivation for adopting new technology
21.
Learner’s technical competency
22.
Online technical support for providers and learners
23.
Positive trend of accepting eLearning by learner
24.
Reduces effort
25.
Rich and various resources
26.
Time saving
27. Trialability (the degree to which lecturers can test the technology before
adopting)
28.
using internet in the classroom
29. Social
Adds value to the existing teaching methodology
30.
Ease inter and intra communication
31.
Equal learning opportunities for all learners
32.
Facilitates both individualised and collaborative learning
33.
High quality of teaching and learning
34.
Improved education process
35.
Improves access to education and training (accessibility)
36.
Improves teaching and learning performance
37.
Increases inter and intra communication and discussion
38.
knowledge enhancement
39.
Modern (cope with the development)
40.
Observability (the degree to which the results of technology are visible)
41.
Security of Intellectual Property
42.
Serves as a catalyst for institutional transformation
69

Latefa Bin Fryan and Lampros Stergioulas
43.
Technology innovation
44.
Wide acceptance of distant learning
No. Categorisations CSFs for eLearning Adoption in KSA Educational Institutions
45
46.
47.
48.
49.
50.
51.
52.
Economic
Governmental
organizational
Alleviating capacity constraints in terms of classrooms and staff
Delivering educational programs to more learners at a lower cost
Scalability in terms of learners’ volume
Capitalising on emerging market opportunities
Establish partnerships with national and international universities
Facilitate and support Distance Learning
Internationally recognised
University's support for eLearning activities
The contribution of this research is a comprehensive list of CSFs for eLearning adoption, identified
through primary and secondary data collections methods, which is significant to this area of research.
Of the total 52 CSFs identified, 25% were highlighted only through interviews and survey
questionnaires and were not found in the existing literature. These new CSFs are:
Table 2: CSFs identified in primary but missing in secondary
No. Categorisations CSFs missing from the primary data
1. Enhanced learner's achievements
2. Enriches learner's experiences
3.
4.
Individual
Interactive way of learning
Instructor’s teaching style
5. Positive trend of accepting eLearning by learner
6.
Reduces effort
7. Ease inter and intra communication
8.
9.
Social
Improved education process
Knowledge enhancement
10.
Wide acceptance of distant learning
11. Economic Scalability in terms of learners’ volume
12. Governmental/
Establish partnerships with national and international universities
13. Organisational International recognition
Some CSFs that were identified through secondary data but were missed in the primary data are
presented in Table 3.
Table 3: CSFs identified in secondary but missing in primary
No. Categorisations CSFs missing from the primary data
1.
2.
Individual
Instructors can share good practice (teaching methodology)
Trialability (the degree to which lecturers can test the technology before
adopting)
3.
4.
Social
Observability (the degree to which the results of technology are visible)
Security of Intellectual Property
5. Governmental/
Capitalizing on emerging market opportunities
6. Organisational
University's support for eLearning activities
7. Reflection on primary data findings and analysis
According to the interviews and the survey conducted in KSA, a prominent inclination observed from
all the academic staff was towards the positive acceptance of eLearning with the main focus on:
improved quality of teaching and learning; increased number of students; and need to cope with 21st
Century development. Undoubtedly, using technology to enhance the educational process will help
KSA to achieve this major transformation of the traditional education. Furthermore, adoption of
eLearning within the educational processes adds value to the existing methods of traditional learning
and facilitates preparing student to the complex economic society. KSA is in the initial stages of
70

Latefa Bin Fryan and Lampros Stergioulas
adopting eLearning in educational institutions compared to other developing countries; however, the
transformation process is quite rapid in order to achieve the goal of technology-enhanced learning.
KSA educational institutions seem to believe strongly that eLearning will help to solve education
issues resulting from rapid population growth and the technology-intensive demands of the future;
moreover, eLearning will facilitate and support distance learning which is very common in KSA
universities.
8. Future research and development
For future research, the number of participant and educational institutions can be increased for
extensible research which will help to investigate more CSFs from different perspectives.
Some universities in KSA are already undergoing the process of implementing the eLearning system
and they are using student feedbacks as an essential resource to determine the ongoing quality and
to ensure improvement and refinement of the eLearning system. Considering this positive trend of
eLearning adoption in KSA educational institutions, findings from this research can prove to be a
significant tool for future enhancement and implementation of the eLearning systems in KSA.
9. Conclusion
eLearning adoption in KSA educational institutions has been considered as a huge and major
transformation from traditional educational system. Although the significance of eLearning cannot be
denied and its impact on the improvement of the traditional educational system is well-recognised,
there are still gaps in the adoption of eLearning due to the lack of understanding about the CSFs for
eLearning adoption. In order to fully realise the potential of eLearning and understand the challenges
in the adoption of eLearning, it is imperative to identify the CSF that affect adoption of eLearning in
KSA educational institutions.
For this purpose, the existing literature was thoroughly reviewed for identifying the CSFs for
eLearning adoption and important issues were explored. It was obvious that limited research has
been conducted in this area and a clear gap was identified through the literature reviewed, which led
to the need for investigating the CSFs for eLearning adoption. There are two perspectives to the
adoption of eLearning: students’ perspective and education providers’ (or teaching staff) perspective.
It was indicated through the existing literature that only some studies consider both perspectives and
are limited in terms of the comprehensiveness of the CSFs identified. Since, considering both
perspectives is imperative while adopting and implementing eLearning, therefore this research
considers both of them.
The primary data was collected through conducting semi-structured interviews and survey
questionnaires in order to extract the CSFs for the adoption of eLearning in KSA. For this purpose,
FIVE different KSA educational institutions including universities and eLearning training centres were
considered. The participants were chosen according to their related jobs in the area of eLearning in
KSA to ensure high quality consistent findings. . The interpretive, qualitative data analysis methods are
used, and findings are presented in tables and figures. Findings show that of the total 52 CSFs
identified, 25% were highlighted only through interviews and survey questionnaires and were not
found in the existing literature.
References
Abouchedid, K. & Eid, G. (2004),'eLearning challenges in the Arab world; revelations from a case study profile',
Quality Assurance in Education, 21(1) pp. 15-27.
Alcock, M. (2007),' Rapid eLearning software: what should we be looking for?', development and learning in
organizations , 21(6) pp. 15-17.
Al-Faisal, L. Al-Hakami, A. & Al-Sabti, K. (2010). Education: Saudi Arabia’s Commitment to its Future. U.S.-Saudi
Business Opportunities Forum, Chicago, Illinois. Available online: http://www.ussabc.org/i4a/pages/Index.cfm?pageID=3799
[last access 2010-11-03]
Al-Jaghoub, S. et al. (2009), 'eLearning adoption in higher education in Jordon: vision, reality and change'
European and Mediterranean Conference on Information Systems, Izmir
AME Info Website, 2009 http://www.ameinfo.com/219649.html [last access 2010-11-02]
Banyard, P. and Underwood, J. (2008). Understanding the learning space. eLearning Papers (9): 1-12. ISSN
1887-1542
Bell, J. (2007),'Feature articles ELearning: your flexible development friend?', development and learning in
organizations, 21(6) pp. 7-9.
71

Latefa Bin Fryan and Lampros Stergioulas
Beyth-Marom, R., Chajut, E., Roccas, S., & Sagiv, L. (2003), ‘Internet-assisted versus traditional distance
learning environments: factors affecting students’ preferences', Computers & Education, 41(1) pp.65–76.
Broad, M.J., Matthews, M. & Shephard, K. (2003), 'Audit and control of the use of the internet for learning and
teaching: issues for stakeholders in higher education', Managerial Auditing Journal, 18 (3) pp. 244-53.
Candy, P.C., Crebert, G. & O'Leary, J. (1994),’Developing Lifelong Learners through Undergraduate
Education’,National Board of Employment, Education and Training, Australian Government Publishing
Service, Canberra..
Fayter, D. (1998), 'Issues in training lecturers to exploit the internet as a teaching resource', Education + Training,
40 (8) pp. 334-339.
Field, M. (1994). Inside the Arab world. Cambridge, MA: Harvard University Press.
Freire, P. (1994). Pedagogy of the oppressed. London, UK: Continuum..
Homan, G. & Macpherson, A. (2005), 'eLearning in the corporate university', Journal of European Industrial
Training, 29 (1) pp. 75-90.
Hsbollah, H. & Idris, K. (2009) ‘eLearning adoption: the role of relative advantages’, trialability and academic
specialisation, 26(1) pp. 54-70.
Inglis, A. (2008) 'Approaches to the validation of quality frameworks for eLearning', quality Assurance in
Education, 16 (4) pp. 347-362
Johnson, R., Gueutal, H. & Falbe, C. (2009), 'Technology, Trainees, Metacognitive activity and eLearning
effectiveness',Journal of Managerial Psychology, 24(6) pp. 545-566.
Kelly, H., Ponton, M., & Rovai, A. (2007), 'A comparison of student evaluations of teaching between online and
face-to-face courses, Journal of Internet and Higher Education, 10(2) pp. 89-101.
Khan, B. H. (2001). A framework for web-based learning. Engelwood CliVs, NJ: Educational Technology
Publications.
Miles, M.B. & Huberman, A.M. (1994). Qualitative Data Analysis: A sourcebook of new methods. Sage
Publications.
Newton, R. (2003), 'Staff attitudes to the development and delivery of eLearning', New Library World, 104 (1193)
pp. 412-25.
NZCER. (2004). ’Critical Success Factors and Effective Pedagogy for eLearning in Tertiary Education’. New
Zealand Council for Educational Research (NZCER), Background paper for ITP New Zealand.
Papp, R. (2000),’Critical success factors for distance learning’ Paper presented at the Americas Conference on
Information Systems, Long Beach, CA, USA.
Park, J. & Wentling, T. (2007),' Factors associated with transfer of training in workplace eLearning', Journal of
Workplace Learning, 19(5) pp. 311-329
Peled, A. (2000), 'Bringing the internet and multimedia revolution to the classroom' , Campus-Wide Information
Systems, 17 (1) pp. 16-22.
Rovai, A. & Downey, J. (2010), ' Why some distance education programs fail while others succeed in a global
environment', Internet and Higher Education, 13 (3) pp. 141–147.
Selim, H. (2007). ‘Critical success factors for eLearning acceptance: Confirmatory factor models’ , computer &
Education, 49(2) pp. 396–413.
Suddaby, G. & Milne, J. (2008),' Coordinated, collaborative and coherent; Developing and implementing
eLearning guidelines within a national tertiary education system', Campus-Wide Information Systems, 25(2)
pp.114-122.
Sun, P., Tsai, R., Finger, G., Chen, Y. & Yeh, D. (2008), ' What drives a successful eLearning? An empirical
investigation of the critical factors influencing learner satisfaction', Computers & Education, 50 (4) pp. 1183–
1202.
Thomas L. Russell, The No Significant Difference Phenomenon, 5th ed. (Montgomery, AL: International Distance
Education Certification Center, 2001).
Underwood, J. (2009) 'The Impact of the Digital Technology; A review of the evidence of the impact of digital
technologies on formal education' Becta, Publication ID: BEC1-15790
Volery, T., Lord, D. (2000),' Critical success factors in online education', The International Journal of Educational
Management, 14(5) pp.216-223
Wang, Y. (2003). Assessment of learner satisfaction with asynchronous electronic learning systems. Information
& Management, 41(1) pp. 75-86.
Wang, Q., Zhu, Z., Chen, L. & Yan, H. (2009),' Guest Editorial eLearning in China', Campus-Wide Information
System, 26(2) pp. 77-81.
Zhang, D., Zhao, J., Zhou, L., & Nunamaker, J. (2004),’ Can ELearning Replace Classroom Learning?’,
Communication of The ACM,47(5) pp. 75-79
72

Challenges in Developing e-Submission Policy and
Practice
Alice Bird
Liverpool John Moores University, Liverpool, UK
a.bird@ljmu.ac.uk
Abstract: In the current economic climate, higher educational institutions are seeking strategies to enhance the
student learning experience and, at the same time, reduce administrative costs. Electronic submission of
coursework, e-submission, is frequently perceived as one approach that can help achieve both of these
objectives. However, this view fails to acknowledge the diverse needs of the key stakeholders (institutional
strategic management, academic staff and the student body) and the increased influence of the latter on
institutional policy development. This shift in powerbase imposes additional challenges for those responsible for
developing and implementing high stakes institutional change initiatives, such as e-submission policy and
practice. This paper begins by reviewing the background to developing and implementing electronic submission
policy and practice, in the context of institutional cultures and orientations to academic development. It
acknowledges increased influence of the student voice and enhanced difficulty in developing policy and practice
when mediating between three equally powerful stakeholders; strategic management, academia and the student
body. It offers an insight into the dynamics of these triumvirate influences through a three-year e-submission case
study at one higher educational institution. The paper charts the progress of the case through three phases
(feasibility study, pilot study and early stage implementation), highlighting the influences of the key stakeholders
on outcomes and attempts to gain consensus on policy and practice. The paper concludes by reflecting on the
experiences and findings from the case study and making recommendations on how other institutions might
approach developing similar e-submission policy and practice.
Keywords: e-submission policy, institutional change, institutional compliance, academic freedom, student voice
1. Introduction
In the current economic climate, higher educational institutions are seeking strategies to enhance the
student learning experience and, at the same time, reduce administrative costs. Electronic submission
of coursework, e-submission, is frequently perceived as one approach that can help achieve both of
these objectives. The affordances of web-based technologies and content repositories lend
themselves well to the administrative aspects of submission receipting, electronic feedback, online
moderation and overall rapid turnaround. However, this technological view of the process fails to
acknowledge the diverse needs of the key stakeholders; institutional management, academic staff
and the student body. It also ignores the current shift in influence on institutional policy development,
brought about by increased attention to the student voice.
Electronic submission of coursework represents a high stakes cultural change in the way student
work is submitted, marked and feedback provided. The three key stakeholders have diverse, but not
necessarily, mutually exclusive objectives. Higher educational management are mindful that
administrative costs need to be curtailed and view the handling of paper-based items of coursework
as a non-essential activity. Their preference is for institutional compliance on e-submission to reduce
costs and offer a consistent student experience. The student body, empowered by National Student
Survey findings, seeks more flexible approaches to assessment and more timely and informative
feedback on assessment. Members of academic staff continue to advocate academic freedom in
choosing how to teach and assess students, with concerns about radical change in practice and
disempowerment. For successful development of institutional policy and practice, these three groups
must come together and agree an appropriate way forward.
This paper begins by considering the background to changes in institutional powerbases before
charting a three-year case study aimed at implementing e-submission at one higher educational
institution. It concludes by reflecting on the experience and making recommendations for other
institutions considering similar development of e-submission policy and practice.
2. Background
The context of the current paper has resonance with many UK higher educational institutions as they
respond to and prepare for changes in funding and anticipated increase in accountability demanded
by students. Institutions are seeking to reduce costs, particularly those not associated directly with
research or teaching. In many institutions, administrative and support activities have been or are
73

Alice Bird
being reviewed under the banner of improving the student experience. This focus on cutting costs
coincides with increased power of the student voice through the medium of the National Student
Survey. In particular, the outcomes of the survey continue to suggest sector-wide issues in meeting
student expectations on assessment and feedback on assessment. In recent years, the National
Union of Students (NUS) has proactively sought to engage institutions in addressing these issues
through a series of campaigns. For many institutions, this has resulted in a three-way split in the
institutional powerbase; institutional strategic management, academia and the student body, often
represented by the local Student Union. At this point, it should be acknowledged that overall power
within this triumvirate will differ from one institution to another depending on the predominant
institutional culture.
The topic of culture in higher education is one that has been widely researched and discussed.
Trowler (2008) presents an exploration into approaches to understanding higher education cultures,
including references to macro, micro and meso level approaches. At the macro (organisational level),
Trowler cites Berquist’s four cultures (1992); collegial, managerial, developmental and negotiating.
Whilst at the micro (individual) level, culture is defined in terms of the institution and its
distinctiveness, e.g. Tierney (1988). An alternative meso level approach acknowledges a multiple
cultural configuration which encompasses more open, dynamic and interactive concepts e.g.
Alvesson (2002, pp.186-187). In an earlier study, Land (2001) also makes reference to organisational
cultures and maps a set of orientations to academic development against conceptions of change.
These orientations, opportunistic/entrepreneurial, reflective practitioner, interpretive-hermeneutic,
romantic (outreach), professional competence activist-modellers, political strategist (pragmatist),
consultant researcher, disciplinary, managerial/HRM, are adopted by practitioners promoting change
in academic practice according to the context that they find themselves in. Groups and individuals
involved in promoting change in academic practice are generally aware of the predominant and subcultures
that exist within a particular institution. However, the shifts in power, with the student voice in
the ascendancy, mean that they may need to review and revisit their approaches to implementing
change.
As with any work-based or social relationship, it is easier to mediate between two parties than three.
With three stakeholder groups influencing policy and practice, the direction of change can be swayed
with one party often feeling disempowered by the perception of the other parties operating in league.
This paper offers an insight into the dynamics of triumvirate influences through a three-year case
study to develop and implement electronic submission policy and practice at one higher educational
institution. The predominant organisational culture of the institution is generally acknowledged as
managerial, with strategic leadership responsible for driving through change, but with sub-culture
influences often aligned to specific subject disciplines. However, student contribution to development
of policy has become more influential through proactive engagement by the Student Union and a
strategic ‘you say, we do’ approach to improving the student experience.
3. Case study
The recommendation to introduce electronic submission of coursework (e-submission) first evolved
from an institutional review of student administrative processes. The recommendations from the
review were translated into a student experience review implementation project aimed at restructuring
student support services and activities across the institution. Part of the review included an
observation of activities in three centralised centres, processing student submissions of paper-based
coursework items. This revealed a negative impact on student experience; in particular, issues
associated with processing submissions at peak submission deadlines. The outcome was a
recommendation to develop institutional policy and practice for the electronic submission of
coursework. This recommendation was approved and strategically driven from the top through an
institutional development project. The progress of the project through three phases, feasibility study,
pilot study and early-stage implementation, constitutes the core of this case study, with references to
triumvirate influences on the direction and outcomes.
3.1 Feasibility study
The feasibility study was initiated in September 2008 and conducted according to institutional
principles based on the Prince2 methodology. A Project Board was convened, with a School Director
appointed as the Chair and membership representatives drawn from each Faculty, relevant Central
Service Teams and the Student Union. Faculty representatives, who were nominated by School
74

Alice Bird
Directors, for their general interest in teaching, learning and assessment, were not necessarily
advocates for e-submission. An initial challenge was defining the project brief and initiation document.
There was no explicit articulation from senior management of the rationale for e-submission policy
and practice, other than fulfilling the recommendation within the student experience review. However,
the general consensus of the project board was that the scope of the feasibility study should not be
limited to exploring administrative benefits but should also cover pedagogical benefits. In particular, it
was considered essential to explore potential improvements in the timeliness and relevance of
feedback to students on assessment, in the light of the sector-wide dissatisfaction expressed in the
2007 and 2008 National Student Surveys.
The project objectives included, as a starting point, outlining the activities associated with the overall
coursework management process. Defining the process was vital to gaining a better understanding of
the potential role for technological support, as a preliminary stage in specifying technical and
functional requirements. The project also sought to assess the needs and perceptions of key
stakeholders, in particular academic staff and students. A final objective was to estimate the resource
implications in moving from paper-based to electronic submission. A project reporting deadline was
set for March 2009, in order to make recommendations for implementation in the academic year
2009/10. Figure 1 provides the final version of the outline process map, agreed after frequent
iterations.
Figure 1: Outline process map
75

Alice Bird
Developing the process map presented a further challenge to the Project Board, in terms of gaining
consensus on the various activities. The process involves much more than submission, feedback and
recording of grades. For example, submission of coursework is used as an indicator of a student’s
continued participation on a course, an essential item of information in tracking local and international
students. The process map also took account of the fact that implementing e-submission would not
necessarily guarantee e-marking, with some staff electing to continue marking printed copies of
submitted coursework items. Refining the start/end points and various routes through the process
generated some debate, with representatives from some subject areas having more clearly defined
conceptualisations than others. However, defining an outline of the process proved invaluable for
highlighting the overall complexity and also in demonstrating that recommendations for submitting
coursework electronically could not be made in isolation from other activities that preceded or
succeeded it. It was also used as the basis for specifying the technical and functional requirements for
the technology
Faculty representatives consulted academic staff on types of assessment items classified as
‘coursework’ and implications for submission file types. This revealed considerable diversity in items,
other than the standard text-based essay, report or portfolio. (See Table 1 below).
Table 1: Types of coursework item
Essays Studio display
Class test (seen and unseen) Video and audio recordings
Documentary exercises Field notebooks
Research reports Posters
Response papers Data authenticity folders
(confidential material)
Presentations (group and individual) Dioramas
Performances Log books/diaries
Portfolios Drawings
Seminar journals Computer programmes
Drawings Mathematical items
Artistic rendering Graphs
Physical models Computer-generated models
The range of possible file types was equally diverse, although common text-based types were likely to
predominate. (See Table 2 below)
Table 2: Example file types
MS Word Web pages/HTML
MS Excel CAD files
MS PowerPoint Computer programmes
Pdf ChemDraw
Video/audio GIS
Image files Other specialist software files
The possible file types indicated the need to identify a technological solution capable of supporting
non-standard submissions. The assignment tool within the institutional virtual learning environment
(VLE) was one solution that met this requirement. Users of an existing plagiarism detection service
were keen to continue using it but the limited range of file types that it supported meant that it could
not be the sole solution for electronic submission.
Student and academic staff consultations were also undertaken to assess perceived benefits and
issues in implementing e-submission. Members of academic staff were also asked to comment on
prior experience of e-submission. The Student Union co-ordinated the student consultation by inviting
course representatives to carry out face-to-face interviews with students that they represented.
Faculty consultation was undertaken through Project Board Faculty representatives sending out a set
of standard questions by Faculty email lists. The total number of respondents was not recorded but a
summary of the views of respondents from each of the six Faculties was collated into a general
statement of perceptions and experiences.
76

Alice Bird
The student consultation collated feedback from 30 course representatives who interviewed a total of
160 students. The majority of students interviewed (83%) suggested they would welcome electronic
submission of coursework. Some perceived advantages included ‘less travel for students who
commute’ and, for international students, the ability to ‘return home and still submit’. However, there
was some concern about the potential robustness and reliability of any system for supporting
submission. In particular, students expressed concern about the ability of a system to handle high
volumes of student submissions, at the same time. Other concerns related to potential failure to meet
deadlines due to problems with their own internet connections or computer systems. When asked to
comment on existing submission and feedback experiences, 76% thought deadlines were scheduled
effectively, although some of the remainder felt that deadlines tended to be very close together.
‘Quicker’ and ‘more detailed’ feedback featured highly as a key area for improvement, with 32%
receiving feedback over a month after submission and almost a quarter (23.4%) never receiving any
feedback on assessment.
The Faculty consultation highlighted a wide variation of prior experience across the institution from
individuals with in-depth experience of e-submission and e-marking to other members of staff having
limited experience of technology, in general. The perceived benefits and issues reflected this
diversity. In most cases, the benefits were considered only realisable with associated assumptions; in
particular, a move to electronic marking and feedback. Tables 3 and 4 below summarise the
perceived benefits and issues with associated assumptions.
Table 3: Faculty perceived benefits
Perceived Benefit Assumption
Faster, more direct access by markers to submitted coursework
‘Greener’ coursework submission approach Overall reduced levels of printing
Reduced manual handling of paper-based coursework Reduced levels of printing
More detailed and timely feedback e-Marking and e-feedback
Easier deterrence/detection of collusion/plagiarism Integration with plagiarism detection
service
Table 4: Faculty perceived issues
Perceived Issues Assumption
Failed submission due to students’ own PCs/connections Off-campus submission
Robustness and scalability of submission/feedback system
Lack of staff/student ICT skills and confidence
Lack of local staff support for changing practice
Staff screen usage health and safety issues e-Marking and e-feedback
Increased departmental printing cost Staff opting to mark paper-based copies
Delays in receipt of coursework/provision of feedback Staff opting to use Print Room service
Ability of technology to fulfil associated ‘marking’ requirements:
anonymous, team, double and external
The feasibility study concluded with the publication of a report and set of recommendations,
incorporating a summary of associated resources and costs. The main recommendation was to
implement a pilot study within Faculties in the academic year 2009/10 to assess the extent to which
perceived benefits and issues were likely to be actual benefits and issues when scaling up to
institution wide implementation. A second recommendation proposed further technological
development and performance testing of submission via the VLE to ensure scalability to institutional
requirements. The report also recommended expansion of central and local support to minimise
institutional risk during the pilot phase. Finally, the report acknowledged the need to consider
alternative approaches to assessment and not to simply replicate traditional modes of assessment
within a technology enhanced learning environment. There was a strong belief that the affordances of
technology could support alternative, more authentic methods of assessment that should be explored
more widely.
The report was presented to the institutional strategic management group in May 2009. The
recommendations to implement the pilot study and to undertake the technological work were
approved. Additional resources for central and local Faculty-based support were not approved at this
stage as a key objective was to reduce overall administrative and support costs.
77

Alice Bird
Reflecting on the feasibility study, the division of power still seemed to be split mainly between
strategic management and academic staff. Project Board meetings frequently returned to discussions
on issues associated with academic staff engagement in e-submission. The project Chair played a
vital role in keeping the project on track in achieving its objectives. The role of the student union was
mainly consultative rather than a force for change. However, there were early indicators of the
increasing influence of the local Student Union through lobbying in response to a National Union of
Student (NUS) Higher Education Campaign. Working in collaboration with the institutional Learning
Development Unit, Mark My Words, Not My Name, NUS (2008), was translated into a
recommendation for institutional implementation of anonymous marking of coursework.
3.2 Pilot study
The second phase of the e-submission project was initiated in September 2009, following on from the
recommendations of the feasibility study. The stepping down of the Chair from the Project Board
resulted in a Faculty Dean being appointed to fulfil this role.
Preparation for the pilot study included the development of online support resources for staff and
student participants prior to the start of the academic year. During August and September 2009,
participating members of academic staff were recruited and identified specific module cohorts for the
pilot. The pilot was implemented in 52 modules from a range of different programmes. A key aim was
to ensure the pilot covered a diversity of subject disciplines, different cohort sizes and programme
levels, different types of submissions (file types, sizes and single/multiple files) and range of student
and staff levels of ICT competencies. In practice, although all Faculties participated in the pilot,
engagement varied considerably with some Faculties involving whole subject areas, whilst others
allowed self-nomination by individual members of staff. The latter tended to be techno-enthusiasts
and this had to be taken into account when evaluating the pilot. Group and one-to-one hands-on
training sessions were provided for participating staff in using the VLE assignment tool. In some
cases, participating staff implemented e-submission for a single coursework assignment with one
cohort, whilst others piloted e-submission with several cohorts in Semester 1 and Semester 2.
Although the pilot required staff to use a specified submission tool, they were at liberty to determine
how they marked coursework and provided feedback to students. Online help resources were
provided for a range of methods for e-marking but instructions were also provided on how to batch
submit coursework items for printing via the university print room.
Evaluation of the pilot study focused on three key areas; submission, marking and feedback. The
methods used included online questionnaires issued to participating staff and students. Students were
invited to participate in a post-submission questionnaire, whilst members of staff were invited to
complete pre- and post-submission questionnaires in order to assess any shifts in their perceptions
after engaging with e-submission. Triangulation of questionnaire findings was achieved through
analysis of staff and student support calls throughout the pilot and more general feedback collated by
the Faculty and Student Union representatives on the Project Board.
In parallel with the pilot study, technological development and performance testing was implemented.
During the pilot, some additional requirements for augmentation of the VLE assignment tool emerged.
In addition, the outcomes from a Higher Education Academy funded study at the University of
Sheffield Hallam, Hepplestone et al (2010), focused on the use of a customisation of the tool that
helped promote student engagement with feedback on assessment. A demonstration of the system to
the Project Board and other members of academic staff resulted in a strong recommendation for
acquisition of the customised version for any future phase of the project.
The participant response rate to online questionnaires was 49% of academic staff (25) and 17% of
students (167). The outcomes of the evaluation suggested that only a few members of academic staff
in the pilot had prior experience of e-submission. Most staff and student respondents rated their levels
of ICT confidence as confident or very confident. However, approximately 5% of staff had limited or
no confidence and expressed some apprehension in engaging in the pilot. Some staff and students
had experienced technical issues during the pilot but both staff and students agreed that there were
benefits for students, in particular, with the submission stage of the process. The main benefits
included savings on travel time and costs and enhanced ability to submit at anytime and from any
location. Some staff also acknowledged benefits through faster receipt of submissions, although some
of those who chose to print out submitted coursework disagreed with this view. Members of staff were
much more divided on the benefits when taking into account other stages in the process. Some were
78

Alice Bird
unhappy about feeling obliged to mark electronically, whilst others had issues in dealing with specific
coursework types. Some members of staff also experienced difficulties in downloading very large files
submitted by students or in dealing with multiple file submissions. The findings, in terms of whether
students thought that submitting coursework electronically had enabled better and/or timelier
feedback, were inconclusive. A slight majority (39%) thought it had but 30% did not and the remaining
31% were unsure.
When asked to rate their overall experience, two thirds of students considered the experience to be
worthwhile or very worthwhile. A further 22% rated the experience as satisfactory whilst 11% believed
the experience to be not worthwhile or (for two students) totally unsatisfactory. Despite the concerns
raised by staff, 48% rated their overall experience as worthwhile or very worthwhile. A further 24%
found the experience to be satisfactory but 28% considered their experience to be not worthwhile or
totally unsatisfactory. In addition, staff continued to reiterate concerns about health and safety, issues
with printing and overall capacity to cope with any scaling up of e-submission activity.
The findings from the evaluation demonstrated a need to accommodate the diversity in needs
between staff and students. Many on both sides acknowledged benefits but the balance was swung in
favour of students, in terms of convenience and savings to them. As a consequence, the Project
Board developed a proposed ‘third way’ forward that would require, as a minimum expectation, esubmission
for some but not all items of coursework. Hence, the main recommendation was for a
phased implementation in 2010/11 based on the types of coursework item identified as most suitable,
in the pilot study, for submitting electronically. This included items meeting the following criteria:
A single file
In Word or pdf format
Of no more than 2000 words
Submitted before the end of main assessment period
It was believed that assessments meeting these criteria would be more appropriate for academic staff
to test out e-marking, to develop other steps in the submission/feedback process and to look at
alternative approaches to delivering feedback electronically, e.g. voice feedback. This
recommendation was approved by the strategic management group along with the agreement to
implement the customisation of the native VLE assignment tool.
During the pilot study, the influence of the student voice on strategic engagement in institutional
decision-making gained greater significance. A second NUS Higher Education Campaign, NUS
(2009), focused on improvements to assessment and feedback and identified ten principles on which
good feedback should be based. In addition, Government sponsorship for defining ‘a reasonable
student experience’, through the National Student Forum, further raised the need for improved
feedback on assessment along with some fundamental changes to teaching and learning organisation
and practice in universities, NSF (2009). (The continued strategic contribution by students, at a
national level, is evidenced in the report commissioned by HEFCE through the NUS on students’
perspectives of technology in learning, teaching and assessment, as part of the Online Task Force
consultation, HEFCE (2010)).
In the current case, the Student Union developed its own version of ten ways to make feedback better
based on consultation with students. This was presented to the Academic Board, in May 2010, in the
form of a printed document with real examples of positive and negative student experiences. The first
three objectives were approved for implementation. These included a requirement for feedback
deadlines to be published along with assessment, all feedback to be made available three weeks after
the deadline and all feedback to relate to assessment criteria. Of these three objectives, the three
week turnaround for feedback proved most contentious, with academic staff feeling that they had not
been consulted in the decision-making process. As a sense of disempowerment emerged, academic
staff became more aware of greater student influence and apparent collusion between strategic
management and students in decision-making processes. This was further enhanced by other
institutional changes to the academic framework and administrative processes which culminated in a
feeling of initiative overload from the member of academic staff perspective.
79

3.3 Early stage implementation
Alice Bird
During the academic year 2010/11, a number of factors combined to transform the proposed early
stage implementation into an extended pilot rather than wide-spread implementation. Academic staff
initiative overload referred to earlier was one factor and there was also a need to align more generally
with other institutional policies, including the proposal for anonymous marking. Participation was wider
than in the previous pilot study, with increased general awareness of a developing e-submission
policy. This led to wider debate amongst the academic community, with the main union representing
academic staff questioning aspects of policy and practice, already raised as perceived issues by staff.
Union concerns about health and safety resulted in a revision of the university’s safety code of
practice on visual display equipment to accommodate laptop and tablet devices. This did not lead to
the identification of any specific e-submission and e-marking safety issue, although staff remained
concerned about their ability to comply with the code and also meet the three week turnaround for
feedback. Another union concern was to ensure the right for staff to print electronically submitted
coursework, although this did conflict with directives in some Schools that electronic submission
should not result in increased departmental costs. A further union concern related to the ability of the
institutional print room to cope with large volume printing a peak deadline times but this was
countered as a historical view of the capabilities of the service.
The early phase implementation is currently being evaluated but the institution has decided to
continue with the ‘third way’ approach for electronic submission. This has been translated into policy
for 2011/12 but in a way that still allows academic staff to choose how to mark and provide feedback.
The decision to maintain the approach, without further amendment, acknowledges the general sense
of initiative overload by academic staff. E-submission does represent a major cultural shift in staff
assessment practices and our experience suggests that it cannot be achieved without agreeing some
form of consensus in approach. The project has also highlighted to strategic management the
complexity of the overall coursework submission process and the need to limit change to what is
practicable at any point in time.
4. Reflection and recommendations
In reflecting on the case study experience, the institution has made progress towards implementing
electronic submission of coursework and to achieving its objectives. The proposal to implement a
‘third way’ was critical in moving the process forward. However, we are conscious that implementing
e-submission is only one aspect of reviewing and improving assessment practices within the
institution. Evidence from existing and emerging research supports the need to redesign assessment
practices across the whole institution, REAP (2011), and within programmes of study, TESTA (2011).
The intention is to continue working with institutional management, academic staff and students to
develop practices that benefit all stakeholders.
In recommending how other institutions might approach development of e-submission policy and
practice, a key starting point is analysis of the predominant culture and sub-cultures. Acknowledging
the increased influence in student power is essential and early inclusion of student leadership does
promote better understanding between key stakeholders. The needs of key stakeholders need to
align but giving stakeholders the opportunity to vent their concerns is also critical in promoting better
understanding. This needs to be conducted in the spirit of listening and moving forward with
agreement to minimise concerns. In our case, frequent consultation throughout the various stages of
the project did not translate into members of academic staff being aware of the introduction of new
policy. Hence, further debate can be expected once any new policy is first implemented. Providing
evidence that perceived issues have already been discussed and solutions considered is essential at
this stage.
Progress can only be achieved through managing the expectations and understandings of all key
stakeholders. Our experience also highlights the need to take an institutional perspective of all
initiatives impacting on academic practice, administrative activities and the student experience, at any
point in time. Finally, any implementation of e-submission policy and practice needs to be considered
in the context of changing assessment practices, in general, and the opportunities that technology
provides for alternative approaches to assessment and feedback on assessment.
80

References
Alice Bird
Alvesson, M (2002) Understanding Organizational Culture. London: Sage.
Berquist, W.H. (1992) The Four Cultures of the Academy. San Francisco: Jossey Bass.
HEFCE (2010) Student Perspectives on Technology – demand, perceptions and training needs. Available online
at http://www.hefce.ac.uk/pubs/rdreports/2010/rd18_10/
Hepplestone, S., Parkin, H., Irwin, B., Holden, G and Thorpe, L (2010) Technology, Feedback, Action! The
impact of learning technology upon students' engagement with their feedback. Available online at
http://www.heacademy.ac.uk/assets/EvidenceNet/TFA_report_final.pdf
Land, R. (2001) Agency, context and change in academic development. International Journal for Academic
Development, 6 (1), 4-20.
NSF (2009) National Student Forum Annual Report 09. Available online at
http://www.bis.gov.uk/assets/biscore/higher-education/docs/n/09-p83-national-student-forum-annual-report-
09
NUS (2008) Higher Education Campaign: Mark my words, not my name. Further information available online at
http://www.nus.org.uk/en/Campaigns/Higher-Education/Mark-my-words-not-my-name/
NUS (2009) Higher Education Campaign: Assessment feedback. Further information available online at
http://www.nus.org.uk/en/Campaigns/Higher-Education/Assessment-feedback-/
REAP (2011) Re-Engineering Assessment Practices in Higher Education. Further information available online at
http://www.reap.ac.uk/Home.aspx
TESTA (2011) Transforming the Experience of Students Through Assessment. Further information available
online at http://www.testa.ac.uk/.
Tierney, W. G. (1988) Organisational culture in higher education. Journal of Higher Education, 59, (1) 2-21.
Trowler, P. (2008) Cultures and Change in Higher Education, 1 -15. New York: Palgrave MacMillan.
81

Enhancement of e-Testing Possibilities With the Elements
of Interactivity Reflecting the Students’ Attitude to
Electronic Testing
Martin Cápay, Martin Magdin and Miroslava Mesárošová
Department of Informatics, Faculty of Natural Sciences, Constantine the
Philosopher University in Nitra, Slovakia
mcapay@ukf.sk
mmagdin@ukf.sk
mmesarosova@ukf.sk
Abstract: Testing in electronic environment is usually considered to be very effective, however, that does not
mean it is objective at the same time, mainly from the students’ point of view. Within three years we carried out a
research based on questionnaire investigation which aim was to explore students’ attitude toward electronic
testing. We found out that students’ perception of this type of testing depends on whether classification is the
main objective of the testing or not. The research also showed that the attitude of students is to a large extent
influenced by the level of their previous experience with e-testing. In spite of the fact that testing is in all
mentioned groups perceived as modern, effective and interesting, it is not completely accepted by all the
students, nor is it considered to be objective. Teachers, on the other hand, perceive e-testing assuredly more
positively. The differences in the opinions were obviously caused also by insufficient interactivity in standard LMS
systems as well as limited support in evaluating the essay tasks. These were the main reasons for us to
concentrate on modification and adaptation of FlashQuestions module implementable into LMS Moodle. The
module allows interactive testing utilizing animations. At the moment, we are carrying out an empirical research
focused on the effectiveness of testing that employs the interactive media objects. So far, we found out that this
method of testing required less time spent on test elaboration by the students in comparison with traditional
“paper” testing. The other interesting feature pertains to the increase of average result (number of points gained
by the students).
Keywords: e-testing, FlashQuestions, interactivity, animations, feedback
1. Introduction
Evaluation and assessment is undoubtedly a very important constituent of teaching and learning
process since the primary education levels. One of the most frequently used didactic means for
assessing the rate of cognitive skills gained by the students is a didactic test. There are several ways
how to perform it in class, one of them being electronic testing – e-testing. Its advantages lie in its
modernity connected with the usage of ICT in education, simplicity of administration, usability in larger
group of students, as well as reducing of subjectivity in testing and assessing the students’ attempts.
Although it is possible to create own authoring applications, after several years of experience in this
field we found out that the most effective way is to use an existing tool – in our case Learning
Management System (LMS) - that has large community of users and at the same time was open to
modification according to the users’ needs. As the most suitable environment the LMS Moodle was
chosen, not only because of cost (free open software) but mainly because it is a system with growing
user community that continuously helps to enhance its possibilities. By now, the university has
already been using it for several years as the common support of education at most of its
departments – being the first university in Slovakia using LMS in such an extent.
In spite of the fact that LMS Moodle enables the users to create new or modify existing modules, we
decided to implement the model of e-testing in the standard Moodle activity – Quiz. This module
allows the teacher to get the view of students’ knowledge acquired during the term. The tests are
often used as elements of particular lessons were they not only let the teacher to check the students’
knowledge but at the same time are also a feedback tool for the students – they can check their own
progress and understanding of the topic. The outcomes of the test attempts may be included in the
final evaluation. Another possibility is to use the tests as a form of the exam necessary for successful
completion of the subject.
Moreover, the LMS provides a quick statistical report and item analysis of tests, with the possibility of
export to different file types. There are two types of item analysis, quantitative and qualitative, and
they are used for exploring the characteristics of particular items of the test. (Kučera et. al 2008).
82

Martin Cápay et al.
The system supports several question types, the multiple choice questions are the most frequently
used. These questions, in which the choice of one correct answer is usually necessary, are mostly
aimed at testing of facts (dates, values, categorisations) – in fact, they are based on memorization.
The learner can only guess the answers – that is why we qualify these questions as low-level ones.
On the other hand, the questions, in which the student has to form the answer or find it using the
application of learnt procedures, are more valuable. These questions can be considered high-level
questions. The questions that require only recitation of facts take less time to ask and answer which
might be the reason for teachers to sometimes avoid asking high-level questions (Kipper & Rüütmann
2010). Unfortunately, mainly because of the above mentioned reasons, multiple choice questions with
one correct answer usually dominate in most of our courses.
However, we do not want to ask only questions which require rote memory for a correct response, the
questions should also facilitate critical thinking and problem solving. At the same time, we are
interested in application of such a way of testing which reduces the degree of subjectivity in testing. In
the paper, we are dealing with the possibilities of various applications of LMS Moodle in testing, the
usage of multimedia and interactive elements in quiz questions to take advantage of wide range of
existing LMS Moodle tools.
2. The first experiments with on-line testing
eLearning can construct an independent and individualized learning environment and break through
the restrictions of conventional learning (Wen and Lin 2007). The online testing procedure using
LMSs is probably the most widely used and the quickest way to test students without prejudice
(Skalka and Drlik 2009). Our first attempt for serious usage of online testing in practice was within the
course of programming. We developed a testing model containing a bank of questions and tasks
adapted for implementation in LMS environment. There are questions that require active application
of knowledge and programming skills for the correct answer – in this case the correctness of
responses could be evaluated automatically (Cápay 2010). We used standard Moodle question
types– multiple choice, short answer and true/false questions. To evaluate this type of testing and its
benefits we carried out a questionnaire investigation during the pilot testing period.
2.1 Methodology of research
The main questionnaire research was carried out in course of three years among the Department of
Informatics students at the end of each term. There were different test procedures used during the
terms in three groups of students (Table 1):
1st group – students attempted only traditional testing – writing programs down on a paper, oral
examination, evaluation without computers,
2nd group – students first attempted traditional examination and in the next term they attempted
on-line testing,
3rd group – students attempted only on-line testing.
The research participants were the university students of the first grade (each year different group of
students) and they passed the testing in subjects Programming 1 and Programming 2.
Table 1: Structure of research participants
Number of respondents – Bachelors students of Gender
Group
Applied Informatics
Teaching Informatics
in combination with another subject sum female male
1st 55 14 69 10 84
2nd 55 39 94 10 84
3rd 14 24 38 3 35
sum 69 63 132 13 119
Another feedback we gained to online testing of programming was realised as a national research
among a selected group of secondary and university teachers of Informatics. (Cápay 2009)
Our questionnaires were composed of several questions regarding students’ preferences in
programming testing. We mostly used multiple choice questions with five or seven choices in the form
83

Martin Cápay et al.
of Likert scales. We compared electronic testing to traditional (“paper”) testing using a scale with
seven point scale between the options stated as antonyms, e.g. we used the scale objective vs.
subjective. The question was: Do you think that traditional testing is
objective 1 2 3 4 5 6 7 subjective
effective 1 2 3 4 5 6 7 ineffective etc.
We used nine scales (Figure 1, Figure 2). If the respondent marked number 4, it meant that he/she
regards the particular statement neutrally. On the other hand, marking any other option from given
scale, denoted respondent’s inclination to certain term (on the left or on the right side of the scale).
The results may be compared to the balance pans - one of the terms tips the scales according to the
value it was given by the respondent while the pans are balanced if the respondent does not prefer
any of the terms (antonym responses to the statement).
2.2 Research outcomes
One of the findings of our survey was that the students did not clearly prefer computer testing and
evaluation to teacher’s assessment.
We asked them about the perception of two terms, electronic testing (Figure 1) and “traditional”
(paper) testing (Figure 2).
Figure 1: Comparison of responses in particular phases. Students’ and teachers’ perception of the
term “electronic” testing
The research showed that the attitude of students is to large extent influenced by the level of their
previous experience with e-testing. In spite of the fact that testing is in all mentioned groups perceived
as modern, effective and interesting, it is not completely accepted by all the students, nor is it
considered to be objective. Students who were tested only traditionally (answering the questions on a
paper) claimed they would like to have the opportunity to be tested electronically. Those who
experienced both traditional and electronic testing partly preferred the traditional way of testing.
Finally, the students tested only using electronic tests expressed mostly positive attitude toward this
testing method (Figure 1 and Figure 2).
Based on the responses to other questions in the questionnaire in which 5-choice Lickert scales were
used, we found out that students’ perception of this type of testing depends on whether classification
is the objective of the testing or not. The students preferred combined form of testing where the
teacher still plays the essential role. In some cases they also required to check the automatically
evaluated tests – in these cases the teacher went through the test again with the student and checked
the answers.
84

Martin Cápay et al.
Figure 2: Comparison of responses in particular phases, students’ and teachers’ perception of the
term “traditional” testing (note: The data of the 3 rd group are not available)
On the other hand, we expected a little more traditional view in teachers’ responses but the answers
exposed their modern thinking. Teachers perceive e-testing assuredly more positively. However, the
survey showed that in spite of positive view of e-testing, this method still remains very doubtful, mainly
in the tests used for grading.
The biggest complaints from the students, and partially from the teachers as well, were directed to the
display of remaining time, which according to their opinion rose their nervousness during the testing.
We also found out several differences between the students’ and the teachers’ point of view on
advantages and disadvantages of electronic testing (Cápay and Tomanová 2010). Some of the
students’ responses on disadvantages of e-testing could be summed up as the lack of multimedia
features – the possibility to demonstrate the problem. From the teachers’ point of view the absence of
the question parameterisation was among the most often mentioned disadvantages. Therefore we
endeavoured to improve these two drawbacks in the tests implemented in LMS Moodle in the
following years.
3. Increase of testing possibilities in LMS Moodle
One of the possibilities to increase the testing effectiveness is creation of models (Balogh at al.)
focusing on the methods of adaptive providing of study materials (Kapusta et al, 2009) and
conditioned tests. LMS Moodle is an open system, which means it should not be a problem to suggest
a concept of involving multimedia elements and/or the parameterised questions and implement it into
the system.
3.1 Parameterisation of the questions
LMS Moodle offers quiz questions with the Calculated answers with a collection of input parameters
generated for each question. These parameters are inserted automatically into the question text and
are unique for each student. Currently it is possible to enter a prescription (function) in the process of
inputting the parameters to one output (problem solution). However, it has some limitations. That is
why we also need to consider some other possibilities, e.g. the response in the form of clicking to a
particular part of a picture map, the question in form of flash application, etc., to solve this problem.
Gangur (2011) shows an example of generating a unique test containing cloze questions in the
selected LMS. He proposes cloze questions to be generated in the XML structure (Figure 3) and later
it can be transformed to a more suitable form.
85

Martin Cápay et al.
Figure 3 Process of questions generation in XML format and its transformation to other output
formats (Gangur 2011)
The creation procedure written in any programming language or in any appropriate environment is the
basis of the generating process. For example, Matlab system is used to calculate the results of the
given problems created from randomly generated input parameters. This system is appropriate for
questions that are aimed to e.g. passive programming (the skill to read and analyse unknown source
codes). There would be an existing algorithm (sorting, searching). A question to this algorithm would
be generated according to the particular input values randomly by the basic process. At the same
time, also the appropriate answer would be generated. In the following chapters we focus on the
implementation of multimedia elements that would serve not only to display the task but they would be
the interface to insert the input (answer) of the user as well.
3.2 Multimedia and interactivity
Interactivity is a wide term that can be specified (in case of ICT usage) as an ability of the device
and/or application to react on the user’s stimuli that are evaluated and processed according to the
options, e.g. the program code. With the development of the ICT together with the increasing
measure of interactivity these terms started to be used also in the field of education – in this case the
stimuli are given by the students (Neo and Neo 2004; Sonwalker 2001).
Implementation and usage of interactive media elements within the teaching-learning process in the
form of eLearning is these days regarded as one of the essential methods of supporting the
interactivity of the student with the study material. Therefore, there is a need to enable the usage of
interactive questions in the quizzes as well as the evaluation of elaborated interactive questions
directly in the Book module in LMS Moodle e-courses.
4. Implementation of interactivity into the electronic testing in LMS Moodle
Regarding that the internal structure of LMS Moodle is quite unintelligible. It was necessary to design
such a technological solution that would be acceptable not only from the programming point of view
but also from the point of view of simple organization and saving the data in the system database.
First, it was necessary to suggest the way of inserting the interactive animation directly into the
content of the study materials. The students had to be able to manipulate with the animations as well
as to use the interactivity to change the input values. Currently, we are able to integrate the Flash
animation and/or applets into the e-courses thanks to a simple application designed to generate the
HTML code necessary to include the animation or applet within the study material (Magdin 2010).
The next step was developing an electronic testing with the features of interactivity. The most suitable
solution (at this time the only one functional in Moodle version 1.9x) is considered to be the
implementation and partial modification of FlashQuestion module created by Dmitry Pupinin.
FlashQuestion module allows interconnection of the data we get during the usage of interactive tasks
with the system database. Thanks to this interconnection, we are able to statistically process and
evaluate the variables representing correct and incorrect solutions of the tasks. Implementation of the
module in education is very impressive. We can say it is a new approach in the field of testing. The
module offers the possibility of rated assessment of a student as well as the storage of the answers in
the Moodle database. There is a possibility to stop working on the task and continue later. After the
86

Martin Cápay et al.
test is submitted, the student’s answers are displayed together with the correct ones which serve as a
feedback for the student.
Another advantage of this type of quiz questions is the possibility to create interactive tests that
include various types of tasks. Test containing such type of questions is not limited to selecting one
correct answer or filling in the correct word(s) because it is possible to create combinations of tasks in
the form of simulations that are solved by the students. We can insert such flash application among
other quiz questions and combine them according to our needs.
Great advantage of this type of testing is the fact that it eliminates former imperfection of the system –
to estimate if the student “only” made a slip – now it points to the process of tackling the task (whether
it is correct or not). From this point of view the interactive tasks give the students the same
opportunity of communication and correction as if he/she were in direct contact with the teacher.
4.1 Didactic effectiveness of interactive animations
The experimental verification of didactic effectiveness as well as the pedagogical and psychological
research indicate that the effectiveness of perception and retention is directly commensurable to the
number of sensors that are activated during the learning process.
According to Lindstrom (1994) the interactivity brings in the ability to remember the information as
follows - when people:
See (20%),
See and hear (40%),
See, hear and can intervene into the action (75%).
Interactive animations, according to the above mentioned definition, do not only function as means of
sense-perception but also as means of cognitive perception as they point to and at the same time
break into the objects’ fundamentals. Furthermore, this continuous increase of the visuality enables
the students to lively look at and understand the basic principles. This is very important especially in
the abstract thinking since the student not only develops his/her concepts but constructs the
fundamentals of logical thinking as well.
Identification of the degree of didactic effectiveness of innovated study materials (i.e. its increase or
decrease) is a relatively problematic step that depends on particular implementation of designed
educational tool into the educational process. The reason is that the question of its function or
effectiveness is always answered in the same way: its function is to support the development of
cognitive and intellectual abilities of the student, i.e. to teach something new. This answer is based
mainly on the proven claim - the more diverse methods of learning, the better retention of information
(Mayer 1997, Hvorecký et al. 2010). It is possible to verify the didactic effectiveness of interactive
animation usage by means of didactic tests in which the expected outcome of educational process is
detected by written exam. The student answers a specific question or solves a particular task and
there is usually only one correct answer. There is a possibility of more correct answers – in this case
the student has to be informed in advance. The following task is an example of electronic interactive
didactic test from the Computer Architecture subject (Figure 4).
4.2 Verification of didactic effectiveness of interactive e-testing
In order to correctly verify the didactic effectiveness of a created interactive question, it was
necessary to carry out a research for which we chose the test method – the students were given the
tests in various forms (electronic and printed). Consequently we compared the results of particular
groups of students. There were two research groups:
Experimental group - the group of students that used the study materials with implemented
interactive animations and interactive problem tasks during their study,
Control group - the students who studied from the materials in the form of PDF files without any
interactivity implemented.
87

Martin Cápay et al.
Figure 4: An example of interactive task – Karnaugh map
Prior to conducting the experiment, the following hypotheses were stated:
H1: There is no statistically significant difference in the elaboration and evaluation of given task
between experimental and control group.
H2: The usage of interactive tests will enforce better results of students than the usage of printed
tests.
The tests used in the research were the following:
Electronic interactive test – students elaborated the task of Karnaugh map in the form of
interactive test, after each elaboration of the electronic task we reduced the time limit of the group
by 3 minutes,
Printed test – students elaborated the task on a paper. They were given a table and a picture of
Karnaugh map and had to fill in the appropriate values. In order to specify the time effectiveness
the maximum time for elaboration was stated, in this case 15 minutes, in both groups.
88

Martin Cápay et al.
The test in the experimental as well as in the control group was realised six times. In all cases the
task was to work with Karnaugh map, the difference was in the basic function that was stated to work
with. During particular testing, all the students in research groups got the same task - the difference
was in the form of the task elaboration (electronic test vs. printed test.
To verify the stated hypotheses, we decided to exchange the form of testing after the third test, i.e.
the control group elaborated the test electronically and experimental group used printed form. This
way we ensured the same conditions at elaborating the problem task and each of the groups had
three attempts to elaborate the test in particular form.
Table 2: The data of the experimental and the control group before the exchange
Experimental group
(used interactive materials during their study)
Average
evaluation
Control group
(used non-interactive materials during their study)
Average
evaluation
(mark A-FX)
Test
Time
Test
Time
no. Form of testing limit (mark A-FX) no. Form of testing limit
1 electronic 15 B 1 printed 15 E
2 electronic 12 B 2 printed 15 D
3 electronic 9 A 3 printed 15 D
4 printed 15 E 4 electronic 15 A
5 printed 15 E 5 electronic 12 B
6 printed 15 E 6 electronic 9 A
In the first and the second test, the experimental group gained B mark as the average evaluation, in
the last electronically elaborated test the average mark was A, in spite of the fact that the time limit
was decreased after each test. On the other hand, the control group had the same time limit in all
three tests (15 minutes) and their average evaluation was E in the first test and two times D in the
following tests.
After we exchanged the form of testing in experimental and control group, we found out that the
results of the students were similar to those before the exchange from the point of view of the form of
testing, i.e. the experimental group that now elaborated the printed test gained mark E at average (the
same result was in the control group in the first three tests), in spite of the fact that previously they
elaborated the test electronically and their evaluation was significantly better. Similarly, in the control
group that now elaborated the test in electronic form the average evaluation was two times A mark
and once B mark which is comparable to the results of the experimental group in the first three tests,
in spite of the fact that the time limit was decreased after each test.
Based on the three realisations of testing in two different forms (electronic and printed) we have to
conclude that there is a statistically significant difference in the evaluation of the given task between
the experimental and the control group (Table 2). Therefore, hypothesis H1 has to be dismissed. On
the other hand, we observed significantly better results when using electronic form of testing in both
the experimental and the control group. That means the hypothesis H1 was confirmed. Furthermore,
we can say that the success in the task elaboration and the following evaluation result is not
dependent on the form of previous study of the learners.
5. Conclusion
Several years ago, we found out the perception of electronic testing is more positive than the
perception of traditional “paper” form. At that time, our courses, as well as the tests, were
characterised by static form to quite a wide extent. In the study materials, there were no animations
nor interactivity, the testing was carried out only within the basic possibilities of the used system. This
fact was obvious not only to the teachers but to the students as well. It was expressed in the final
questionnaires they filled in at the end of each term. After some time, the interactive animations were
implemented in several courses. The illustrative value of the study materials was increased and the
testing of particular topics was more clear and transparent. Consequently, we were able to verify the
didactic effectiveness of designed and created tests.
We found out that the implementation of interactive animations and interactive problem tasks really
facilitates the development of intellectual and cognitive abilities. At the same time, based on the
results of the experiment, we can see that the usage of interactive problem tasks is effective from the
point of view of time necessary for the elaboration of the task, as well as the successful task solving.
89

Martin Cápay et al.
On the other hand, there is still the question of problems the students face while elaborating the same
task on a paper (in printed form) even if they are able to solve the task easily in electronic form. After
the discussion with psychologists, we suppose that the reason lies in higher acceptability of electronic
form of testing which, unfortunately, eliminates students’ critical thinking as well as the ability of
formally correct writing. The elaboration of given task is in this case faster but only mechanical.
The observation also showed that electronic tasks, in which there is the possibility to “undo” or “clear
the desktop” (e.g. delete the configurations in Karnaugh map), are more likely to be better solved by
the students. In case of a bad solution in printed test, it is not possible to go back or change the
direction – this significantly shortens the time as well as decreases the willingness to revise the
solution process.
Interactive animations and interactive problem tasks are undoubtedly a very interesting possibility of
the educational process enhancement. However, there are several problems occurring in their usage
that need to be investigated in greater detail and eliminated in the future. We can assume that
utilization of this testing method may help to mobilize the students but, on the other hand, it can also
cause partial elimination of critical thinking as well as the ability to follow the formalism in writing.
Acknowledgements
This paper is published thanks to the financial support of the ESF project ITMS 26110230012 “Virtual
faculty – Distance Learning at FSVaZ UKF in Nitra”, and the national project KEGA 080-019SPU-
4/2010 “Development of creative potential in eLearning”.
References
Balogh, Z., Klimeš, C. and Turčáni, M. (2011) “Instruction Adaptation Modelling Using Fuzzy Petri NETs.“
Distance Learning, Simulation and Communication 2011. Brno. pp. 22-29.
Brooks, I. and Weatherston, J. (1997) The Business Environment: Challenges and Changes, Prentice
Cápay, M (2010). “Testing the programming knowledge and skills in electronic environment.“ Problems of
Education in the 21st Century, Vol. 23.
Cápay, M. (2009). Development and use of intelligent computer systems to support the teaching of informatics
subjects. Disertation thesis.
Cápay, M. and Tomanová, J. (2010). “Enhancing the Quality of Administration, Teaching and Testing of
Computer Science Using Learning Management System.“ WSEAS Transactions on Information Science &
Applications. Vol. 7, No. 9, p. 1126-1136.
Gangur, M. (2011). “Automatic generation of cloze questions.“ CSEDU 2011: Proceedings of the Third Internation
Conference on Computer Supported Education. ISSN-ISBN: 978-989-8425-49-2 .p. 264-269
Hvorecký, J. Drlík, M. and Munk, M. (2010) “The effect of visual query languages on the improvement of
information retrieval skills.“ Procedia - Social and Behavioral Sciences. Vol. 2, No. 2, pp. 717-723.
Kapusta, J., Munk, M. and Turčáni, M. (2009) “Experimental comparison of adaptive links annotation technique
with adaptive direct guidance technique. “ International Conference on Web Information Systems and
Technologies, pp. 250-256.
Kipper H. and Rüütmann, T. (2010). “Strategies and techniques of questioning effectuating thinking and deep
understanding in teaching engineering at Estonian centre for engineering pedagogy. “ Problems of
Education in the 21st Century, pp.36 – 44.
Kučera, P., Kvasnička, R. and Vydrová, H. (2008). “Evaluation of test questions using the item analysis for the
credit test of the subject of mathematical methods in economics in the Moodle LMS.“, Efficiency and
Responsibility in Education. Praha p. 145 - 153.
Lindstrom, R. (1994) The Business Week Guide to Multimedia Presentations: Create Dynamic Presentations
That Inspire. New York: McGraw-Hill.
Magdin, M. (2010).“Integrating interactive flash animations ana java applet in LMS Moodle“. Journal of
Technology and Information Education. Vol 2, No 1, http://www.jtie.upol.cz/10_1.htm.
Mayer, R. (2004) “Multimedia learning. Are we asking the right questions? “ Educational Psychologist [online] [cit.
2010-12-07], http://www.ameprc.mq.edu.au/docs/conferences/2003/EileenChau.pdf.
Skalka, J. and Drlík, M. (2009). “Avoiding Plagiarism in Computer Science ELearning Courses.“ Information &
Communicaton Technology in Natural Science Education. Vol. 16.
Sonwalker N. (2001) A New Methodology for Evaluation: The Pedagogical Rating of Online Courses. Campus
Technology from Syllabus Media Group.
Tse-Kian Neo, Mai Neo. (2004). “Integrating multimedia into the Malaysian classroom: Engaging students in
interactive learning“, The Turkish Online Journal of Educational Technology – TOJET Vol 3 No 3 [cit. 2011-
08-08], http://www.tojet.net/articles/334.pdf
Wen, T.S. and Lin,H.Ch. (2007). “The Study of ELearning for Geographic Information Curriculum in Higher
Education. “ 6th WSEAS International Conference on Applied Computer Science (ACOS '07).
90

e-Assessment Using Digital Pens – a Pilot Study to
Improve Feedback and Assessment Processes
Tim Cappelli
University of Manchester, UK
Timothy.cappelli-2@manchester.ac.uk
Abstract: Manchester Medical School is the largest medical school in the UK, with over 2000 students on the
MBChB programme. During the final three years of the programme all students undergo regular assessments of
their clinical skills through a series of Objective Structured Clinical Examinations (OSCEs). The OSCEs require
students to carry out a series of simulated exercises in front of an examiner. The examiner completes a scoresheet
for each student, giving a mark between 1 and 7 for each of four criteria together with a ‘global mark’, again
between 1 and 7. The examiner also leaves a small piece of written feedback on the bottom of each form.
Following the exam, all the score-sheets from each of the four teaching hospitals attached to the University are
scanned using an optical reader. This involves a large amount of effort and provides many opportunities for
errors. Due to the work involved and logistical problems, student feedback from the OSCEs is currently limited to
a single mark. Despite the examiners providing the piece of text on the score-sheet, this is only made available to
students scoring less than 4 on their global mark. The students and the School are increasingly motivated to
allow all students access to the written feedback. Hence, in an effort to increase efficiency of the OSCE process
and enable the delivery of student feedback, the Medical School has piloted the use of digital pens as a method
of capturing and processing scoring and feedback. This case study presents the process and evaluation of the
pilot. The study examines the choice of technology, the aims of the pilot and an evaluation of the technology to
assess whether objectives have been achieved. An impact analysis of the use of the pens over a five year period
also shows the return on investment.
Keywords: e-assessment, digital pens, feedback
1. Background
Manchester Medical School is the largest medical school in the UK, with over 2000 students across a
five year MBChB programme. The final three years of the programme are spent based at one of five
teaching hospitals across the North-West of England. With so many students dispersed across so
many sites, the logistics of curriculum delivery and examination is immensely challenging. This is
particularly so when it comes to the assessment of students’ clinical skills. In common with most
medical schools, a student’s clinical skills are assessed using a series of Objective Structured Clinical
Assessments (OSCE) (Harden and Gleeson 1979). OSCEs have been used for over three decades
as a way of providing a reliable and objective method for testing students’ clinical abilities. OSCEs
consist of series of short tests – or stations – that are designed to test a student’s clinical performance
along with competences in history taking and clinical examination. Each of the stations has a different
examiner and students rotate through the stations, to complete all the stations on their circuit. In this
way, all candidates take the same stations, with the same set of examiners, providing the basis of an
objective, structured assessment. The students at the University of Manchester undertake OSCEs
twice a year from year three onwards, with each of the four base hospitals delivering 16 stations per
circuit, 4 times a day over a week to assess all the students in a year group. Each examiner has one
form for each of the students; so with 16 stations and 16 students per circuit, four circuits per day per
hospital and four hospitals, it’s clear that the number of forms soon mounts up. At present all the
forms are marked, collected and checked manually and the forms returned to the Medical Exams
Office (MEO) at the University for scanning and collating. The opportunities for making this process
more efficient and secure through the introduction of electronic marking are clear; several studies
show that e-assessment improves information management and ‘avoids the meltdown of paper
systems’ (Buzzetto-More and Alade 2006, Ridgeway, McCusker and Pead 2007). However, efficiency
savings where not the driving-force behind the Medical School’s commission of a pilot scheme in June
2010; instead it was the lack of feedback given to students after the exam. Each form completed by
the examiner consists of a series of criteria on which the student is given a score between one and
seven, with four representing the minimum standard required. The examiner then gives a ‘global
mark’ for the student’s performance, again between one and seven, and is encouraged to write one or
two lines of feedback in a text box at the bottom of the form. However, due to difficulties in making the
manual forms available to students, only those students who have scored less than four on any given
station are provided with this feedback, and then only sometime after the exam. This is clearly not in
keeping with best practice for summative feedback, with most authors agreeing that feedback should
be constructive, timely and freely available (Nicol and Macfalane-Dick 2006, Gibbs 2010). Increasing
91

Tim Cappelli
demand from students to be able to view this feedback, and in good time, prompted the Medical
School to task a small group within the school to examine, pilot and evaluate the use of electronic
marking by OSCE examiners. The use of information technology to provide immediate feedback from
assessments is well documented (Ridgeway et al 2010), the challenge was to determine the most
appropriate technology.
2. The challenges
The main objective of the pilot was to improve the delivery of feedback to students, both in terms of
the number of students and the time taken for them to receive their feedback. This information was
already being captured, and therefore required no change to the practices of the examiner, but rather
the process by which it was captured, collated and delivered needed to be made more efficient. In
order to determine how this process might be improved through the use of technology, the team
started by detailing the current process. This is shown as a process diagram (Figure 1).
Figure 1: Current OSCE process
92

Tim Cappelli
In order to determine this process, the team carried out a series of interviews with each of the relevant
stakeholders and in doing so also identified issues inherent in the process that stakeholders wanted
to improve or eliminate. From this a set of key objectives were articulated, namely:
Provision of faster results and feedback to students.
Reduction of the opportunities for errors in data handling and erroneous results.
Provision of timely textual feedback to all students.
Reduction of time-pressured manual handling of the marking forms.
Less labour-intensive and faster preparation of forms for the different OSCE exams with less
manual form preparation.
More efficient and faster preparation and transmission of data for analysis by the different roles
involved in administration, management and quality assurance.
3. The solution
According to Love and Cooper (2004), assessment systems should consider the interface,
accessibility, security, reliability and the information to be captured. In considering a potential
technology solution, it was therefore important to consider these issues alongside the system’s ability
to achieve the pilot’s objectives. In particular, the following features where considered crucial:
Data security: transmission of data must be secure and storage of exam scenarios, forms and
results is also secure.
Performance: OSCE exams run on a tight time schedule. The system must not introduce risk of
time delays due to slow response times.
Connectivity: the system must offer solutions for the communications infrastructure that is in
place in the hospitals. Wireless internet availability or provision for wired networks cannot be
guaranteed.
Data integrity: it must be ensured that no data is lost in transmission and that there can be no
mix-up between fields in a form or between forms. It would be an advantage if mandatory fields
could be checked in real-time so that only complete forms are submitted to the system.
Scalability: the system must be able to support the volume of forms needed in the annual cycle
of OSCE exams.
Human Computer Interaction: the examiners’ interaction with the system must be intuitive and
not alter the marking process for the examiner. For example, the examiner must be able to amend
his marks on a form prior to submission.
Training: the device chosen must also be intuitive enough to avoid the need for lengthy or
expensive training for examiners.
Robustness: the chosen technology must be robust and reliable, with a failsafe backup system,
as any delay in the OSCE exam is untenable.
When the team looked at potential technology, there emerged two principal approaches to supporting
the OSCE process. These were:
Hand held devices together with an electronic forms software application. The types of devices
considered were:
Mobile phone
iPad
Tablet PC
Netbooks
Digital Pens together with form management software.
Digitised forms and digital pens
Digital pen and sensor with non-digitised forms
Preliminary research into the technologies and their suitability to support the above process indicated
that this list could be narrowed down still further. The consensus was that the screens available on
mobile phones are too small for marking sheets, allowing examiners to see only a fraction of the sheet
93

Tim Cappelli
at a time and increasing the possibility for incorrect data entry. Feedback from stakeholders indicated
that interaction with the small screens would be difficult for examiners.
Further, a review of the digital pens and sensor solution suggests that, whilst extremely useful for
capturing notes, this approach would be problematic for forms such as the marking sheets. It would
be very difficult to ensure that the student and OSCE data were correctly matched, to verify that the
marks were as intended or to easily compile the marks into a global spreadsheet for analysis.
Removal of these options left the comparison of hand-held devices such as Tablet PC, Netbook or
iPad systems with the digital pen and digitised forms systems. An analysis of both these approaches
against the objectives of the pilot revealed that all these technologies could potentially achieve the
objectives. Hence, the choice of technology rested on which offered the cheapest and most practical
solution. Each approach was assessed against the set of criteria above and the results are shown in
Table 1 below.
Table 1: A comparison of touch based devices with digital pens
Criteria Tablet PC based solution
Handheld device with electronic
forms software on the client and/or
server. Examiner interacts with the
Data security
Performance
Connectivity
Data integrity
Scalability
Human
Computer
Interaction
Training
Robustness
device with stylus and/or finger
Data can be cached securely on the
local device and transmitted or
downloaded in encrypted format
Data can be captured locally on the
device without the need for
connection to the network. Devices
generally reliable but are at risk from
battery or other failure
Data can be stored on the local
device and transmitted or
downloaded when possible.
Data can be stored on the local
device and could be checked on
each device, or downloading to a
local PC prior to transmitting to
Central Office for collation
Scaling to all base hospitals would
require the purchase of 16x4 (64)
devices with sufficient reserves as
back up
Touch devices are dependent on the
software that presents the form. The
software could be designed to be
intuitive and allow for corrections,
etc.
Some training would be required for
any examiners unfamiliar with touch-
based technologies.
The devices tend to be reliable
though a major device failure that
results in all data loss cannot be
ruled out.
Set up Costs Cost per device: £600+
Software: £20k +
Licence cost (per device): £180
Maintenance
Costs
Training costs: £3500
Ongoing support: £2500
Forms cost: £0
Digital Pen based solution
Digital pens and digitised forms. Forms
management software on the server. Examiner
interacts with the system with the pen and paper
in the same way as currently
Data can be cached securely on the local device
and transmitted or downloaded in encrypted
format
Data can be captured locally on the device
without the need for connection to the network.
Devices generally reliable but are at risk from
battery or other failure, though the use of hardcopy
forms ensures the test can continue and
data captured later.
Data can be stored on the local device and
transmitted or downloaded when possible.
Data can be stored on the local device and could
be checked by downloading to a local PC prior to
transmitting to Central Office for collation
Scaling to all base hospitals would require the
purchase of 16x4 (64) devices with sufficient
reserves as back up
The pens look and feel like an ordinary pen and
are therefore already familiar to examiners. The
software that comes with the pen covers the
required features.
No training should be required, but rather a
highlighting of the pen’s features in the current
briefing
The devices tend to be reliable though a major
device failure that results in all data loss cannot
be ruled out. Fortunately, the continued use of
hard-copy forms and ink with this approach
provides a back-up without further investment.
Cost per device: £120
Software: £500
Licence cost (per device): £315
Training costs: £0
Ongoing support: included
Forms cost: £250
On the basis of the data gathered, the digital pen based solution appeared to offer a number of
advantages, namely:
94

Tim Cappelli
It was closest to the examiners’ current way of working so does not require acceptance of new
technology and training is negligible.
If the technology fails for some reason, there is always a backup in the form of the paper forms
that have been completed.
The technology is lightweight, simply stored and maintained.
The solution is scalable with no additional cost for software on multiple servers.
The cost of the solution overall is the lowest.
Based on the team’s recommendation, the Medical School went ahead with a pilot of the digital pens
and forms in November 2010.
4. The pilot
It was decided to run the pilot with the selected technology at one of the base-hospitals in order to
assess the viability and practicalities of the pens, how well the stakeholders responded and how well
the devices fulfilled the original objectives.
The team invited proposals from a number of digital pen providers. From a series of email exchanges
and follow up interviews, Ubysis Technologies (http://www.ubisys.co.uk/) were chosen based on their
experience in the Health Service and their accompanying software, Formidable. Formidable is a
server-based application that lets users design, create, print and manage digital forms. It also
interacts with the pen-software, Pen Pusher, to enable the upload and automatic checking of forms
prior to exporting the results to Excel or another data analysis tool. This end-to-end support appeared
to offer the best solution in supporting the OSCE process and a pilot was agreed.
The pilot consisted of an initial dummy run of the pens using a number of dummy OSCE forms
provided by the School and created in Formidable by Ubysis. Members of the development and
OSCE administration teams played the roles of students and examiners. The purpose of the dummy
run was to identify any issues or pressure-points in the process that might occur in the real exam, but
also to reassure OSCE administrators that the technology was reliable and fulfilled their requirements.
Following the dummy run, the School decided to go ahead with a pilot on a real OSCE. Year four
OSCEs were chosen at one of the hospitals and it was decided to run the pilot across 8 stations, with
the other 8 using traditional pens and forms to act as a direct comparison. The pilot would stretch
across 4 cycles per day for 4 days; in total, 1024 forms would be marked using digital pens. The
Medical School purchased 10 digital pens (8 for the pilot and 2 back-up) along with the necessary
licences and a ‘shelf’ of printing patterns that was required to print the required number of unique
forms.
The OSCEs took place in December 2010, and during November, Ubysis worked with the
development team to create and print the forms, train staff on the use of Formidable and ensure all
the technology was ready prior to the OSCEs. They also provided a member of staff to be present on
the first day of the pilot in case of any software or hardware issues occurring.
For the pilot, Formidable was installed on a development server, and two network-connected laptops
were placed in an IT room where the exams took place. Both laptops had docking devices linked via a
USB port, in order to upload the data from the pens to the server. Although it is possible to transmit
the completed forms directly from the pen to the server using a linked mobile phone, it was felt this
would result in additional cost, more confusion for the examiners and a higher risk of data loss.
Instead, it was decided the forms would be downloaded via the laptops at each break, after which
they could be checked for completeness both automatically by the software, and also by the
administrators as a fail-safe.
Forms downloaded automatically once the pens were placed in the docking device and an image of
the mark sheet, alongside a table of the marks gathered from the form, then appeared on the screen.
The software automatically checked for any missing marks and for the presence of text in the
feedback field if the overall score was less than 4. A visual inspection by the administrators ensured
that wherever there was more than one mark for any criteria, that the intended score had been
accepted. Following these checks, each form was saved on the server. At the end of each day, all
95

Tim Cappelli
forms were processed and the data (pdf copies of all the forms and a csv file of all the scores) made
available to the Medical Exams Office (MEO) over a secure link.
Prior to every OSCE cycle, all examiners attend a compulsory briefing session on how to complete
the forms. During the pilot, a single slide was added to this briefing to inform examiners about the
technology and that the pens would be collected during the break. They were not required to change
their practice in any way.
5. Evaluation
The pilot was observed and key points recorded throughout by the development team. In addition,
participating examiners were given questionnaires to complete and examination and support staff
were interviewed two weeks after the event.
5.1 Feedback from stakeholders
Examiners: 54 examiners took part in the pilot and 42 completed the evaluation questionnaire (78%).
91% of examiners felt that using the pens was the same as normal OSCE exams and 7% felt they
were better. 50% of examiners felt positive and 36% neutral about providing written feedback for all
students. Two examiners (5%) reported feeling displeased about this citing extra burden for
examiners and questioning whether the purpose of an OSCE exam was to provide teaching for the
students. There was also a desire for more information about the technology i.e., what it is for, how
accurate it is. In terms of using the pen, examiners commented that, apart from being a bit bulky and
therefore perhaps altering the readability of their handwriting, the pen was just like using any other
pen.
Administrators: Examination support staff reported that the examiners were positive but surprised
when they were given the briefing in the morning. They reported that there seemed to be initial
concern from the examiners but that the examiners appeared reassured when it was just a ‘pen and
paper’. Staff felt that they needed two more members of staff on exam days than usual for docking the
pens and checking the mark sheets as they docked. However, this would not be the case if the whole
circuit had been completed using pens, since pen docking and form checking occurred at the break,
when staff traditionally collect and check the hardcopy forms. Staff did express enthusiasm to trial the
mobile connection as this would decrease their workload further (no need to collect and dock the
pens) and would allow checking to be done whilst the OSCE was ongoing. Exam staff reported that
instead of sending marks to central exams office on the Tuesday following a week of OSCEs, they
were able to send marks at the end of each day of exams. This would speed up the data analysis
and would enable data analysts to have extra time to perform statistics prior to the exams board.
5.2 Achievement of objectives
Pulling together the observed data and feedback from stakeholders, the outcomes of the pilot were
assessed against the original objectives. These are shown in Table 2.
5.3 Return on Investment
Given the potentially high costs of implementing the digital pens across all the hospital bases, it was
decided to complete a Return on Investment impact analysis, using the potential efficiency savings in
time and resources identified in the pilot. This was designed to sit alongside the evaluation of the
objectives in order to demonstrate the period of time required before any financial savings on the
initial investment were realised. Although financial savings were not seen as a driving force for the
pilot, any evidence of efficiency savings could be used to justify the large initial outlay required for
technologies to be adopted.
The impact analysis was carried out using the JISC InfoNet Impact Calculator
(http://www.jiscinfonet.ac.uk/impact-calculator) which allows articulation of the business processes,
the nature of the benefits to be achieved, the capture of performance data for each of the benefits and
recording of the costs associated with realising the change. The calculator was completed, focusing
on the savings in staff time and resources afforded by the adoption of the technology across the
hospitals. Given the large number of forms involved, the few minutes saved in checking each form
and the few pence saved in not photocopying each form soon accrued to a substantial amount.
However, the largest savings came from no longer having to transport the forms manually from each
96

Tim Cappelli
hospital back to the MEO, and from MEO staff no longer having to manually input rejected forms. In
total, the annual savings from introducing digital pens across all the hospitals was calculated to be
over £12,500 per year. Given the initial set up costs and ongoing support, maintenance and licensing
fees, the pens would therefore become funding neutral after four years, after which they would
continue to save the Medical School, year on year.
Table 2: An evaluation of the digital pens pilot against the original objectives
Objectives
Evaluation Impact
Faster results and
feedback to students.
Provision of timely
textual feedback to all
students, not just
those who performed
poorly.
Reduction of timepressured
manual
handling of the
marking sheets
(checking correct
completion, dealing
with exceptions,
manual data entry,
transfer of forms).
Less labour-intensive
process and faster
preparation of forms
for the different OSCE
exams with less
manual form
preparation.
The pilot demonstrated that it is possible
to provide the mark sheets, scores and
text feedback immediately to the MEO on
completion of an exam.
Once checked by MEO and exams
board, the forms with the scores and
feedback (as pdfs) could be made
available to students in electronic form.
The pilot was designed to provide a pdf
image of the text feedback for students
Having the possibility to provide an image
of the mark sheet or of the feedback text
alone provides the facility to give
feedback to all the students, whatever
their overall score.
Initially, the digital pens were taken from
examiners during ‘breaks’ in the cycle to
upload and check forms on the laptop.
However, it became apparent that
examiners often returned to a student’s
form later in the cycle to amend the entry,
meaning the form had to be re-checked.
Hence, it was decided to leave the
upload and checking to the end of each
cycle.
It was not necessary to photocopy the
mark sheets. Once checked and saved,
each exam batch of results was
converted to a CSV file and sent to MEO.
The necessity to scan forms, deal with
exceptions and manually input incorrect
forms was removed, saving significant
time.
The forms for this pilot were created
using Formidable, The forms preparation
was done by the Ubisys technical support
team for this pilot. They also prepopulated
the forms with student data
provided to them a short time before the
exam.
The same number of forms for the OSCE
stations using the pen was created as
they would be using the manual process
– i.e. one form per student per station.
97
Students would receive results faster
than at present and could receive the
feedback without having to come to
exam administrators or their tutors to
see their mark sheets.
Provision of feedback with the digital
pens will require some additional
training for OSCE examiners. They
must keep the text within the box
provided and it may require them to
write more legibly. This can be included
in the current training and briefing of
examiners.
Providing feedback legibly appears to
be a small change to current practice
and the impact of providing timely
feedback to students could be
extremely positive.
Once the administration staff were
familiar with the process, the time taken
to upload and check the forms on the
laptops was significantly less than the
time taken to collect, collate and check
the forms manually.
The business rules written into the form
software made checking the mark
sheets easier, and in the future
additional rules could be included to
further aid this.
There is a significant saving in the time
and cost of photocopying the mark
sheets.
The time taken for collection and
collation of data is significantly reduced,
saving central office staff considerable
effort.
The Formidable software allows
preparation and pre-population of
forms. Some further analysis is required
to establish the training and skills
needed for MEO to do this in future.
There is a significant saving in the use
of paper (only one copy is created and
no photocopying done) and in the
moving and storage of paper forms.

Objectives
More efficient and
faster preparation and
transmission of data
for analysis by the
different roles involved
in administration,
management and
quality assurance.
6. Conclusion and next steps
Tim Cappelli
Evaluation Impact
The exams data was sent from the base
hospital to MEO on the day of each
OSCE exam. It was sent electronically
over a secure link rather than as bundles
of paper forms, reducing the time, cost
and security of data transmission back to
the centre.
Because the data is already in
electronic format, the data can be easily
collated and assimilated into existing
systems for analysis and distribution.
Having business rules in the software
that instantly flag incorrect forms or
missing data before the examiners
leave the site reduces the number of
errors and facilitates the QA of the
process.
The pilot of the digital pens was evaluated and reported on to the Senior Management Team of the
School together with a set of recommendations for large-scale roll out across all the base hospital.
The recommendations were:
Clarification of the data requirements of the Medical Exams Office (MEO).
Demonstration of the process of using the pens to MEO staff.
Assessment of the value of providing timely feedback to students: Another pilot, similar to the
first, should be run. This time the results from the pens, including the textual feedback, should be
passed directly to the students once the data has been collated, checked and approved for
distribution.
Test the viability and reliability of the mobile phone upload.
Exploration of vendor support: Although Ubysis provided adequate support and advice prior to,
and during, the pilot, it would be prudent to seek assurances that this level of support will continue
in any future
Training of Base administration staff in downloading and checking the pen’s content.
Training of OSCE examiners: Some examiners expressed concerns over lack of training to
provide appropriate feedback.
The first two points were particularly important as there had been a great deal of resistance to the
introduction of the technology from the MEO. This was partly due to a lack of communication and
resulting misunderstanding of the technology and partly due to concerns about the technology making
certain roles redundant. The result was that despite the pilot, MEO staff were still opposed to the
technology and promoted the view the pilot had been unsuccessful. This ‘dis-information’ rapidly
became the prevailing view and work was required to assure the concerns of the MEO staff. This
demonstrates the importance of embracing all stakeholders at the start of any technology change.
At present, the SMT are reviewing the results of the pilot, together with other initiatives in student
feedback, to determine the most appropriate way forward. A final decision will be based on available
resources and the views of staff and students.
Acknowledgements
Thanks to Dr’s Hilary Dexter and Lucie Byrne-Davies from the Manchester Medical School for their
work on the student evaluation that informs much of this study.
References
Buzzetto-More, A. and Alade, J.A. (2006) “Best Practices in e-Assessment”, Journal of Information Technology
Education, Vol 5, pp 251-269.
Gibbs, G. (2010) Dimensions of Quality, The Higher Education Academy,
http://search3.openobjects.com/kb5/hea/evidencenet/resource.page?record=12nH2AFIYcc [accessed 12
August 2011]
Harden, R and Gleeson F (1979) “Assessment of clinical competence using an objective structured clinical
examination (OSCE)”, Medical Education, Vol 13, No.1, pp39-54.
Love, T and Cooper, T (2004) “Designing online information systems for portfolio-based assessment: Design
criteria and heuristics”, Journal of Information Technology Education, Vol 3, pp65-81.
98

Tim Cappelli
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.77.6803&rep=rep1&type=pdf [accessed 15
August 2011]
Nicol, D. & Macfalane-Dick, D. (2006) Formative assessment and self-regulated learning: a model and seven
principles of good feedback practice. published in Studies Higher Education 2006, Vol 31(2), pp99-218.
http://tltt.strath.ac.uk/REAP/public/Resources/DN_SHE_Final.pdf [accessed 12 August 2011]
Ridgway, J. and McCusker, S. and Pead, D. (2007) ’Literature review of e-assessment.’, [online].
Futurelab, Bristol http://dro.dur.ac.uk/1929/1/Ridgway_Literature.pdf?DDD29+ded0kmt [accessed 15 August
2011]
99

Digital Educational Resources Repositories in Lower and
Middle Education in Portugal: Quality Criteria in the
International Context
Cornélia Castro 1 , Sérgio André Ferreira 1 and António Andrade 2
1
School of Education and Psychology, Portuguese Catholic University, Oporto,
Portugal
2
School of Economics and Management, Portuguese Catholic University,
Oporto, Portugal
corneliacastro@gmail.com
sergioandreferreira@gmail.com
aandrade@porto.ucp.pt
Abstract: The thematic of Digital Educational Resources (DER) and digital educational repositories at primary
and secondary education levels is of growing centrality in the educational themes in the international debate. In
order to make it easier for teachers and students to find the best and relevant learning resources and to
encourage teachers’ uptake of innovative materials and learning styles, authorities and/or companies in
European countries have launched web based educational repositories. Following this trend, about two years
ago, three major institutions in Portugal have launched their repositories: Portal das Escolas (ME), Casa das
Ciências (FCG) and BOA (INESC-ID). After examining the state of the art of educational repositories in Europe, it
was found that those three Portuguese repositories are not mentioned in recent European reports and so the
study here presented seemed to be very relevant. This work aims to develop a descriptive and comparative
study, in order to characterize the reality of the Portuguese repositories and compare it with European examples
of good practices. To guide this investigation, the following question was theorized: “How are the Portuguese
institutional repositories, for primary and secondary education levels, positioned in what concerns international
quality criteria?” Our methodology design is a qualitative one and of type i) descriptive, as it is intended to
characterize the main Portuguese institutional DER repositories for primary and secondary school education,
aiming to assess the key dimensions that affect their quality and ii) comparative since it has in view comparing
those repositories with reference European ones. In this sense, Lektion in Sweden and Scoilnet, Portal for Irish
Education in Ireland are considered. The study was undertaken from the view of the DER producers and users.
As a tool for data collection it was used a framework of analysis built on the Guía para la Evaluación de
Repositorios Institucionales e Investigación (GERII), a joint publication of the Spanish Ministry of Science and
Innovation and other organizations in the fields of Education, Science and Technology (Villar et al, 2010). The
GERII guide includes a set of guidelines for the creation and evaluation of a repository, bringing together a
battery of 31 evaluation criteria, distributed over seven dimensions that any repository must accomplish to meet
quality and therefore was considered an excellent basis for our analysis. Comparing the three Portuguese
repositories, target of this study, with two reference European repositories in order to contextualize the national
reality in the international arena will stand for the Portuguese institutions that hold the repositories as an
opportunity for improvement. Thus, Portuguese teachers will have an increased chance to access an eclectic
collection of quality resources, in order to diversify their teaching strategies.
Keywords: comparative study; digital educational resources; evaluation; quality; repositories
1. Introduction
The 2010 edition of the Horizon Report (Johnson et al., 2010) refers to the Open Content as an
emerging trend. In fact, we are witnessing a growing volume of digital content from different sources
(OECD, 2007) in the education field. It is shared in the cloud DER which: i) contain inherently
educational purposes; ii) are framed on the needs of education systems, iii) have identity and
autonomy in relation to other objects and iv) meet the defined quality standards (Ramos et al., 2010).
This environment of sharing and reuse makes the access, by teachers, to a wide range of resources
easier, increasing the exploitation of more diversified contents (OECD, 2007), which is considered as
an important condition for the improvement of quality in education.
In this study we aimed to develop a descriptive and comparative study in order to characterize the
reality of three Portuguese repositories and compare it with European examples of good practices in
order to evaluate how Portuguese institutional repositories stand in the international arena. To
achieve that we established the following research question: “How are the Portuguese institutional
repositories, for primary and secondary education levels, positioned in what concerns international
quality criteria?”
100

Cornélia Castro et al.
After examining the state of the art of educational repositories in Europe (EdReNe 2009, Poulsen,
2008; Dumitru et al, 2009), we verified that the variety in DER repositories in Europe is huge and the
approaches taken by the various countries (22, altogether) differ. However, all countries and
developers share the objective of making resources available and visible to the users and face the
same challenges of reaching many users, managing rights and adapting standards (EdReNe, 2009:
4). The same state of the art showed us that three major Portuguese repositories: Portal das Escolas
(Schools Portal – ME), Casa das Ciências (House of Sciences – FCG) and BOA (Learning Objects
Exchange – INESC-ID), were not mentioned in recent European reports although the 2 nd Strategic
Seminar of EdReNe took place in Lisbon on June 9 th – 11 th , 2008 [exception for Portal das Escolas
once, under the Portuguese Technological Plan for Education (PTE) and its three axes, “(…) the
content area will be supported by a Schools Portal to be used by teachers and students in schools all
over the country (…)” (EdReNe 2009: 68)]. We considered these to be the main Portuguese
repositories for the following reasons: 1 st – a varied typology of resources; 2 nd - curriculum areas
covered; 3 rd - number of publications; 4 th - number of downloads made; 5 th - number of visits.
Lektion, a web community for teachers, created in 2002, was chosen because it is Sweden’s largest
teachers’ site, a so called “(…) meeting place for teachers and all who work in Swedish schools (…)”
(Lektion.se, 2011). Scoilnet is the department of Education and Skills official portal for Irish education
(Scoilnet.ie, 2011) and was selected because it is an institutional repository whose resources are
constantly being reviewed by a team of over 30 curriculum experts. Besides, it allows the users to
meet the repository at their local school web portal. Both Lektion and Scoilnet were still selected
because i) their url was available; ii) their diversity of resources and iii) they were launched well before
the Portuguese ones.
The relevance of this study is also justified by the fact that the results could prove to be inputs to the
three institutions holding the Portuguese repositories analyzed, with the goal to continue acting
towards the improvement of their quality.
2. Repositories drivers and barriers
The repositories are online systems intended to archive, preserve, make available and disseminate
the intellectual output of a community, available in a digital space where data and information are
stored and updated (Ramos, 2010; Poulsen et al., 2008 ) or "digital store boxes that host collections
of digital resources in a learning object format: i. e. resources that are designed to be integrated,
aggregated and sequential in an efficient way to produce “units of learning” that are meaningful to
learners” (Margaryan, 2007: 3).
2.1 Drivers
As there are several potentialities in a repository, we have decided to stress some of those
established by the Organization for Economic Co-Operation and Development (OECD, 2007):
To facilitate the change of teaching practices;
To stimulate more interactive and constructivist teaching practices;
To promote the use of DER as a complement or substitute of teaching in the classroom;
To make collaborative approaches in teaching easier and faster (OECD, 2007; GEPE, 2009);
To induce and streamline the production and usage of tools, content, resources and information
on digital support;
To minimize the digital divide, allowing remote access at low cost to content, modules and
courses;
To enhance the inclusion in teaching and learning of people with special needs;
To develop and strengthen a culture of lifelong learning
2.2 Barriers
Studies identify several barriers that limit the access and use of repositories (Collis, 1995; Davis et al.,
2010; OECD, 2007, Pawlowski and Zimmermann, 2007), such as:
Technical: lack of broadband availability;
101

Cornélia Castro et al.
Economic: lack of resources to invest in hardware and software required to develop and share
DER;
Social: lack of skills to use technical inventions; communication style;
Cultural: resistance to share or use the resources produced by other teachers or other institutions,
cultural perceptions of the teacher and student’s roles;
Political;
Legal: intellectual property and copyrights.
3. Digital Educational Resources repositories
The objects of this study are the three most visible institutional repositories of DER in Portugal, in
primary and secondary education: Portal das Escolas, Casa das Ciências and BOA, which are
compared with the Swedish Lektion and the Irish Scoilnet. The three countries – Ireland, Portugal and
Sweden – are members of EdReNe thematic network (edrene.org, 2011) which was established in
2007 with support from the European Union's eContemplus programme (ec.europa.eu, 2011). The
overall goal of EdReNe "is to improve the provision of and access to learning resources" (Poulsen and
Lund, 2009: 86).
3.1 Portugal
3.1.1 Portal das Escolas
The Portal das Escolas, launched on the 22 nd June 2009 and addressed to the educational
community (teachers, students, parents and non-teaching), intends to establish itself as the
institutional platform par excellence in the educational community, through which the Portuguese
teachers can produce and access, anytime and anywhere, a wide range of online services that add
value to the teaching and learning process. This repository provides "material and reliable
information", adopting the ME mechanisms for validation of educational resources (ME, 2010).
3.1.2 Casa das Ciências
The Casa das Ciências project was created by the Fundação Calouste Gulbenkian (FCG), a
Portuguese private institution of public utility whose statutory aims are Art, Charity, Science and
Education.
It stands as "an integrator and amplifier vehicle of current efforts in using Information Technology in
teaching and learning, made by many different agents and whose results are scattered” (FCG, 2010),
establishing itself as a subject repository. This repository "aims to give visibility and usefulness to the
efforts of many teachers, recognizing the merit they actually have and becoming a website of
reference for all Portuguese Science teachers" (FCG, 2010). All the materials submitted are assessed
by referees from a scientific and educational point of view, in an approach to peer-review. This quality
control of the submitted materials complies with a set of rules stated in a regulation, which considers
the awarding of prizes to the DER considered of higher quality, as a way to encourage the
submission.
3.1.3 BOA
The BOA/VemAprender (INESC-ID, 2010) project is a research initiative sponsored by the Information
Systems Group of INESC-ID at Lisbon with the aim of designing, developing and evaluating the
implementation of collaborative platforms within teaching and learning contexts. Its main purpose is
the creation of a web platform to which the educational community of various levels of education can
access. This way it aims to enable everybody to participate collaboratively in building a repository of
learning objects, with a significant set of associated information. This initiative intends to foster the
creation of a community of practice that is a facilitator and an enhancer of quality in the teaching
profession. In this repository, when submitting a DER, the author gives a certain value – credits – to
its object. The value initially given is regularly updated according to the number of downloads made,
this being the mechanism that allows the evaluation of the object submitted.
102

3.2 Sweden
Cornélia Castro et al.
3.2.1 Lektion
Lektion, the largest Swedish teacher site, created in 2002, is a repository owned and developed by a
web company named Lektion Sweden AB. This company, located at Helsingborg, took the initiative of
starting the Lektion.se site to connect Swedish teachers in an internet community. Lektion.se is an
idea with suggestions for lessons by teachers, for teachers and the lesson database is referred to be
gigantic, containing thousands of tips and ideas for teaching. This concept of community is favoured
by the fact that the teachers can share ideas about the DER through a forum. There is a Vendor
Archive where teachers can find everything to do relating school and even find a job in the Jobs
Archive.
3.3 Ireland
3.3.1 Scoilnet
Scoilnet is the official education portal of the department of Institutional Education and Science in
Ireland. This repository is responsible for the promotion and use of the internet in education under the
Government Information and Communication Technologies in Schools Programme. Launched in
1998, the website is managed by the National Centre for Technology in Education (NCTE, located at
Dublin City University). Its current interface was launched in 2003 with design tweaks in September
2009 (scoilnet.ie). Scoilnet contains a database of over 11 800 digital resources (theme pages,
reviewed websites, quizzes, lesson plans, crosswords and other multimedia). The content of the
Encyclopaedia Britannica Online and of Word Book Online was specifically authorized for educational
use within the site Scoilnet. Curriculum focused, it acts as a support to teachers, students and
parents. Scoilnet promotes actively the integration of Information and Communication Technologies
(ICT) in teaching and learning.
4. Methodology
The type of study followed in this research fits the descriptive and comparative ones. The main
institutional repositories of DER for primary and secondary levels of education in Portugal are
characterized, the key dimensions that affect the quality are assessed and they are compared with
other European repositories. To do this study it was required to previously identify the dimensions to
be evaluated (Vilelas, 2009).
With that purpose, the Guia para la Evaluación de Repositorios Institucionales de Investigación
(GERII), published jointly by the Ministry of Science and Innovation of Spain and other organizations
in the field of Education, Science and Technology, was adapted (Villar et al., 2010). The GERII
includes a series of guidelines for the creation and evaluation of a repository. For that function it
gathers together a set of 31 evaluation criteria, distributed over seven dimensions that any quality
repository must meet and therefore it was considered an excellent base for our analysis.
In the present study, we chose to make some adjustments to the evaluation model followed in GERII.
Thus, we considered five of its seven dimensions and it was decided not to include the dimensions:
Interoperability and Security, Authenticity and Integrity of Data because: i) since they are essentially
technical dimensions, they do not fit in the methodology of data collection defined and ii) it is intended
to perform an evaluation of the repositories only in the perspective of producer and user.
In table 1, and according to the guide GERII, we present the criteria to be considered in any
evaluation of the five dimensions considered in this study:
Table 1: Dimensions to be evaluated
Dimension Evaluation Criteria
Visibility
Visibility in the major search engines; existence of a standard name and a
proper url; presence of incentives for participation and information on the
mode of submission.
103

Cornélia Castro et al.
Dimension Evaluation Criteria
Policies
Disclosure of the mission, goals and function; information on who can submit,
what can be submitted and in what formats; information on documents
archive policies and preservation of content; forms of contact and support.
Legal aspects Information on the intellectual property of DER present in the repository.
Metadata
Statistics
Information on formats used; use of classifiers or definition of a policy of
indexing content.
Mechanisms to record access logs to the server and access, use and
download of DER.
The methodology of data collection was based on the analysis of the repositories websites. Thus,
each repository was classified for each criterion of the five dimensions and the evaluation was made
from the point of view of the producer and user of DER.
5. Results
The study results allowed a global assessment of the quality of the five repositories of DER for
primary and secondary levels of education, from the perspective of the author and the user of DER
and are presented in a comparative summary for each of the dimensions assessed.
The following table 2 summarizes the Visibility dimension results.
Table 2: Dimension: Visibility
Advantages Limitations
Excelent visibility in the search engines.
The website homepage of the Portuguese
Standardized name of repositories (except BOA, institutions makes no reference to the
facilitating its identification).
repository.
Apropriated url.
Lack of public disclosure in the case of BOA
Incentives for participation in the repository.
(limitations not applicable to Lektion).
Clear information on how to submit RED.
In this dimension, there aren’t significant differences between the Portuguese repositories and the foreign
ones.
In a search for repositories, by "denomination", in the three most used search engines (Google, Bing,
Yahoo) (SEO, 2011), they all appear as the first result. The exception is Lektion that in Yahoo
appeared in third place.
The fact that the ME, FCG and INESC-ID homepages do not make reference, in their internal search
engines, to their repositories, may lead the target audience to consider the repositories as not worthy.
This may be evidence that the repositories are not central in the policies of the institutions that hold
them.
In all the analyzed repositories, information on how to submit resources is published and messages
encouraging the participation and collaboration in them are promoted. It is noteworthy the dynamics of
the repositories Lektion and Scoilnet which present a video explaining how to find a resource and/or
how to make a submission. The Scoilnet repository also presents additional information on 10 ways to
use Scoilnet.
The main results of the dimension Policies are summarized in Table 3.
104

Table 3: Dimension: Policies
Cornélia Castro et al.
Advantages Limitations
The mission, goals and functions of the repository are present.
It is established how users can upload the content, what kind of
content is accepted and in what formats.
There is a requirement to register in order to download a
resource (Portal das Escolas and Scoilnet are
exceptions 1 ).
Several ways of contact and support are available.
There isn’t established a policy of
preserving content.
Unlike the Portuguese, the Swedish and Irish repositories have a video demo of their features.
1 Navigation is not subject to registration but some features are only accessible to teachers who are registered users.
It appears that institutions do not declare their commitment to make content available on an ongoing
basis and to take measures for their preservation (as in the case of migration, for example) to ensure
access to them and create and maintain file formats. This condition can contribute to the reduction in
the confidence of authors, inhibiting their collaboration. This less favorable aspect occurred in the
Portuguese repositories as well as in Lektion and Scoilnet.
The Legal Aspects were compared and with regard to compliance with the fundamental requirements
of authors’ rights guarantee, we present a summary in table 4.
Table 4: Dimension: Legal Aspects
Advantages Limitations
Confirmation by the author during submission, that the material
does not violate any intellectual property right (BOA is an
exception and it was not possible to confirm in Scoilnet).
Request for an authorization from the author that allows the
Not identified.
distribution of content (not possible to confirm in Scoilnet).
Existence of information on copyright like Creative Commons or
any other.
Like Lektion, the Portuguese repositories adopt the Creative Commons philosophy in response to the
protection of copyright. Scoilnet refers to Irish and European legislation, not materializing the type of license.
The safeguard policy of intellectual property and copyright encourages the dissemination of ethically
correct behaviour in the production and use of DER.
The enormous amount of DER available in the repositories makes it important that they are described
by metadata (table 5). These allow the recognition of the repositories both by humans and computers,
so that they can be located by using different criteria. This is a capital issue since the resources will
be of little use if they cannot be easily found.
Table 5: Dimension: Metadata
Advantages Limitations
There are standard procedures for indexing DER (indication
of the author, description, topic, subject, school grade,
keywords, target audience, resource type or time).
Not identified.
The metadata allow search by standard criteria.
Metadata facilitate research of DER into the archives of the
repository.
In this dimension, there were no significant differences between the Swedish and Portuguese repositories
(couldn't be confirmed in Scoilnet).
Upon uploading the authors are asked to categorize the resources through a series of standard
criteria such as author, description, topic, subject, school grade, keyword or time, for example.
Similarly, the repositories enable a search of DER by the same criteria.
Finally, the dimension Statistics (table 6) is also important, both from the perspective of the user or
from the author.
When you use a repository, the statistics for a given DER, allow you to realize, for example, which
resources were the most voted, information that can help you on the decision to download a particular
resource, helping to give it visibility. From the author’s point of view, this knowledge will give him
105

Cornélia Castro et al.
feedback on the interest that his own DER created in the community of peers and will encourage him
to produce and share more resources. Knowing the number of visitors to a repository helps the
teacher/user to understand its importance and so he starts seeing it as a tool that he might consider
using to facilitate his work and diversify his practices. The existence of various statistics relating to the
access to the repository is also valuable to the institution that holds the repository, as it provides
information on the dynamics of the repository which is very important to its management.
Table 6: Dimension: Statistics
Advantages Limitations
They report on the number of available and/or published objects.
They refer to the number of downloads and/or visits made.
They indicate the number of visitors (Casa das Ciências and Portal
das Escolas have several historical data).
Not identified.
They present the latest, top rated, most popular and/or most visited
DER.
The authors that share more learning objects are highlighted (BOA).
In this dimension, there were no significant differences between repositories, although it was noted that
Scoilnet does not indicate some of the items evaluated in this dimension; in addition, by not admitting the
login, it did not allow the identification of possible limitations.
6. Conclusions
Repositories contribute to the opening of the classroom to the world. The DER online availability
allows them to be accessed from anywhere, anytime and by anyone who has appropriate training and
an Internet connection.
The data presented in table 7, concerning the implementation of each one of the repositories
evaluated in this study, as well as the discussion of the results, allowed us to answer to our initial
research question: “How are the Portuguese institutional repositories, for primary and secondary
education levels, positioned in what concerns international quality criteria?”.
Table 7: Data on the implementation of the evaluated repositories
PORTUGAL SWEDEN IRELAND
Portal das
Escolas
Casa das Ciências BOA Lektion Scoilnet
Inhabitants 10 647 763 1
9 354 462 2
4 470 700 3
Teachers 115 680 4
Registered Users 83 000 7
Published DER 2 620 7
Teachers/registered
teachers
Registered
teachers/published
DER
1,4:1
8 226 7
1 407 7
Not applicable for being a
subject repository
326 7
168 7
355:1
32:1 6:1 2:1
78 400 5 55 052 6
229 123 7 4 000 7
Not available 11 800 7
No possible
evaluation
No answer from
request to
Lektion
14:1
1 Estimate, National Statistical Institute, February, 2011; 2 2009, most recent census; 3 estimate
2010, CSO; 4 GEPE, ME, 2009; 5 2003; 6 2007; 7 Data repositories, May, 2011.
The data shows that Portugal is, as far as population size and number of teachers is concerned,
closer to Sweden than to Ireland. Although Ireland is the smallest country in terms of population and
number of teachers, it has the highest number of published DER. The reasons for that might be: i) the
fact that the repository is quite user-friendly, allowing the teacher to file his preferences in the tab My
stuff, ii) it is possible to submit any resource without peer-review (the teacher is the sole responsible
for the material he submits and he can leave it in the repository for as long as he wants or take it off
when he wants) and iii) the age of the repository.
The number of registered users in the Portuguese repositories is substantially smaller than in the two
foreigner ones. This probably happens because of the youth of our repositories. However it is
106
1:3

Cornélia Castro et al.
noticeable that there has been an effort to keep increasing levels of quality since their creation. The
ME Portal das Escolas requires the registration of teachers so that they can be certified in terms of
ICT skills, which possibly explains the ratio 1,4:1.
Taking into consideration the few limitations detected for the Portuguese repositories in the
dimensions evaluated on the basis of GERII, it appears that the solution of some of them
(unavailability of broadband and the lack of resources to invest in hardware and software, for
example) does not depend directly from the institutions that control them and others (the institutions
homepages that make no reference to their repositories or the lack of definition of a public policy of
preserving data) can be easily minimized.
The Portal das Escolas has recently launched (March 10 th , 2011) an area of educational blogs and on
May 30 th , 2011 the Casa das Ciências launched a bank of images and the first scientific encyclopedia
written in Portuguese: the WikiCiências with a collaborative basis and constantly quotable on the
Internet. We should highlight the uniqueness of this repository which, being related to a subject, is
totally different from any of the two European repositories analyzed. For the sustainability of DER
repositories in Portugal it will be important to foster communities of practice of which the Casa das
Ciências seems to be the most paradigmatic one.
As a result of this comparative study, it was found that the foreign repositories meet certain
requirements not found in the Portuguese ones. Thus, we present the following proposals for
inclusion in the Portuguese repositories:
Video demonstration on the process of submitting a resource and/or all the repository functions;
Different tabs with information for parents and students;
Forum for sharing ideas among users;
Diversify the user generated content like blogs, wikis, twitter and podcasts in a logic of Web 2.0
technology;
Engage in the provision of a web hosting service to all schools (in Portal das Escolas). This would
enable schools to easily develop and host their website on a school’s broad band network and to
assume as a repository that actively promotes the integration of ICT in teaching and learning,
under the Portuguese PTE.
Navigation in the five analyzed repositories could only be made in the language of the country of
origin. It would be important to consider the development of interfaces, at least bilingual, in the
language of the country of origin and in English. The same applies to metadata. We consider this to
be a relevant step since in the European Union there are several projects of mobility of both teachers
and students and exchange programmes between schools. This would also enable all teachers to
access and adapt DER from other countries other than their own, thus creating a European network
of repositories.
The comparative study held, whose goal was to verify how the Portuguese institutional repositories, in
lower and middle education, are positioned in what concerns international quality criteria, allowed us
to conclude that the Portuguese repositories are facilitators of authors collaboration and general
users’ access. They also contribute to promoting the pursuit of new practices in the classroom by
teachers in primary and secondary schools in Portugal, as they make accessible DER with scientific
and teaching quality.
In short, it seems that the Portuguese experience, in spite of being a recent one, when compared with
other European experiences, is on the right track to achieve educational innovation. Repositories
focused on people, on the community of education stakeholders and on teaching practices will help to
overcome individual and organizational inertia and other barriers, towards the success among its
users.
References
Dumitru et al. (2009) “D1.2 European State-of-Art-Report”, [online], Lifelong Learning Programme, Education and
Culture DG,
http://www.ea.gr/ep/metaschool/deliverables/Metaschool_WP1_EuropeanStateofArtReport_V1.0_30Apr200
9_EUN.pdf (acessed on 20/01/10).
107

Cornélia Castro et al.
EdReNe (2009) “State of the art – II Educational Repositories in Europe”, [online] eContentplus,
www.edrene.org/results/deliverables/EdReNe D 26 SoA-II.pdf (accessed on 20/01/10).
EdReNe (2011) “EdReNe”, [online], http://edrene.org/ (acessed on 20/05/2011).
European Commission – CIP (2011). “Competitiveness and Innovation Framework Programme (CIP)”, [online],
http://ec.europa.eu/cip/index_en.htm (acessed on 20/05/2011).
Fundação Calouste Gulbenkian - FCG, "Casa das Ciências", [online],
http://www.casadasciencias.org/index.php?option=com_content&view=article&id=16&Itemid=46&menu=2&i
ntro=1 (acessed on 07/12/10).
GEPE (2009), Portal das Escolas. Estudo de Implementação, ME, GEPE, Lisboa.
Grupo de Sistemas de Informação do INESC-ID (2010, Dez, 17), [online], "Bolsa de Objectos de Aprendizagem,
BOA", http://www.vemaprender.net/Acerca@12.aspx (acessed on 17/12/10).
Johnson, L., Levine A., Smith, R. and Stone S. (2010) The 2010 Horizon Report, Austin, Texas, The New Media
Consortium, USA.
Margaryan, A., Milligan, C. and Douglas, P. (2007) “Structured Guidelines for setting up Learning Object
Repositories”, [online], CDLOR, http://www.academy.gcal.ac.uk/cd-lor/documents/CD-
LOR_Structured_Guidelines_v1p0_001.pdf (accessed on 17/05/11).
Lektion.se (2011) “Lektion.se”, [online], http://www.lektion.se/index.php (acessed on 20/05/2011).
Milligan C. (2007) “CD-LOR Final Project”, [online], http://www.academy.gcal.ac.uk/cd-lor/documents/CD-
LOR_Final_Report_v1p0.pdf (accessed on 17/05/11).
Ministério da Educação da República Portuguesa - ME, "Portal das Escolas", [online],
https://www.portaldasescolas.pt/portal/server.pt/community/p%C3%A1ginas/243/perguntas_frequentes/152
12#Funcionalidades%20do%20Portal%20das%20Escolas (acessed on 07/12/10).
OECD (2007) Giving Knowledge for Free. The Emergence of Open Educational Resources, Paris. France.
Pawlowski J. and Zimmermann V. (2007) Open Content: A Concept for the Future of ELearning and Knowledge
Management, in Conference Proceedings Knowtech.
Poulsen, L. J., et al. (2008) “State of the art – I Educational Repositories in Europe”, [online], eContentplus,
http://edrene.org/results/.../EdReNe%20D%202.5%20SoA%20-%20I.pdf (acessed on 20/01/10).
Poulsen, L. H. and Lund, T. B. (2009) “EdReNe – Educational Repositories Network”, Vol. 12, No. 1, pp 85-91.
Ramos, J. L., Teodoro, V. D., Fernandes, J. P. S., Ferreira, F. M. and Chagas, I. (2010) Portal das Escolas:
Recursos Digitais para Portugal. Estudo Estratégico, ME, GEPE, Lisboa.
Scoilnet (2011). “Scoilnet Portal for Irish Education”, [online], http://www.scoilnet.ie/Default.aspx (acessed on
20/05/2011).
SEO Consultants Directory (2010) "Top search engines for 2010", [online], http://www.seoconsultants.com
(acessed on 18/05/11).
Vilelas J., (2009) Investigação – O processo de construção do conhecimento, Edições Sílabo, Lisboa.
Villar C. A., et al. (2010) "Guía para Evaluación de Repositorios Institucionales de Investigación", [online],
Ministério da Ciencia e Innovación de España, FECYT, Recolecta e CRUE,
www.acessoaberto.usp.br/guiavaliacaorepositorios2011/ (accessed on 11/02/11).
108

eLearning: Roles in Distance Tertiary Education
Ivana Cechova, Dana Zerzanova and Jana Berankova
University of Defence, Brno, Czech Republic
ivanacechova@unob.cz
dana.zerzanova@unob.cz
jana.berankova@unob.cz
Abstract: Information and communication technologies (ICT) have undergone a rapid development in
accordance with advances in the society. This state has to be reflected in the teaching process, while teaching
languages is no exception. Language training at the University of Defence (UoD) is carried out on both a full-time
as well as a distance study basis. E-learning is an integral part of both forms of language study programmes. The
possibilities of students’ involvement into the process of creating study supports, especially for teaching and
learning specific terminology were employed to determine the influence of technology-supported materials on the
students’ language learning. The co-operation with students enables their more active participation in study
material preparation and during the lesson itself, the content of which responds to the needs of particular
specialisations. This approach opens a different perspective to the teachers who prepare the study materials for
teaching special terminology in co-operation with individual departments of the faculty. The students’ involvement
motivates both sides of the educational process and influences them qualitatively. The basis for the research
were semi-structured interviews the aim of which was to find out how Information and Communication
Technologies (ICT) help the distance students in their study, how distance students assess ICT in their language
training and especially how the students assess LMS Barborka. The second part of the research was a
questionnaire survey aimed at creating a distance learner profile dealing with technology usage for language
learning. We also wanted to find out if our distance students prefer to: use up-to-date technologies, the Internet,
an in-house Study Portal and on-line sessions in language lessons be taught in a classical way (without computer
technology programs). The paper describes our research, the methods used and our respective findings and
suggests implications of the research for a broad range of educational fields. It will be of interest to stakeholders
in education and those who seek to be informed users and or promoters of computer technology for learning
purposes.
Keywords: distance learning, LMS Barborka, students’ participation, electronic study material
1. Distance education
“Foreign language is crucial to a nation’s economic competitiveness and national
security. Multilingualism enhances cognitive and social growth, competitiveness in the
global marketplace…, national security, and understanding of diverse people and
cultures. As we approach a new century where global communication will be essential for
survival, we cannot afford the luxury of international ignorance…”– The United States
Congress, Foreign Language Assistance Act of 1994
Advances in information and communication technologies (ICT), coupled with changes in society,
have had a tremendous impact on educational systems. This provides an opportunity to develop new
flexible learning environments to facilitate the learning process by providing more efficient ways of
learning and teaching. Many learners, teachers, stakeholders and institutions are coming to distance
education as a tool allowing access to education to a number of those adult students for whom the
full-time study is not possible for many different reasons (e.g. time, distance or financial constraints).
Distance education aims at delivering a quality tertiary education to students who are not able to be
physically present on campus. Although the label "distance learning" could be applied to any situation
where students are learning at remote sites, the term is normally restricted to teaching via satellite or
other long-distance telecommunication technologies. Shelley defines distance education as: “an
educational system in which learners can study in a flexible manner in their own time, at the pace of
their choice and without requiring face-to-face contact with a teacher” (Shelley 2000). According to
Cassell distance learning is "a field of education focusing on the andragogy and pedagogy,
instructional systems, and technology which endeavor to deliver an education to students who are not
physically in a classroom or campus setting" (Cassel, 2010). However, White emphasizes the
following starting points for a definition of distance education: most begin with structural concerns, a
few begin with pedagogical concerns and fewer begin with learner-based perspectives.
Organisational or structural concern includes:
The separation of teacher and learner in time and/or place.
109

Ivana Cechova et al.
The influence of an educational organisation, especially in preparing, delivering and providing
support materials.
The use of a range of media, such as print, audio, video and computer-based applications, CD or
MP3 recordings or Webcasts.
The use of communication devices to facilitate an effective communication (Internet or corporate
Intranet).
The possibility of face-to-face contact usually in the form of tutorials and courses (online courses,
web-conference courses).
The provision of a range of support services, which may include opportunities for interaction and
response to individual learners, as well as guides (White, 2003).
Pedagogical concerns comprise the extent to which “written teaching dominates in contrast to spoken
teaching and so learning by reading is stressed rather than learning by listening” Peters (1998). This
has a significant impact on the development of oral and aural skills as well as interactive skills in
distance language learning.
The learner concern includes the characteristics that a learner brings to learning. This includes the
learners’ beliefs, attitudes, experiences and any prior knowledge how to learn.
Traditional forms of distance learning that used print materials in combinations with materials on CD
ROMs are being replaced by opportunities for collaboration and interaction online (Cohen and Lippert
1999). In the case of distance language learning, course designers must consider not only how to
help learners gain communicative competence but also language learner strategies that support
success.
According to Wohlert the keys to successful distance learning are:
Live interaction between the instructor and the students during the course.
The presence of a classroom teacher in the remote sites who is involved in the learning process.
In some cases, these cooperating or coordinating teachers are studying the language along with
the students with the intention of completing a teaching minor.
The regular use of other media, such as computers, speech recognition devices, audiotapes, and
workbooks in a comprehensive approach to distance learning.
The involvement and support of school administrators.
The use of electronic mailboxes (which all students and cooperating site teachers use), or a tollfree
phone number with recording machine (Wohlert 1989).
Grubb adds that successful distance learning is impossible without honesty, responsiveness,
relevance, respect, openness and empowerment (Grubb. 2010).
Distance education allows a student to arrange the study around work and family commitments.
Understanding of the subject can be enhanced by access to online lessons, CDROMs, DVDs, text
books and other reference sources. Support and communication between a teacher and students is
possible via email, online sessions and telephone and provides web-based facilities which allow a
student to send questions, submit assignments, check the grades and access other resources, 24
hours a day, seven days a week.
2. Distance language learning
Distance language learning includes a lot of elements and practices ranging from print-based
correspondence courses to online courses with extensive opportunities for interaction, feedback,
support between a teacher and learners as well as support among learners themselves (White 2003).
Distance language learning consists of synchronous and asynchronous learning. Asynchronous
distance language learning is commonly facilitated by media such as eMail and discussion boards,
thus supporting work relations among learners and learners with teachers, even when participants
cannot be online at the same time. It is thus a key component of flexible e-learning. In fact, many
people take online courses because of their asynchronous nature, combining education with work,
family, and other commitments. Asynchronous e-learning makes it possible for learners to log on to
an e-learning environment at any time and download documents or send messages to teachers or
110

Ivana Cechova et al.
peers. Students may spend more time refining their contributions, which are generally considered
more thoughtful compared to synchronous communication (Hrastinski 2008). Synchronous learning,
commonly supported by media such as videoconferencing and chat, has the potential to support
learners in the development of learning communities. Learners and teachers experience synchronous
learning as more social and avoid frustration by asking and answering questions in real time.
Synchronous sessions help learners feel like participants rather than “isolates”.
The debate about the benefits and limitations of asynchronous and synchronous learning seems to
have left the initial stage, in which researchers tried to determine the medium that works “better”—
such studies generally yielded no significant differences (Gunawardena, McIsaac 2004). According to
Hrastinski instead of trying to determine the best medium, the e-learning community needs an
understanding of when, why, and how to use different types of e-learning. Note also that the users
decide how to use a medium (Hrastinski 2008). For example, in some instances eMail is used nearsynchronously
when users remain logged in and monitor their eMail continuously. Thus, the
difference between asynchronous and synchronous e-learning is often a matter of degree. A solution
is offered by Mason, who uses the new term multi-synchronous learning. The term multi-synchronous
learning refers to the combination of both synchronous and asynchronous learning with the emphasis
on advantages of both of them. Multi synchronous learning may include the following elements:
Web-based conferencing/chats to hold online lessons
eMail to send messages, homework and feedback and for announcements
Face-to-face lessons/meetings to start and finish a course (Mason 1998b, Cassell, 2011).
However, language teachers must be fully aware of the following facts: who their students are, what
their needs and wants are, and what their preferences in the language training and their motivation
might be.
3. LMS Barborka
The teachers at the Language Training Centre (LTC) of the University of Defence (UoD) are aware of
the contemporary trends in distance language learning and the involvement of ICT in teaching. We
have implemented the Learning Management System (LMS) Barborka into the managing and
teaching of distance study programme students. LMS Barborka allows the creation, administration,
and use of the study supports included, which are distributed by the Internet or on the intranet of the
UoD. In addition to tools for the creation, administration and distribution of courses, LMS contains
tools for communication both among students and between students and the teacher. It consists of
the following four basic subsystems:
Author – it serves for designing appropriate study supports, which include texts with various
pictures, animations, video sequences, or sounds to give the students structured material as
comprehensible and clear as possible.
Student – this subsystem enables learners to go through the texts, communicate with other
students and teachers/tutors via the Barborka mail system. Students can use electronic study
supports at any time and place if there is Internet access available. It is also possible to print the
materials and study without Internet access.
Tutor – this subsystem enables the managing of the teaching process; it serves for administration
of students and courses, assigning homework, giving basic information to students about the
study course, and communicating with them.
Administrator – it enables the registering of courses, students and their results, it also records
study groups and teachers.
Electronic study supports deal with courses of general English, professional English – military,
economic and management topics and vocabulary, and for the higher grades also specific vocabulary
of their specializations. The electronic supports are supplemented with exercises both for reflection as
well as those which enable feedback, and thus allow monitoring of the comprehension of the studied
materials, either in communication with the tutor, or with independent work. Instantaneous feedback is
ensured by the interactive exercises created by the ‘Hot Potatoes’ program. Study supports are
appreciated by students because they help them to prepare both for the final tertiary exams as well as
for the STANAG 6001 (SLP 2-3) standardized exams.
111

4. Methodology
Ivana Cechova et al.
In academic year 2010/11 we designed a research study the task of which was to find out if and how
ICT help the distance students in their study, how distance students assess ICT in their language
training and especially how the students assess LMS Barborka. For the first step of our research we
chose the method of interviews which were carried out via Skype. All respondents were our distance
students and graduates. The interviews were semi-structured, using an 'interview plus' approach to
enhance discussion (Bloom 1953; JISC 2009). The interview questions focused on four main themes:
likes and dislikes dealing with English study; language learning and ICT; LMS Barborka as a tool for
language learning; LMS Barborka and its evaluation from the students’ point of view. Ten interviews
were undertaken, with each lasting approximately one hour. Data were fully transcribed and entered
into INVivo. Initial coding was inductive (Thomas 2006) and categories that reflected the question
schedule and main trends were developed. The coded data were then reviewed by the research team
and a series of key themes were determined by consensus. The data were then re-coded according
to the agreed themes, and through this iterative process the project's findings emerged (Strauss,
Corbin 2004). We tried to achieve answers to the following research questions:
What are the students’ likes and dislikes dealing with English?
How do the students evaluate LMS Barborka as a tool for studying English and how do they
evaluate the study supports presented in LMS Barborka?
What are the advantages and disadvantages of ICT in language learning?
The second part of the research is a questionnaire aimed at creating a distance learner profile by
establishing baseline data on technology use. The questions elicited prior experience and
expectations of using technology for learning, and the impacts of e-learning. Then we wanted to find
out if our distance students prefer to use up-to-date technologies (the Internet, an in-house Study
Portal and on-line sessions in language lessons) or to be taught in a classical way (without computer
technology programs), furthermore we wanted to know the students’ preferences and needs while
using ICT in the language classes and the LMS Barborka utility. We wanted either to confirm or reject
the following hypothesis:
The students prefer the use of up-to-date technologies to a classical way of learning.
ICT can influence English training in a positive way.
Study materials and communication via LMS Barborka can influence English training in a positive
way.
The respondents were distance students of the University of Defence, the Faculty of Economics and
Management.
5. Findings
In some answers the opinions of distance study students do not differ from those of full-time ones
(research carried out in 2004-2009). All students highly appreciate the access to LMS Barborka,
which offers interesting texts from different economic and military areas full of various specialized
terminology. They like most especially those texts which differ from commercial textbooks and the
Internet materials. In this context they often express a demand for topics regarding their
specializations or military-political ones.
“Topics dealing with military and terrorism, as I can’t find them anywhere else.” (R4,
March 2011)
“Study supports for individual topics, they are well arranged and written in readable
English.” (R1, May 2011)
Students asses positively the communication with their tutors, tutors’ fast response to the questions,
the correction and evaluation of their homework and they also appreciate the possibility of electronic
testing, mainly in terms of the necessity to pass the STANAG 6001 exam.
“The evaluation of homework was a big surprise for me, it is accomplished at a high
technical level.” (R9, April 2011)
“The study materials enable to revise, possibly to deepen knowledge of the topics. They
are useful for the STANAG 6001 exam preparation as well.”(R10, April 2011)
112

Ivana Cechova et al.
Using ICT while studying language the students see as the necessity, and moreover it breathes fresh
life into teaching. The choice of the appropriate tool is connected with the activities which it facilitates.
They use the Internet for eMails, while working with texts they use a dictionary, they download study
materials from web pages and they keep in touch with their peers. They appreciate electronic
dictionaries, electronic testing and interactive forms of most study materials. The possibility to choose
the place, time and pace of study plays a very important role when choosing the ICT tool.
“In the electronic dictionary you look everything up much quicker than in the paper one.
There are lots of teaching programs.”(R2, March 2011)
“In my job I don’t use English very often; it means that ICT is the only means how to
brush up the language (at least dealing with listening and writing).” (R1, May 2011)
Studying English with the help of ICT is assessed positively by 7 out of 10 respondents. One
respondent is negative about anything which is connected with e-learning; another one prefers the
classical teaching method, i.e. face-to-face one.
“Studying English with the help of Barborka is the new way of study for me; no doubt it is
more up-to-date and comfortable. Textbooks are not bad. Nevertheless, I prefer
ICT.”(R1, May 2011)
“I appreciate assigning and evaluating tasks in electronic form. It is quicker and more
comfortable for me.”(R2, March 2011)
“I prefer teaching in a classical way.” (R4, March 2011)
Students underline the study of English as a prerequisite of their own personal development and
future practice as well. Acquiring the language is understood as a necessity for the future and a
means of promotion at a job.
“In the case of mastering the language, there are possibilities which the knowledge offers
(job - career ladder, etc.)” (R1, May 2011)
“English is nowadays used in a wide range of areas.” (R7, April 2011)
While studying English, the students appreciate mainly the variety, interestingness and topicality; on
the other hand the least attractive activity is studying grammar and vocabulary, mainly idioms and
phrasal verbs
“Present perfect and past perfect tenses are difficult for me.“(R3, April 2011)
“Some texts are demanding in terms of vocabulary.”(R9, May 2011)
According to the analysis of the students’ interviews it is possible to state that distance students can
see ICT/LMS Barborka as a tool which facilitates their study and at the same time as a simpler form of
teaching. They like especially the chance to study any time/anywhere and the offer of unique
materials, which are accessible only in this application (LMS Barborka). On the other hand they are
fully aware of some limitations, e.g. the lack of communication and listening exercises. They focus
repeatedly on the increase of military and management materials accompanied with suitable
exercises and glossary, which can be emphasized as the most important information from all
interviews.
6. Recommendations
The findings of the research suggest the following to improve the students' use of ICT/LMS Barborka
for maximum educational benefit:
Involve students in the creation of military materials. This will help teachers and students to
cooperate closely and to use the students’ special military knowledge.
Encourage students to use web-based support by clearer signposting of online tutorials, guidance
and discussion boards. This will help to deepen comprehension.
Enhance students' awareness and understanding of meta-learning strategies in an online
environment (and elsewhere).
Motivate students to search the Internet for updated reading and listening materials (news, video,
radio reports, written briefings, etc,) and share them with other students as well as teachers.
Reward Internet-based research skills within assessment, ensuring that students use critical
thinking skills to evaluate Internet resources.
113

Ivana Cechova et al.
Although this research was undertaken with distance students, the recommendations suggest several
possibilities that could include other student groups. The research determines the strengths and
weaknesses of ICT/LMS Barborka in language study and identifies the specific skills which distance
students possess, enabling them to use technology effectively for learning English. However, the
fundamental question remains: How to achieve balanced communicative competence of distance
students and how to motivate them to practice all skills (speaking, listening comprehension, writing
and reading comprehension) via the Internet in order to feel confident in English?
7. Material development
The research findings show that the students predominantly miss integrated file of materials of military
and management terminology with easy access. First we prepared authentic reading materials
according to students needs. In the process of choice we have cooperated with teachers from
specialized departments and methodologically processed them. We supplemented interactive
exercises for vocabulary practice and comprehension check to provide students with appropriate
feedback. This was only the first step. The following one in accordance with constructivist theory is to
enable students’ active participation in the lessons. It is reflected by their own choice of vocabulary
areas needed for their future specializations. Students of the 1st form of the master study programme
were asked to use their knowledge of specialized military subjects as well as military training and help
us with the creation of study supports for the courses of Military English and English for Management.
They agreed with great enthusiasm because they can utilize the gained information and the
knowledge from practical military training. In addition the chosen students have a very high level of
English (Stanag SLP 3), they are interested in the military and have access to a plenty of authentic
literature. Teachers and these students co-operate on a methodological processing of the authentic
texts to tailor them for use in e-courses and a future e-book. The students’ involvement motivates
both sides of the educational process and influences them qualitatively. Teachers profit from students’
specialized updated knowledge of the topics; students combine their knowledge from specialized
subjects with English and ICT, and thus their knowledge becomes complex. From the pedagogical
point of view, teachers and students become active partners in the educational process, so they both
share the responsibility for students’ study, which is one of the basic goals of tertiary education
(Berankova et al. 2011). In Figures 1, 2, and 3 there are examples of students’ involvement in
creating study supports.
Figure 1: LMS Barborka – military text
114

Figure 2: LMS Barborka – page with exercises
Figure 3: LMS Barborka - Exercise
Ivana Cechova et al.
115

8. Conclusion
Ivana Cechova et al.
This paper has documented the experience of a group of distance students. The findings demonstrate
that the majority of students in this study are able to use technology to meet their own learning needs.
The distance students assessing their learning opportunities often mention quality, speed, diversity,
timeliness, clarity, transparency, efficiency, openness, frequency to achieve their learning and
professional goals.
It is clear that the teaching of foreign languages presents special instructional challenges. The
teachers and researchers at the LTC of the UoD continue to address such issues as the development
of communicative competence, as well as participation and interaction in the distance learning
environment. However, current trends within the distance learning may affect many decisions,
processes and emphasis within the distance learning environment.
For further development of the monitored area it is recommended to carry out an additional research
to prove the above mentioned findings and to find out if the achieved information is also valid outside
the sample of the UoD respondents. The teachers at the LTC of the UoD are fully aware of this
issue´s complexity and are trying to implement distance learning strategies sensitively, bearing in
mind Plato’s quotation: “Someday, in the distant future, our grandchildren’s grandchildren will develop
a new equivalent of our classrooms. They will spend many hours in front of boxes with fires glowing
within. May they have the wisdom to know the difference between light and knowledge.”
References
Berankova, J. Cechova, I. and Zerzanova, D. (2011) Contemporary trends in language training - practical
implementation of ICT in tertiary education. Distance, Learning, Simulation and Communication 2011. Brno,
pp. 36 -41.
Bloom, B.S. (1953) ‘Thought processes in lectures and discussions’, Journal of General Education, vol. 7, pp. 55-
71.
Cassell D. Distance Learning. (2010) [online]. Available at: [Accessed 3
October 2011].
Davis, J.N. (1988) Distance education and foreign language education: Towards a coherent approach, Foreign
Language Annals, vol. 21 (6), pp 547-550.
Grubb A. Building Cyber-Learning Communities in WebCT. [online]. Available at:
[Accessed 1 October
2011].
Gunawardena Ch. N., McIsaac, M. S. (2004) Distance Education. In: David H. Jonassen, ed. Handbook of
Research on Educational Communications and Technology. Mahwah, NJ: Lawrence Erlbaum, pp. 355–395.
Hrastinski, S. (2008) Asynchronous and Synchronous e-learning. [online]. Available at:
[Accessed 6 June 2011].
JISC (2009) JISC learner experience phase 2: Interview plus. [online]. Available at:
[Accessed 3 June 2011].
Mason, R. (1998b) Models of online courses. Journal of Asynchronous Learning networks 2 Available at:
[Accessed 30 May 2011].
Peters, B.G. (1998) Comparative politics: Theory and methods. New York University Press.
Shelley, M.A. (2000) Distance education. In M. S. Byram, ed. Encyclopedia of Language teaching and Learning.
London: Routledge, pp. 183-185.
Strauss, A. and Corbin J. (2004) Basics of Qualitative Research: Grounded Theory Procedures and techniques.
Newbury Park: California Sage.
Thomas, R. (2006) ‘A general inductive approach for analysing qualitative evaluation data’, American Journal of
Evaluation, vol. 27 (2), pp. 237-246.
Thinkexist.com (2008) [online]. Available at:
[Accessed 27 May 2011].
White, C. (2003) Language Learning in Distance Education. Cambridge University Press.
Winter, J. et.al. (2010) ‘Effective e-learning? Multi-tasking distractions and boundary management by graduate
students in an online environment’. Research in Learning Technology, March 18 (1), pp 71 – 83.
Wohlert, H. S. (1989) German by satellite: Technology-enhanced distance learning. In: W.F. Smith, ed. Modern
technology in foreign language education: Applications and projects. Lincolnwood, IL: National Textbook
Co, pp 107-115.
116

Independent Learning in Need or in Crisis? Independent
Learning Under the new Four-Year Undergraduate
Curriculum in Hong Kong
Yin Ha Vivian Chan, Delian Dawn Gaskell, Mei Ah Tan and Lip Yan Felix Chao
The Chinese University of Hong Kong, China
chanyh@cuhk.edu.hk
b123499@mailserv.cuhk.edu.hk
meiahtan@cuhk.edu.hk
felixchao@cuhk.edu.hk
Abstract: Accompanied with the rapid development of technology, independent learning, especially in the format
of multi-media, has inspired a great deal of enthusiasm and energy in academia over the past few decades. This is
certainly the case in Hong Kong since the number of university students has continued to increase dramatically.
One particular point that has placed Hong Kong in the spotlight is that there will soon be a complete switch from
the previous 3-2-2-3 education curriculum to the 3-3-4 curriculum in 2012. Under this new system, university
students will be required to spend four years rather than three to obtain an undergraduate degree. In response to
this, Hong Kong teachers and administrators have been propelled to reconsider the role self-access centres
should play in this situation. The eight tertiary institutions currently funded by the University Grants Committee
have all been running language centre or self-access centre, sometimes both. The majority of these centres
provide language learning resources and language support, some of them are directly under language centres.
The Independent Learning Centre (ILC) of the Chinese University of Hong Kong is a special case. It serves more
or less as a standalone unit, and there have been instructions from higher authorities to broaden its scope from
language to subject-related matters. The ultimate goal is to create an online platform that contains comprehensive
teaching/learning materials on various disciplines for students to study on their own. This paper uses the ILC as a
point of investigation. Through discussing the challenges and difficulties it faces in the process of transition, this
study explores the role self-access centres should play, including what is practical and reasonable, and what is
theoretical and idealistic. It begins with a review of the concept of independent learning and the ideology behind it,
followed by a discussion of autonomous learning in Hong Kong and the functioning of the self-access centres,
proceeds with the challenges the ILC faces, and concludes with possible solutions to these challenges in the face
of the upcoming four-year curriculum. It is hoped that this research can shed light on what independent learning
means, if technology is the ultimate solution to budget strain, and how self-access centres should perhaps function
with the benefits of students in mind.
Keywords: independent learning, autonomy, education reform, financial imperatives
1. Independent learning: A contested concept
Independent learning has inspired a great deal of enthusiasm and taken up a large amount of energy in
academia in the past few decades. In Hong Kong, this has been driven by a combination of the fact that
an increasing number of students have been admitted to local universities, while the development of
technology has become ever more influential in local education. However, independent learning is a
thorny concept because though it is not new, there is no universal understanding of what exactly it
means. In fact, researchers such as Broad (2006) list several major definitions. The earliest may be
Moore’s (1984: 27) “broad and accessible” definition from the Open University: “working with
increasingly less structured teaching materials and with less reliance on traditional kinds of tutor
support”, which is understandable when considering the nature of adult education. There are also the
social factors behind the surge of independent learning. Instead of pointing to the structure and setting,
Kesten (1987) emphasizes the autonomy of the learner to make the decisions necessary to meet one's
own needs. She also notes the social aspect of learning by iterating that “[t]hese decisions ought to be
made within the bounds of social acceptability and by self-directed, self-motivated, willing learners.”
(1987: 5) Broad (2006: 121) himself states that the ultimate goal of independent learning is to teach
students “to learn for themselves and in turn empower them in their learning whatever the context."
Summing up these definitions may give us a rough idea of independent learning but what exactly is
understood by “independent” is still ambiguous and open to many interpretations. In fact it is interesting
to view the plethora of results from Kesten’s (1987) audit of the terms used synonymously to describe
this type of learning: autonomous learning, independent study, self-directed learning, student initiated
learning, project orientation, discovery and inquiry, teaching for thinking, learning to learn, self
instruction and lifelong learning.
117

Yin Ha Vivian Chan et al.
What is shared by these interchangeable terms is the concept of autonomy. Researchers generally
agree that that there should be a certain degree of autonomy which learners enjoy or acquire in their
learning that these terms describe. Unfortunately, to define what autonomy in learning means is equally
difficult. Among the many different definitions of autonomy in learning, there are two primary schools of
thought: one maintains that the term should describe the “capacity of the learner”, inspired by Holec
(1981: 5), while the other (Riley and Zoppis 1985; Dickinson 1987) refers to autonomy as more of a
situational context. Researchers on autonomy are aware that this situational definition goes against the
fact that learners who are forced to study in isolation do not necessarily develop autonomy. Instead,
autonomy should entail some interdependence (Kohonen 1992), and collaboration (Little 1996) as the
learner works with others toward shared goals, following a more communicative approach to learning.
When autonomy is seen as the capacity to take control of one’s own learning, the question arises of how
much control being referred to? Holec (1981: 3) draws an almost ideal picture of an autonomous learner
by saying that he “is himself capable of making all these decisions concerning the learning with which
he is or wishes to be involved.” These decisions include determining the objectives, defining the
contents and progress, selecting methods and techniques, and monitoring and evaluating any progress
and achievements. Benson (2001: 103) adds that autonomy without control over content is actually an
inauthentic form of autonomy. Few would refuse this tempting image of the ideal autonomous learner.
The question is this: Is it realistic to expect to have such an ideal autonomous learner in any present
education system? If this type of ideal learners do exist, what is the proportion of them at an university?
When education has become more and more an industry focused on accommodating thousands of
students, does such an individualistic concept of learning have a place? How can this individual control
over one’s learning be possible in collective education practices? Benson (1966: 33) in an earlier paper
argued that: “[c]ontrol is a question of collective decision-making rather than individual choice.”
However, in reality, perhaps such absolute control over one’s learning is theoretical and may serve as
more of an ultimate goal to pursue.
Although there is still no concrete agreement of what learner autonomy means, there are some
common basic beliefs. Benson (2001: 183) summarises three hypotheses: first, autonomy is available
to all; second, learners who lack autonomy are capable of developing it; and third, autonomous learning
is more effective than non-autonomous learning. The first two are less disputable after the paradigm
shift of a genuine belief in students' ability and responsibility occurred several decades ago. Yet the
third hypothesis remains debatable. Researchers try to argue for its validity but invariably admit that it is
difficult to measure the effectiveness of autonomous learning. Some research collects positive
feedback for self-access centres as a means to prove this effectiveness, but unfortunately the success
of a self-access centre does not guarantee the enhancement of learner autonomy. Nor does the
improvement of learner autonomy necessarily result in more effective learning. On the contrary, some
research findings even work against learner autonomy. Ferugson and Tryjankowski (2009) conducted a
study comparing online and face-to-face learning in Master’s-level courses. The research looked at
whether these two modes of the same course resulted in two equivalent groups of MA students
achieving the same level of learning success. Results did not show this to be the case. In fact, students
in the face-to-face courses did better than those in the online courses. This was especially the case
when students were struggling academically.
The setting up of independent learning centres when funding is abundant may illustrate foresight and is
often ambitious. However, in the case of Hong Kong, the higher education system is in the process of
returning to a four-year undergraduate curriculum and funding is tight, so the major concern of the
funding authorities right now is whether these centres can share the increased face-to-face teaching
load in a more economical way with an emphasis on providing online resources for students.
2. Autonomous learning in Hong Kong higher education
The concept of autonomy in learning can be traced back as early as to the education philosophy of
Jean-Jacques Rousseau (1712–1778) and John Dewey (1859–1952). However, the concept did not
enter into the field of language learning until the late 1960s. Early interest in autonomy in language
learning is said to have been part of a response to the social and ideological changes of the time. A
respect for the individual in society (Holec, 1981), an emphasis on the meaning and value of personal
experience (Gremmo and Riley, 1995), and the notion of student power (Cockburn and Blackburn,
1970) all have informed the research and practice of autonomy in the field. Self-access resource
centres that encourage self-directed learning have proliferated over the last three decades in higher
education throughout the world. It is also understandable that autonomy arouses interest in the field of
118

Yin Ha Vivian Chan et al.
language learning as language is one of the primary tools used to acquire knowledge, and is thus an
essential aspect of independent learning.
This emphasis on languages within self-access centres is particularly true for Hong Kong education. As
an international city, Hong Kong takes pride in its bilingualism: both English and Chinese (the latter
often referring to written Chinese and the local dialect of Cantonese for everyday spoken
communication). Since the handover of Hong Kong’s sovereignty, Hong Kong’s Education Bureau has
promoted various approaches for the population to cope with the new role the city plays in relation to
China and the rest of the world. One of the major tactics has been the encouragement of “bi-literacy and
tri-lingualism” in education. While the written languages refer to standard English and Chinese, the
spoken refer to English, Cantonese, and Putonghua. In 1998, the government decided to require that
the “mother-tongue” be implemented as the medium of instruction in schools, which triggered a
vigorous debate about whether such a change would lead to a decline in the English proficiency of local
students. This concern reveals the public awareness that the mastery of languages can lead to success
in study and vice versa.
The importance of language in Hong Kong was further reinforced in 2005 when the Education Bureau
presented the document “Reforming the Academic Structure for Senior Secondary Education and
Higher Education – Actions for Investing in the Future.” It announced the full implementation of a new
high school education system in 2009 and the complete switch from the 3-2-2-3 to 3-3-4 curriculum in
2012 1 . The switch poses great challenges to all parties concerned. As a result, large-scale curriculum
revisions at the high school level have been ongoing since 2009; the most significant of which is
probably that of Chinese language and the incorporation of liberal studies into the curriculum. By
broadening the scope for these two subjects, the Bureau aims to encourage extensive reading and
nurture youngsters to become well-informed, well-educated, and well-cultured members of society.
The emphasis on language enhancement is not only at the high school level. Various universities have
already launched pilot courses on languages and general education to cope with the change in the high
school curriculum. Take, for example, The Chinese University of Hong Kong where newly admitted
students will have to take 21 credits for general education, six credits for Chinese, and nine credits for
English. These new language requirements are meant to prepare students to handle the voluminous
reading requirements in both Chinese and English and to sharpen their language abilities in preparation
for the future job market. According to Bachman’s 2008/09 review of the efficiency of language
education in Hong Kong tertiary institutions, language enhancement is an urgent need for local
university students. Bachman notes:
Virtually all the research literature on achieving high-levels of proficiency in a second or
foreign language suggests that once learners have reached a given level, increasing larger
amounts of time are required for them to move from intermediate to advanced levels.
(2010: 2)
His remark illustrates why universities need to invest more resources to cater for students’ language
enhancement needs in preparation for serious academic studies and research. Bachman (2010: 2)
further notes that the present resources available to students are insufficient and suggests that local
universities should radically increase the number of language education courses they offer in order to
help students to advance their language proficiency.
What makes the situation in Hong Kong even more daunting is that the number of first year university
students will increase by an estimated 40 per cent in 2012. This “double cohort” includes the last group
of students under the former education system as well as the new group under the revised curriculum,
both of whom will enter local universities in the same year, thus creating more budgetary strain. In view
of this, it is not surprised that the university administration and those in charge of the budgets see
independent learning as the most simple and direct means to deal with this surge of student intake and
alleviate potential financial pressure. However, their understanding of independent learning remains
unclear. For many, a popular view is that independent learning simply means transferring face-to-face
teaching to self-study and thus the responsibility would be put onto universities’ self-access centres.
However, this view has been deeply questioned by the recent research in the field. Sturtridge (1997: 70)
calls attention to this intention of funding authorities to see one-off capital expenditures on self-access
1 Originally, students had to spend three years to complete junior high, two years for high school, two years for university
preparatory courses, and three years for university. After the reform, students will take three years to complete junior high, three
years for high school and four years for university.
119

Yin Ha Vivian Chan et al.
as a viable alternative to more expensive ongoing expenditures on teacher salaries. Ferugson and
Tryjankowski (2009) in their research conclude that simply transferring a face-to-face syllabus to an
online medium does not result in an equal learning environment. The lack of support, community and
connectedness, and a dearth of specific skills and attitudes needed for online teaching and learning in
particular tend to lead to this unfavourable result. In addition, Gardner and Miller (1999:31) remind us
that “a key element in success SALL [self-access language learning] is pedagogical input and this is
relatively expensive”. Littlejohn and O’Dell both have similar observations. Littlejohn (1985) suggests
that the effective use of self-access centres depends on the extent to which each teacher views
self-access work as valuable. O’Dell (1992) observes that students who made full use of the resources
of the Eurocentre Cambridge in the United Kingdom were those in classes where the teacher was
confident and well-informed of the centre. These researches indicate that attempts to use self-access
centres as a means to cut costs is the result of a misunderstanding of their purpose, and also brings
about little improvement in student learning, which should be the focus of a university.
The eight local tertiary institutions currently funded by the University Grants Committee (UGC) have all
established either a language or self-access centre, many times both. The establishment and facilitation
of these centres are both in line with the goal set down by Chief Executive, Donald Tsang, and the UGC
to increase Hong Kong’s global competitiveness. This blatant embrace of the global economy is deeply
disturbing as research has pointed out that in these circumstances the concern of language education
becomes “a service to global economy in which language skills represent a form of economic capital”
(Benson, 2001: 19), and that “[b]roader political concerns about autonomy are increasingly replaced by
concerns about how to develop strategies for learner autonomy” (Pennycook, 1997: 41). The push for
the economically-driven nature of these centres illustrates a larger problem in terms of the
commercialization of Hong Kong education and is worth more research and attention.
The major role of Hong Kong’s self-access centres is to help students to master language. So, it makes
sense that their two common features are the provision of language learning resources in different
formats and language consultations. Approximately half of these centres also run workshops. In
addition, the teachers at these centres act as facilitators and mentors rather than simply as teachers.
They direct students to relevant self-learning materials and guide them through their plan to improve
their language abilities, which includes providing advice at opportune times. These centres are
therefore significantly different from libraries in the sense that they not only provide materials, but also
provide tutorial support to help users develop language and study skills and design a suitable
independent study strategy.
The practice of providing human support in the form of teachers has been implemented with discretion
in these centres. In 1994, instructors from different Hong Kong institutions published research on the
self-access centres at their respective universities. This research revealed that self-access centres run
most effectively with the guidance of teachers and most properly utilized when the teaching and
learning behaviors changed from the giver-receiver mode to that of trainer-trainee. Star’s (1994)
research at the Hong Kong Institute of Education revealed three major points. 1) Users prefer some
guidance to save time and effort in searching for suitable materials; 2) they prefer some training in
cognitive as well as meta-cognitive strategies; and 3) they express the need for materials to help check
progress. She concludes that “learner training” is a key to the success of a self-access centre (Star,
1994: 165). While Star proposes various ways to promote self-access centres, she also notes that it is
necessary to improve consultations “by explaining their purpose to both students and tutors and asking
tutors to be more proactive in the area.” (1994: 166) Contrary to the nonprofessional belief that a
self-access centre does not need teachers, in fact research and actual practice reveal that tutors play
an important role in facilitating and promoting self-access centres in tertiary education.
Star’s findings do not represent an isolated case. Pang (1994: 37) from Lingnan University comments
on the potential of self-access centres as follows: “The SAC [self-access centre] acts to provide much
needed consultation and support — moral, pedagogical, linguistic and tactical.” While Miller and
Gardner (1994) from the City Polytechnic of Hong Kong and the University of Hong Kong respectively at
that time both point out the importance of learner training and teacher training, how the learner functions
as an independent learner and the teacher as learner trainers.
At present, in preparation for the new curriculum in 2012, some local self-access centres seem to be
narrowing their focus to writing, which may result in their apparent similarity to writing centres. However,
120

Yin Ha Vivian Chan et al.
the Independent Learning Centre (ILC) of The Chinese University of Hong Kong (CUHK) is also worthy
of mention due to recent developments and transition of its focus.
3. Independent Learning at CUHK: Challenges and solutions
The ILC of CUHK was originally run by the English Language Teaching Unit (ELTU) when it first started
in October 1993 and became a bilingual centre shortly after its opening. Staff from the Chinese
Department together with those from the ELTU formed a task force to help students enhance their
language proficiency through independent learning. However, the ILC became an independent unit in
2005 and has its own staff. This independence gives the ILC more freedom in fostering autonomous
learning among students, but at the same time poses significant challenges to it.
Challenge 1:
The fact that most of the self-access centres (SAC) affiliate to their university language centre(s) is not
without reasons. Some of them may even share teaching staff like the ILC did. Such close connection
gives the language teachers a better position to understand the missions and resources of the SAC.
Research has already shown that teachers’ attitude towards the centre and their familiarity with the
resources of the centre are crucial to the effective use of it (Littlejohn, 1985; O’Dell, 1997). Therefore, it
is not surprising to know that the major challenge the ILC faces seems to come from the Centre’s
autonomous status outside the university’s language units. Though many efforts have been made to
establish routine and productive engagement between the ILC and the two largest language units, there
have been some issues that seem to delay or negate this effort. These issues seem to stem from the
differences in understanding of what “self-access” or “independent learning” is among the parties
concerned. This is perhaps understandable considering the difficulty practitioners have in defining
these terms. Many within the language units and top administration appear to be of the opinion that the
ILC’s role should be restricted to that of a resource centre, where students can take out resources or
receive lists of links to use in their language study. Though this kind of provision is definitely one of the
aspects of assisting students in enhancing their language ability and learning autonomy, simply
providing these resources without being involved in discussions with the instructors from the language
units in terms of how to best support their students’ needs within specific courses is not making full use
of the ILC’s capacities, nor is it fully supporting the students’ needs. Therefore, there have been a
number of attempts to work more closely with the language units, of which a recent example may serve
to illustrate the current challenge.
In response to the upcoming four-year undergraduate curriculum, ten per cent of the assessment of a
few required language courses will go to online learning. In order for this to take place, a website will be
set up and the ILC and the corresponding language unit are meant to collaborate to design part of the
associated online learning materials. The language unit head and the instructors concerned have been
invited to communicate with the ILC to share their goals and ideas for the website and online learning,
and in preparation, the ILC has introduced its services and resources in extensive detail. Suggestions
from the ILC have included the inclusion of interactive language exercises for the students, learner
support for assigned readings, and the possibility of linking up the ILC’s website to this new site for easy
access to more independent learning materials. The idea behind this exchange of opinions and
information is to encourage instructors who may still practice face-to-face teaching predominantly to
understand more about the concept of learner autonomy and enable the Centre to gather more
information about what sort of independent learning activities these instructors would consider effective
in enhancing students’ language development.
One of the key factors Sturtridge (1997) identifies in the effectiveness of self-access is whether activities
of the SAC are integrated into the curriculum and teaching timetable. The ten per cent online learning
requirement written in the new language curricula could be an advantage to foster learning autonomy, if
the online learning activities are carefully designed to achieve the goal. However, a couple of meetings
have been held but no serious discussion on the concept of autonomy has been successfully started.
The message implies by the top authorities of the university that independent learning simply means to
shift paper-based materials to a digital format, and ultimately a way to cut cost, seems predominant.
The online learning component is seen mainly as a means to cut down contact hours and to save
marking time. In addition, mutual trust in certain academic circles seems difficult to establish. In a subtle
and indirect way, there seem to be reservations expressed by some parties involved about whether and
how the ILC can academically facilitate language learning, though our instructors all come from the
same academic background and experience level as those in the language units. In other occasions,
121

Yin Ha Vivian Chan et al.
the ILC has been seen as a threat and rival. Perhaps it is difficult to expect standalone language
teaching-related units to work smoothly and with complete trust if they are not officially linked.
Ultimately, the progression of this collaboration is at a standstill. There is currently no working schedule
to move the collaboration forward.
The potential solution to this challenge could perhaps come from a change in the culture currently in
existence between the ILC and the language units, and in the University in general. This may be best
facilitated by the involvement of the related directors and higher administration to require the staff in all
the units to work together to integrate the principles of independent learning into the new four-year
curriculum for the relevant language courses that will be offered. This would require the staff to meet on
a routine basis, rather than simply leaving it up to the staff to participate if and how they see fit.
Sturtridge’s (1997) argument that all staff should be involved as stakeholders in the development of
self-access facilities perhaps is not too ideal or ambitious. In the long run, if we see the need for the
independent learning facilities and learner autonomy to develop and grow, a deeper understanding of
the concept and widespread support must be established within the University.
Challenge 2:
The ILC has experienced some recent developments and a transition of focus over the last few years in
the run-up to the new curriculum that will be put in place after September 2012. Over the previous two
years, the Centre has experienced numerous changes initiated from management above. In 2009,
these higher authorities decided to both drop the word yuwen 語文 from the ILC’s Chinese name and to
encourage staff members to focus more on study skills rather than language improvement. The ultimate
goal has been for the ILC to develop an online resource platform where students can access
information and learning resources of their programmes of study. It is hoped that by doing so the ILC
can assist the development of their study skills and learning strategies while also promote lifelong
learning. This requires the ILC to work in partnership with teachers and units across the University in
providing resources and activities to support students’ various learning needs. This type of independent
learning would be much broader than the original focus on language enhancement and is undoubtedly
ambitious. If successful, it could expand the practice of independent learning to various academic
programmes in the University. However, this plan faces many obstacles, the most apparent of which is
to gain teachers’ support. The discussion of student’s learning needs is a sensitive, sometimes political,
issue, especially when it is initiated by an outside unit as the ILC not to mention that language teachers
are traditionally less respected in the academic circle.. Despite the various attempts at different levels
by the ILC Director and the higher authorities to introduce the ILC support to students’ learning needs to
different departments, there are very few responses that suggest genuine interests.
Apparently it is not that students do not have problems in their studies, nor do the teachers not care. It is
the question of whose responsibility it is to take care of students’ learning needs. Perhaps it is like
opening up a Pandora's box of problems of responsibility which entails the investment of extra funding,
manpower, time, etc. There may also be an unwillingness to upset the status quo. The intended move
away from the traditional role the ILC to assist students in their independent language enhancement
work has also had the secondary effect of causing confusion amongst university staff, who are quite
rightly uncertain of the educational role the ILC currently plays, and will play in the future.
The solution to this challenge would seem to involve an open discussion amongst all the stakeholders
involved: the ILC, the heads of all the different academic departments, student union representatives
and directors of the two language units. Perhaps if this sort of a dialogue could take place on a routine
basis, it would be possible to determine what the academic departments require most from their
students, what the students feel they need in terms of support and what the language units and what the
ILC is capable of offering by the 2012 deadline before the new four-year curriculum begins.
4. Conclusion
Hong Kong’s university system will face an unprecedented challenge when it switches to a four-year
curriculum in 2012. The pressure will be high on academic staff and freshmen entering universities in
the next few years when the new curriculum is in effect. Therefore, there is the potential for independent
learning and self-access centres to play a beneficial role in alleviating some of this pressure, though
what this role will be exactly must be thought through carefully.
122

Yin Ha Vivian Chan et al.
The role of the CUHK’s ILC has been under some scrutiny in the run-up to the new curriculum’s
implementation. Traditionally, the Centre has functioned as a centre to support students’ independent
learning of Chinese and English, as these are the two modes of communication required for students to
acquire and communicate knowledge in a local tertiary environment. In addition, the Hong Kong
government has continued to emphasize the importance of becoming bilingual in at least these two
languages, and outside experts such as Bachman have reported that students need more support in
these languages, not less. However, the ILC faces challenges from different levels within the University
to abandon its traditional focus on language enhancement, and to act as more of a resource centre
while working with a wide variety of departments to create online versions of course materials for
students to access without teacher support. There are also challenges to overcome in surmounting
some misunderstandings of what independent learning is and how it is best implemented for students to
improve their ability to learn and improve their language skills independently. Potential solutions have
been suggested, but it should be remembered that all of these challenges exist while the university
readies itself for the upcoming influx of students who will represent a 40 per cent increase in student
intake in September 2012, so the stakes are high.
References
Bachman, Lyle F. (2010) Language enhancement in Hong Kong universities: Some observations and
recommendations, review report on language enhancement activities, Hong Kong: University Grants
Committee, pp. 1–19.
Benson, P. (1996) “Concepts of autonomy in language learning”, in R. Pemberton et al. (eds) Taking Control:
Autonomy in Learning Learning, Hong Kong: Hong Kong University Press, pp. 27–34.
Benson, P. (2001) Teaching and researching autonomy in language learning, Harlow: Pearson Education Limited.
Broad, J. (2006) “Interpretations of independent learning in further education”, Journal of Further and Higher
Education, vol. 30, No. 2, May, pp. 119–143.
Cockburn, A. and Blackburn, R. (eds.) (1970) Student power: Problems, diagnosis, action, London: Penguin.
Dickinson, L. (1987) Self-instruction in language learning, Cambridge: Cambridge University Press.
Gardner, D. and Miller, L. (1999) Establishing self-access: From theory to practice, Cambridge: Cambridge
University Press.
Gremmo, M.-J. and Riley, P. (1995) “Autonomy, self-direction and self-access in language teaching and learning:
the history of an idea”, System , 23 (2), 151–64.
Holec, H. (1981) Autonomy and foreign language learning, Oxford: Pergamon.
Kesten, C. (1987) Independent learning: a common essential learning, A study completed for the Saskatchewan
Department of Education Core Curriculum Investigation Project, Saskatchewan Department of Education,
University of Regina.
Kohonen, V. (1992) “Experiential language learning: Second language learning as cooperative learner education”,
in D. Nunan (ed.) Collaborative Language Learning and Teaching, Cambridge: Cambridge University Press,
pp. 14–39.
Little, D. (1996) “Freedom to learn and complusion to interact: Promoting learner autonomy through the use of
information systems and information technologies”, in R. Pemberton et al. (eds) Taking Control: Autonomy in
Language Learning, Hong Kong: Hong Kong University Press, 203–18.
Littlejohn, A. (1985) “Learner choice in language study”, ELT Journal, 39 (4): 253–61.
Miller, Lindsay and Gardner, David, “Directions for research into self-access language learning” in Gardner, David
and Miller, Lindsay (eds.) (1994) Directions in Self-access Language Learning, pp. 167–175.
Moore, M. G. (1984) “Concepts of adult education”, in: R. Bell (ed.) Education for Adults, Milton Keynes: The Open
University Press.
O’Dell, F. (1992) “Helping teachers to use a self-access centre to its full potential”, ELT Journal, 46 (2): 153–9.
O’Dell, F. (1997) “Confidence building for classroom teachers working with self-access resources”, in P. Benson
and P. Voller (eds.) Autonomy and Independence in Language Learning, London: Longman, pp. 150–63.
Pang, Terence T. T. (1994) “A self-directed project: A critical humanistic approach to self-access”, in Gardner,
David and Lindsay Miller (eds.) Directions in Self-access Language Learning, Hong Kong: Hong Kong
University Press, pp. 29–38.
Pennycook, A. (1997) Cultural alternatices and autonomy, in P. B. (eds.) Autonomy and Independence in
Language Learning, London: Longman, pp. 35–53.
Riley, P. and Zoppis, C. (1985) “The sound and video library”, in P. Riley (ed.) Discourse and Learning, London:
Longman, pp. 286–98.
Star, Marian. (1994) “Learning to improve: Evaluating self-access centres”, in Gardner, David and Miller, Lindsay
(eds.) Directions in Self-access Language Learning, Hong Kong: Hong Kong University Press, pp. 157–166.
Sturtridge, G. (1997) “Teaching and language learning in self-access centres: Changing roles?” in P. Benson and
P. Voller (eds.) Autonomy and Independence in Language Learning, London: Longman, 66–78.
123

The Development and Application of a web Based
Metacognitive Mapping Tool
Serdar Çiftci 1 and Mehmet Akif Ocak 2
1
MS. Department of Educational Technology, Institute of Educational Studies
of Gazi University, Turkey
2
PHd Department of Computer and Instructional Technologies Teaching, Gazi
University, Turkey
serdar@gazi.edu.tr
maocak@gazi.edu.tr
Abstract: Teachers support metacognition by following different teaching methods in face to face learning
environments. However, literature shows that web based learning environments lack of the use of tools which
support and guide the students’ metacognitive activities. The primary purpose of this study is to give insight about
development of a metacognitive mapping tool (MMT) supporting the college students’ metacognitive learning
abilities. The participants of this study were 25 third grade college students selected from Gazi Faculty of
Education, Gazi University. The students participated in a six week qualitative study. They studied an online
Database Management Course by getting support from the tool. The tool was developed on the four components
of metacognitive skills which were planning, monitoring, evaluation and revision. The tool was utilized for an
asynchronous web based course. The tool allowed students to plan their studies, monitor and evaluate their
progress. After monitoring and evaluation, students could revise their planning and change their thinking patterns.
A content analysis was carried out to examine students’ interviews. The focus of the interviews was to investigate
the students’ perceptions and purpose of utilization. This study argues that using the tool gives the students an
opportunity to choose exact content of the course and to plan the content for individual aims. The tool used in this
study can assist time management and tracing of the students’ personal study progress.
Keywords: metacognition, metacognitive mapping tool, web based education, eLearning, metacognitive learning
process
1. Introduction
The biggest advantage of web based learning is that it gives students active learning opportunities
(Tsai 2009). Together with the increase in the demand for web based learning environments, today,
this kind of environments shows the rapid development. The causes such as regional distances,
equal educational opportunities and time constraints are among of the causes of this increase in
demand. While the studies conducted in the first place dealt with the development of learning
environments and transfer of learning activities from the form of face-to-face environment to the web
based environments. They, now, focuses on how web based learning can be given effectively. To
provide an effective presentation of learning environments, supporting tools and advanced
technologies in context must be provided and integrated. Collis and Meeuwsen (1999) indicated that
students need to take responsibility for increased control of learning process and show positive
attitude to achieve active learning in such an open-ended learning environment. Visual and written
representation of the content on the web is not just enough to teach and learn. Students may be
required for new learning strategies and skills so that they can become effective and successful online
learners (Tsai 2009). Using learning tools help students to increase interaction with each other, the
content and the teacher and are used more effectively. Over time, difficulties of web based learning
were overcome and studies have largely concentrated on how additional gains can be achieved from
web based learning.
2. Metacognition
Metacognition, first, was defined by Flavell (1976) as knowledge about knowledge. Subasi (2003)
defined the metacognition as the individual's own cognitive processes and knowledge. Schraw and
Brooks (1998) and Zimmerman (1989) collected metacognitive processes under the heading of
learning based on self-regulation. Prior research suggests that metacognitive strategies in face-toface
learning improve students' achievements. Metacognitive knowledge of strategies consists of
knowledge about cognitive strategies for enhancing learning and performance (Pifarre and Cobos
2009) (Figure 1). For constructivist learning, students' better use of self-regulation skills facilitates the
process of information processing and the recall. Learners having deficiencies in key metacognitive
and self-regulatory skills will not learn much from open-ended learning environments without the
implementation of scaffolds to help them along (Azevedo, Moos, Greene, Winters and Cromley,
124

Serdar Çiftci and Mehmet Akif Ocak
2008). Thus, students' metacognitive awareness is important. Jones, Farquhar and Surry (1995)
suggest that with improved learner awareness of metacognitive knowledge, learning efficacy could
also be enhanced. Especially, in web based learning environments, the students have a sense of
loneliness. In order to overcome these difficulties, different supporting tools for this purpose might be
structured. Learning in a hypermedia environment requires from the learner to regulate his or her own
learning in order to construct higher and deeper levels of knowledge; that is, it provides to make
decisions about what, how, how much to learn, how much time to spend on it, how to access other
educational materials, how to realize whether he or she understands the material, how to modify
plans and strategies to learn better and when to increase one’s effort (Azevedo et al. 2008). This
research is shaped on the basis of the metacognitive grounds.
Figure 1: Metacognition development strategies used in face-to-face learning environments
2.1 Metacognitive mapping tool
Metacognitive mapping tool (MMT) was developed based on the principles of the students’ planning
of a web based content, monitoring of their activities and evaluation cycle (Figure 2). Students
engaging in web based learning environments plan their own learning activities. This planning, mostly,
takes place in the student's mind and cannot be followed actively. One of the aims of the MMT is to
record planning, monitoring and evaluation activities shaped in the students' mind while guiding the
students in a metacognitive way at the same time. In this way, students’ behavior can be observed.
As a result of the monitoring, based on the students’ process, learning environment, mapping tool and
the content can be formed in a more effective way.
Figure 2: Four components of metacognitive skills
Four components of metacognitive skills are explained below:
Planning: Students in this section will be able to choose which headings to cover, which sequence to
use and how much time they will spend on each topic. Planning consists of two sections. In the first
section, students choose their topics that they will study and determine the study sequence. In the
second section, students decide time planning of each topic.
Monitoring: The students will be able to monitor the process during the instruction. The stage of the
process which was planned, how much time was spent and remaining time might be accessible to all
students from this section. The current situation might be supported with visual help and it is visible
whether the student follows the plan which is part of the whole process. This section provides the
infrastructure to the next sections which are evaluation and revision.
Self-evaluation: The students monitoring the current state with monitoring section of the tool will be
able to evaluate themselves whether the situation they are in is correct. In this section, students
evaluate whether they obey the plan (in terms of time, content and sequence) with "yes / no" answers.
The answer, "Yes”, shows that the student followed the plan in the learning process and did not need
any editing. "No" response offers students revision stage by which the students get the opportunity to
correct mistakes and see the lacking points in the process.
125

Serdar Çiftci and Mehmet Akif Ocak
Revision: In revision stage, opportunities for changes are given to the students to plan the whole
process again and allow them to see what they did before. Students finding themselves insufficient
and are not confident with the current situation might return to previous topics and change their plans
in revision.
2.2 Development and application of the MMT
In application, MySQL Database Management Course was administered to the students through the
web based learning environment. Students, with a period of 6 weeks of training, received instruction
via the web and used metacognitive mapping tool as a means of support. Learning environment and
metacognitive mapping tool were constructed via PHP programming language and MySQL database
management system. Throughout the development of the MMT, expert opinions on content, learning
environment and the MMT were obtained. Some revisions have been made based on the feedback
obtained from the experts. Figure 3, 4 and 5 show screenshots of the MMT tool. Figure 3 shows the
main screen of the web based environment and the MMT.
Figure 3: MMT in the web based learning environment
Figure 4 shows the first face of the planning section of the tool.In this section, students are allowed to
choose the topics they will study from the existing course content by using drag and drop method.
Figure 4: First phase of the planning section of the MMT
126

Serdar Çiftci and Mehmet Akif Ocak
Figure 5 shows the second phase of the planning section of the tool. Students decide on study time of
chosen topics. For example, in Figure 5, there are 3 topics chosen by the student and 50 minutes
were given to each topic. Totally, the student plans to work 150 minutes to study on chosen topics.
Figure 5: Second phase of the planning section of the MMT tool
3. Purpose of the study
This research aims to give insight about development and implementation of MMT, planning to help
college students' metacognitive activities to organize a web based course content, course work and
activities. In web based learning environments, a learning tool is needed to improve students'
metacognitive awareness and help them in planning, monitoring and self-evaluation procedures.
Considering this requirement, the purpose of the study is to develop and implement a supporting tool
to help students to arrange metacognitive activities.
4. Method
4.1 Participants
The participants of the study were of 25 third grade college students studying in the Department of
Computer and Instructional Technologies Teaching, Gazi Faculty of Education, Gazi University. The
research was conducted in spring semester of 2010-2011 academic year. Of the students in the
study, 16 (65.38%) were male and 9 (34.61%) were female.
4.2 Learning environment
Before the study begins, the researchers conducted one week orientation session for all students in
the computer laboratory. In this orientation, students were explained to use the tool and web based
learning environment. Additionally, there were no pre-information or sample planning provided to the
students. Using the tool takes approximately 6 to 10 minutes to complete. Any additional information
was not needed for utilization of the tool.
Students were given learning tasks for each week. Total, 6 tasks were administered to students.
Tasks were related to topics that students studied in accordance with learning goals. Students were
asked to complete one task for each week. Aim of the tasks was to engage students to use the MMT
and study the chosen topics in web based learning environment.
The tool was used in the MySQL Database Management Course. Third grade college students
planned their study in terms of content, time and sequence. In other words, “A plan” means that
students choose which headings to cover, which sequence to use and how much time they will spend
on each topic. This progress lasted 6 weeks. Students were able to monitor their study activities and
check their plan if they follow the plan efficiently.
127

4.3 Data collection
Serdar Çiftci and Mehmet Akif Ocak
In this study, in order to provide students a metacognitive support in a web based learning
environment, a MMT has been developed. The intent of developing a separate tool was to offer
support to students while studying in the web based environment. In the data collection process, the
students were asked open-ended questions by using the interview form. All students participating in
the application answered every question. A total of 8 questions administered to students were asked
online.
Interview questions presented to students as follows:
1. In the planning stage, please explain how MMT helped you?
2. In the monitoring stage, to what extent MMT helped you?
3. In the self-evaluation stage, to what extent MMT helped you?
4. In the revision stage, to what extent MMT helped you?
5. With the MMT, did your awareness towards planning, monitoring, self- evaluation and revision
activities increase in your mind?
6. With the MMT, did you arrange your own work activities more efficiently?
7. Briefly describe the points that you find useful when using MMT.
8. Briefly explain the missing points that you see in MMT.
9. While studying in web based environment, do you need this kind of tool/ support / guidance?
4.4 Data analysis
The data obtained from the study were examined by content analysis. Common patterns were
identified in content analysis. As a result, the frequency and percentage of given responses were
obtained. Based on the frequency of students' usage of metacognitive mapping tool, common
keywords were identified and patterns were classified.
5. Results
As a result of the interview responses of students on open-ended questions, frequency and
percentage of responses were calculated and used. Results are presented separately for each
question below:
1. In the planning stage, please explain how MMT helped you?
Most of the students indicated, with the help of planning phase of the tool, it was helpful to plan their
learning. Students reported that, by using planning section of the tool, they prepared a plan in the web
based environment according to their own learning styles. It shows that metacognition is not a linear
process that moves from preparing and planning to evaluating (Anderson, 2002). When analyzing the
answers given by the students, frequency and percentage of the prominent answers are shown in
Table 1.
Table 1: Students’ responses to question 1
Keywords f %
Planning 23 44,23
For assistant 16 30,77
For efficient study 6 11,54
Adaptation 3 5,77
I did not use the tool very much 3 5,77
It provided guidance 1 1,92
Total 52 100
128

Serdar Çiftci and Mehmet Akif Ocak
Some of the sample answers given by students to question 1 were as follows: “In order to choice the
topics, it created a specific order and depending on this order, it provided easiness to study the topics.
Therefore, with the planning phase of the MMT, it provided planning to study the topics effectively and
efficiently with a particular sequence.”, “In a specified period of time, it helped to study the topics
which I chose more effectively and in a disciplined way.”, “It helped me to follow the topics
respectively; it assisted me on how much time I will spend for each subject.”, “It helped me to plan
and arrange the topics according to the order of importance.”. Similarly, Simons (1994) found that one
aspect of planning that has been studied is whether students plan different learning activities when
learning for recall and when preparing for comprehension.
2. In the monitoring stage, to what extent MMT helped you?
Of the coded responses, 55.56% specifically indicated that the students were able to monitor their
study progress. 29.63% of the coding indicated that the MMT helped students to go through the
learning, to see learning deficiencies and to estimate the required time for each topic. Examining the
data obtained from interviews, it was identified that the students, with the monitoring section of the
MMT, could trace their own study activities and give direction to their study. The tool, also, allowed
the students to monitor their study speed and re-shape their own work schedules. When analyzing the
answers given by the students, prominent answers are shown in Table 2.
Table 2: Students’ responses to question 2
Keywords f %
Monitoring 15 55,56
It gives help 8 29,63
Step by step learning 1 3,70
Disciplined work 1 3,70
I did not see its effect 1 3,70
Guidance 1 3,70
Total 27 100
Sample answers given by students to question 2 were as follows: “I saw my deficiencies by looking
the time spent on the topics and studied those topics again.”, “I watched the steps in a more
disciplined way. I performed the work in a better way than I have done before.”, “I see the difference
between my application and my plan. It helped me to see where I stayed behind on my work.”, “By
seeing how much time I complete the topics, now I estimate the time required to split up for each
subject better.”. Prior research also indicates that accurate monitoring is crucial in learning and
training contexts where students have to master a great deal of new knowledge (Tobias, Everson and
Laitusis, 1999).
3. In the self-evaluation stage, to what extent MMT helped you?
Of the coded responses, 76.19% indicated that the students received reminders for self-evaluation. In
this way, students controlled their planning (14.28%) and study process (4.76%). When analyzing the
answers given by the students, prominent answers are shown in Table 3.
Table 3: Students’ responses to question 3
Keywords f %
Checking points for self-evaluation 16 76,19
Re-planning 3 14,28
Controlling my self-study 1 4,76
Evaluation 1 4,76
Total 21 100
Sample answers given by students to question 3 are as follows: “In the evaluation stage, I got
opportunity to control my own studies.”, “It gave me chance to re-plan when I needed”, “It remind me
of self-evaluation by myself.”
4. In the revision stage, to what extent MMT helped you?
129

Serdar Çiftci and Mehmet Akif Ocak
Of the coded responses, 72.73% specifically indicated that the students revised their study plan.
13.64% of the coding shows that students did not see any benefit of the revision stage. Students who
could not make proper planning in the first place processed their work more accurately. In particular,
by starting to study the content, their study plan was much more clearer. When analyzing the answers
given by the students, prominent answers are shown in Table 4.
Table 4: Students’ responses to question 4
Keywords f %
Revision 16 72,73
I did not see any benefit 3 13,64
Plan arrangement 1 4,55
Guidance 1 4,55
Time saving 1 4,55
Total 22 100
Sample answers given by students to question 4 are as follows: “My pre-planning was random,
because I did not know the content of the topics. After seeing and studying a little, I re-edited.”, “I saw
unnecessary time planning on some topics and did re-planning to avoid loss of time by using the
MMT.”, “I had the opportunity to re-edit the time spent more or less on the topics which I studied.”, “It
helped me to arrange the topics which I had to study and arrange the time which must be spent.”.
Thomas (2001) found that the effect of intervention on students’ metacognition was variable. Some
students became increasingly metacognitive and reported evidence of revision of their learning
processes.
5. With the MMT, did your awareness towards planning, monitoring, self-evaluation and revision
activities increase in your mind?
The answers to this question showed that students’ metacognitive activities in planning, monitoring,
self-evaluation and revision stages with MMT converted their mental plan into writing. Concrete plans
laid out by the students have enabled them to plan more accurately. When analyzing the answers
given by the students, prominent answers are shown in Table 5.
Table 5: Students’ responses to question 5
Keywords f %
It raised my awareness 16 53,33
Helped to plan 5 16,67
I converted my mental plan into writing 4 13,33
My awareness increased partially 3 10,00
I saw my deficiencies 1 3,33
It was useless 1 3,33
Total 30 100
Sample answers to question 5 are given as follows: “Normally, when I study a lesson, I make a plan in
my head. Since this plan was in my head, it was difficult for me to implement. With this tool, I know I
used this plan in my mind; otherwise, trying to implement my plan would be very difficult. This issue
has increased my attention more.”, “With MMT; my awareness has increased towards planning,
monitoring, evaluation and revision activities. While learning the topic, it needed to draw the map from
the beginning. While processing and also at the end, it was useful to use this tool to increase
efficiency.”
5. With the MMT tool, did you arrange your own work activities more efficiently?
The students found MMT useful to do the plan. Deficiencies in the first planning have been corrected
by the revision in the next stages. When analyzing the answers given by the students, prominent
answers are shown in Table 6.
130

Table 6: Students’ responses to question 6
Serdar Çiftci and Mehmet Akif Ocak
Keywords f %
It supported my arrangement 20 64,52
Problems with time arrangement 3 9,68
There should be orientation 2 6,45
I did not need it 2 6,45
It provided adaptation 1 3,23
Supportive for efficient planning 1 3,23
It should be improved 1 3,23
It helped partially 1 3,23
Total 31 100
Sample answers to question 6 are given as follows: “I did not have difficult time to choose the topics
in order; however, there were different from the first done. I reduced these differences over time.”, “It
helped me to edit my studies. Thanks to this kind of tools that made it easier to plan and implement.”,
“I could not arrange in my first planning. With the help of revision, I was able to arrange efficiently.”
7. Briefly describe the points that you find useful when using MMT.
The students found planning section of the tool most useful. Monitoring and evaluation stages,
recognizing their own metacognivite abilities and adaptation were some of the other important points
indicated by the students. When analyzing the answers given by the students, responses are shown
in Table 7.
Table 7: Students’ responses to question 7
Keywords f %
Planning 16 61,54
Monitoring 6 23,08
Self-evaluation 2 7,69
Revision 2 7,69
Total 26 100
Some students' answers given to question 7 are as follows: “By providing me an opportunity to sort
the content according to my pre-knowledge, I completed my own studies and this allowed me to
recognize myself in the planning.”, “To present my study plans saved me from the chaos of
complexity.”, “The most important point was time I saved.”, “It helped me to see how much time I
spend to which content, how much I studied each content and whether I passed the time that I
specified for each content.”, “For me, the most useful point was re-editing, because I had trouble time
to spend the time given to chosen contents. Although I understand that content, I repeated two or
three times.”
8. Briefly explain the missing points that you see in MMT.
Although some of the students pointed that MMT does not contain any deficiency, the data was
obtained about some technical problems such as visuality and need of development of
personalization points. When analyzing the answers given by the students, some of the important
points are shown in Table 8.
Sample students' answers given to question 8 are as follows: “I think MMT is enough for students;
however, I believe it should contain more information for students.”, “In the course planning section,
sample times might be specified.”, “Parts of the subject content studied might be presented visually.
In other words, visuals might be added.”
131

Table 8: Students’ responses to question 8
Serdar Çiftci and Mehmet Akif Ocak
Keywords f %
There is no deficiency of the MMT 5 29,41
There are problems with design 2 11,76
Some constraints exist 2 11,76
Time management problems 2 11,76
Technical problems 2 11,76
Detailed information is needed 2 11,76
It should be developed 1 5,88
Personalization problems 1 5,88
Total 17 100
9. While studying in a web based environment, do you need this kind of tool/ support / guidance?
Findings show, while students study in web based environment, it was observed that they need such
a tool. When studying in an individual lesson via the Internet, one of the important considerations was
distracting variables affecting the students’ interest and attention. The MMT made a positive effect on
planning to deal with such issues. When analyzing the answers given by the students, responses are
shown in Table 9.
Table 9: Students’ responses to question 9
Keywords f %
Yes (For learning) 15 65,22
Yes (For the web) 4 17,39
Partially yes 2 8,70
No 2 8,70
Total 23 100
Sample answers given by students to question 9 are as follows: “I have not used this type of a web
based tool before. It caught my attention and I started to use. I think it is great for planning and
monitoring.”, “When using web based course, it is very easy to lose my interest since the internet is
always under my hand. But thanks to this tool that helped to develop a plan and according to the plan,
I was able to focus on to what I really have to do.”, “I think it is necessary to have such a tool not only
in web based environments but also in other learning environments. So far, I was not aware of the
need to use such a tool. Moreover, I think everyone needs such a tool.”
6. Conclusion and recommendations
The purpose of the study was to examine students’ metacognitive activities in a web based learning
environment. The MMT was developed to guide and allow students to achieve structured learning
task. Results found that the MMT constructed according to metacognitive components (planning,
monitoring, self-evaluation and revision) was used by the students efficiently and positively.
Thinking about online courses, Henning (2003) indicated that we need “more than data of
performance in the technology is needed to interpret the learners’ social position comprehensively” (p.
304). Hence, supporting students’ metacognitive activities with learning tools would be a solution to
this obstacle. The American Psychological Association (APA), in its report for a framework for school
redesign and reform (APA, 1997), concluded that metacognition is one of the most important
principles for learning. It is important to note that that basic concept behind the metacognition differs
from adaptive instructional systems. In this study metacognition mainly deals with students’ selfregulated
learning strategies and perceptions on metacognitive skills. As indicated by Carrier and
Jonassen (1988), adaptive learning includes following variables: (a) aptitude variables (b) prior
knowledge (c) cognitive styles and (d) personality variables. The structure of an adaptive system
needs collection of information about individual user. This study did not focus on the variables
indicated above; however further research will be needed to investigate the relationship between
metacognition and adaptive system in terms of learning tools.
132

Serdar Çiftci and Mehmet Akif Ocak
The results showed that the use of MMT resulted in supporting student awareness and monitoring of
thinking during complex tasks. Martinez (2010) found that similar metacognitive tools provided
common vocabulary to make students’ thinking explicit to teachers and other students and providing a
generic but powerful process that could be used by students for other learning tasks. Hartman (2001)
noted that metacognition is especially important because it affects comprehension and retention of
what is learned, in addition to affecting learning efficiency, critical thinking and problem solving.
In web based learning environments, computers and the Internet might lead to distraction of attention.
In this regard, the MMT was observed to make a positive impact on students. In addition, to make the
metacognitive activities in the students’ mind concrete was one of the best advantages provided by
MMT. The visual development of the tool and some of the sample plans for students as a
presentation were seen as limitation. Further research might concentrate on the improvement of the
tool. To present a sample plan to students is a condition to be investigated in terms of limiting the
students’ own plan activities. As limitations of the study, there was a limitation on generalizing the
study result because of the number of participants. Also, the MMT was limited to principles of
metacognition: planning, monitoring, self-evaluation and revision. Other principles about
metacognition might be included for further research.
References
American Psychological Association (1997). Learner-Centered Psychological Principles: A Framework for School
Reform & Redesign. Retrieved on 15/08/2011, from http://www.apa.org/ed/governance/bea/learnercentered.pdf
Anderson, N.J. (2002). The role of metacognition in second language teaching and learning. ERIC Digest, April
2002, 3-4.
Azevedo, R., Moos, D. C., Greene, J. A., Winters, F. I. and Cromley, J. C. (2008). Why is externally regulated
learning more effective than self-regulated learning with hypermedia?, Educational Technology Research
and Development, 56, 45–72.
Carrier, C., Jonassen, D. H. (1988). Adapting courseware to accommodate individual differences. In D. Jonassen
(Ed.), Instructional desings for microcomputer courseware. Mahwah, NJ: Lawrence Erlbaum Associates.
Collis, B. and Meeuwsen, E. (1999) Learning to learn in a WWW-based environment, In D. French, C. Hale, C.
Johnson and G. Farr, (Eds.), Internet based learning: Higher education and industry, Sterling, VA: Stylus
Publishing, pp. 25-46.
Flavell, J. H. (1976), Metacognitive aspects of problem solving, In L. Resnick (Ed.), The nature of intelligence,
Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 231-235.
Hartman, H. J. (2001). Metacognition in Learning and Instruction: Theory, Research and Practice. Dordrecht:
Kluwer Academic Publishers
Henning, E. (2003). ‘I click, therefore I am (not)’: Is cognition ‘distributed’ or is it ‘contained’ in borderless
eLearning programmes? International Journal of Training and Development, 7(4), 303-317.
Jones, M. G., Farquhar, J. D. and Surry, D. W. (1995). Using metacognitive theories to design user interfaces for
Computer-based learning, Educational Technology & Society, 35(4). 12-22.
Pifarre, M. and Cobos, R. (2009), “Evaluation of the development of metacognitive knowledge supported by the
knowcat system”, Education Tech Research Dev, 57, 787-799.
Scraw, G. and Brooks, D. W. (1998), Helping Students Self-Regulate in Math and Sciences Courses:Improving
the Will and the Skill, University of Nebraska-Lincoln, Lincoln, NE, Retrieved on 08/06/2011, from
http://dwb.unl.edu/Chau/SR/Self_Reg.html.
Simons, P. R. J. (1996). Metacognition. In: E. DeCorte and F. E. Weinert, Editors, International encyclopedia of
developmental and instructional psychology, Pergamon Press, Kidlington, pp. 436–441.
Subaşı, G. (2000), “Etkili Öğrenme: Öğrenme Stratejileri”, Milli Eğitim Dergisi, 146, Retrieved on 02/06/2011, from
http://yayim.meb.gov.tr/yayimlar/146/subasi.htm.
Thomas, G. P. and McRobbie, C.J., 2001, Using a metaphor for learning to improve students metacognition in
the chemistry classroom, Journal of Research in Science Teaching, 38(2), 222-259.
Tobias, S., Everson, H. and Laitusis, V. (1999). Towards a performance based measure of metacognitive
knowledge monitoring: Relationships with self-reports and behavior ratings. Proceedings from the American
Educational Research Association proceedings on Metacognitive Monitoring in Montreal, Canada.
Retrieved August 10, 2011 from http://www.fordham.edu/gse/faculty/tobias/aera99.html
Tsai, M. J. (2009). The Model of Strategic eLearning: Understanding and Evaluating Student eLearning from
Metacognitive Perspectives, Educational Technology & Society, 12(1), 34-48.
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning, Journal of Educational
Psychology, 81(3), 329-339.
133

An Exploratory Comparative Study of Distance-Learning
Programmes
Marija Cubric, Karen Clark and Mariana Lilley
University of Hertfordshire, Hatfield, UK
m.cubric@herts.ac.uk
k.clark@herts.ac.uk
m.lilley@herts.ac.uk
Abstract: In this paper we describe an exploratory study of distance learning practice. We review five different
distance learning programme models from five different schools at the University of Hertfordshire, each varying in
production and presentation. We situate the programmes in an extension of Weller’s pedagogy-technology
space, and we further qualify their pedagogy, using Chickering and Gamson’s principles as a basis for
evaluation. The results of our analysis show that while the flexibility offered to students and economics of
distance learning are indeed important drivers for implementation and adoption of distance learning, the quality of
teaching and students’ learning experience is less well understood and frequently overshadowed by the abovementioned
factors. Moreover, we found that certain principles for ‘good teaching’ become more important than in
the face-to-face scenarios, some principles assume different meaning in distance learning situations and new
principles related to effectiveness and ‘affordability’ of on-line communication emerge and gain in importance.
The study aims to help develop a framework for analysis to be a tool for programme planners in a dynamic
education environment. It is already helping in formulating implementation of the ambitious distance learning
strategy at University of Hertfordshire but can also help other higher education institutions that aspire to provide
quality distance learning education in the future as well as in informing other providers of distance learning
materials and tools.
Keywords: distance learning, distance learning programmes, distance learning pedagogy distance learning
models, Chickering and Gamson’s principles
1. Introduction
A simple, commonly agreed definition of distance learning is that of ‘planned learning that occurs in a
different place from teaching…’ (Moore & Kearsley, 1996:2). This definition is broad enough to
include all evolutionary stages of distance learning from early correspondence courses to the present
generation based on increased use of the internet. The media used to present learning materials has
changed over time and as a consequence so has communication between teacher and learner, with
many more opportunities for shared and collaborative learning emerging in the modern age.
In addition to the separation of learner and teacher in space (or time), Sherry (1996) identifies two
more ‘hallmarks’ of distance learning: control of learning by students and non-contiguous
communication between student and teacher.
We argue that neither of the above characteristics is specific to distance learning, nor necessarily
implied by contemporary distance learning. Active, student-controlled learning is desirable attribute of
any type of learning, Also, more and more researchers and practitioners are emphasising the
importance of ‘pacing’ of learning materials (Sherry, 1996, Galusha,1997, Cohen, 2002) as well as of
increased teacher ‘presence’ (Garrison, Anderson and Archer, 2000) in distance learning processes.
Advancing ICT capabilities such as audio and video conferencing meant from the 1980s onwards
communication no longer had to be asynchronous. More and more synchronous interactions became
feasible through the use of technologies such as Skype, virtual classrooms (e.g. Elluminate) and
similar. Classroom-based courses are also increasingly encompassing asynchronous means of
communication between the teacher and students, through the use of e-mails, chats, discussion
forums etc.
In addition to the medium used for learning materials and communication, further distinction can be
made based on the unit of learning, that in the context of higher education can be a single module (or
course), degree programme or an entire institution.
Through the rest of this paper we will adopt Moore & Kearsley’s (1996:2) definition in the context of
internet-enabled distance learning and we will consider five different distance-learning programmes
offered at five different schools at University of Hertfordshire, that vary in scope, production and
presentation.
134

Marija Cubric et al.
The aim of the study is to help develop a framework for analysis to be a tool for distance learning
programme planners. The main objective is to identify principle dimensions and attributes of distance
learning programmes that can be used to compare different programmes and to contextualize them
for evaluation purposes. Another objective is to assess applicability of some of the course (module) –
based evaluation tools, such as Chickering and Gamson’s (1987) principles, in the context of distance
learning programmes.
The approach used in the paper is based on semi-structured interviews with programme tutors and
teachers from the five programmes. Programmes were selected based on their match with the
definition of distance learning adopted in this paper, and do not include some other approaches
regarded as distance learning by the University administration, such as ‘dissertation top-up’
programmes, ‘fly-in faculty’ programmes, or ‘work-based learning’ programmes.
Pre-prepared interview questions, included basic programme descriptors, such as number of
students, type of assessment, etc., as well as specific questions on the extent of implementation of
specific Chickering and Gamson’s (1976) principles in teaching and programme implementation.
Those questions were augmented with additional questions that came out as a result of the
interviewees’ responses. These additional responses were ‘normalized’ and sorted into additional
categories, such as, drivers and barriers for staff engagement, effectiveness of distance learning and
measures of it, degree of interaction between classroom based (CB) and distance learning (DL)
students etc.
The paper is organized as follows. We start with a review of the literature in section 2. This is followed
by a summary of cases (section 3), comparison of cases and relevant discussion (section 4) and
conclusions (section 5).
2. Background
The literature review on distance learning (or distance education, as it is sometimes referred to)
reveals a large body of work on issues, barriers, benefits and strategies for implementation. Equally,
it reveals a lack of standards for classifying different modalities of distance learning.
Dillenbourg (1999) argues that collaborative learning cannot be discussed unless the learning is
contextualized: similarly, within distance learning the ‘space’ must be defined. In other words, an
object in ‘distance learning space’ defines one type of distance learning with specific effects that
cannot be automatically generalized to other objects in the space. For example, the effects of
distance learning taking place with a small group of part-time local students with a residential tutoring
option cannot be extrapolated to distance learning taking place with a massive 100+ globally
distributed cohort. Therefore it is important to understand different defining characteristic
(‘dimensions’) of distance learning approaches, in order to contextualize results.
Bélanger and Jordan (2000) provide an overview of the learning variables applicable to distance
learning that includes learning objectives from Bloom’s (1956) cognitive, psycho-motor and affective
domains, interactivity (between learners, and between content and learner), content navigation and
search capabilities of the learning materials, and synchronicity of communication.
Weller’s (2002) framework for classifying distance-learning courses extends Bélanger and Jordan’s
(2000) set of pedagogical variables with technology-related attributes. This approach is based on the
premise that technology and pedagogy are ‘intertwined’ in any online course, i.e. actively and
iteratively influencing each other. The framework has two dimensions representing the influence of
technology or pedagogy on the course.
The ‘technology’ axis represents the degree of technological sophistication in the design and delivery
of a course. Weller (2002) proposes the following criteria for assessing the technology of a course:
Range of media (audio, video, animation)
Interactive tools such as quizzes, games, simulations
Degree of personalization offered
Sophisticated back-end (tracking progress, logs, annotations)
Web-page design including navigation, interactivity, search
135

Marija Cubric et al.
Web 2.0 tools (wikis, blogs, RSS, sharing of slides, images etc.)
Communication environment to facilitate dialogue e.g. discussion forums, chats, etc.
Additionally, the following more recent net-centric applications (Anderson, 2009) should be added to
the criteria for technical ‘richness’ of a course:
Awareness mechanisms, such as notifications (e.g.RSS), online presence and status updates
(e.g. Twitter) etc.
Tools for supporting virtual communities, based on the ‘wisdom of the crowd’ idea such as wikis,
Digg, Facebook, wePapers, Course Hero, Elgg, Ning, VoiceThread.com etc.
Emerging network-centric applications for aggregating the information and extracting knowledge
(e.g. Slashdot, Omgili)
Mobile learning
Virtual worlds such as Second Life etc.
Two different poles of the ‘pedagogy’ axis represent a ‘didactic’ or teacher-directed approach, and a
‘constructivist’ or student-centred, collaborative approach. Rather than considering learning
objectives, Weller uses following criteria for characterising how close is the pedagogy of the course to
either the didactic or constructivist end of the scale:
Focus on content vs. focus on students’ interaction
Assessing retention of content vs. assessing student’s interpretation
Traditional lecture-based teaching vs. ‘conversational’ teaching and active learning
Teacher as an expert vs. teacher as a facilitator or mentor
Learning as a knowledge acquisition vs. learning as a construction of knowledge through social
activity.
The extent to which the above attributes prevail in the course, will determine its proximity to either
side of the pedagogy axis.
The two-dimensional framework results in four different categories of distance learning:
Low technology/didactic approach, based around streaming video lectures, and some form of
CMC (computer mediated communication) such as email, suitable for initial adoption and low
investment
Low technology/constructivist approach, includes simple websites with more substantial CMC
capabilities; suitable for small scale university courses in non-technical subjects that involve
discussions and debating e.g. online courses in theology, philosophy, history and similar
High technology/didactic approach, also know as ‘web-based training’, often aimed at individuals,
who may or may not be supported by a tutor; suitable for CPD, professional certifications,
accreditations, life-long learning and work-based learning
High technology/constructivist approach, encompasses virtual environments, different online
spaces that promote collaboration; particularly useful for engineering and scientific subjects.
Weller (2002) adds that the total cost, including production of materials (production cost) and staff
time for course delivery (presentation cost) differs significantly between the four models, with the
lowest cost being associated with low technology/didactic approach and highest with the high
technology/constructivist approach. He concludes that balancing the trade-off between the cost and
technological and pedagogical sophistication of the course is the key factor for success of online
courses.
One interesting observation with cost implications noted by Lozier, Oblinger, and Choa (2002) is that
centralized services are generally used to support development and technology solutions while
responsibility for core academic decisions, including course content, conferring degrees and faculty
workload remains within individual departments.
Similarly to Weller (2002), Cohen (2002) proposes a model for evaluation of distance learning
courses, based on the combination of pedagogical and technological factors. He further distinguishes
136

Marija Cubric et al.
between different pedagogical criteria, such as: the process of learning and teaching, the community
of learners, the role of teacher, the role of students and implementation of the course.
Gaspray, Dardan and Legorreta (2008) consider the effectiveness of distance learning through the
‘lens’ of different learning theories, such as objectivist, constructivist, collaborative, socio-cultural,
cognitive and computational models of learning. They suggest that each of the learning models,
implies different meaning of learning ‘effectiveness’ i.e. student grades, student satisfaction,
perceived interaction difficulty, perceived flexibility, learning climate, perceived knowledge and skill
development respectively. They conclude that distance learning has characteristics that are important
to all learning models, and while, the first three models are well-understood and widely accepted by
practitioners, the last three need more attention, in order to ‘enable minority perspectives (sociocultural
model) as well as individualized perspectives (cognitive and computational models)’
(Gaspray, Dardan and Legorreta, 2008:58).
This suggests that constructivism in Weller’s framework should be placed somewhere in the middle of
pedagogy axis, while the pole opposite the ‘didactic’ end of the scale should in fact be characterized
as socio-cultural, cognitive and computational.
Sherry (1996) identifies two main models of distance education, based on the starting point or
‘philosophy’ of the design. The ‘Iowa model’ starting point in design is classroom–based teaching, and
here distance learning tries to recreate this classroom environment via mediating technologies such
as virtual classrooms, audio-visual interactions etc. Alternatively, the ‘Norwegian model’ starts with
distance teaching that can be computer-mediated and combined with some local ‘face-to-face’
support. A third model, not mentioned by Sherry, is based on the Iowa model but in addition provides
local face-to-face support in a form of residential sessions or local tutoring (branded as ‘tutored elearning’
by University of Hertfordshire - http://www.herts.ac.uk/courses/schools-of-study/computerscience/online-courses/supported-elearning.cfm).
Beldarrain (2006) explores the affordances of different new technologies, such as wikis, blogs
podcasts etc. in the context of distance learning, and suggests the use of Chickering and Gamson’s
(1987) principles as a starting point for defining the purpose and rationale of integrating specific
technology into the distance learning curriculum.
Chickering and Gamson’s principles were published in 1987 in a bulletin of the American Association
for Higher Education & Accreditation, as a direct response to a call made by the association for easy
to understand, practical and general principles that would guide further reforms of higher education
and lead to better student experience. The principles state that good practice in undergraduate
education:
Encourages contact between students and faculty
Develops reciprocity and cooperation among students
Encourages active learning
Gives prompt feedback
Emphasizes time on task
Communicates high expectations
Respects diverse talents and ways of learning.
Our choice of the principles as a framework for comparison was based not only on their simplicity and
practicality, but more importantly because they were founded on more than 30 years of research on
how we teach and how students learn.
Research findings especially support the principle of staff contact with students, active learning and
delivery of feedback, all three of which are said to have a positive impact on students’ learning and
engagement. Critiques of the principles argue that their importance varies across different disciplines,
teaching methods, learning styles or organisations (e.g. Sorcinelli, 1991). For example, they are better
suited for humanities and social sciences and are intended for traditional (18-21 year-old) students.
Dalton & Tharp (2002) argue that the principles are incomplete and suggest additional constructivistbased
requirements such as that teacher and students should join in productive activities and that
‘meaning’ should be generated by linking the curriculum to students' lives.
137

Marija Cubric et al.
Despite all the issues, the principles have survived the test of time, and remain a popular tool for
guiding curriculum design (Beldarrain, 2006) and evaluating the quality of learning and teaching in
online courses (Graham et al., 2001).
As observed by Merisotis and Phipps (1999) the research literature is focused on modules (or
courses) and lacks in evaluation of distance learning programmes. In this paper we start to address
that question.
We use Weller’s framework as a basis for classification of distance learning programmes and we
extend it with some other categories that appeared in course of discussions with the interviewees.
Once the programmes are ‘situated’ in this ‘extended’ Weller’s space (Figure 2), we further qualify
their pedagogy, using Chickering and Gamson’s principles as a basis for evaluation and comparison
(section 4).
3. Summary of cases
In this section we summarize (anonymously) details of five programmes from five different subject
areas, based on data collected in December 2009, excluding any subsequent programme changes.
Case 3.1
This MA programme was established in 2007/8 and initially targeted international students though it
also includes a few home students. It is relatively small (less than 20 students). The learning and
teaching model of the programme (including assessment) is based on a similar classroom-based
programme (Iowa model) and employs a range of technologies such as (proprietary) MLE, Flickr (for
uploading students’ work), discussion forum, email, Skype and Facebook for discussions and
feedback, as well as recorded guest lectures. Teaching is done via guest lectures and students are
supported in developing projects through frequent interaction with a tutor. Learning effectiveness
measures are not decisive due to the small size of the cohort. The entire programme is supported by
one (enthusiastic) member of staff! The main barriers for further staff engagement are the fear of
extended workload, inadequacy of the current workload model for distance learning provision as well
as a doubt among staff members that distance learning is an adequate method for learning creative
subjects. In terms of Weller’s (2002) classification, the model can be described as low
technology/constructivist approach, with low production cost (as there are no teaching materials
specifically developed for the programme) and high presentation cost, due to the intensity and
frequency of interactions between the students and staff.
Case 3.2
This BSc programme was established in 2004, and since then more than 1,200 students have
enrolled and more than 600 have graduated. The programme runs in two different modalities: online
(Iowa model) and online with local (face-to-face) tutoring support (‘tutored e-learning’). The student
population is derived from over 35 countries across the world.
Online students are tutored by UH staff predominantly through the University’s MLE (purpose-built
learning materials, discussion forums, blogs, wikis, group work, electronic journals, e-books). The use
of the MLE is supplemented by a suite of applications to support synchronous collaborative work,
presentations, vivas and online tests. Tutored e-learning students have access to the same online
facilities and resources but also receive local tutor support from staff at a partner institution.
Independent study is supported through sequenced learning activities. The programme’s pedagogical
framework (Pyper, Lilley & Hewitt, 2009) is based on learning activities comprised of tasks and
resources and a narrative component to provide rationale for the work.
Although there is no formal interaction between classroom-based and distance learners, some online
learning materials are currently being used by classroom-based modules, enhancing the flexibility for
these learners. Learning effectiveness is high compared to similar classroom-based programmes,
with distance learners achieving slightly better grades and expressing higher level of satisfaction with
the course. The retention rate is comparable to similar classroom-based programmes. Drivers and
barriers for staff engagement are related to individual preferences i.e. some staff prefer face-to-face
teaching, others prefer the flexibility offered by distance learning.
138

Marija Cubric et al.
Both modalities can be classified as medium to high on the technology scale, and in the ‘middle’ on
the (constructivist) pedagogy scale. The production and presentation cost will also be somewhere in
the middle, with the strong economy of scale effect, as the programme continues to grow.
Case 3.3
This postgraduate programme offers a flexible route, where students can choose the duration
(12/15/18 months) and mode of study (classroom-based or distance learning). The student population
consists of more than 70 early career professionals. It is designed as a combination of classroombased
teaching (induction weekend and another extended weekend) with online activities. All
technologies used are based on the (proprietary) MLE extended with additional features for online
delivery, designed by a dedicated member of staff. Pedagogy is content-driven with some elements of
experiential learning (activities) and collaborative learning (discussions).
Case 3.4
The postgraduate programme is aimed at working practitioners who need to complete specific
professional training. It was established in 1996, with currently more than 150 students enrolled of
whom 50% are home with the rest mainly from EU. It is designed as a combination of distance
learning (14 weeks) and residential 3-day workshop/conference after the first six weeks. It can be
classified as the ‘Norwegian’ model. Apart from residential sessions which are based on tutor-led
problem-based and collaborative learning activities, tutor support is limited to on-demand contact and
extra support for weaker students. Technologies used are limited to (proprietary) MLE and telephone
communication. Although the learning effectiveness data are not provided, students are highly
motivated as they are expected to report back at their workplace. The majority of teaching staff are
visiting lecturers i.e. working practitioners from industry.
Case 3.5
This postgraduate programme is aimed at students who wish to convert a first degree into a
professional qualification. The distance-learning route runs in tandem with the classroom-based and
was established in 2007. It is relatively small with around 60 students enrolled about one third of
whom are distance learners based mainly in the UK. It follows the Iowa model: lectures and plenary
sessions are recorded in the classroom and uploaded on the (proprietary) MLE. Initially pedagogy
was didactic, but a staff project established in 2010 was set to move closer to the constructivist end of
the scale. Technologies used include (proprietary) MLE, audio podcast, virtual classroom (rarely),
discussion forums, and email. Achievement and retention for distance-learning students is
comparable to that of classroom-based peers. Student satisfaction is hard to administer, but
comments are usually highly positive. All students are enrolled on the same programme and have
access to all materials.
4. Comparison of cases
We start to differentiate between the five programmes (Figure 1) using the following attributes for
comparison:
Level of study
Student population
Years running to-date
Scale (number of students)
Learning and teaching model (Iowa, Norwegian, face-to-face%)
Technologies used
Leading pedagogy i.e. the pedagogical approach used by the majority of modules in the
programme
Learning effectiveness (measured through students’ performance, satisfaction and retention rate)
Interactions between classroom-based (CB) and distance learning (DL) students
Drivers and barriers for engaging teaching staff
139

Marija Cubric et al.
Staff profile i.e. permanent (CB), visiting lecturers, specially employed staff etc.
While the first four attributes correspond to simple ‘demographic’ aspects of a programme, others
result either from the reviewed literature (5-7) or emerge as important aspects of DL programmes in
discussions with the interviewees. An additional criterion for differentiation is the subject discipline,
which is not considered in this paper because of anonymised data.
Case Scale/Student
population
3.1 Low/
UK + some
O/S
3.2
3.3
3.4
3.5
Very high/
35 different
countries
Low-medium/
UK
Medium/
UK+ some EU
Medium/
mainly UK
Basic Face-toTechno- Pedagogy Staff Learning
model face % logy
effectiveness
Iowa 0 Low Constructivist Dedicated
to DL
programme
NA
Iowa 0/50% Medium
- high
Iowa Inductio
n day +
Norwegia
n
3 days
3 days
per
module
Middle
between
didactic/
constructivist
Combinatio
n of CB
staff, tutors
at partner
institutions
and visiting
vecturers.
Low Didactic Same as for
CB
Very low Middle
between
didactic/
students
Mainly
visiting
lecturers
Iowa 0 Low Didactic Same as for
CB
students
Better results
and higher
satisfaction
than CB
students
NA
Highly
motivated
students
Comparable
results; CB
students
switching to
DL mode
Figure 1: Comparison of distance learning programmes
While the programmes clearly differ in scope, use of technology and pedagogy, they each offer
specific value to a specific, target population of learners: continuous conversation guiding
development of student’s practice (3.1), focus on self-study for professional qualifications (3.3 and
3.4), support for large diverse cohort of international students (3.2) and integrating distance learners
with CB students into a more cohesive learning community (3.5).
By comparing and contrasting the pedagogy and technology of programmes under investigation, we
are able to situate each programme in Weller’s distance learning space (Figure 2), extended with new
dimensions such as subject discipline, scope (number of students) and Sherry’s (1996) distance
learning categories, represented by shapes of different size in Figure 2.
In the rest of this section, we continue to differentiate between programmes’ pedagogies using
Chickering and Gamson’s (1987) principles as a framework for discussion. To refer to a specific case,
we use the corresponding section number i.e. 3.1, 3.2 etc.
4.1 Good practice encourages contact between students and faculty
Examples supporting this principle include: daily staff availability via emails (all cases), Facebook chat
(3.1), tutor’s feedback on Flickr uploads (3.1), tutor engagement in discussion forum (3.3), personal
tutoring via e-mail, phone or chat (3.3), and use of Skype/Elluminate/Facebook/individual online work
areas (3.2).
In case 3.4, a three-day residential session is setup as a conference, providing formal and informal
interactions. As learners are adults, there are fewer barriers between students and teachers.
Additional social activities are organised in case 3.5, where locally-based DL students are invited to
public formal and social events with staff.
140

Marija Cubric et al.
4.2 Good practice develops reciprocity and cooperation among students
This principle is supported with mechanisms for awareness (‘see who is online’), use of (online)
discussion forums (3.1), discussion group ‘meeting rooms’ (3.2) etc.
Figure 2: Distance learning programmes space
In case 3.3, students are encouraged to post on discussion boards, with some less frequent use of
wikis and blogs.
Residential workshops offer opportunities for collaboration and cooperation – some assessment in
case 3.4 is a group-based workshop: students are purposely not introduced, encouraging them to find
out about their colleagues.
In case 3.5, students are asked to post an online biography to create a sense of community. Group
formative assessments require collaborative activities between CB and DL students.
Although the literature emphasises the importance of the social component of distance learning
(Garrison, Anderson, & Archer, 2000) , it can be difficult to achieve (at least synchronously) in cases
where the majority of students on programme do not share same time zone (3.2).
4.3 Good practice encourages active learning
In case 3.1, the tutor indicates various (online) information sources required for a live project.
In case 3.2 this is facilitated through a pedagogical framework (Pyper, Lilley & Hewitt, 2009) that
guides tutors in designing various online tasks to keep students active.
Case 3.4 is based on extensive self-directed study periods before and after the residential session.
In case 3.5, online quizzes offer structured feedback and encourage cooperation between learners,
discussing particular questions or answer rationale etc
4.4 Good practice gives prompt feedback
Examples include: feedback on Flickr images (3.1), annotated essays or sample answers after the
residential session in case 3.4, response to email enquiries and use of audio for feedback i.e.
recorded comments in case 3.5.
141

Marija Cubric et al.
In case 3.2, a ‘service level agreement’ is established regarding the expectations of staff and
students’ engagement with discussion forums and emails. In addition to that, automated tools,
annotations and sample answers are used to provide feedback.
4.5 Good practice emphasizes time on task
This principle was not considered in cases 3.1 and 3.4, where students are expected to conduct selfdirected
study, supported by subject tutors, (when requested). In case 3.5, module guides specify
session outlines, including formative activities planned for the session. More granular time on task is
implemented in cases 3.2 and 3.3, where students are asked to complete weekly individual or group
tasks with deadlines, using variety of methods and technologies (proprietary MLE groupware,
Facebook etc). In addition, in case 3.3 all units are broken into timed tasks.
4.6 Good practice communicates high expectations
According to most interviewees, the implementation of this principle relies on the type of learning
activities used in different modules. For example, in case 3.1, students are expected to work on a ‘live
project’ and to publish the results of their work (as well as work in progress) on a public website.
4.7 Good practice respects diverse talents and ways of learning
With regards to this principle, various programmes offer various approaches. In case 3.1, programme
tutor acts as a personal tutor, and provides one-to-one support to individual students. Students are
accepted based on the portfolio and set of learning objectives that they want to achieve. The
programme does not provide teaching of techniques but instead aims to support students in
developing individual practice.
In case of 3.2, from its inception, the programme was designed to support diversity and flexibility, by
offering two modalities of distance learning (with or without face-to-face contact).
In case 3.3, students are offered flexibility not only in where and how they learn, but also in the
duration of their studies (12/15/18 months). In case 3.5 students can choose to study wholly or partly
by distance over one or two years.
In cases 3.2 and 3.4 assessment variety helps support different types of learners.
4.8 Discussion
The principles used for comparison are useful in situating the pedagogy of different DL models,
however they seem to mean different things to different people. These differences seem to have been
even greater across programmes.
Weller (2002), Chickering and Gamson (1987), and indeed most evaluation studies, concentrate on
specific courses (e.g. Graham, 2001; Buckley, 2003) or usability of technological solutions (e.g.
Tselios et al, 2001). This, perhaps inevitably, means that the usefulness and applicability of some of
the principles varies when extrapolated for DL programme analysis.
Based on the data from our study, it may be anticipated that some principles are usable at
programme level (e.g. 4.1, 4.4 and 4.7) while others are perhaps expected to be more
module/teacher-centric (4.3, 4.5, 4.6) or cohort-dependent (4.2).
Amongst the ‘programme-level principles’, principle 4.1 (‘staff-student contact time’) and 4.4 (‘prompt
feedback’) could be formulated within the programme service level agreement, while 4.7 may be
answered by the intrinsic flexibility of distance education.
Although Weller’s framework was intended for classifying online modules, it can be applied equally
well in the context of distance learning programmes, as they tend to exhibit higher ‘cohesion’ with
respect to pedagogies and technologies used across different comprising modules (Lozier, Oblinger,
and Choa, 2002). Indeed the experience in case 3.5 is that this consistency is not just a feature but a
requirement for student satisfaction in a DL model at the didactic end of the spectrum.
142

5. Conclusions
Marija Cubric et al.
Comparing distance learning programmes is clearly non-trivial; for example, different pedagogies
imply different meanings for learning effectiveness, as suggested by Gaspray, Dardan and Legorreta
(2008). There are also issues with gathering and interpreting appropriate primary data, which our
own experience highlights; the design of different research instruments – for example, a questionnaire
to be consistently administered across programmes – could be considered. It is very difficult to draw
definitive conclusions from the current data analysis under Chickering and Gamson (1987), although it
has been possible to identify which of their principles are useful at programme level. it is easy to see
the value of an overarching framework for evaluation not least at the design and implementation stage
of programmes. Such a framework could provide a mix of quantitative and qualitative data to support
stronger conclusions in the future.
So, the value of this work is in sketching such a framework for analysis to be a tool for programme
planners in a dynamic education environment. In particular, one of its key strengths is the combination
of technological and pedagogical axes – the literature indicates that historically there has been an
over-emphasis on technology and this framework offers an integrated way to move forward. This
framework, including accessibly-formulated principles also offers a chance for programme planners to
expound a practical pedagogic rationale to staff who may well be, as many have noted, reluctant to
reflect on their own teaching practice.
References
Anderson T (2009) ALT-C 2009 - Keynote Speech on 10 September 2009, Available online at:
http://www.alt.ac.uk/altc2009/keynotes.html [07/06/2011]
Bélanger, F. and Jordan, D.H. (2000) Evaluation and implementation of distance learning: technologies, tools,
and techniques. UK: Idea Group Publishing
Beldarrain Y. (2006) ‘Distance education trends : integrating new technologies to foster student interaction and
collaboration’, Distance Education 27(2) 139-153
Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David
McKay Co Inc.
Boyed-Barrett O (1999) ‘European and North American Models of Distance Education’, The Cal Poly Pomona
Journal Of Interdisciplinary Studies
Bullen M. (1999) ‘What’s the Difference?: A Review of Contemporary Research on the Effectiveness of Distance
Learning in Higher Education (Authors: Jamie P. Merisotis and Ronald A. Phipps) ‘, Journal of Distance
Education, 14:1, 102-114
Buckley K.M., (2003) ‘Evaluation of classroom-based, Web-enhanced, and Web-based distance learning nutrition
courses for undergraduate nursing’, Journal of Nursing Education, 42(8):367-70
Cohen , V.L. (2003) ‘A model for assessing distance learning instruction’, Journal of Computing in Higher
Education, 14(2), 98-120
Chickering, A. W., and Gamson, Z. F. (1987) ‘Seven Principles for Good Practice in Under- graduate Education.’
AAHE Bulletin, 39(7), 3–7
Dalton S.S. & Tharp R.G. (2002) ‘Standards For Pedagogy: Research, Theory And Practice’, In Learning for life
in the 21st century: Sociocultural perspectives on the future of education. G. Wells & G. Claxton (Eds.)
Oxford: Blackwell, 181-194
Dillenbourg P. (1999) ‘What Do You Mean By Collaborative Learning?.’, In P. Dillenbourg (Ed) Collaborative-
Learning: Cognitive And Computational Approaches, 1-19. Oxford: Elsevier
Galusha J. (1997) ‘Barriers to learning in distance education’, Interpersonal Computing and Technology, 1997
Garrison, D. R., Anderson, T., & Archer,W. (2000) ‘Critical Inquiry In A Text-Based Environment: Computer
Conferencing In Higher Education’,The internet And Higher Education, 2(2-3), 87-105
Gaspay A., Dardan S., Legorreta L. (2008) ‘Distance learning through the lens of learning models’, Review of
Business Research, 8(4)
Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy T. M. (2001) ‘Seven principles of effective teaching: a
practical lens for evaluating online courses’, The Technology Source Archives at the University of North
Carolina. Available online at: http://bit.ly/atSX7
Lozier, G., Oblinger, D., Choa, M. (2002) ‘Organizational Models for Delivering Distance Learning’, Educause
Center for Applied Research Research Bulletin, 2002(2)
Merisotis, J.P., Phipps, R.A. (1999) ‘What’s the Difference?: A Review of Contemporary Research on the
Effectiveness of Distance Learning in Higher Education’, Washington, DC: The Institute for Higher
Education Policy
Pyper, A., Lilley, M. Hewitt, J. (2009) ‘A Framework to Support Students in Their Individual Studies’, In
Proceedings of the First International Conference on Computer Supported Education, Lisboa, Portugal,
March 23-26, 2009 - Volume 1 2009
Sherry, L. (1996) ‘Issues in Distance Learning’, International Journal of Educational Telecommunications, 1(4)
337-365
143

Marija Cubric et al.
Sorcinelli, M. D. (1991) ‘Research Findings on the Seven Principles’, New Directions for Teaching and Learning,
47
Tselios N., Avouris N., Dimitracopoulou A., Daskalaki S. (2001) ‘Evaluation of Distance-learning Environments:
Impact of Usability on Student Performance’, International Journal of Educational Telecommunications,
7(4), 355-378
Weller, M. (2002) Delivering Learning on the Net, Routledge_Farmer
144

The Optimal Teaching Style Based on Variability of Study
Materials
Blanka Czeczotková, Kateřina Kostolányová and Jana Šarmanová
Ostravská univerzita, Ostrava, Česká republika
D09434@student.osu.cz
Katerina.Kostolanyova@osu.cz
Jana.Sarmanova@osu.cz
Abstract: Teaching by standard eLearning is being gradually replaced by a new form - personalized eLearning.
Personalized eLearning is understood as not only an instruction tailored to each student according to his
characteristics, but it is also adaptable according to the actual conditions under which the learning takes place.
Pilot testing of students and subsequent analysis determined a group of student´s characteristics to which the
eLearning study environment can be adapted. These characteristics must be put into accord with forms and
variants of created learning materials. This paper deals with the assignment of an appropriate method of learning
management to students' individual learning styles.
Keywords: adaptive eLearning, virtual teacher, teaching styles, learning styles, teaching methods
1. Introduction
Learning of an individual is affected by many factors - in particular by student learning styles, their
personalities and motivation to learn. From this perspective, each student can be seen as an
individual. During a standard instruction, the student is a part of a team and the teacher adapts
teaching to suit the majority of students in the classroom. An experienced teacher notices during faceto-face
instruction when the students lose their train of though.(Barker, 2005,2009)
Let us move from the conventional instruction in a classroom to electronic education. Electronic
education requires the teachers prepare all study materials for the students and organizational
guidelines in order to minimize student's discomfort when following the course. Of course there is the
possibility of consultations, either in person or by means of communication technologies (telephone,
e-mail, chat, etc.) If such a kind of instruction is supported by information and communication
technologies (ICT), we are talking about eLearning, managed by a Learning Management System
(LMS). Even in such a case the learning process is not adapted to the specificity of each individual.
The curriculum is presented to students in an integral coherent form, with the course walk through
exactly planned. However, not every student may find such prepared procedures convenient.
When we upgrade eLearning to the next quality level, we get to the personalized eLearning. Such an
environment is predestined to consider each student’s individuality.(Brusilovský, 2001)
By personalised eLearning we mean an automated way of teaching which adapts to different qualities
of students which are characteristic for their learning styles.
In case students learn without a direct contact with a teacher, they usually use textbooks. A good
textbook can be regarded as another form of “a teacher”. Its author has put in it his optimal
instructional approach, scope and detail of the information presented. It is known that the classic
textbooks supplement direct interpretation given by a teacher, whereas textbooks for independent
learning, so-called “distance education”, are supposed to substitute not only an interpretation of a new
topic itself, but also the communication with the teacher, practicing and reviewing of the content, etc.
To create an adaptive eLearning environment that will ensure a personalized (individual) approach for
each student, we need to provide study materials processed in different variants.
The system we have designed may be illustrated by the following schema (see Figure 1: Adaptive
eLearning system schema).(Šarmanová, 2009)
145

Figure 1: Adaptive eLearning system schema
2. Adaptive eLearning system
2.1 Student module
Blanka Czeczotková et al.
The basic presumption for the proper functioning of the adaptive system is a student who has been
appropriately and properly tested. To create a tuple of his basic characteristics, each student must be
a subject of continuous testing at several levels.
The most important characteristic is the student's learning style. Learning style consists of a student´s
individual features and characteristics, bearing a significant impact on the learning process. This set
of characteristics is obtained before the study commenced, using an appropriate questionnaire.
Further testing is carried out during the study and focuses mainly on the level of his current
knowledge. A final set of characteristics is obtained by a long-term monitoring of the student´s study
activities. This set of characteristics may contribute not only as a feedback during the current learning,
but also can be used for an adjustment of the characteristics of the student, or to monitor his
development.(Dunn, 2004; Entwistle, 1996; Gregorc, 1979).
The choice of suitable student characteristics that we will observe was carried out on the basis of an
initial analysis of several well-known and verified studies that focused on the student learning styles.
We have selected such characteristics that can be taken into account when creating an eLearning
environment. (Kostolányová, 2010a, 2010b). The analysis of the forecasting of the characteristics of
students is described in more detail (Takács, 2009 ). The student analysis was conducted at 400 high
schools and universities. Those are the following:
Sensual perception: visual - auditive - kinaesthetic - verbal;
Social aspects: likes to work alone - in a pair - in a group;
Affective aspects: motivation to study - internal and external;
Learning tactics:
Systematic approach : studies and solves problems systematically, according to the instructions,
needs regulations/ heuristic style (requires freedom, solves problems creatively);
Method: prefers theoretical derivation / experimentation;
Learning approach: perception of details / holistic;
Learning tactics: deep - strategic – surface;
Self-regulation: high, the student is able to organize his own learning low, needs teacher’s
guidance.
2.2 Author module
A possible approach to create the variable supports may be when the author creates several books to
teach the same topic, applying, however, each time a different teaching style. Such a method is not
very suitable. First, the author with his own distinctive style of teaching may find it difficult to use a
146

Blanka Czeczotková et al.
number of quite different learning styles. The second reason is the fact that there are too many
students with various learning styles.
2.3 Three bases for the creation of adaptive learning supports
When designing the methodology for creating adaptive learning materials, we have based our
approach on general principles of teaching and learning and teaching methodology of distance
education textbooks.
The principles were adapted from J. A. Komenský,(Lánský, 1997) the events of instruction by Gagne
(Gagné, 1975) and the Bloom's (Schaink, 2002) taxonomy of learning objectives. As few other works
by renowned scholars werw also referred to.
J. A. Komenský (Čapková, 1992) defined a number of pedagogical principles in his work that need to
be applied so that the process of teaching is carried out successfully and efficiently. Out of these
principles, we have selected the following five that are considered to be the most important: the
principle of sense-impression, awareness, steady and systematic approach, adequacy and durability
(see Table 1: Application of the principles by J. A. Komensky when creating an adaptable textbook).
Table 1: Application of the principles by J. A. Komensky when creating an adaptable textbook
principle way of application in an adaptable textbook
senseimpression
Using multimedia components and objects
awareness Different ways of motivating, demonstrations of practical application of the learned matter
steady and
systematic
approach
Linking to the student’s prior knowledge, sequential and systematic learning
adequacy Content and extent of the curriculum according to the current levels of knowledge and
competency
durability Goals determination according to the study programme
The instruction generally consists of learning (the activity of the student) and teaching (the activity of
the teacher), and is carried out within an educational environment (in this case an electronic
environment).
Let us summarize the facts that are known to us: we know the student’s learning style (tested before
the instruction itself).(Šarmanová, 2010; Takács, 2010) The student will be tested during the course of
instruction (feedback of the presented content and its comprehension) and generally we are trying to
create a user friendly environment (for the student not to feel helpless and isolated). The learning
environment´s ability to support a democratic way of teaching is still to be resolved. The democratic
climate gives support to the independency development, self-control, planning and responsibility. All
of these features are welcome and important in an electronic environment. Changing different
teaching methods according to the student’s present mental state and his preferred learning style
comes as an option. This area of adaptive environment design will be handled by adaptive algorithms,
designed through cooperation of teams of psychologists, pedagogues and informatics.
A necessary condition for the implementation of these algorithms is to structure the study material into
small parts. They will be arranged and offered to the student based on the algorithms in concorde with
his preferred learning style.
Teaching methods consist of a sequence of elementary steps of instruction. In the process of
instruction, there is a certain order in the sense of procedure: the beginning of instruction, lecturing,
exercising, testing and ending of the process. To be able to apply this principle in the field of
adaptable teaching, we based our efforts on the distribution of the learning process according to R.
Gagné. On the basis of this principle, we have assigned to each teaching activity a subset of adaptive
support - so called layers. The following table shows the distribution (see Table 2: Gagné’s theory
applied on an adaptable study support).
For the designed methodology to be as general and as universal as possible, we have also included
the theory of Bloom's taxonomy of learning objectives. According to this theory, in adaptive learning
materials there are learning objectives defined with an emphasis on the cognitive domain of
147

Blanka Czeczotková et al.
knowledge with regard to curriculum level of detail and with regard to the particular subject. For
example, the content of a course teaching office applications will be different for potential
administrative staff from the one for analysts or programmers. Therefore, the objectives are
formulated on the basis of the character of the taught content for 6 categories – to remember,
understand, apply, analyse, evaluate, and create.
Table 2: Gagné’s theory applied on an adaptable study support (Kostolányová, 2011)
Event (according to the Gagné’s theory) Layer of the framework in an adaptive support
Gain attention Motivational layer
Inform learners of objectives Goals layer
Stimulate recall of prior learning Entry knowledge testing, eventually demands of the framework
Present the content Theoretical layer
Provide „learning guidance“ Semantic layer
Elicit performance (practice) Practical examples
Provide feedback Auto testing – a layer of questions and tasks
Assess performance Test results, key to unsolved tasks
Enhance retention and transfer to the job Fixation layer
When considering the features of teaching adaptability, we based our efforts on the methodology of
creating distance learning materials. This is where the idea of dividing the study material into
individual, smaller units comes from. The curriculum of the subject was divided into chapters and
subchapters. Within subchapters, we call the thematically coherent elementary parts frameworks.
A framework is the basic educational unit, which explains one sub-theme. And it is the framework we
focus on most in structuring an adaptable textbook and to which more variants of instruction will be
designed.
Let us sum up the basic information on the author module. When designing an adaptive textbook, we
build on the methodology used to create textbooks for distance education. The study material is
divided into sub-chapters, called frameworks, and these are divided into layers representing particular
instruction phases. As the most obvious characteristics of the learning style we have accepted the
type of sense perception, and another quality is the depth of understanding. We chose two basic
criteria for the creation of different variants of frameworks – the form of instruction for the student's
sensory perception type and depth for the level of curriculum detail. There will be four sensory
variants: one with a high amount of text (for the verbal type of student), with many pictures, graphs,
tables, animations (for the visual type), with spoken word, audio recordings, communication activities,
discussions (for the auditive type) or creative tasks, constructions, etc. (for the kinaesthetic type).
The depth of instruction will be represented by three levels:
Basic depth (which is used as the primary) contains the entire basic curriculum, the student is
required master.
Detailed depth for students who would not understand the curriculum presented in basic depth, a
modified version (more detailed, from a different angle) will be ready.
Extended depth for students with high motivation to the given topic, the variant will be enriched
with matters of interest, peculiarities of the topic.
3. Linking student´s module and the author´s module
Now there is a question to be answered which teaching style best suits to each student. To be more
specific which frameworks (sensory perception types of the student) and with layers (enhance,
standard, detailed) will be the most suitable for the particular student. Let us have a look at what
148

Blanka Czeczotková et al.
reputable pedagogues say about teaching styles. Different pedagogues provide different definitions
specifying the term “teaching”.
J. Mares in his book Students Learning styles inclines to a definition by V. Kulic, who defines learning
as a process during which a man changes his complex of knowledge of natural and human
environment, changes his forms of behaviour and procedures, characteristics of his personality and
self-image. A man changes his relationships with people around him and society he lives in – and all
of these towards development and greater efficiency. The fore mentioned changes are made mainly
on the basis of experience, i.e. the results of previous activities, which are transformed into systems
of known information - the knowledge. These experiences can be individual or assumed and adopted
from the social experience.(Mareš, 1998)
Teaching methods can be viewed from different perspectives - from a psychological perspective,
procedural perspective, learning phase perspective, logical and organizational aspects perspectives.
Currently the so-called activation methods have become popular, which include - discussions,
learning games, the project method, the experiential learning, and others. Teaching methods are also
divided according to the nature of the information sources. This division will be further dealt with.
Among the best known teachers who have dealt with the division of teaching methods, according to
the nature of the information sources were I. J. Lerner, Z. Pesek, J. Manak, D. O. Lordkipanidze, S.
Vrana, E. Stracar M. Korinek, M. A. Danilov, B. P. Jesipov, O. Kadner, I. A. Kairov, A. Valisova, J.
Valenta, E. J. Golant ...
Some of the pedagogues were inspired by their colleagues and based their work on the already
existing types of teaching methods. In 1938 D. O. Lordkipanidze divided teaching methods according
to the nature of the information sources to-methods of verbal communication (monologic and dialogic
methods), methods of work (practical work, lab work, ideal tasks of exploratory character) and
methods of observation (demonstration, image and film observation, excursional demonstrations,
illustrations, etc.). In 1959 S. Vrána divided teaching methods according to the nature of the
information sources to: verbal methods (lecture, meeting, working with a book, written assignments,
seminar method, and interview), visual methods (demonstration, observation) and practical methods
(laboratory work, technical works, practical works, working with text, etc.). And so we can continue
with a long list of the same teaching methods divided according to the nature of the information
sources, until we get to J. Manak. J. Manak in 1967 divided the teaching methods into various
aspects. The didactic division includes verbal methods (monologic, dialogic, methods of written work,
methods of working with a textbook, a book, a text, a material), illustrative and demonstrative methods
(observation of objects and phenomena, demonstrations, static images demonstration) and practical
methods (physical training and working skills training, laboratory student activities, students, working
activities, graphic and crafts activities).After mapping out the work of most pedagogues, we have
decided to base our theory on J. Manak´s division of teaching methods.
4. Rules for assigning a teaching style to the student individuality
To keep a good orientation among the amounts of described views, classifications, types of teachers,
and to be able to work with those teaching methods and learning styles, we have designed a table as
a suitable working tool (see Figure 2: Linkage between the teacher and student characteristics and
teaching methods). There are 3 columns containing teaching methods, student characteristics and
characteristic features of teaching styles. Through an analysis of pedagogical principles and student
characteristics, we have designed the assignment of appropriate teaching methods that best suit the
various types of students. Figure 2 shows only a partial result of the whole analysis outcome.
The result of this analysis is a set of rules of a following structure:
If the student possesses (learning) characteristic ABC,
then it is appropriate to use teaching methods XYZ and
learning management type UVW.
This set of rules will be used by the so-called virtual teacher, which is a learning management
program, integrated into the adaptive LMS. The virtual teacher first assigns a teaching style
appropriate to the student learning style, and then will follow this style and adapt the interaction with a
student within the eLearning instruction.
149

Blanka Czeczotková et al.
Figure 2: Linkage between the teacher and student characteristics and teaching methods
150

5. Conclusion
Blanka Czeczotková et al.
This paper deals with an analysis of existing teaching methods and teaching styles and their optimal
choice for students with various learning styles.
After the mapping of all teaching methods, we eliminated such teaching methods that are identical or
very similar. We also unified analogical characteristics of teaching styles. Through another research
student characteristics were assigned with appropriate forms of teaching methods and teaching
styles, in order to be adaptable to each individual student while at the same time being usable in
eLearning.
The result of our efforts is a series of rules, assigning an optimal teaching style to a particular student.
These rules will be the basis for the construction of several algorithms known as the virtual teacher, a
learning management program, capable of adapting instruction to each student.
Acknowledgments
This publication is supported by the ESF project OP VK CZ.1.07/2.3.00/09.0019 titled Adaptive
individualization of study through eLearning.
References
Barker, P. (2005), Knowledge management for eLearning , Innovations in Education and Teaching International.
Vol. 42, No. 2, pp. 111-121.
Barker, P., Schaik, P. (2009) Electronic Performance Support: Using Digital Technology to Enhance Human
Ability, London, Gower Publishing.
Blinov, V. M. (1976), Effektivnost´ obučenija. Moskva: Pedagogika, 1976.
Brusilovsky, P., Kommers, P., Streitz, P. A. M., Berlin, N. (1996), Multimedia, Hypermedia and Virtual Reality;
Models, Systems and Applications, Springer Verlag.
Brusilovsky, P. (2001) Adaptive hypermedia. User Modeling and User Adapted Interaction, Ten Year Anniversary
Issue (Alfred Kobsa, ed.) 11 (1/2), 87-110
Dostál, A. M. (1959) Vyučovací metody. Praha: Krajský pedagogický ústav, 1959.
Dunn, K., Dunn, R., Price, G.E. (2004), Dotazník stylů učení, Praha, Institut pedagogicko-psychologickeho
poradenstvi.
Entwistle, N., Tait, H. (1996) Approaches and Study Skills Inventory for Students, Centre for Research on
Learning and Instruction, University of Edinburgh.
Gagné, R. Podmínky učení. Praha: SPN, 1975.
Gardner, H. (1999), Dimenze myšlení: teorie rozmanitých inteligencí, Portál, Praha.
Gregorc, (1979) A. Learning/Teachins Styles: Their Nature and Effects, Reston, Christian Education Journal, Vol
4, No.1, pp 62.
Kolb, D. A. (1984) Experiental learning: Experience as the source of learning and development, Engelwood Cliffs,
NJ, Prentice Hall.
Čapková, D. Učiteľ učiteľov: J.A. Komenský a učiteľská profesia. Bratislava: SPN, 1992.
Kostolányová, K. (2011). Design of study materials structure for adaptive instruction. ICTE2011;Ostrava.
Kostolányová, K., Šarmanová, J., Takács, O. (2010a). Intelligent Tutoring in ELearning, ECEL2010, Porto, pp.
794-796.
Kostolányová, K., Šarmanová, J., Takács, O. (2010b), Adaptive individualized education in eLearning.
CBLIS2010, Warsaw, Poland, pp. 119-125.
Kucharev, N. V. (1979) , Effektivnost´ obučenija i vospitanija. Minsk: Izdatel´stvo BGU, 1979.
Kulič, V. (1992), Psychologie řízeného učení. Praha: Academia, 1992. ISBN 80-200-0447-5.
Lánský, M. (1997), Didaktická technologie ve světě idejí J. A. Komenského. In Technologické otázky vzdělávání.
Praha: KAVA-PECH, 1997. ISBN 80-85853-28-0.
Lerner, I. J. (1986), Didaktické základy metod výuky. Praha: Státní pedagogické nakladatelství, 1986.
Maňák, J. (2003), Výukové metody. Brno: Paido, 2003. ISBN 80-7315-039-5.
Mareš, J. (1998), Styly učení žáků a studentů. Praha: Portál, 1998. ISBN 80-7178-246-7 pp. 47
Mojžíšek, L. (1988), Vyučovací metody. Praha: Státní pedagogické nakladatelství, 1988.
Schaik, P., Pearson, R., Barker, P., (2002), “Designing Electronic Performance Support Systems to Support
Learning”, Innovations in Education and Teaching International, Vol.39, No.4, p. 289-306.
Šarmanová, J., Kostolányová, K., Takács, O. (2010), The use of adaptive individualized eLearning at teaching,
IMSCI 2010, Florida, USA, pp. 147-152.
Takács, O., Kostolányová, K., Šarmanová, J. (2009), Results of analysis of learning styles. ICTE2009, Ostrava.
pp. 205-210.
Takács, O., Kostolányová, K., Šarmanová, J. (2010), Adaptable educational supports. ICTE 2010, Ostrava, pp.
117-121.
151

Changing Academics, Changing Curriculum: How
Technology Enhanced Curriculum Design can Deliver
Strategic Change
Christine Davies
University of Glamorgan, Pontypridd, UK
cpdavies@glam.ac.uk
Abstract: This paper describes a case study resulting from a JISC Building Capacity Project at the University of
Glamorgan. The case study indicates that curriculum design can be used a vehicle to engage staff with
technology for learning and teaching, and provide the means to initiate sustainable staff development. At the start
of the project, a research survey revealed that whilst academic staff were reasonably proficient in the use of the
institutional VLE, they were unaware of many of the tools and resources that could be useful within their subject
disciplines. Some staff identified training needs, and indicated a preference for one-to-one support over group
training. The survey also revealed evidence that some staff clear use technology in innovative and effective ways
within their courses. In response to the research data the project team took a three-pronged approach to building
the capacity of academic staff to use technology in their learning and teaching in a sustainable way: Remove the
barrier of lack of awareness of useful technologies – this was undertaken through in-faculty drop-in sessions to
demonstrate hardware and software and answer queries; and seminars and blogs to provide more detail of the
ways in which the vast range of available technologies could fit into subject teaching and research activity
Encourage staff already competent in using technology – by organizing regular ‘self-help’ groups to allow sharing
of new ideas and good practice Reach staff with limited engagement with technology – by arranging one-to-one
interviews using appreciative inquiry approaches to explore their subject learning and teaching traditions and
identify the ways in which technology enabled tools and resources could be integrated Emphasis was placed on
the curriculum rather than on the technology. Tools and resources –particularly Open Educational Resources
(OER) - were discussed in the context of improved curriculum design and established tangible benefits (JISC
Infonet, 2008). Learner benefits were also a focus, with emphasis on the key role of the tutor in guiding learners
towards the effective use of technology in their learning (JISC, 2009). This case reveals that an institutional
approach to curriculum design can be implemented through sustainable approaches at subject level. Thus a top
down mandate for change can be implemented through a bottom-up engagement with practitioners in the
language and approaches of their own discipline. Such an approach moves from central support into the common
practice with departmental and course team debate.
Keywords: technology-enhanced learning; staff development; curriculum design
1. Introduction
Over the past decade, The University of Glamorgan has followed a path from Blended Learning to
Technology-Enhanced Learning, via several key projects including the HEA’s GWELLA (University of
Glamorgan, 2011) and Change Academy (University of Glamorgan, 2010) projects which focused
particularly on assessment and feedback methods. These initiatives, steered by the University’s
Centre for Excellence in Learning and Teaching (CELT), have had a significantly positive effect on the
institution, which serves an economically challenged geographical area and a student community that
includes several minority and protected–status groups. These benefits linked to predicted
expectations (Garrison and Kanuka, 2004) to some extent, but it was felt the institution needed to
reflect on the impact resulting from previous interventions (Conole, 2006). It was also believed that
barriers still existed in some cases that were preventing academic staff making full use of the benefits
that technology can bring, and in some cases inhibiting them from making their first tentative steps
across the digital ‘chasm’ (Rogers, 1962). The JISC Building Capacity Project at the University
provided the resource to investigate these barriers more fully and to identify ways of removing them.
The approach was strategic in that it covered the whole organization and was directed by the
institutional Teaching and Learning Strategy (University of Glamorgan, 2006); however, it was also
‘bottom up’ in the sense that there was significant direct contact with academic in the context of their
teaching curricula. The project has constituted a key advance in an on-going programme to make
best use of technology for the benefit of learners.
2. Methods
This paper outlines the methodologies adopted during the project in the form of a case study to
enable the key project milestones to be viewed in context and to provide an ethnographic
development of the project results and its outputs. This phenomenological research began by
considering the barriers to engagement with technology for learning, teaching and assessment in
152

Christine Davies
more detail, and subsequently ways of removing these barriers were devised that were tailored to the
constituencies of academic staff that we observed.
2.1 Identifying barriers
Butler and Sellbom (2002) observed that barriers to engagement with educational technology
included inadequate or unreliable hardware; a belief that technology cannot make any difference; and
lack of time to learn new techniques. We felt that these might also apply to the University of
Glamorgan, but we believed it was important to ask more searching questions about the technologies
in use and how they were viewed in the context of University policies and procedures, and individuals’
teaching and/or research roles. A survey was therefore developed to ask about academics’ existing
experience and perceptions of technology for learning, teaching and assessment which was sent out
to academic staff in several faculties. We also had the intention of utilizing survey responses (with
permission) as a pool of contact information to allow follow up of individuals for more detailed
investigation.
The survey response rate was disappointing at approximately 10%, but nonetheless included
respondents from a wide range of disciplines and roles showing varying levels of experience of the
use of technology. Responses indicated that technology was generally deemed to be important in
education, but the questions that asked for specific examples of useful technologies were often left
blank, as were suggestions for technologies that were perceived as desirable. Further answers
showed that there was a good understanding of the institutional Virtual Learning Environment (VLE)
and its capabilities, though a limited awareness of other learning technologies, especially ‘Web 2.0’
and social media. There was also little experience of the many high-quality online Open Educational
Resources (OER) available in virtually all subjects.
Answers relating to training needs indicated that approximately 50% of respondents identified a need
for training, and also showed a preference for one-to-one approaches. A question specifically asking
for reasons for non-engagement was largely left unanswered, but follow-up interviews discovered that
several of the anticipated barriers were involved, particularly lack of time, and issues such as
imminent retirement, temporary posts, and worries about copyright were also raised. Another barrier
identified was the noticeable lack of awareness of what technology could achieve.
2.2 Raising awareness
Given the results of the survey, we felt it was important to embark on a campaign of raising
awareness, highlighting a wide variety of technologies that could be incorporated into the curriculum.
This was implemented in several ways:
2.2.1 Promotional campaign
A colourful postcard was designed in-house to make lecturers and academics more aware of
technologies that could benefit their students, and this was distributed to all academic staff. The card
also notified them about forthcoming in-Faculty drop-in sessions
2.2.2 In-faculty drop-in sessions
Members of the Technology-Enhanced Learning (TEL) team at CELT visited each Faculty in turn for a
three-hour period over lunchtime to enable academic staff to approach them for ideas, information
and help relevant to their subject teaching. Though the proportion of the staff population who
interacted at these events was quite low, in each Faculty several staff discussed issues affecting their
courses, and subsequently further interviews and projects were arranged in several cases
2.2.3 Accessible OER
The survey indicated that few lecturers were making use of the many high-quality OER that are
current available online. A key initiative within the JISC Building Capacity Project was therefore to
produce templates for use on the institutional VLE to provide important information about key
resources such as iTunesU and Openlearn to make them more accessible for academic staff.
153

Christine Davies
2.2.4 Seminars
CELT regularly hosts seminars that are advertised to the academic community and well-attended by
both external and internal staff. CELT seminars were used to disseminate information about the JISC
Building Capacity Project, and the wide range of technologies relevant to teaching, ranging from
interactive whiteboards to QR codes, and including OER. An interactive session during one seminar
was also an useful opportunity to find out about technologies that the audience deemed useful and
hence to discover some important pockets of innovative practice within the institution.
2.2.5 Blogs and Twitter
Information about the project and its key milestones was also disseminated via a series of blogs on
the CELT website, and appropriately-times tweets.
2.3 Staff already using technology
A ‘Blended Learning Champions’ network was already in existence within the University, and as a
result of the project many other individuals were identified as using technology in innovative and
effective ways. Examples include the use of mobile learning resources for languages; screen-capture
for feedback in science; audio-feedback in sports science. A groups of these ‘leaders’ in the use of
technology were invited to meet and share their good practice, with the TEL team chairing and
facilitating. The first meeting proved to be a great success, with new relationships forged and new
ideas shared. Termly meetings of the group continue, and are effectively ‘self-help’ forums for anyone
who already has some experience and interest in educational technology and wishes to explore
further. The TEL team is available to provide support if necessary.
2.4 Reaching staff with limited engagement in technology
Survey information helped to inform a staff development plan for less-engaged staff that closely
aligned to subject teaching and hence would directly inform curriculum re-design. Certain individuals
were particularly targeted on the basis that they might be ‘key-holders’ in their subject areas or
Faculties (JISC Infonet, 2006). The staff development plan featured a highly staff-focused (Mainka,
2007) approach comprising one-to-one interviews lasting approximately one hour. The interviews
were informal in context and interviewees were asked to suggest where and when they wished to
meet. Appreciative inquiry techniques (Cooperinder et al, 2008) were used to explore lecturers’
teaching methodologies and ascertain any personal barriers to engagement with technology, and care
was taken to respect their expertise both in the context of their subject and their learners. There was
also a discursive element to the interviews following the example of Laurillard’s work (2002), with the
interviewer making suggestions about technologies that would meet specific needs. Technologies
were selected on the basis of their previous tangible benefits as outlined by JISC Infonet publications,
for example (JISC Infonet 2009), and included: online mind-mapping as a tool for planning and
collaboration; blogs within Blackboard for reflection on practice; text-to-speech software to assist
literacy and extend accessibility. The interviewer also made reference to the important role the
teacher plays in guiding learners in their use of technology for their learning (JISC, 2009).
After each interview, an eMail was sent to the interviewee with more information about the
technologies discussed and a request for feedback after one or more had been tried out. As a result
of these activities, a project has been initiated to use YouTube integrated into the institutional VLE as
evidence for a course in the performing arts, and introductory modules in Business subjects will adopt
mind-mapping as a key tool for revision and planning. Many of the comments and ideas resulting from
these interviews are being further explored as part of an on-going action plan for the TEL team.
3. Conclusion
This project was ambitious in its scope and aimed to build capacity for the use of technology for
learning and teaching. It could not be said to have touched all academic staff, but it has certainly
encompassed all Faculties and has involved individuals from almost all subject areas. This
institutional approach to curriculum design implemented at a localized, subject level, will help to
sustain the key initiatives of the project, and will help ensure that a wide range of technologies
continue to be discussed at departmental and course team level.
Throughout the project, emphasis was placed on the curriculum rather than on technology, and tools
were discussed in the context of curriculum design and the needs of specific groups of learners and
154

Christine Davies
teachers. The important role of academic staff in directing the use of technology was emphasized, for
teaching staff with little personal experience of technology often assume that they cannot, or should
not, get involved in educational technology. A dialogic, personalized, approach was also appreciated
by academic staff who are accustomed to being treated as experts and sometimes find it
uncomfortable to be in the position of novices. As a result, technologies that had been previously
unknown are being adopted for the benefit of lecturers and learners. This approach does, however,
require a significant input of time, and additionally, resource needs to be allocated for training needs
that may ensue from the introduction of new technologies.
The project has made a significant impact on raising awareness of the merits of technology and has
helped to steward the University in its effective use of technology (Wenger at al, 2011). It has
enhanced the curriculum, changed academic practice, and helped deliver an important and
sustainable strategic change.
References
Apple (2011) iTunesU. Available at: http://www.apple.com/uk/education/itunes-u (Accessed: 21 April 2011)
Butler D, Sellbom M (2002) Barriers to adopting technology for teaching and learning. Educause Quarterly 2, p22
Conole, G. (2006), ‘What impact are technologies having and how are they changing practice?’, I. McNay (ed),
Beyond Mass Higher Education: Building on Experience, Open University Press/ McGraw-Hill Education,
81-95.
Cooperider, DL, Whitney, D, Stavros, JM (2008) Essentials of Appreciative Inquiry. Brunswick OH: Crown
Custom Publishing Inc
Garrison,R, and Kanuka, H (2004) Blended learning: Uncovering its transformative potential in higher education.
Internet and HE, 7 (2) p 95-105.
JISC (2009) Managing Curriculum Change. Available at:
http://www.jisc.ac.uk/publications/programmerelated/2009/managingcurriculumchange.aspx jisc 09
(Accessed: 29 January 2011)
JISC (2009) The University of Glamorgan Smoothies: the recipe for effective blended learning. Available at:
http://tinyurl.com/6z7ogbx ( Access: 12 May 2011)
JISC (2009) Responding to Learners. Available at:
http://www.jisc.ac.uk/publications/programmerelated/2009/respondingtolearners.aspx (Accessed: 2
February 2011)
JISC Infonet (2006) Change Management Infokit. Available at: http://www.jiscinfonet.ac.uk/infokits/changemanagement/index_html
(Accessed: 19 January 2011)
JISC Infonet (2008) Exploring tangible benefits of e-learning: does investment yield interest? Available at:
http://www.jiscinfonet.ac.uk/publications/info/tangible-benefits-publication (Accessed: 18 January 2011)
Laurillard, D (2002) Rethinking University Teaching: A Conversational Framework for the Effective Use of
Learning Technologies. 2nd edition. London: Routledge Falmer
Mainka C (2007) Putting staff first in staff development for the effective use of technology in teaching. Br. J. Ed.
Tech. 38 (1), p138
Open University (2011): Openlearn LeaningSpace. Available at: http://openlearn.open.ac.uk (Accessed: 20
February 2011)
Rogers, E M (1962). Diffusion of Innovations. Glencoe: Free Press.
University of Glamorgan (2006). Learning, Teaching and Assessment Strategy, 2007-2012. Available at:
http://celt.glam.ac.uk/sites/default/files/LTA%20Strategy%202007-2012.pdf (Accessed: 12 March 2011)
University of Glamorgan (2011) GWELLA project: Enhancing Learning & Teaching through Technology for
Higher Education in Wales. Available at: http://learning.weblog.glam.ac.uk/2011/2/2/gwella-reportenhancing-learning-teaching-through-technology-for-higher-education-in-wales
(Accessed: 22 January
2011)
Wenger E, White N and Smith JD (2011) Digital Habitats: Stewarding technology for communities. Available at :
http://technologyforcommunities.com (Accessed 20 May 2011)
155

Web Conferencing for us, by us and About us – the Leeds
Met Elluminate User Group
Mark de Groot, Gill Harrison and Rob Shaw
Leeds Metropolitan University, Leeds, UK
m.degroot@leedsmet.ac.uk
g.harrison@leedsmet.ac.uk
r.shaw@leedsmet.ac.uk
Abstract: How do you implement a strategy for institutional adoption of web conferencing? What are the
potential benefits for students, tutors, support and administrative staff? What strategies can maximise those
benefits? How do you provide the relevant staff development? At Leeds Metropolitan University answers have
been sought by bringing together current users - meeting in web conferences naturally - to shape, to share and to
develop their ideas. This case study first describes the environment in which it makes sense for higher education
institutions to fast track the implementation of appropriate aspects of web conferencing. It next describes the
establishment and remit of the University’s web conferencing group. It analyses web conferencing benefits and
implementation recommendations as presented in the group and in core related activities. Finally it identifies both
the quick wins on offer and the challenges to be met from the institutional, student, and staff points of view. This
paper explores what is possible when the principles that underlie technology supported student centred learning
are applied to technology supported user centred staff development. It specifically addresses the suggested
conference theme: This
promising and productive experiment in both user driven staff development and, more generally, the
dissemination of ideas and sharing of good practice will be of interest to all responsible for shaping and
supporting their institutional web conferencing policies; providing insights into: The benefits and quick wins for an
institution prepared to adopt web conferencing The challenges and ways in which they can be addressed
Effective staff development strategies in a changing and economically restrained environment
Keywords: web conferencing, staff development, Elluminate, online learning, change management
1. Context and background
1.1 Changes in higher education, nationally and internationally
Increased flexibility of provision and efficiency of business processes will be essential in the response
of Higher Education Institutions (HEIs) both to the Browne Review (Browne 2010) and the
Comprehensive Spending Review (HM Treasury 2010). The most recent government white paper
(BIS 2011) makes clear the government’s intention to promote student choice as a key way to drive
forward innovative forms of delivery.
This approach is entirely consistent with an earlier report, Higher Education in a Web 2.0 World, from
the UK’s Joint Information Systems Committee (JISC 2009). In challenging the inertia of the
established system the report provides an important analysis of the need for and the means by which
HEIs can adapt to meet the needs and expectations of students who are increasingly comfortable with
technology. The report states that the process of engagement, especially with those technologies that
support collaborative networking, develops a skill set that matches both to views of 21 st century
learning skills and to those of 21 st century employability skills –collaboration, creativity, leadership and
proficiency with technology.
Pressures of student numbers, increased competition and constrained resources are not unique to
the UK. Rapid advances in educational technologies, particularly internet based technologies, are
blurring the boundaries between institutions and between countries. Predicting the future virtualization
of international higher education Sohail Inayatullah suggests that as costs go down, over the longer
term current distinctions between virtual and real will disappear, with an important shift taking place
from merely more technology in the class room to actual, digital pedagogy (Inayatullah 2011).
1.2 Web conferencing in higher education
Web conferencing is increasingly available across the Higher Education sector and beyond (Murphy
and Ciszewska-Carr 2007; de Freitas and Neumann 2009).
156

Mark de Groot et al.
Evidence from the sector however indicates that the variables impacting successful synchronous
learning (Park and Bonk, 2007, de Freitas and Neumann, 2009) remain relatively unexplored. In their
discussion of pedagogic models appropriate to synchronous conferencing, especially communities of
enquiry, task design and multimodal theories, De Freitas and Neumann (2009) can be said to
establish a framework, in the dedicated context of web conferencing, for the future ‘digital pedagogy’
envisaged by Inayatullah (2011).
However, despite increased availability of practical and technical guidance in the use of web
conferencing (Chatterton, 2011), the already significant investment made in web conferencing
remains largely in the hands of specialists.
1.3 Approaches to staff development
What this suggests is that in moving beyond awareness raising in exploiting our use of virtual
conferencing spaces we need also to find more efficient ways to develop and to capitalise on the real
knowledge, experience and insights of staff and students who involve themselves with web
conferencing. The core proposal here is that mirroring collaborative climate of Web 2.0 technologies,
web conferencing itself can be the vehicle to provide more appropriate, targeted staff development.
The need for such adjustment is indicated by MacDonald (2006). She argues that effective staff
development must adapt to accommodate the preferences and working habits of both full and part
time tutors. Further, Garrison & Vaughn (2008) are clear about the limitations of “one off” staff
development sessions, however well they may be planned, executed and evaluated. The collegiate
network they identify as fundamental for effective professional development in education needs to be
mirrored by an evolving network of resources and opportunities that is in sympathy with changing
institutional needs, participant constraints and aspirations and pedagogic principles.
Smyth et al (2007) report on the established social constructivist principles that underlie the pedagogy
of their TESEP project. Their learner centred staff development evidences their core belief that the
best way to encourage transformation in teaching practice is to immerse practitioners in the kinds of
learning experiences they would ultimately support for their own students.
This paper reviews the running of a collegiate staff development web conferencing network. It
describes the immersive web conferencing activities of this learner centred staff development forum. It
provides a thematic analysis of web conferencing benefits and recommendations as discussed by the
group. It then tracks related spin off events from the group’s activity. It concludes by summarising the
challenges involved.
2. Web conferencing at Leeds Met
Even as Leeds Met institutional strategies (Leeds Met, 2010) have evolved to reflect both the
demands and constraints of a changing national and international environment certain core priorities
have remained constant. These include:
The continuous review and improvement of teaching and assessment practices across Leeds Met
The development of new, cost-effective and flexible ways of working that make optimal use of our
estate and technology
The empowerment of staff, students and visitors to reduce their environmental impact by
encouraging, and celebrating environmental improvements
In this context funding was sought in early 2008 to assess the possible value of making a common
web conferencing system available to all staff and students. A comprehensive evaluation was
conducted, involving some 45 representative stakeholders drawn from students and from staff
responsible for teaching, administration and technical support. This thorough evaluation process
resulted in a three year site licence for an externally hosted web conferencing system then known as
Elluminate.
All involved in this process agreed that the chosen system could best and most reliably handle all
predicted situations for academic and administrative use. Users can set up as many sessions as they
need, whenever they need them and for as long as they need them. Web conferences can be
accessed through the University’s virtual learning environment or, independently by clicking on a
URL. Equally, if required, there is controlled access to recordings of all sessions. An Elluminate white
157

Mark de Groot,
paper (Elluminate 2010) describes the early, positive engagement of both staff and students at Leeds
Met in this collaborative process.
Since the early adoption of this system, the experience of staff and students at Leeds Met has
provided increasingly compelling evidence that web conferencing offers benefits in the spheres of
flexible online and distance learning. Importantly there are also clear economic and environmental
savings through reductions in travel time and costs for staff attending what are typically shorter and
more focussed meetings.
Figure 1 below illustrates the year on year growth in web conferencing at Leeds Met over this period.
Figure 1: Number of Elluminate sessions 2008 - 2010
As of Jan 2011 there are 163 members of staff who have between them set up / run more than 1500
web conferencing sessions. We currently average about 60 sessions a month.
The graph tracks the use of web conferencing from the processes of awareness raising through to
take up by early adopters. This is healthy progress. Having established the context the remainder of
this paper will describe the dynamics of staff development activity that can help bring web
conferencing into the main stream of University activity. It explores what is possible when the
principles that underlie technology supported student centred learning are applied to technology
supported user centred staff development.
3. Web conferencing user group - overview, aims and operation
When new software has been introduced to University it has proven consistently challenging to
provide sufficient familiarization in a timely and appropriate manner for both individuals and course
teams. This is an issue recognized across the sector and is associated with the difficulties in
establishing flexible mechanisms to provide the follow up local network of support and resources
necessary to put new ideas into practice (Garrison and Vaughn 2008).
Setting up a web conferencing user group would, it was hoped, address some of these difficulties by
making it possible to provide a series of staff development sessions that made minimal demands on
people’s time (all sessions lasted 60 minutes or less), made it possible to catch up should a given
session of interest be missed, and required no travel. There was no compulsion, or expectation to
attend all or any particular session; just the understanding that all information about the work of the
group would be available to all. More importantly the intention was that session content while
coordinated by the three nominal leaders would be discussed, agreed and largely delivered by
members of the network. Importantly this would guarantee immediacy and relevance and flexibility of
content. Sessions would only deal with what was of current interest.
By the end of March 2010 some 70 staff signed up to Leeds Met’s ‘Elluminati’ web conferencing user
group. A request to state which activities they expected to be interested in produced these results:
158

Table 1: Activities
Mark de Groot et al.
Activity Number interested
Monthly email 44
Monthly meeting 35
Promotional events 24
Practical support 23
Leeds Met group 16
Centre of Excellence 15
Licence rationale 14
Impact analysis 12
Other 7
The group agreed to meet online once a month and at the first meeting agreed to convert these
expressed interests into an overall remit in three areas:
Good practice – provide a forum to share, experiment with and promote good practice
Resources – develop accessible, straightforward guidance especially for staff and students new
to web conferencing
Licensing – based on an analysis of current and potential benefits across the university, promote
the group’s consensus view on an appropriate way forward with future web conferencing
licensing.
An initial 12 month time limit was proposed and participation in any or all of these three activities was
entirely voluntary. Overall the intention was to find an effective and practical way to support the 160
registered moderators and promote web conferencing generally.
Meetings proved to be lively. Their general format included warm up activities, reports on progress,
discussions of strategy, experimental activities, and guest speakers demonstrating their own good
practice. Additionally a series of 12 master classes was organised. This was delivered, at times
suitable for UK colleagues, from across the Atlantic by Elluminate experts. The short, 30 minute
sessions covered a range of techniques for engaging, effective presentation online. All sessions were
recorded as were all user group meetings. They are now a lasting resource to help develop the digital
pedagogy (Inayatullah 2011) skills of new converts to web conferencing.
A monthly email newsletter summarized what was happening in the group. A public folder was setup
within the University’s Outlook email system to enable open access to records of all the group’s
activities.
4. Web conferencing user group – activities
These are here considered under the same headings as given for the remit of the group.
4.1 Good practice
The analysis of good practice discussed in the group meetings presented in the table 2 makes clear
the three contexts under consideration: Student meetings, Staff Meetings and Virtual Conferences. It
lists the subjects of web conferences held in each of these categories and then collates the benefits
and recommendations for each category. The consensus in all three categories is that web
conferencing does not replicate the face to face environment. It is necessary for all involved to be
open to different behaviours…and to have the time and space to develop those behaviours.
4.2 Resources
Asked to suggest what would be helpful to support their use of Elluminate 52% of respondents put
readily available resources as their first priority. Figure 2 illustrates the user group’s response to this
perceived need. The links on the right hand column predict the most frequently needed tasks and lead
to step by step instructions to complete the task selected. The examples of good practice are a
growing collection of 3 min (max) recordings describing best practice as first presented to the user
group. The alphabetical list of videos and paper guides allows the creation of reference resources as
159

Mark de Groot,
requested. The target here is to present users first with straightforward guidance on the tasks they
need to do first.
Table 2: Good practice
Good Practice Context Benefits and Recommendations
Student Meetings
Benefits:
Real time communication adds dynamic element to
Running tutorials for MSc Distance Learners
otherwise purely asynchronous experience
Increases tutor availability to students…especially
Small group activities in breakout rooms with placement / dissertation and distance learning
students
Student office hours Web Tour demonstrations Possibility of recorded presentations of speakers who
can’t be present
Working with guest speakers
Recorded sessions valued for revision
Staff Meetings
Setting up subject group meetings
Running course review meetings
Running project support meetings
160
Recommendations:
Expect and prepare for students to have initial access
problems
Especially effective for one to one and small group
meetings
For larger groups if feasible prepare students in a pc
lab beforehand (30 mins orientation)
For distance learners set meeting time at end of
working day
Consider use of video…to confirm identity of student
tutor is talking to
Develop tutor skills to encourage interactivity eg:
Using Chat room for Students to paste URL of what
they have found
Web tour to access online images for discussion.
Practice with exercise in breakout rooms
Challenge staff to be prepared to be available in
these ways…change working practices
Doesn’t replicate face to face…be open to different
behaviours.
Benefits:
Better time management
Better attendance (esp. pt tutors
More focussed / reflective discussion
Better participation
More inclusive (e.g. course administrators).
Save money. Save travel time. Saved meeting time
Clearer agreed action points.
Better note taking …recording available
Green & efficient
Recommendations:
Planning and preparation needed
Expect early technical problems
Encourage peer support.
Consider alternating face to face meetings with web
conferences
Encourage interactivity
Encourage use of Video / Chat
Doesn’t replicate face to face…be open to different
behaviours

Virtual Conferences
Organising an online conference
Supporting guest speakers
Figure 2: Elluminate resources
4.3 Licensing
Mark de Groot et al.
Benefits
Access to recorded sessions following conference
Opportunity for collegiate, reflective discussion pre
and post conference
Savings in conference running costs
Savings in delegate attendance costs
Greener
Opportunity to provide a different, deeper experience
Recommendations:
Support website needed
Manage registrations (consider e.g.
http://eventbrite.com )
Keep sessions short (20 mins)
Prepare video for backup
Speaker practice area needed
Speaker support/guidance needed
Clear timelines needed
Conference practice session needed
Doesn’t replicate face to face…be open to different
behaviours
The initial site licence for Elluminate expired at the end of May 2011. This licence has now been
extended until December 2012 based on the group’s report on the on benefits gained and conditions
under which future potential could be harnessed.
From the beginning of 2012 a sub group will re-evaluate this commitment in the light of the growing
availability of competitive alternatives. What is established however as central to this future decision is
the support and commitment of the core stakeholders as represented by the user group.
5. Web conferencing user group - impact
In this section evidence of the impact of the work done by the user group is provided first by samples
of feedback from students whose tutors are either members of the group or have been supported by
161

Mark de Groot,
members of the group. Further evidence of impact is then provided by ‘spin off’ events supported by
the activities of group members.
5.1 Student feedback
Distance learners have consistently made clear that web conferencing creates for them an active
sense of belonging to their study group, adding a dynamic and personalised element to what would
otherwise necessarily be a purely asynchronous, online experience. Further, they value access to
recordings the flexibility, convenience and discipline inherent in this resource. Evidence here is taken
from two MSc courses offered to distance learners (Construction Law Dispute and Resolution and
Parkinson’s Disease respectively)
It really helps us communicate and get comfortable seeking support from other members of the
group
I’ve found it very difficult to motivate myself to study other modules which haven’t used this
method of teaching
The recordings are brilliant and even when I have attended the live sessions - I have used them
for revision and to go over any parts I missed
This for me offers the best balance of being able to interact whilst still maintaining a lot of the
flexibility of a distance learning course, which given my work commitments is crucial.
I look forward to the sessions - they act as discipline to keep pace with the delivery of the course
and without them I might also feel that I was studying alone.
Exceedingly impressed with the system. .. It avoids issues of having to travel to attend a fixed
location (travel time, commuting and parking etc) which considering that everyone has already
had a long day at work… is a very significant benefit for myself and no doubt others also.
Elluminate gave me the sense that I was not on my own
The live meetings gave me support throughout the module
It’s invaluable because…you feel less isolated
It has to be said that technical problems accessing web conferencing are a significant risk most
especially for students working off campus where neither internet connection nor pc setup can be
predicted or standardised. This feedback finds many echoes but also evidences student willingness to
accept initial problems in favour of a perceived value in the resource.
In principle, the Elluminate sessions are a good idea. Unfortunately I was unable to attend many
of the sessions because of a variety of reasons: a) I could not use the software on my Apple Mac
b) I could not use the software on my Apple Ipad c) I could not use the software on my work
laptop or at work as I do not have the software privileges to install it.) The final option was that I
used my own windows laptop and this is what I used successfully. Unfortunately, when I was
staying away with work I then had problems connecting to the internet in a hotel / anywhere else.
Despite all the above, in better personal circumstances, I would be happy to continue using it in
future.
Student enthusiasm is further (and cheerfully) evidenced from answers written on an Elluminate
conference whiteboard by students on a campus based course gathered in a pc lab who had worked
with an expert speaker working from her home in Somerset.
5.2 Related events and activities
Events and activities that have informed or spun off from the activities of the user group are
characterised by innovative forms of staff development. In all cases their fundamental provision is a
collegiate environment that fosters collective consideration of the pedagogic implications of web
conferencing.
5.2.1 Web conferencing with Elluminate – Faculty of Health and Social Sciences
One of the founder members of the web conferencing group, the learning technologist from the
Faculty of Health and Social Sciences, ran very successful 4 week on line courses in the use of web
conferencing. The courses entailed a weekly one hour synchronous session with planning and
reflective tasks carried out by asynchronous discussion in between. A number of participants went on
162

Mark de Groot et al.
to use web conferencing within their courses and reported positive outcomes. Many participants
stressed the need for an ongoing network of Elluminate users and their experiences have in turn been
fed in to and shared across the University by their involvement in the wider user group.
Figure 3: Student feedback
5.2.2 Let’s talk – Faculty of Arts, Environment and Technology
A member of the group from the Faculty of Arts, environment and Technology secured external
funding to research different ways in which web conferencing might support the work of his
colleagues in the School of the Built Environment. The resource was adopted on 6 courses within the
school with results reported in July 2011 at an online conference, Let’s Talk. Speakers again reported
positive outcomes for both staff and students in using web conferencing to:
Communicate and provide individual support for placement students and students writing
dissertations.
Conduct meetings and exam boards with overseas partners
Deliver inductions, online tutorials and seminars to distance learners
5.2.3 Show and Tell events
Two face to face ‘Show and Tell’ events have been run by members of the group. These have been
as part of our annual Assessment Learning and Teaching conference and at the user group’s
anniversary party. In addition to those outlined in section 5 above, themes covered at these events
included:
Interviewing prospective international postgraduate students
Feedback and assessment of student presentations
The academic coaching of students
Running taster sessions for first timers.
6. Recommendations from the group
Insights from the experiences of those involved in the activities of this Leeds Met user group will be of
interest to all responsible for shaping or supporting their institutional web conferencing policies. The
high payback areas, or quick wins, are here identified along with challenges and suggested strategies
with which these challenges can be addressed.
163

6.1 The quick wins
Mark de Groot,
Students have consistently been alive to the added value afforded by web conferencing. It can put
them in touch with experts in their field, give them more access to their tutors or simply provide a
convenient, collaborative learning and social environment. Distance learners have been particularly
appreciative of the group contact and stronger cohort identity afforded through live conferencing. All
have appreciated the access to revision material provided by recorded sessions.
For tutors the most tangible benefits come with minimum overheads where web conferencing is used
in one to one situations. These include supervision of dissertations and research projects, support for
placement students and tutor contact through scheduled office hours.
Overheads increase as group sizes become larger; nevertheless small groups (5 to 15) of distance
learners have been particularly appreciative of the contact afforded through live conferencing and the
access to revision material provided by recorded sessions.
For administrators and support staff web conferencing has proved to be especially rewarding both
where part time staff are involved in course delivery and where full time staff have needed to travel
between campuses. Benefits have included better attendance, better use of time, more focussed
discussion and agreement on subsequent actions if key points are written and displayed to all during
the meeting.
External projects have also benefited, especially where travel and meeting time has been an explicitly
costed expense.
For the institution the experience of staff and students at Leeds Met provides increasingly cogent
confirmation that web conferencing can contribute to the rethink in the administration, support and
delivery of student learning that is necessitated by increases in student fees from September 2012.
There is clear potential to deliver better value to online and distance learners, better support for
individual students, and reduced travel and environmental costs for those travelling to meet in
University buildings.
6.2 The challenges
Technical challenges need to be expected, recognized and addressed. They may vary from simple
(headsets muted) to more complex (Firewall / Java issues) They are a potential barrier for all new
users but especially for those working off campus. Our licence includes access for all to a 24/7
technical helpline. This has helped distance learners with access problems. It is an especially
valuable resource for students working in different time zones. Student tolerance of technical
difficulties has consistently been greater than that of staff. Possibly the stakes are higher for staff
where issues they cannot always control directly can and have meant cancellation of scheduled
sessions. Guidance generally has been to ensure sufficient early preparation, to encourage practice
sessions and to test access in good time before a session starts.
Challenges relating to pedagogy have tended to arise where the assumption has been made that face
to face approaches will transfer directly to online web conferences. The identified need is to provide
for staff sufficient good experiences and examples of engaging, interactive web conferences.
Web conferencing presents a direct challenge to existing patterns of staff development. Rather than
schedule content driven sessions prepared by support staff consider staff development characterised
as: time limited, task focussed, resource light, technology enabled and user driven. Our experience is
that it makes absolute sense to introduce web conferencing itself as the core means of supporting
such staff development.Software upgrades and company buyouts are ever present and not always
welcome. What is certain however is that change is constant. Staff investing time to become familiar
with particular web conferencing software need to understand and be confident of the benefits, and
inevitability, of major upgrades. The consultation and consensus building that is possible within a user
group goes far in addressing this challenge, empowering users as contributors and advisers on key
decisions such as selection of resources and licensing. The challenge to existing work habits is
perhaps the most far reaching challenge of all. Web conferencing provides a natural, inviting
environment for students schooled and familiar with sophisticated communication technologies. This
needs to be reflected in institutional strategies, especially teaching and estates strategies. Leaders
164

Mark de Groot et al.
need to encourage their staff to challenge established methods and procedures if the needs and
expectations of tomorrow’s students are not to be frustrated (JISC 2009).
7. Conclusion
The web conferencing user group has:
Identified the high payback areas
Taken on board the need for careful induction for new users of web conferencing
Established forward looking models of online staff development and support
Agreed recommendations to respond practically to the future development of an enhanced web
conferencing system.
This group has in summary given practitioners at Leeds Met a collaborative space in which their
individual development needs could be explored and addressed, proving to be a promising and
productive experiment in user driven staff development. It has also, most importantly, proved to be an
influential forum that has helped and will continue to help shape web conferencing practice across the
institution. The dissemination of ideas and the sharing of good practice across the group have proven
effective and valued components of a movement towards institutional change.
References
Browne, J (2010) “Securing a sustainable future for higher education in England”, [online],
http://hereview.independent.gov.uk/hereview/report/ (Accessed 25/3/11)
BIS (2011) “Higher Education: Students at the Heart of the System”, [online],
http://www.bis.gov.uk/news/topstories/2011/Jun/he-white-paper-students-at-the-heart-of-the-system
(Accessed 27/7/11)
Chatterton, P (2011). “Designing for Participant Engagement with Elluminate Live”, [online]
http://www.jisc.ac.uk/elluminateguidance (Accessed 11/11/11)
de Freitas, S. and Neumann, T. (2009) “Pedagogic strategies supporting the use of Synchronous Audiographic
Conferencing: A review of the literature”. British Journal of Educational Technology, Vol 40, No 6, pp 980-
998.
Elluminate (2010). “Leeds Metropolitan University Embraces Technology Enhanced Learning”, Implementing
Elluminate Technology in the United Kingdom,. [online],
http://www.elluminate.com/Resources/White_Papers/White_Paper_request/25/?id=312 [Accessed 15 /6/11)
Garrison D. R. & Vaughn N.D. (2008) Blended learning in Higher Education. Framework, Principles, and
Guidelines. San Francisco, Jossey-Bass.
HM Treasury (2010) Spending Review. [online], http://www.hm-treasury.gov.uk/spend_index.htm (Accessed
21/4/11)
Inayatullah S (2011) “Open Learning Comes of Age”, Observatory on Borderless Higher Education: Perspectives
on the Future, [online],
http://www.obhe.ac.uk/documents/2011/Reports/Borderless_2011_Perspectives_on_the_Future (Accessed
27/7/11)
JISC (2009) “Higher Education in a Web 2.0 World”, [online],
http://www.jisc.ac.uk/publications/generalpublications/2009/heweb2.aspx#downloads (Accessed 20/1/11)
Leeds Metropolitan University (2010) “Strategic Plan 2010 -1015”, [online]
http://www.leedsmet.ac.uk/strategicplan/Leeds-Metropolitan_Strategic-Plan_2010-2015.pdf (Accessed 7
Sep 2011)
MacDonald J. (2006). Blended Learning and Online Tutoring – A Good Practice Guide. Aldershot, Gower.
Murphy, E. and Ciszewska-Carr, J. 2007. Instructors' experiences of web based synchronous communication
using two way audio and direct messaging. Australasian Journal of Educational Technology. 23(1), pp. 68-
86.
Neumann, T. (2010). “Practice and Pedagogies of Synchronous Online Learning”, ExcelDL – Synchronous
Online Learning: Bridging the Divide. Recording [online],
http://www.exceldl.com/ (Accessed 20 /6/11)
Neumann, T. and Carrington, A. (2007). A mass collaboration approach to e-learning: multiple venue production.
Association for Learning Technology Newsletter, April 2007. [online]
http://newsletter.alt.ac.uk/e_article000783770.cfm (Accessed 26 /5/11)
Park, Y. & Bonk, C. (2007) Synchronous Learning Experiences: Distance and Residential Learners’ Perspectives
in a Blended Graduate Course. Journal of Interactive Online Learning Vol 6, No 3. [online] <
http://www.ncolr.org/jiol/issues/pdf/6.3.6.pdf > (Accessed 5/1/11)
Smyth, K., Comrie, A., Foulis, L., Greatorex, D. & McCran, J. (2007). From bruised to enthused: tackling the
challenges of championing online learning for personal and institutional change. Paper presented at JISC
Innovating e-Learning 2007: Institutional Transformation and Supporting Lifelong Learning. [online] <
http://tiny.cc/4y5aa > (Accessed 10/12/10)
165

Tools for Evaluating Students’ Work in an Interactive
(Open) Virtual Space: Case Study of an eLearning Course
in an International Network of Universities
Jana Dlouhá, Martin Zahradník, Jiří Dlouhý and Andrew Barton
Charles University Environment Center, Charles University, Prague, Czech
Republic
jana.dlouha@czp.cuni.cz
Abstract: This article presents a brief analysis of changes in educational practices associated with the “third role”
of higher education institutions (HEIs) that are occurring not only at an institutional level, but also within the
learning process taking place at lower levels (individual, course, program) and could be effectively combined with
the introduction of eLearning methodologies into teaching that stress the social aspects of learning. Teaching in
the open space provides the opportunity to use active forms of teaching / learning and creates conditions for
social learning. Conceptual and practical shifts would also involve methods of assessment to justify their benefits
and stress certain qualities in higher education (HE). Based on these theoretical considerations, practical
experience with the eLearning course “Multiple Perspectives on Globalization and Sustainable Development”
operated as part of the international Virtual Campus for Sustainable Development (VCSE) eLearning program is
analyzed. The method of teaching was geared toward independent and collaborative student work in a wiki
environment and the development of key competences necessary to understand and be active in the complex
field of sustainability. The authors show how to practically apply the pedagogical principle that educational
objectives, methods, learning environments and assessment procedures must be aligned. For assessment, a
combination of evaluation tools was introduced, such as rubrics evaluated by teachers and questionnaires
completed by the course participants which provided feedback on course outcomes in comparison with its
educational goals. The method of assessment focused on “students’ approaches to learning” is described, and
the possibilities for promoting and evaluating social learning processes that would contribute to the development
of capabilities to communicate across disciplinary and academic boundaries within higher education are
discussed.
Keywords: higher education, eLearning, competences, social learning, Wiki, assessment
1. Introduction – change of practices in higher education
With regard to their potential engagement in society and active participation in its transition toward
sustainability, universities have to develop a third role based on interaction with other societal players,
and mutual interdependence with society (“co-evolution” with its systems) (Ferrer-Balas et al., 2009).
In this article we will argue that the envisaged change might be applied at an individual level and thus
brought into the heart of education - through transformation of the knowledge generation process that
will emphasize its social functions, the development of participation and to involve a “system’s
thinking, joint learning, open communication…” (Lukman et al., 2009). In practice, an expanded set of
educational outcomes is required: besides knowledge (that should reside in an individual’s head),
competences that are developed and are subsequently demonstrated in meaningful practice are
important. An operational definition of competences is based on a combination of skills, knowledge
and attitudes that are appropriate to particular situations (Dlouhá, 2009a, 2009b); competences
relevant to sustainable development (the theme of the course described further on) involve:
“competency for using, shaping, handling and sharing different sets of information and knowledge”,
“competency for dealing with uncertainties and thinking proactively”, and “process-oriented and
structural knowledge” (Burandt and Barth, 2010).
This perspective is associated with constructivist theories, focusing on the learning process as a
distinct field, situational understanding of it (the process is affected by a particular situation), and
research that is no longer focused on student characteristics such as intelligence, but rather on their
individual preferences, perceptions of the learning environment and motivations. This focus brought
about not only considerable methodological gains but also possibilities to influence the learning
process and resulting student achievements through teacher interventions (Entwistle, 2000).
1.1 Learning approaches
Learning processes started to be researched in Britain and Sweden in the 1970s and the studies were
focused on the learning environment factors - these factors are perceived differently by students and
result in a combination of student motives and strategies which was defined as student approaches to
166

Jana Dlouhá et al.
learning: “surface”, “deep” and “achieving learning approach” (Biggs, 1987). Research proved that
student approaches to learning considerably affect learning processes in terms of their efficiency and
have an impact on the final student’s performance. The learning environment factors are explored in
studies that concentrate on the educational requirements, the quality of the teaching and the nature of
the assessment, but also student interests and support, the enthusiasm shown by the instructor and
the opportunity for students to manage their own learning process (Richardson, 2009).
New methodologies for assessment of the learning environment and its psychical (motivations) and
social determinants were subsequently developed and numerous psychometric measurement
techniques based on questionnaires assessing student learning experiences have already been
tested so that they can be applied in practice as evaluation instruments for educational management
purposes. The interest of researchers is mostly determined by the attempt to improve student
performance by influencing in particular the learning environment as an external prerequisite for a
good learning process and consequent success.
1.2 Assessment process
Assessment is a rather complex task both at an institutional or system level, and at the level of
learning processes. Different factors that play a role in these processes should be aligned: not only
educational goals and content, but also tasks and assessment. Evaluators of higher education
teaching agree that the most commonly misaligned factor is assessment and “most assessment
strategies tend to focus on what is easy to measure rather than what is important.” For success in
learning, all different aspects of the learning process are relevant – for example, social interactions
play a role in which students are engaged when they carry out their tasks, such as collaboration and
teamwork, relationships with their tutors, and also learning environment factors. Best teaching
assessment practices in higher education should capture these aspects and therefore focus on
“critical thinking, problem solving, creativity, curiosity, concern for ethical issues” as well as “breadth
and depth of specific knowledge”; they also reflect the “methodologies and standards of evidence
used to create that knowledge”. Attention paid to these factors is a requirement of active,
constructivist pedagogy, but in practice, in most cases assessment is based upon multiple-choice
tests or academic essays (Reeves, 2006).
Higher education has therefore many specific features that make the assessment task even more
complex than at lower educational levels. Rather ambitious evaluation goals include not only
assessment of outcomes, but also reflection of the learning process itself by teachers AND students
themselves; assessment techniques comprise peer assessment, self-assessment, authentic
assessment and other sophisticated methods. All of these assessment techniques require that
students evaluate work of the same kind that they themselves are producing, and thus also obtain
experience of realistic evaluation (Sadler, 2005). Besides this, they are aware of assessment criteria
that are transparent and could apply to their own performance.
1.3 Social learning perspective
A social learning definition will be used further as an explanatory framework for the case study
(Wenger, 2000); it describes this phenomenon as “an interplay between social competence and
personal experience; it is a dynamic, two-way relationship between people and social learning
systems in which they participate. In this constellation, the necessary competences are negotiated
through the experience of direct participation: engagement in joint enterprise requires that members
of a community are competent to contribute to it, to interact and mutually reflect on this interaction;
and they also share a common repertoire of communal resources such as language, routines, etc.
According to Reed, social learning “occurs through social interactions and processes between actors
within a social network, either through direct interaction, e.g., conversation, or through other media,
e.g., mass media, telephone, or Web 2.0 applications” (Reed et al. 2010). A role for the open virtual
space such as Wiki in education is also reported by other authors (Wheeler et al, 2008; Wheeler &
Wheeler, 2009).
2. Case study - MPG&SD course in the VCSE network
The Multiple Perspectives on Globalization and Sustainable Development (MPG&SD) course ran over
the 2009/2010 European winter semester. The course was part of the international VCSE network of
European universities and participated in by students from Germany and the Czech Republic. The
167

Jana Dlouhá et al.
course was distant (fully eLearning) and the planned workload was 150 hours (5 credits). Twelve
students of the 15 who enrolled successfully completed it.
2.1 Educational goals
In the course, students had to develop three literacies – environmental & SD, academic writing, and
ICT literacy. Environmental & SD literacy was related to the theme of the course and students had to
learn about environmental problems and sustainable development policies very generally, and study a
theme of their choice in more depth. Writing literacy was considered to be an important metacognitive
skill necessary to carry out the writing assignment. Students should proceed through all the stages of
the academic writing process; their progress from one stage to the next was monitored. ICT literacy
was a prerequisite and also a byproduct of learning. Students’ written assignments as the main
product of the course were displayed in the students’ wiki space, and finally, the tutor compiled a
Globalization Handbook as a common virtual publication available for a future “generation” of students
as a starting point (Dlouhá, 2010). The rules and formal customs of research and academic writing
had to be followed while stressing a creative and critical approach.
A wiki environment as an open space for students’ creative work on the course themes was used:
they wrote their assignments, discussed their content (including a peer review) and shared their views
there (Dlouhá & Macháčková-Henderson, 2008; Dlouhá & Dlouhý, 2009). The role of the learning
environment was specific; it should stress interdisciplinary aspects of the learning process, and
support active learning approaches and solutions. These goals were achieved thanks to its more fluid
character which offered different ways of going through content, and provided an interactive character
and a three-dimensional structure (hyperlinks opening successive themes).
2.2 Rules and requirements
No online testing occurred (cheating is difficult to prevent in distant education). Students experienced
academic discourse and the principles of communication in a community of researchers: its
possibilities (freedom of choice of the research theme), rules (research ethics and norms of writing,
respect for the critical procedures of a peer review, formal requirements) and limitations (the necessity
to narrow the theme of interest and to proceed with discipline and efficiency towards justified
conclusions).
The relationship between the different processes could be tracked (peer review versus changes in the
article). Moreover, the learning processes could also be assessed – they were documented in the
Wiki environment (where the writing process was realized) in the page history. This option allowed a
comparison of subsequent versions of the article and was used to analyze the influence of a student’s
review on the quantity of the text (supposedly affecting its quality) – see Figure 1.
Figure 1: Comparison of student essays in the wiki environment before and after the peer review
168

Jana Dlouhá et al.
The number of bytes before and after the peer review (documented in the page history) was
compared; the results showed changes especially in the articles that were underdeveloped before
peer review.
The role of communication was important – a negotiated standpoint from participants and a common
decision on a hypothetical problem were required. The evidence of communication was available in
the fora; it could be analyzed and served as a resource for further research. The mutual peer review
process was undertaken in the Wiki space and its results played a role in the quality criterion. The
course outcomes were demonstrated in student portfolios which consisted of: fora discussion
contributions (evidence available in Moodle), student essays and mutual critical reviews of them
(evidence in Wiki).
All actors were actively involved in the learning processes and were in permanent interaction with
each other as well as with the learning environment – it was not only a resource base, but also a
factor in changing their roles. As subsequent research proved, the electronic environment (the Moodle
system) itself helped to monitor student activity, or control the quantity and timing of assignments
handed in, and thus influenced the performance of authority in the learning process. The virtual
electronic environment thus provided some almost invisible powers and thus took over some functions
of the authority that teachers use in face-to-face interactions (Dlouhá, M., 2010).
2.3 The role of, and the tools for assessment
The learning goals in the MPG&SD course were focused on the skills and competences needed for
self-regulation of the learning process, such as understanding, critical judgment, complex thinking
(awareness of the interdisciplinary context), commitment (the values behind scientific work), and a
constructivist and action-oriented approach (in theory and practice). Assessment goals were process,
not outcome oriented; the assessment options were adjusted to these goals and closely related to the
features and qualities of the articles and other texts produced by students – their essays and all other
contributions in fora were carefully analyzed in each stage of the course. New assessment tools for
evaluation of the specific qualities mentioned above had to be developed – evaluation rubrics were
used for all texts produced in the course. The rubrics assessed to what extent educational goals in
terms of (sustainability oriented) competences demonstrated by the specific properties of the texts
written by students were met (see Table 1).
Table 1: Example of the assessment rubric for a student’s essay – Wiki text (shortened)
Criteria/levels of
fulfillment
Content
Focus
Context
Clarity
Critical approach
10 0
quality of resources & well-founded
argumentation
strong conclusions, values behind
the topic are clear
wide context & core of the problem
identified
logical structure of the text, clear
ideas in sentence, paragraph etc.
balanced text: opposing views
presented
Commitment ethics (writing) & length (text)
Individual input &
risk-taking
Formal features
initiative in researching topic,
originality, independent work with
resources
respecting academic genre, citation
format respected
information is limited; argument is
biased
various themes with no clear
priorities, value orientation not
certain
narrow context, problems not
interrelated
ideas unclear, disconnected;
structure not logical
Competence under
consideration
Knowledge (deep)
Understanding
(research question)
Knowledge (broad)
Information
management
one-sided, demagogic Critical thinking
ethical principles not satisfactorily
respected
no new perspectives; conclusions
not original
mixed genre, citation format and
ethics insufficient
Values
Action competence
Respect for formal
requirements
Assessment tools designed for specific evaluation needs (competences) and online environment
options exploited the evidence of the learning outcomes – a portfolio of students’ work was
transparently available in the learning space.
169

2.4 Feedback – student perceptions
Jana Dlouhá et al.
Because of the complex character of the skills and competences to be developed within the course,
the principle of student interactive involvement and the rule that the learning process should be
reflected by them was applied (due also to the low number of course teachers/tutors and a presumed
lack of evaluation objectivity), while particular attention was paid to the students’ reflection of the
learning environment, requirements and outcomes.
Students’ perceptions of the academic quality of courses in distance education are (according to
relevant research) strongly associated with the approaches to studying that they adopt on those
courses (Richardson, 2009). Moreover, by comparing desirable educational goals in terms of
competences with characteristics of the students’ approaches we have identified a deep study
approach, in contrast to a surface approach, as the necessary condition for the development of
desirable skills and competences. The deep study approach can be described as “looking for
meaning in the matter being studied and relating it to other experiences and ideas with a critical
approach. Students adopting a deep approach aim to understand the subject and are intrinsically
interested in, and derive enjoyment from, studying. A surface approach can be thought of as a
reliance on roteLearning and memorization in isolation to other ideas. Surface learners perceive the
task of learning as an external imposition and they are externally motivated. They typically treat parts
of the subject as separate entities and fail to integrate topics into a coherent whole.” (Duff, 2004).
It has been argued that students must usually “take a deep learning approach in order to develop an
effective understanding of the multidimensional meaning of the complex topic of sustainability or
sustainable development” because complexity of the problem itself requires an active approach and
deep understanding (von Blottnitz, 2006).
To assess the extent to which students applied a deep/surface approach to studying, the
standardized Experiences of Teaching and Learning Questionnaire (ETLQ) was used to explore
student approaches to learning and studying (Richardson, 2009). Even though the questionnaire
explores solely the process of learning, the nature of this process relates strongly to final educational
outcomes, as has been proved by several studies (Entwistle et al., 2000; Duff, 2003; Mattick et al.,
2004). At the same time, researchers conclude that the studying approach is in turn influenced by the
context of learning (Duff, 2004; Parpala et al., 2010). The MPG&SD course is designed with the goal
of supporting the development of students’ competences using web 2.0 tools in an eLearning
environment and we expected that the strong emphasis put on their involvement in communication
and self-directed action will be associated with a deep study approach in the process of learning. The
ETLQ questionnaire designed by Entwistle to distinguish different student approaches was applied to
verify this assumption: we used its short 18 item version that considers four dimensions/approaches
to learning: the deep approach, monitored studying, organized studying, effort management, and the
surface approach (Mattick et al., 2004). The first three of them relate to desirable ways of studying
and refer to deep involvement in the learning process characterized by interest and understanding.
The results of testing are presented in Table 2 - students apparently refer to the prevailing deep
approach and reject the surface learning statements. This inquiry (application of a quantitative
questionnaire to a low number of students) was based on the application of a standardized tool whose
validity and reliability has been proved by previous surveys (Mattick et al., 2004; Parpala et al., 2010;
Duff, 2004). A median was used instead of a mean to represent students’ opinions as a more
appropriate measure from such a low number of respondents. The fact that most of the positive
statements resulted in 4 = agree and the prevailing answer to the surface approach was 2 = disagree
supports the hypothesis about encouraging course environment, i.e. context of learning.
Color
scale
1 1.5 2 2.5 3 3.5 4 4.5 5
Color scale used in all following tables: 5-point Likert scale (from Strongly disagree = 1 point to
Strongly agree = 5 points)
170

Jana Dlouhá et al.
Table 2: Results obtained in final standardized Experiences of Teaching and Learning Questionnaire
(ETLQ): DA = Deep approach, MS = Monitoring studying, OS = Organized studying and
effort management, SA = Surface approach. 5 item scale, 1 = strongly disagree, 5 =
strongly agree
Indicate to what extent you agree with the
following statement...
Data 2009/2010
(12 students)
median
Demonstration of the learning approach
I have usually tried to understand for myself the
meaning of what we had to learn
4 DA: Intention to understand for oneself
In reading for this course, I’ve tried to find out for
myself exactly what the author means
4 DA: Intention to understand for oneself
In making sense of new ideas, I have often
related them to practical or real life contexts
4
DA: Relating ideas (including
constructivist learning)
Ideas I’ve come across in my academic reading
often set me off on long chains of thought
3.5
DA: Relating ideas (including
constructivist learning)
I’ve looked at evidence carefully to reach my own
conclusion about what I’m studying
4 DA: Use of evidence
It has been important for me to follow the
argument, or to see the reasons behind things
When I’ve been communicating ideas, I’ve
4.5 DA: Use of evidence
thought over how well I’ve got my points across
(so that others understood)
4 MS: Monitoring generic skills
I’ve tried to find better ways of tracking down
relevant information in this subject
3 MS: Monitoring generic skills
I have been revising the work I’ve done to check
my reasoning and see that it makes sense
4 MS: Monitoring understanding
If I’ve not understood things well enough when
studying, I’ve tried a different approach
4 MS: Monitoring understanding
Concentration has not usually been a problem for
me, unless I’ve been really tired
4 OS: Concentration
I have generally put a lot of effort into my studying 5 OS: Effort
On the whole, I’ve been quite systematic and
organized in my studying
4 OS: Organised studying
I’ve organized my study time carefully to make
the best use of it
4 OS: Time management
Much of what I’ve learned seems no more than
lots of unrelated bits and pieces in my mind
2 SA: Fragmented knowledge
I’ve often had trouble in making sense of the
things I have to remember
2.5 SA: Memorizing without understanding
I’ve just been going through the motions of
studying without seeing where I’m going
2 SA: Unreflective studying
I’ve tended to take as guaranteed what we’ve
been taught without questioning it much
2 SA: Unthinking acceptance
This finding was further supported by the results of a questionnaire designed to explore specific
competences developed through the course in a post-course survey using a 5-point Likert scale. The
answers were also compared with students’ expectations about achieving the competences in a
similar pre-course survey and thus provided feedback on the course achievements related to the
learning outcomes expected by students entering the virtual environment. For this subjective
reflection on the degree to which students developed a certain competence or have found some
process of learning useful see Figures 2 and 3, and for those who are interested in detailed wording
see Tables 3 and 4. Comparison with an objective assessment of students’ achievements undertaken
by the teacher using an evaluation rubric (see above) was not done due to non-comparable
categories used in both cases; nevertheless, visible outcomes in terms of a portfolio of students’ work
(essays, peer reviews, discussions) could not be achieved without using the surveyed competences in
practice.
Students’ perception of learning outcomes in terms of competences is in accordance with findings
about the good quality of the learning process (manifested as students’ approaches to learning).
These findings might support the hypothesis that the method of teaching used in MPG&SD was
appropriate for sustainability-oriented education and the high expectations placed on social learning
processes in a virtual space were, at least in this limited case, justified. We can state that an open
171

Jana Dlouhá et al.
virtual learning environment that promotes independent work, communication and social interaction
fulfilled the teacher’s and students’ expectations to a great extent.
Figure 2: Results obtained in the final questionnaire developed for assessment of competences
Comparison of means with relevant questions in the pre-entry questionnaire (see these questions in
Table 3.)
Table 3: Questions in final questionnaire developed for assessment of competences.
Learning Outcomes Data 2009/2010
Indicate to what extent do you agree
with the following statement:
I would be able to explain the main
topics of the seminar to a third
person.
I increased my ability to be able to
understand and interpret global
issues from multiple perspectives
(disciplinary perspectives e.g.
sociology, economics, philosophy
etc).
I developed confidence in my own
ability to express my own wellinformed
arguments in globalization
and sustainable development
debates.
I increased my ability to plan my time
and study independently.
I developed my ability to write texts
which express my points clearly.
Description of the skill /
competence
Final
12 students
Comprehensive understanding 5
Expectations Preentry
15 students
median median
Multiple perspectives 5 5
Confident argumentation 4.5 4
Planning & studying 4 4
Academic writing 4 5
I developed my capacity to evaluate
whether information is biased or
credible.
Evaluation of info 4 4
I developed my computer skills.
I developed my ability to
ICT 4 3
communicate with people from other Cross-national communication 4 4
countries.
I developed my ability to study and
communicate with people from other
disciplinary backgrounds.
I developed my ability to provide
constructive criticism on fellow
students’ work.
I developed my ability to reflect on
the quality of my own work.
Cross-disciplinary
communication
4 4
Constructive criticism 4.5 4
Reflection on own work 5 4
172

Jana Dlouhá et al.
Results compared with previous year of the course and with relevant questions in pre-entry
questionnaire
In the pre-entry questionnaire, modified wording was used: “My expectations of the course can be
characterised by the following sentences …” “I expect to...“
Figure 3: Results received in the final questionnaire developed for assessment of the learning
process
Comparison of means with relevant questions in pre-entry questionnaire
(see these questions in Table 4.)
Table 4: Questions in the final questionnaire developed for assessment of the learning process.
Learning Process Data 2009/2010
Indicate to what extent do you agree
with the following statement:
I developed new ideas and learned
new information through
communication with my fellow
students.
I developed new ideas and learned
new information through discussion
with my tutors.
I developed new ideas and learned
new information through reading
provided materials.
I developed new ideas and learned
new information through researching
my own topics of interest within the
course.
My tutor helped me to find my own
answers about the content of the
course when I needed it.
My tutor provided answers to my
questions about the content of the
course when I need it.
Description of the skill /
competence
Final
12 students
median median
Communication - students 4 4
Discussion - tutors 5 4
Reading - provided materials 5 5
Own research 5 5
Supportive assistance 4 4
Instructive assistance 5 4
Expectations Preentry
15 students
Results compared with previous year of the course and with relevant questions in the pre-entry
questionnaire.
173

3. Discussion
Jana Dlouhá et al.
Qualitative feedback is more appropriate for courses such as MPG&SD. It provided information about
prior knowledge of the course topic, detailed feedback on eLearning tools, course content, discussion
topics, workload, and satisfaction with students’ own performances. Regarding evaluation of trained
skills and competences, we can identify the following items as what students perceived they gained:
language skills (English – all students were non-native speakers), writing (by creating their own wiki
article, developing the proper formal structure of an article, use of citations and references, a true
peer review process, communication: discuss and consider other opinions, and time management.
Looking at students’ open comments on course quality we can again find mostly positive reactions
like: “In total I really liked this course and I am happy that I participated in it. This course I really found
fun to handle. Hope to participate in other course like it.” The last comment was elicited from a
specific statement: “I would recommend this course to my colleagues/fellow students.” Eight students
agreed, while 4 of the 12 provided no response. Critical opinions also appeared. The intellectual
demands of the course were perceived as high. In addition, the whole course was seen as too timeconsuming
and sometimes deadlines were not 100% clear. Nevertheless, the positive reaction to the
course environment resulted in a unanimous appreciation of the tutors’ involvement, which is always
crucial for the creation of a well-functioning eLearning course.
Other researchers combine the results of an analysis of learning approaches with other analytical
tools focused on students’ reactions to teaching. This is only possible when a certain number of
students is analyzed. For example, Noel Entwistle, Velda McCune and Jenny Hounsell (2002) report
on research that looks for relationships between approaches to studying and the teaching-learning
environment carried out within several courses. They find the concepts and categories used to
describe general differences in studying to be a valuable analytical framework that allows
consideration of how individuals study and shows how various aspects of a teaching-learning
environment may affect student engagement in the course. Thus, the effects of the teaching-learning
environment, as perceived by the students, in terms of their deep approach to it might be assessed to
a certain extent. In addition, students’ perceptions of a particular course might be compared with their
general approach to studying, so that a pedagogical effect would be indicated. These authors
concluded that the ETLQ questionnaires are working effectively and are able to detect changes in
approaches to studying depending on changes in course design aimed at enhancing teachinglearning
environments to support high-quality learning, or compare the responses of students who
had first-hand experience of the innovation with those who had not.
In general, the deep studying approach should result in good quality outcomes and academic
performance (Entwistle et al., 2000; Duff, 2003). We have not overestimated the results of the
evaluation of on our course with a limited number of participants due to which we were not able to
proceed with a deeper statistical analysis; we have simply presented feedback on possible evaluation
tools that could be used for the assessment of social learning processes that give educational
outcomes different to those achieved in traditional HE settings.
4. Conclusion – the applied social learning concept in an eLearning course
The HE course presented as a case study is a model of the applied concept of social learning in
eLearning. In the MPG&SD course, students had to learn from the social environment with the
support of an electronic environment. The manner of involvement should follow the principles of (and
communication in) the academic community in general; the teaching method used and the eLearning
environment selected were mediators of the educational aims. The eLearning environment was
flexible enough to provide the options required, and, moreover, it provided evidence so that the
learning processes could be researched in different stages, as the work of the students was
continuously documented. Evidence of participants’ contributions in the forum discussion space
provided the opportunity to observe the social aspects of scientific work and the principles of
academic dialogue – the roles of participants in the dialogue, the distribution of authority, possibilities
of communication across boundaries, the added value of a multicultural environment, etc. The
research dimension was highlighted in an eLearning environment that makes external evaluation
possible.
Some of explorations undertaken and experiences obtained would not be possible without the
multicultural and interdisciplinary learning environment that was achieved within the VCSE
174

Jana Dlouhá et al.
international network of cooperation. International collaboration generally promotes intercultural
understanding, fosters the exchange of information on global problems, supports the transfer of
experiences and provides a broader sustainability context (Vann et al., 2006).
The course represented an experiment in teaching, with well documented and transparent outcomes
that were tenable with regards to academic skills – student academic essays compiled in the final
edition of the Globalization Handbook, and their reviews of colleagues’ articles in the discussion part
of the wiki page (and also numerous fora contributions). It was an example of the applied “community
of practice” model in teaching where the learning environment played a crucial role, especially with its
social aspects (actors and their roles, communication tools and means), but also the information
management options, etc. The social learning methodology – teaching through involvement in the
community, accepting and applying its discourse – brought about an educational experience that
should be explored further.
References
Biggs, J.B. (1987) Student Approaches to Learning and Studying. Research Monograph. Australian Council for
Educational Research Ltd., Radford House, Australia, 14–19.
von Blottnitz, H. (2006) Promoting active learning in sustainable development: experiences from a 4th year
chemical engineering course. Journal of Cleaner Production 14(9-11), 916-923.
Burandt, S., Barth, M. (2010) Learning settings to face climate change. Journal of Cleaner Production 18(7), 659–
665.
Dlouhá, J., Macháčková-Henderson, L. (2008) ELearning as an opportunity for virtual mobility and competence
development within European Universities. In: Williams R; Remenyi, D. (ed.) The Proceedings of the 7th
European Conference on eLearning (ECEL). NR Reading: Academic Publishing Limited. Book 1, pp. 297–
307
Dlouhá, J., Dlouhý, J. (2009) Use of Wiki Tools for Raising the Communicative Aspect of Learning. In: Remenyi
D. (ed.) Proceedings of the 8th European Conference on eLearning. NR Reading: Academic Conferences
Ltd. 165-173
Dlouhá, J. (2009a) General educational qualities and the term “competence”. Envigogika 4(1) Online. Retrieved
on 2009-06-16, from http://envigogika.cuni.cz/index.php/cs/texty/20091/299-obecne-vzdlavaci-kvality-apojem-kompetence
Dlouhá, J. (2009b) Competences in environmental education. Envigogika 4(1) Online. Retrieved on 2009-06-16,
from http://envigogika.cuni.cz/index.php/cs/texty/20091/300-kompetence-v-environmentalnim-vzdlani-
Dlouhá, J. (2010) Globalisation Risks and Opportunities. VCSEWiki. Online. Retrieved on 2010-08-31, from
http://www.czp.cuni.cz/vcsewiki/index.php?title=Globalisation_Risks_and_Opportunities&oldid=1350
Dlouhá, M. (2010) Authority in virtual education. Charles University in Prague, FSV, ISS, Praha.
Duff, A. (2003) Quality of Learning on an MBA Programme: The Impact of Approaches to Learning on Academic
Performance. Educational Psychology 23, 123-139
Duff, A. (2004) The Revised Approaches to Studying Inventory (RASI) and its Use in Management Education.
Active learning in Higher education 56(5), p. 57
Entwistle, N. (2000). Promoting deep learning through teaching and assessment: conceptual frameworks and
educational contexts. TLRP conference, Leicester.
Entwistle, N., Tait, H. & McCune, V. (2000) Patterns of Response to an Approaches to Studying Inventory Across
Contrasting Groups and Contexts. European Journal of the Psychology of Education 15, 33-48
Entwistle, N., McCune, V., & Hounsell, J. (2002). Approaches to studying and perceptions of university teachinglearning
environments: Concepts, measures and preliminary findings. Enhancing Teaching and Learning
Environments in Undergraduate Courses Occasional Report, 1.
Ferrer-Balas, D., Buckland, H., de Mingo, M. (2009) Explorations on the University's role in society for
sustainable development through a systems transition approach. Case-study of the Technical University of
Catalonia (UPC). Journal of Cleaner Production 17(12), 1075-1085.
Lukman, R., Krajnc, D., Glavic, P. (2009) Fostering collaboration between universities regarding regional
sustainability initiatives - the University of Maribor. Journal of Cleaner Production 17(12), 1143-1153.
Mattick, K., Dennis, I. & Bligh, J. (2004) Approaches to learning and studying in medical students: validation of a
revised inventory and its relation to student characteristics and performance. Medical Education 38(5),
535-543
Parpala, A., Lindblom-Ylänne, S., Komulainen, E., Litmanen, T. & Hirsto, L. (2010) Students’ Approaches to
Learning and their Experiences of the Teaching-learning Environment in Different Disciplines. British
Journal of Educational Psychology 80, 269-282
Reed, M. S., A. C. Evely, G. Cundill, I. Fazey, J. Glass, A. Laing, J. Newig, B. Parrish, C. Prell, C. Raymond, and
L. C. Stringer (2010) What is social learning? Ecology and Society 15(4): r1. [online] URL:
http://www.ecologyandsociety.org/vol15/iss4/resp1/
Reeves, T.C. (2006) How do you know they are learning? The importance of alignment in higher education.
International Journal of Learning Technology, 2(4), 294–309.
Richardson, J.T. (2009) Face-to-face versus online tutoring support in humanities courses in distance education.
Arts and Humanities in Higher Education 8(1), 69.
175

Jana Dlouhá et al.
Sadler, R.D. (2005) Interpretations of criteria-based assessment and grading in higher education. Assessment &
Evaluation in Higher Education 30(2), 175–194.
Vann, J., Pacheco, P., Motloch, J. (2006) Cross-cultural education for sustainability: development of an
introduction to sustainability course. Journal of Cleaner Production 14(9-11), 900-905.
Wenger, E. (2000) Communities of practice and social learning systems. Organization 7(2), 225.
Wheeler, S., Yeomans, P., Wheeler, D. (2008) The good, the bad and the wiki: Evaluating student-generated
content for collaborative learning. British Journal of Educational Technology, 2008, 39(6), 987–995.
Wheeler, S., Wheeler, D. (2009) Using wikis to promote quality learning in teacher training. Learning, Media and
Technology, 2009, Vol. 34, No. 1, March, 1-10
176

Putting Things in Context - Designing Social Media for
Education
Jon Dron, Terry Anderson and George Siemens
TEKRI, Athabasca University, Canada
jond@athabascau.ca
terrya@athabascau.ca
gsiemens@athabascau.ca
Abstract: The rich promise of social software in formal education can be offset by a clash between hierarchical
organisational structures and the bottom-up, distributed nature that characterizes network development and
growth. Learners often experience confusion when using social networking systems within formal education
systems, negating many of the potential benefits of sharing, collaboration, communication and personal
ownership associated with social networking systems. This often leaves learners and their teachers lost or at
least disorientated in social space. Social networking systems are mainly based on explicit individual social
connections, while students and staff in academia constantly shift between overlapping but delineated
hierarchically organised community contexts like classes, committees, research groups, centres and schools as
well as less formal person-to-person networks. Each context presents different needs for communication, shared
resources and connectivity. Most existing educational social software systems blur these contexts into a single,
confused and confusing sub-optimal space that is neither fully social and user controlled nor fully institutionally
controlled. In this paper we describe a set of partial solutions that we are evolving for the Elgg system, providing
multi-faceted profiles for both users and groups to allow control of content, presentation and audience for shared
artefacts, catering for different social, organisational and personal/group contexts. A facet is represented as a
page comprised of draggable widgets. Unlike the static views often associated with e-portfolios, these facets can
be interactive, inviting comment, assessment and other responses in specific contexts. Facets can be moulded to
fit the shifting contexts of academic activities and leisure lives, thus reducing the confusion of network, group, set
and personal spaces that besets current social software use in education, without losing the personal control,
sociability and ownership that makes it valuable in the first place.
Keywords: social media, higher education, technology-enhanced learning
1. Introduction
We and others (Dron and Anderson, 2009a; Ford et al., 2011; Sebastion et al., 2009) have observed
the enormous potential of social media (including but not limited to blogs, file sharing, link sharing,
social tagging, wikis, social networking, microblogging) to enable methods, pedagogies and forms of
learning that earlier technologies have failed to support. A clear set of affordances makes their use
compelling, including:
Content
Sharing of found and created objects
Emergence of patterns, computer augmented and visible
Authenticity as education activity aligns with business and social activity
Connection
Discovery of others with whom to learn
Leveraging networks that go beyond the formal classroom or workplace community
Serendipity as networks and sets interconnect
Sustainance of sociability with a positive association of social network use and traditional forms
of social contact (Hua Wang and Wellman, 2010)
Control
Empowerment to be both a reader and a writer
Adaptability to varying needs due to flexible and mashable (soft) technologies
Communication
Collaborating in teams and groups
Engagement and motivation brought on by persistence, visibility to and interaction with others
177

Jon Dron et al.
Social networking is a feature, not a destination, of most social systems (Anderson, 2007) and, apart
from in a few high-profile sites, it is a necessary but not sufficient condition for them to operate. The
largest educational benefit arguably lies not in the ability to connect with friends and colleagues but in
the ability to own, build and create a stake in a shared social space, to create and content, to interact
with others, in an equal space where all have the same capabilities and in which there is no inherent
hierarchical structure. The benefits relate not only to the potential to be constructors and consumers
of valuable learning content that makes learning easier, but to improve intrinsic motivation and
engagement through the enabling of pedagogies that are based on learner control and the sense of
connectedness with others that is entailed (Ratelle et al., 2007; Shroff and Vogel, 2009).
For all their potential benefits they present new problems – issues of trust, lack of centre, privacy and
information overload can weaken the efficacy of social software systems, especially for intentional
learning. This is not the kind of space in which learners in educational institutions have become
habituated. For many it is a bewildering and potentially frightening or frustrating new context (Kop,
2011; Mackness et al., 2010; Turkle, 2011). We will argue that this is mainly because the primary
social and educational form within formal institutions is the closed group, such as the class, tutorial or
seminar, whereas social systems occupy shifting social spaces that include social networks and sets.
This leads to a mismatch of organisation, activity and technology that can deter, demotivate and
sidetrack learners. We have addressed the nature of social forms in such systems in previous papers
(Dron and Anderson, 2007; Dron and Anderson, 2011, under review) but provide a summary here for
convenience.
1.1 Groups
The group has, historically, been the primary social and organizational form in formal education.
Classes, tutorial groups, seminars, faculties, schools, centres, and the institutions themselves embed
and embody this form deep in their DNA. Groups are formed by people, intentionally and explicitly.
Groups tend to have roles and rules, usually involving leadership, often delegating aspects of
responsibility to members that differentiate them from non-members. Groups in education, notably
courses, tend to have a defined purpose and often operate over a defined period. Groups tend to
have implicit and explicit rules of acceptable and unacceptable behaviours that differ or represent a
subset of rules that exist outside the group. Groups are defined as much by who they exclude as who
they include (Shirky, 2008). It is very rare to be a member of a group without knowing that one is a
member. Groups rely on trust and knowledge of sameness and inclusion to create a safe, but
cloistered learning context.
1.2 Sets
An overlapping but distinct kind of social entity common in education is the set. In learning situations,
we often find ourselves interacting with an indeterminate number of typically unknown individuals that
are bound by a shared attribute. For example, when we publish books or papers they are seldom
aimed at specific people or a known group but relate to a subject matter and/or audience
demographics: we thus treat potential readers as a set of people with shared characteristics. Sets are
not only about publication. A show of hands in a classroom might be asked for simply to count the set
of those hands in order to make a decision. When we tag a blog post we are explicitly categorising
that post into a set, and we generally have an expectation that potentially unknown others with an
interest in that tag will find it useful. Sets are about categories: humans and some programmed
machines make decisions about what category is of interest in order to define a set. Having made that
decision then interesting emergent features may be observed and analysed, especially when we look
at intersections between them – people with doctorates tend to buy more scholarly books than people
with high school diplomas, for example.
1.3 Networks
A network is formed of connections between nodes. To be part of a network means to be linked to
one or more of the nodes within it. Unlike the group but like the set, we may not be aware of
membership of a network: we join one just by connecting to one of its nodes. Local connections may
be all that we ever see of it – our friends and perhaps their friends, for instance, may link to a network
of friends that extends across much of the planet. Networks often have no fixed size limits but, like
sets, can occur within other social organisational forms, especially within groups. Networks can
connect us directly or indirectly, not only through people but also through objects. For example, a rich
178

Jon Dron et al.
network of connections may be linked through a popular blogging site and the blogs themselves may
be connected through trackbacks and blogrolls to each other, allowing people to traverse a network
structure in order to explore connected ideas, conversations and people. Networks and sets may
overlap and help to form and inform each other. For example, through set-oriented tags displayed as
a tagcloud, we may follow links and discover networked connections. We may treat a network as a set
by counting similar objects within it, and treat a group as an analysable network. We may analyse the
structure of the network itself to discover the patterns and shapes of connections within it – who is
most connected, how strong are the different kinds of connections and so on. Learning in networks is
quite different from learning in groups (Downes, 2007).
Despite our efforts above to differentiate these aggregations of the many we do not wish to imply that
these are hierarchical or exclusive categories. It is useful to think of them as being part of emergent
organization that Kontopoulos (Kontopoulos, 1993) describes as a heterarchy. "Hierarchy means
complete inclusion and supersession; heterarchy means partial inclusion and tangledness." (p.64)
2. Why this matters
Different virtual social spaces are implicitly or explicitly designed to support different social forms. A
system designed to support collaboration and centrally managed closed groups may have different
functions and be informed by different design decisions than one built to support bottom-up network
formation, even though they may share many of the same tools. This profoundly affects the nature of
interactions within them. ‘We shape our dwellings and afterwards our dwellings shape our lives’
(Churchill, 1943) or, as McLuhan said it, ‘we shape our tools and then our tools shape us’ (McLuhan,
1964). Using a system that supports a particular social form may often lead to difficulties for those
trying to use it to support a different form, even though the tools and affordances can be used that
way. By way of analogy, a lecture theatre may be used to support small group work, but the design of
the tiered forward-facing seats makes it more difficult to conduct smaller team discussions than to
conduct lectures.
As McLuhan (ibid) also noted the content of any new media is (at first) always the content of the older
media. The design of early learning management tools was based upon the existing shapes and
forms found in educational and training institutions and enables the shapes and forms of pedagogies,
processes and communities that had arisen in those physical settings. Unfortunately, many of those
forms had little to do with pedagogy or process and everything to do with physical constraint,
economies of scale and path dependencies. Universities created libraries because books were hard
to come by, lectures to share the scarce resource of learned scholars, classes and timetables to
efficiently allocate those resources, and group-oriented pedagogies because of the constraints of
physical space. Distance universities are, despite freedom from many of those restrictions, equally
driven by constraints: pedagogies of isolation, mainly behaviourist or cognitivist in style, have been
necessary in a technological landscape where communication was slow or prohibitively expensive.
They also inherited the limitations of traditional universities because they had to compete in the same
ecology and thus had to satisfy expectations (and prejudices) about forms of accreditation, roles,
organisational structures and so on that stretched back to medieval forebears (Norton, 1909).
Whether at a distance or face-to-face, the combination of structural constraints that have formed
universities made the group by far the most natural form in traditional institutional learning. It was the
only plausible form of organisation that allowed sharing of scarce resources and afforded the benefits
of both individual didactic teaching and social learning.
In contrast, educational social software typified by systems such as Elgg, Mahara, and BuddyPress is
predominantly network-oriented. Although all such systems do have support for groups (often
explicitly) they are architecturally predisposed to network interactions. They also usually offer strong
support for indeterminate sets (implicitly through features like tags, ‘like this’ buttons, ‘what’s hot’
indicators, as well as through publication tools). Content and interaction are discovered and shared
through links with individuals and their content or through set-based features like popularity metrics
and tags, less through formal groups. Even when formal groups are available, it is generally
impossible to ignore or hide the set and net structures that are parallel, overlaid or overlapping, and
which compete constantly for attention, diffusing and blurring the boundaries that are innate in group
forms. This allows for more open, connected learning that is not bound by the limits of the classroom,
that can better fit changing and shifting needs, that allows richer knowledge construction,
serendipitous knowledge discovery and great opportunities for self-guided, motivated learning.
179

Jon Dron et al.
However, it is also the main source of complaint from learners who are sometimes forced into these
social systems through a formal class requirement (Kop, 2011; Dron and Anderson, 2009c; Mackness
et al., 2010). The innate blurring of focus in their set/net orientation means that those who are seeking
a structured, usually teacher directed, path are constantly struggling with the mass of emergent
structure and diffuse connections that co-exist in that environment. Finding relevant content and
people is hard, keeping track of (creating) a beaten path is difficult. Leaders of groups (typically
teachers) have to constantly intervene, advise, guide and manipulate in order to allow learners to
maintain a focus and to avoid the sense of placelessness that characterises network or set-oriented
social spaces.
3. Problems and solutions
3.1 Problems of control
If teachers give control of the social space to learners then their own control is diminished. Without
the formal structures and cues that overcome such inherent egalitarianism in traditional classes
(timetables, lecture halls, etc) nor the structural technologies that replicate such forms in learning
management systems (course areas, hierarchical roles, formal announcement systems etc) teacher
contributions have no more inherent weight than those of their students. While teachers, classes and
administrators are recognisable entities that carry some weight, the design of social systems does not
reflect and magnify such roles. This means that the use of social systems for formal education can
confuse learners because their structure does not reflect that of an academic context in which they
are comfortable. To tweak this to match the organisational structure would to diminish the value of the
very things for which we use such systems: if we create hierarchies, roles and privileged content (or,
less regulating, an interface that enhances such things) we take control from the learners and
reinforce power structures that may affect social interaction considerably and greatly diminish the
unique learning affordances of networked contexts.
3.2 Problems of context shifting
Academia is a world packed with rapidly shifting contexts in which different activities, groups,
networks and sets constantly change. This relates to at least three facets, the social, the personal and
the group.
3.2.1 Social facets
As social beings, we adopt different facades in different circumstances (Goffman, 1959). This
subsumes but is a looser concept than that of ‘identity’ as described by (Wenger, 1998). We may
simply choose to show different things to different people at different times. A profile meant for one’s
friends will often be very different from a profile for one’s employers or classmates, for example. In
academia, we constantly and continually shift between different social and working contexts, usually
many times a day. The academic world is massively discontinuous, bounded not only by the different
needs of different classes but also of social, clubs, committees and so on, as well as social or work
networks and sets that may interest them. Few other contexts exhibit the same formalised and
frequently changing diversity that is found in almost every academic setting.
3.2.2 Personal facets
Some tools, such as Elgg or Wookie, are at least partly presented as personal learning environments
(PLEs) which provide a single, configurable learning space in which we might aggregate the things we
need when we are learning. However, in academia, personal needs change with alarming regularity.
The resources and toolsets that we may need in one class may be entirely different from those in
another, or overlap to different degrees. The groups that interest us when engaged in hobbies are not
the same as those that we might look for when working on a research project. A single personal
learning environment is too broad and non-specific in the context of most formal education.
3.2.3 Group facets
A similar problem besets collaborative groups working in social spaces. On the one hand they use
such spaces to support their working practice while, on the other, they often represent the group as its
external face to the world. Unfortunately, the needs of process are typically different from the needs of
product, though there will usually be overlaps – publications, for example, may be both objects to
180

Jon Dron et al.
work on and outputs of the group. Many academic groups (notably classes) also shift context on a
regular basis as they change activities and topics: they are not static objects, but exist in
discontinuous phases.
3.3 Current solutions
There are many ways that the structural conflict between learner control and teacher guidance can be
overcome, but all current solutions have their weaknesses. We discuss these in the following
subsections.
3.3.1 Pick and mix
Until now, the most effective way to present different social facets was through creating identities or
‘sockpuppets’ on different systems – a professional profile on LinkedIn, a profile for friends on
Facebook, an academic profile on Academia.edu, for example. Similarly, the simplest way to deal with
the conflict between social and formal systems is to use both, using the formal system to guide and
the informal systems to give the social interaction, connectivity, serendipity and other benefits.
Unfortunately this can be no less confusing, adds greatly to the cognitive and technical load for
learners and teachers, and often remains a very teacher-controlled process.
3.3.2 Group tools
A similar concern relates to the use of group tools provided by a social system. Because they are
situated in a space that emphasises local connection and collective crowd-mining, learners are a click
away from chaos. For instance, it is common to aggregate personal blogs by group members in the
group environment, but these exist outside the group in the individual’s networked space, where the
surrounding context is divorced from the guided space of the group, with nothing but browser
navigation buttons to lead back to the centre.
3.3.3 Collective tools
Using the system itself to mine the collective activities of its users, typically through analytics of sets
or nets, such as through collaborative recommendations, reputation-mining, automated clustering, tag
clouds and so on, can help to provide some structure. Unfortunately that structure, usually based on a
single dimension of implicit or explicit preference or pre-stated goal, is almost bound to be at odds
with the formal structure of a course and is as likely to lead to confusion as it is to good learning. And,
again, once the visitor has followed links determined by that structure, they are in a networked space
where that structure plays no further role.
3.3.4 Hybrids
Content management systems with social elements such as Drupal, Joomla, Plone, LifeRay, etc,
tread a middle ground between the highly formal learning management system and the informal social
network. Roles and hierarchies are built into such systems, allowing them to be as controllable as
needed, from an administrator’s perspective. Unfortunately, for the end-user, the benefits of bottomup
network and set structures are heavily constrained by what those administrators, decide is of
value. Although such systems allow an element of social networking and crowd processes to emerge,
the boundaries are regulated. This can be leveraged to provide a more flexible learner-centric
approach than a system that directly embodies traditional teaching models, but it does not provide the
benefits of ownership, control, serendipity and connection of a full-blown socially networked system.
3.3.5 Filtering
Existing social learning environments mostly treat an individual as a single entity defined by
structured/semi-structured data in a profile, but most offer some filtering. Systems such as Elgg,
Orkut, Google Plus and Diaspora provide fine-grained access control at an individual post level,
allowing users to easily specify who gets to see what of a user’s content. Other systems such as
BuddyPress, Facebook and LinkedIn work at a level of user rights rather than individual permissions
so, for the most part, make it possible to specify kinds of things that can or cannot be seen by certain
people. Either way, rights or permissions enable control over what is seen, but not how it is seen: the
single profile is filtered, not transformed to cater for different visitors. This does not fit social reality nor
the discontinuous nature of formal educational systems, and it makes it difficult to separate out the
many roles and groups of related activity that a typical learner is engaged in. Mahara’s Views go
181

Jon Dron et al.
further in enabling the selective publication of diverse content from a profile, but these are solely
meant for special-purpose presentations of portfolios and are not in themselves meant to assist the
dynamic process of social learning.
4. Context switching
The problem of overlapping social spaces could be reduced considerably if we were to reduce the
overlap, not just in an aggregated group space but also in the network and set spaces to which that
space leads, maintaining continuity and connection throughout. However, to do this without imposing
a pre-ordained hierarchy is difficult to manage without breaking the fragile and egalitarian
connectedness of the network space. Ideally, learners themselves should be able to control the way
that social spaces overlap and shift between their changing contexts as and when desired.
We have built a tool to extend the Elgg social framework that allows learners, both individually and in
groups to shift easily between different personal and social contexts, to aggregate different things in
different ways for each aspect of their learning (and, if they wish, social and working) lives. The tool
can present different facets to different, specified, groups of people, from entirely private workspaces
to entirely public facades. Each facet can have an entirely different content, appearance and layout
from that of every other, and they can overlap as much as needed.
4.1 Configurable facets
From an individual’s perspective, facets appear as a list of pages of a personal profile. Individuals can
add as many of these pages as they wish, and adjust permissions, layout and appearance as they
wish. By moving pages up and down the list, they can make different facets appear by default to
different people. A visitor to their profile will see the first facet to which they have access. So, for
example, a learner might create a personal page limited only to friends at the top of the tab set, then
professional portfolio accessible to the general public next down the list. Friends will see the personal
page, anyone else will see the public portfolio.
Figure 1: A configured facet showing option to choose other facets
4.2 Configurable widgets
The content of facets is constructed using configurable widgets. Elgg and the Elgg community provide
a wide range of these and they can be used to display many different kinds of content from within and
beyond the system, including blog posts, group membership, pictures, wiki pages, bookmarks,
external RSS feeds, arbitrary text, podcasts, files and so on. In order to provide greater control for the
182

Jon Dron et al.
individual, we have extended the functionality of many of these widgets to allow fuller control over
what is shown and how it is presented. For instance, rather than simply showing a user’s own posts, a
blog widget can show selected individual posts belonging to friends and shared group members, in
different sort order, with its own title, with varying-sized snippets of the original post, displayed
differently. In effect, this offers a kind of desk-top publishing tool for the social web, drawing dynamic
and/or static content from across the social site and beyond into different pages for different
purposes.
Figure 2: onfigured widget
Figure 3: Widget configuration options
183

4.3 Differentiated friends
Jon Dron et al.
Elgg lets users enable access control that is private, public, logged-in users, people they are
following, and the various groups of which they are members. It also supports the functionality of
collections, which enable an individual to subdivide those they are following (in some systems these
are known as ‘friends’ or ‘contacts’) into subsets that can be used to control access to posts. We have
built a plugin to make it explicit at the time of following what kind of relationship is being specified
using relationship tags, or ‘r-tags’ – friend, colleague, classmate, etc. There are no limits on the tags
that people may use to describe these relationships and use these to grant or deny access. This
makes it very simple for access to be granted to (for instance), friends, teachers, classmates in a
course, employers, family and so on.
Figure 4: Relationship tags
5. The benefits
Context switching provides a centre for learners and groups to organise not only groups but also
networks and sets in the learners’ own manageable spaces. Contextual profiles are only ever one
click away within the system so it is impossible to get lost when following links to personal, networked
or set spaces. Each facet adjusts dynamically to the social content provided by its widgets, updating
with new relevant posts as they are posted, yet the learner retains control over exactly which
elements are displayed. When visiting another learner’s profile, visitors can choose which facets they
see, if they have permission to do so. So, should they visit the profile of someone who is a friend as
well as a member of other classes or communities, they are able to see only those facets the friend
wishes them to see in each specified context, greatly reducing the confusion and potential for
becoming lost in social space. Similarly, owners of groups can create facets for users, networks, sets
activities, roles, presentations to the outside world and much more, with widgets displaying specified
activities of users outside the group and within it, making them far more self-contained and intuitive to
navigate, combining the parts of networked and set space that are most relevant to the group in a
given context. The result is that a community with the context switcher in place can switch between
networked, group and set modes of engagement without any central authority determining how it is
organised.
6. Conclusions
The system we have presented here is just one of many possible solutions to the problem of shifting
and overlapping social and learning contexts. As we implement, evaluate and refine the toolset we
expect that the solution will evolve and we expect to see increasing recognition of the importance of
context in other social systems that may lead to approaches that are better or more suited to learner
needs than what we have described here. Our purpose in highlighting the importance of shifting
contexts is to help others to recognise ways that the current generation of social systems for learning
need to move beyond a naïve, single-faceted view of the people that use them to a richer more
human set of tools that support the complex context(s) in which we find ourselves, in which multiple
184

Jon Dron et al.
groups, networks and sets coexist and intertwine, shaping and shaped by the activities and
organisational forms their members engage with.
References
Anderson C. (2007) Social networking is a feature, not a destination. Available at:
http://thelongtail.com/the_long_tail/2007/09/social-networki.html.
Churchill W. (1943) HC Deb 28 October 1943 c403.
Downes S. (2007) Learning Networks in Practice. In: Ley D (ed) Emerging Technologies for Learning. BECTA,
19-27.
Dron J and Anderson T. (2007) Collectives, Networks and Groups in Social Software for E-Learning. In:
Bastiaens T and Carliner S (eds) E-Learn 2007. Quebec City, Canada: AACE, 2460-2467.
Dron J and Anderson T. (2009a) How the crowd can teach. In: Hatzipanagos S and Warburton S (eds) Handbook
of Research on Social Software and Developing Community Ontologies. Hershey, PA: IGI Global
Information Science, 1-17.
Dron J and Anderson T. (2009b) Lost in social space: Information retrieval issues in Web 1.5. Jounral of Digital
Information 10.
Dron J and Anderson T. (2009c) Lost in social space: Information retrieval issues in Web 1.5. Journal of Digital
Information 10.
Dron J and Anderson T. (2011, under review) Groups, Sets and Nets in Online Learning. Journal of Computer-
Assisted Learning.
Ford N, Bowden M and Beard J. (2011) Learning together: using social media to foster collaboration in higher
education. In: Wankel L (ed) Higher Education Administration with Social Media (Cutting-edge Technologies
in Higher Education, Volume 2). Bingley, UK: Emerald Group Publishing, pp.105-126.
Goffman E. (1959) The Presentation of Self in Everyday Life, New York: DoubleDay.
Hua Wang and Wellman B. (2010) Social Connectivity in America: Changes in Adult Friendship Network Size
From 2002 to 2007. American Behavioral Scientist 53: 1148-1169.
Kontopoulos K. (1993) The logics of social stracture, Cambridge: Cambridge Unievrsity Press.
Kop R. (2011) The challenges to connectivist learning on open online networks: Learning experiences during a
massive open online course.
Mackness J, Mak SFJ and Wiliams R. (2010) The ideals and reality of participating in a MOOC. In: Dirckinck-
Homfield L, Hodgson V, Jones C, et al. (eds) 7th International Conference on Networked Learning. Aalborg,
Denmark: University of Lancaster, 266-274.
McLuhan M. (1964) Understanding media: The extensions of man, Toronto: McGraw-Hill.
Norton A. (1909) Readings in the History of Education Medieval Universities, Cambridge:
Harvard University.
Ratelle CF, Guay F, Vallerand RJ, et al. (2007) Autonomous, controlled, and amotivated types of academic
motivation: A person-oriented analysis. Journal of Educational Psychology 99: 734-746.
Sebastion V, Namsu P and Kerk FK. (2009) Is There Social Capital in a Social Network Site?: Facebook Use and
College Students' Life Satisfaction, Trust, and Participation1. Journal of Computer-Mediated
Communication 14: 875-901.
Shirky C. (2008) Here comes everybody: the power of organizing without organizations, New York: Penguin
Press.
Shroff RH and Vogel DR. (2009) Assessing the factors deemed to support individual student intrinsic motivation
in technology supported online and face-to-face discussions. Journal of Information Technology Education
8: 59-85.
Turkle
S. (2011) Alone together, New York: Basic Books.
Wenger
E. (1998) Communities of Practice: Learning, Meaning and Identity, New York: Cambridge University
Press.
185

Experimental Assessment of Virtual Students
Michaela Drozdová 1 , Ondřej Takács 2 and Jana Šarmanová 1
1 Department of Information and Communication Technologies, University of
Ostrava, Ostrava, Czech Republic
2 Department of Computer Science, Faculty of Electrical Engineering and
Computer Science, VSB - Technical university of Ostrava, Ostrava, Czech
Republic
l09430@student.osu.cz
ondrej.takacs@gmail.com
Abstract. In this fast developing society full of changes is learning getting more and more important. How can we
make it more effective? One of the approaches uses electronic learning with its capability to adapt learning
process to individual students needs. Students' characteristics important for adaptive learning must be known in
order to achieve this. In this paper the most frequently used method – the questionnaire is used to identify the
students' characteristics. The problem is that even small number of student's properties leads to vast number of
its combinations. To make the design of adaptive learning easier, we try to narrow this combinations by setting
few virtual students with given values of respective properties that represent most common combinations of
student properties. In this paper we describe the methods and results of virtual student assessment using
quantitative analysis of filled learning styles questionnaire. We created new questionnaire that measures sensual
perception, motivation and learning styles. Sensual perception describes which form of information suits students
best: visual, auditive, verbal or kinesthetic. Social aspects concerns with type of company that student prefers
when learning. The systematic approach describes sequence of learning, which can be either in logical sequence
of steps or almost randomly, without connections, in great steps. We divide way of learning on deriving and
experimenting. In accordance to sequence of learning we can divide students on detail oriented, which focus on
small pieces of information and compose them to the global picture, and holistic, which focus on big pieces of
abstract information from which it works through to details. The approach of learning can be divided into: deep,
where student's main goal is to fully understand the curriculum, strategic, where are prioritized results and
effectiveness and surface approach, where students only try to meet basic requirements. Degree of student's
ability to independently control his learning is given by his self regulation of learning. This questionnaire was filled
by 500 students from different fields of study. We analyzed these results using clustering, decision tree and
principal component analysis. Cluster analysis tries to discover groups of mutually similar objects which differ
from other groups. We set virtual students using this method. Decision tree analysis focuses on one property and
tries to discover other properties that affect its values. Several interesting dependencies between properties were
discovered using this method, for example students have high motivation if they do not possess surface learning
style.
Keywords: learning styles, eLearning, data mining, virtual students
1. Introduction
Learning process can be influenced by a number of aspects. For our field of interest, aspects
characterizing learning style, preferences, motivation and the way particular students learn using
eLearning materials have been chosen. Our aim is to prepare a learning environment in eLearning
domain respecting students’ differences and adapting to them accordingly.
In a traditional teaching process, the educator teaches all of the students in the same way despite the
fact that the educator realizes that each student is different. Each student can be talented in different
subjects, has a different level of previous knowledge, possesses different learning styles, different
memory, motivation for learning, different family background, different habits concerning when and
how to learn, etc. Unfortunately, a teacher in the class lacks the capacity to respect and consider such
preferences as well as the instant dispositions of all individuals in the class (Šarmanová, 2010).
This can be different in eLearning. In this learning form the student studies through the Internet and
represents an individual person to a tutor, who communicates with the student individually, gives
advice and guides the student through the learning process. The tutor is thus able to adapt to the
preferences of the student. Ideal assistants for such adaptation are modern information and
communication technologies, vast databases, multimedia equipment, internet, etc. We asked
ourselves a question, if it is possible to simulate the process of going through a course, which would
suit each student with his individual properties. Our intention is to find methodology and algorithms to
search for the optimum procedure. This procedure shall respect the differences among students on
the basis of detected learning style and also consider the altering level of knowledge and skills of
186

Michaela Drozdová et al.
each individual student in the course. On the basis of students’ personal characteristics identification,
they will be provided with learning material in the form which shall be most convenient to them.
The model of learning being created is based on a new paradigm – the personalization of learning
environment. In an ideal case, this environment consists of three parts: a student part, a learning
materials database part and a part representing the system of adaptive algorithm.
Our goal is to create adaptive eLearning environment – an environment in which the student learns
through directed self-study form. Suitable instruments for this form of teaching are so called Learning
Management Systems - LMS. These are systems incorporating teaching supports, features to support
learning management and, finally, an information system registering students, their activities and
results (Kostolányová, 2010).
For an adaptive system being able to respect the diversity of users, it cannot be anonymous (Takács,
2010). The collection of data about a student will be implemented in several phases. The most
important part is the student self-assessment, i.e. testing before entering the course. The second type
of students testing during the study course (testing will be classified as dynamic changing
characteristics, crucial for adjustments of the proposed course route).
In such an extensive project we first concentrated on students and their characteristics, as they are
the key subjects. Especially their learning preferences which will influence the creation of the adaptive
course route the most.
2. Student part of adaptive system
To enable the learning management system to react on different students, we have to choose,
describe and suitably store the student’s qualities and other attributes, which influence the process of
his/her learning.
We can gain one group of qualities straight from students with the help of a suitable questionnaire and
the second group of qualities we gain by long time monitoring of their study activities. The second
group can serve as feedback not only during the current learning, but also for the alteration of the
student’s qualities, possibly for monitoring of his/her development.
It can be generally said that each person is a truly an individual with various points of view. There is
an abundant literature available dealing with learning styles and also describing their classification. By
analyzing and studying the already published classifications by various authors (Gregorc, 1979),
(Kolb, 1984). (Mareš, 1998) we have come to the following characteristics that may be taken into
account in the area of eLearning. We have divided these characteristics into several groups:
The group of characteristics called the sensual perception describes the form of information most
convenient for the student – visual type, auditive type, kinesthetic type and verbal type. For this group
of characteristics we used the VARK questionnaire (Fleming, 2009) based on (Fleming, 1992). The
acronym VARK stands for Visual, Aural, Read/write, and Kinesthetic sensory modalities that are used
for learning information.
Another group of characteristics called the social aspects deals with the company most convenient to
the student within the learning process – whether the student prefers learning with his classmates,
with a teacher or by himself. For this group of characteristics the Czech version of LSI (Learning Style
Inventory) questionnaire was used (Mareš, 1993).
The group of characteristics called the affective characteristics deals with student’s feelings and
attitude which influence the learning process. The most important characteristic of this category is
motivation. For this group a part of the Czech version of LSI questionnaire was used (Mareš, 1993).
The most extensive group of characteristics is called the tactics of learning. These describe the
“technology“ of the way the student learns. The method of the student’s learning is described by the
order of learning, which can be done either subsequently in mutually related blocks (pole rule) or in
almost random manner, without connections, with large skipping (pole latitude). To describe this
characteristic a part of the ILS (Index of Learning Styles) questionnaire was used (Felder, 2009).
According to the way of learning we divide the tactics to theoretical deduction and experimentation.
To describe these characteristics a part of the ILS questionnaire was used (Felder, 2009). According
187

Michaela Drozdová et al.
to the order of learning there can be either detailistic or holistic tactics. For this characteristic a part of
the TSI (Thinking Style Inventory) questionnaire was used (Sternberg, 1999).
The conception of student’s learning can be divided into three levels: deep, strategic and surface.
This characteristic is tested by ASSIST (Approaches and Study Skills Inventory for Students)
questionnaire (Tait, 1996).
The learning auto-regulation defines, to which extent the student is able to manage his learning
independently. This implies his need of external supervision of the processes of learning where, on
one side, there are those who appreciate punctual instructions and on the other side those who prefer
to manage the learning processes themselves. For this characteristic a part of the LSI questionnaire
was used (Mareš, 1993).
2.1 Testing and data analysis of individual learning styles
To measure aforesaid characteristics, it is best to use questionnaires. In the pilot phase we used the
above mentioned questionnaires, but their combination was not applicable due to their length. This
led to inaccurate filling of the questionnaires, shown by the results of their analysis (Takács, 2009).
These reasons led to compiling a new shortened Learning Profile Questionnaire - LPQ (Novotný,
2010). The usual duration of filling the LPQ is only ten minutes while measuring all properties needed.
This questionnaire is inspired by following questionnaires: LSI, ILS, TSI and ASSIST. Only three to
five questions measures one property of the student. This questionnaire was not verified on students
yet.
Most questions in the LPQ questionnaire are made of statements together with scales of agreement
to this statement. These scales are usually from 1 (I don’t agree) to 5 (I agree). Some questions have
four variants, one for each type of perception.
This questionnaire was converted into electronic form. Before filling the questionnaire, its purpose was
introduced to the students. A group of university students from different fields of study were asked to
fill this questionnaire, as well as a group of high school students. A total of 508 students filled this
questionnaire, 190 from those being grammar school students, 196 pedagogical university students,
68 high school students and 62 informatics university students. The duration of filling the
questionnaires varied between two minutes and half an hour. To exclude suspiciously fast filling
students we sat aside 45 questionnaires filled in less than five minutes.
From filled questionnaires we saved data about students’ answers and questionnaire results that were
computed according to a scoring key of this questionnaire. Values of all properties ranged from 0 to
100, zero meaning that the student does not possess this characteristic and one-hundred meaning
that he does.
For the data analysis association rules method, which looks for dependencies between attributes and
conveys them in form of rules, was used. Each rule consists of a condition and a conclusion and is
assigned with reliability and support. Reliability provides information about how many percent of
records found with this condition being true meet also the conclusion. Support provides information
about for how many percent of records the condition is true.
Principal component analysis was used to identify possible dependent attributes measuring the same
or very similar student’s characteristics. This method searches for principal components – new
attributes which values are determined by linear combination of former attributes. We can distinguish
composition of each component by the size of its coefficients. Every principal component is also
described by standard deviation that indicates significance of the component.
Decision tree analysis was also performed. This analysis finds numbers of interesting rules for one
goal attribute. Rules are then represented in a form of a tree. Leaves gives value of the goal attribute,
nodes are conditions of rules and branches are values of conditions. Finally, a cluster analysis was
performed, which tries to find groups of similar objects that are different from other group‘s objects in
data. For the computation of the distance was used Euclidian metric and then two grouping methods
were used: single link and Ward-Wishard Ward’s minimum variance method (Ward, 1963) aims at
finding compact, spherical clusters. The single linkage method adopts a ‘friends of friends’ clustering
strategy.
188

3. Results of learning style analysis
Michaela Drozdová et al.
In this part, results obtained during the analysis are presented.
3.1 Association rules
Association rules found many dependencies between properties. We were looking for rules with
minimal support of 10% and minimal reliability of 70% and found total of 147 rules. The majority of
these rules have very low level of depth learning in conclusion. Conclusions also contained visual
type, strategic learning and motivation. Some of these rules are shown in Table 1. Of most interest
are the rules with high support. For example almost half of non-visual type of students mostly do not
possess depth learning style (see rule number 1, where support 43% means that this rule is applied to
43% of students and reliability 71% means that 71% of students with non visual learning also do not
possess depth learning). Unfortunately 70% of all students do not possess depth learning style, so
many of these rules are influenced by this fact. Because of this we cannot make any valid conclusions
based on this analysis. But we can use its results to determine goal attributes for the decision tree
analysis: depth and strategic learning, visual perception and motivation.
Table 1: Selected rules with 70% minimal reliability and 10% minimal support, ordered by support
Num Condition Conclusion Support Reliability
1 visual = very low depth = very low 43% 71%
2 strategic = very low depth = very low 38% 76%
3 social (alone - group) = middle depth = very low 38% 72%
4 verbal = low depth = very low 31% 73%
5 theoretical – experimental = high depth = very low 31% 76%
… … … … …
32 motivation = high, theoretical – experimental = high depth = very low 17% 73%
33 social (alone - group) = middle, verbal = low depth = very low 17% 75%
34 auditive = middle visual = very low 16% 80%
… … … … …
depth = very low, systematic (sequential - random) =
69
very low
strategic = very
low
13% 72%
visual = very low, motivation = high, strategic = very
70
low
depth = very low 13% 73%
… … … … …
depth = very low, visual = very low, field =
87
University, pedagogical
motivation = high 12% 70%
88 strategic = very low, visual = middle depth = very low 12% 79%
… … … … …
depth = very low, surface = very low, autoregulation =
107 visual = very low
low
11% 71%
… … … … …
147 autoregulation = low, social (alone - group) = low depth = very low 10% 87%
3.2 Decision trees analysis
This method was used for four goal attributes based on the association rule analysis results: depth
and strategic learning, visual perception and motivation. Further on we describe results of these
analyses.
By analyzing depth learning (see figure 1) we found out that group learning (left branch of node
“social: alone/group”), holistic (left branch of node “detailistic/holistic”) students mostly have depth
learning style (yellow color in leaf labeled “high”). Other results derived from this tree cannot
unfortunately be used because 70% of students in this data sample do not have depth learning style.
189

Michaela Drozdová et al.
Figure 1: Decision tree for depth learning
The analysis of strategic learning style revealed (see figure 2) that most experimenting students, not
learning alone and randomly do not possess the strategic learning style. Other rules provided by this
tree cannot unfortunately be used because of their low reliability.
Figure 2: Decision tree for strategic learning style
190

Michaela Drozdová et al.
By analyzing motivation (see figure 3) we discovered that students have high motivation if they:
Do not possess surface learning style,
Possess surface learning but do not have high auto-regulation,
Possess neither surface learning style, nor high auto-regulation nor strategic learning style.
Figure 3: Decision tree for motivation
Through the analysis of visual perception (see figure 4) we found out that students don’t possess
visual perception if they:
Are auditive types,
Are not auditive and are kinesthetic types,
Are neither auditive nor kinesthetic, but are verbal types.
3.3 Cluster analysis
Agglomerative clustering with single-link method didn’t find any significant clusters. So, the Ward-
Wishard method was used and the results are presented here. The final dendogram (see figure 5)
was cut on the height 75 which created 5 clusters within the data. In bottom part of this figure are
more than 500 vertical lines, where each line represents one student. Vertical lines connected by
horizontal lines forms clusters. The longer the vertical line is, the more dissimilar are the objects in the
cluster. The thick horizontal line on the height 75 represents cut and chooses five clusters.
191

Figure 4: Decision tree for visual perception
Michaela Drozdová et al.
Figure 5: Dendrogram of agglomerative clustering analysis
Figure 6 presents detail description of the five clusters created. Each cluster is represented by one
color in the boxplot chart. The most interesting results are at first four properties. Red cluster is typical
by average level at all properties except kinesthetic, auditive and verbal property. Yellow cluster is
described by very high values of auditive property. Green cluster is characterized by low value of
kinesthetic property, high value of auditive and verbal properties. Blue cluster is described by higher
value of kinesthetic, low value of auditive and above average value of verbal property. Black cluster is
typical by its high value of visual and auditive property and low value of kinesthetic, social and verbal
property. The main finding is that the clusters of students depend mainly on type of perception of
students.
192

Figure 6: Box plots of five clusters
3.4 Principal component analysis
Michaela Drozdová et al.
Principal component analysis found 13 components. The last component has much lower standard
deviation compared to the other components (see table 2), so this component can be removed from
the analysis. Most of the principal components have high coefficient in the covariance matrix (see
table 3) at only one property, so this property is independent. Significant components with more than
one high coefficient have high values mostly at sensual properties. This means that sensual
properties depend on each other, but that was already known. All other properties are almost entirely
independent.
Table 2: Standard deviation table of principal components
PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 PC11 PC12 PC13
3,46 2,45 2,26 2,09 1,51 1,25 1,08 0,75 0,34 0,31 0,29 0,26 0,00
Table 3: Covariance matrix of principal components
PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 PC11 PC12 PC13
visual 0,0 0,1 -0,1 0,0 -0,8 0,0 0,1 0,0 0,0 0,0 0,0 0,0 0,5
kinesthetic 0,0 0,7 0,2 -0,1 0,3 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,5
auditive 0,0 -0,6 0,6 -0,1 0,1 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,5
verbal 0,0 -0,2 -0,7 0,3 0,3 0,0 -0,1 0,0 0,0 0,0 0,0 0,0 0,5
social -1,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0
motivation 0,0 -0,1 -0,3 -0,9 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0
order 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 -0,7 -0,7 0,2 0,0 0,0
theor/exper 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,1 -0,4 -0,9 0,2 0,0
detail/holistic 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 -0,7 0,6 -0,2 0,4 0,0
surface 0,0 0,0 0,0 0,0 -0,1 1,0 -0,1 0,1 0,0 0,0 0,0 0,1 0,0
strategic 0,0 0,0 0,0 0,0 0,1 0,1 0,9 -0,3 0,0 0,0 0,0 0,0 0,0
deep 0,0 0,0 0,0 0,0 0,1 -0,1 0,3 0,9 0,0 0,0 0,0 0,0 0,0
auto-regulation 0,0 0,0 0,0 0,0 0,0 0,1 0,0 0,0 -0,3 0,2 -0,3 -0,9 0,0
4. Conclusion
The newly created LPQ questionnaire has many advantages compared to the group of questionnaires
used in our previous research. Its main benefit is lower filling time that led to more honest filling.
Analysis of filled questionnaires found only minor dependencies between chosen learning styles, so
there is no need to change the set of measured student characteristic. One potential problem is that
193

Michaela Drozdová et al.
very few students possess the depth learning style. This can be explained by wrongly designed
questions in the questionnaire or by the composition of the set of the tested students.
Results of cluster analysis which discovered five dominant clusters divided according to type of
perception tell us that this property is very important for definition of virtual students: set of properties
that describes group of students with similar properties. Finding of these virtual students will be the
goal of the future work. The definition of virtual students can be very useful in eLearning. It is much
easier to design the course for only five types of students instead of trying to design it for every
individual student and all possible combinations of his properties.
Analysis of principal components revealed that evaluated properties except type of perception are
independent. For practical use it is more appropriate to distinguish the four types of perception,
because there is clear way of using this information in eLearning.
LPQ questionnaire proved to be suitable for using directly in adaptive learning process. It will be
presented to untested students at the beginning of the lecture. We plan to do more analysis of results
of this questionnaire on students that will participate in adaptive learning.
Acknowledgments
This publication is supported by the ESF project OP VK CZ.1.07/2.3.00/09.0019 title Adaptive
individualization of study through eLearning.
References
Felder, M. R., Soloman, A. B. (2009) Index of Learning Styles [Online], Available: http://www.ncsu.edu/felderpublic/ILSpage.html
[2009-05-10].
Fleming, N. D. and Mills, C. (1992) “Not Another Inventory, Rather a Catalyst for Reflection“, To Improve the
Academy, vol. 11, pp. 137-155.
Fleming N. D. (2009) VARK -- A Guide to Learning Styles [Online], Available: http://www.varklearn.com/english/index.asp
[2009-05-10].
Gregorc A. (1979) “Learning/Teaching Styles: Their Nature and Effects”, Christian Education Journal, vol. 4, pp.
62
Kolb D. A. (1984) Experiental learning: Experience as the source of learning and development, Engelwood Cliffs,
NJ, Prentice Hall
Kostolányová, K., Šarmanová, J., Takács, O. (2010) “Learning Characteristics”. Proceedings of the 9th European
Conference on eLearning, Porto, pp. 107—107.
Mareš, J., Skalská, H. (1993) “Česká verze dotazníku LSI, zjišťující styly učení u žáků základních
škol”, Pedagogický výzkum a transformace české školy, pp. 54.
Mareš, J. (1998) Styly učení žáků a studentů, Praha, Portal
Novotný, J. S. (2010) “Individualization of teaching through eLearning: Development of Students? Learning
Profile Questionnaire”, Theoretical and Practical Aspects of Distance Learning, Cieszyn.
Sternberg, J. R. (1999) Thinking styles, Cambridge: University Press.
Šarmanová, J., Kostolányová, K. and Takács, O. (2010) “Intelligent Tutoring in ELearning System“, 9th European
Conference on eLearning, Porto, pp. 794-796.
Tait, H. & Entwistle, N. (1996) “Identifying students at risk through ineffective study strategies”, Higher
Education, vol. 31, pp. 99-118.
Takács, O., Šarmanová, J. and Kostolányová, K. (2009) “Results of analysis of learning styles”, Proceedings of
ICTE2009, Ostrava, pp. 205-210.
Takács, O., Kostolányová, K., Šarmanová, J. (2010) “The use of adaptive individualized eLearning at
teaching”, IMSCI 2010, Florida, pp. 147—152.
Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American
Statistical Association, 58, 236.
194

Priming for Modules: A Case Study Evaluation of ‘Pre-
Workshop’ Online Resources for an Executive MBA Course
Glenn Duckworth
Bristol Business School/e-Learning Development Unit, The University of the
West of England, Bristol, UK
glenn.duckworth@uwe.ac.uk
Abstract: This case study looks at a series of 'pre-workshop' resources and tasks developed for the Executive
MBA (EMBA) at the University of the West of England (UWE). Students on the EMBA course at UWE are
generally in full time employment and therefore do not have the opportunity to follow a traditional semester - or
year - long module structure. For each module on the EMBA programme, the taught classroom based component
runs as a five day intensive block. It is therefore very important for the students to get to grips with concepts and
terminology for each module as quickly as possible so that the time spent in class can be used for high level
discussions and debates as well as the more traditional lecture formats. This is common of a variety of executive
education and CPD courses across the sector. A set of online resources were developed for a range of modules
on the EMBA course and delivered online. Their aim was to ‘prime’ the students for the modules in advance of
the classroom based components. As well as providing readings, the resources provided a range of online
interactive exercises to introduce the topics covered on the module and to help the students assess their level of
learning. Student evaluations were conducted on three modules to assess the usefulness of these pre-workshop
resources. These took the form of surveys with likert-type questions and more open ended questions, as well as
informal group discussions. Views of the academic staff teaching on the modules were also sought. For all three
modules evaluated, the pre-workshop resources were seen as being a very valuable resource. There were fears
that students would not complete the exercises and readings before the classroom based sessions began, but
the majority of them did so. They reported that the exercises helped them to structure their learning and that they
felt more confident and prepared for the first day of classroom based teaching. Lecturers also felt that the
resources and exercises were useful – they felt that they were able to engage with students at a higher level from
day one of the classroom based sessions. This case study will look at the evaluations in more detail and go on to
assess whether a ‘pre-workshop’ component may be of use on a variety of other executive and CPD intensive
courses.
Keywords: priming, pre-workshop activities, online learning, interactive exercises
1. Introduction
This paper is concerned with intensive courses - also known as block, compressed or time-shortened
courses – and in particular a programme developed for the part-time Executive MBA (EMBA) at the
University of the West of England (UK). Sets of online learning resources were developed to
complement the more traditional classroom based learning and teaching. They were provided in
advance of the classroom based sessions and became known as ‘pre-workshop activities’. They were
developed with the intention of ‘priming’ students to the subject matter before beginning the intensive
block of classroom based teaching. Post-workshop activities were also provided – generally with a
focus on the assignment submission.
The paper will take the form of a case study examining the views of students and lecturers as to how
useful these online priming activities were seen as being. It will draw together evaluations of the preworkshop
activities used on three modules on the EMBA programme. It will also examine the views of
module leaders relating to the usefulness of these activities.
There has been debate about the efficacy of intensive courses for many years, although research on
the topic has been somewhat patchy. As Kasworm (2001) put it, “While there is recent significant
growth of accelerated degree programs, there is little empirical research regarding the quality and
impact of accelerated degrees on adult learning” (cited in Davies 2006). According to Davies (2006)
“a considerable amount of the literature … appears to exist in academic areas where skill acquisition
is paramount, rather than discursive, conceptual learning. This point may be critical in assessing the
value of intensive teaching in various subjects.”
Where research into this topic does exist, the findings have been somewhat mixed. Many studies
have found that compressed courses compare favourably with traditional semester - or year - long
courses. Daniel (2000) in a review of relevant literature suggested that intensive courses offer as
effective a learning experience as do traditional courses – and in fact they may sometimes lead to
superior outcomes (although she did go on to question methodologies used in many such studies).
195

Glenn Duckworth
Other studies have questioned the usefulness of such accelerated courses with regards to levels of
higher order thinking and analysis skills, finding that students performed worse in assessments
requiring these higher order skills (e.g. Petrowsky 1996).
Although much research points to positive outcomes, there may be a number of threats to the
effectiveness of intensive courses. A common belief is that “some material and problems require an
extended period of time to ‘digest’ or ‘simmer.’ There is evidence in the psychological literature that
setting a problem aside for a while (a process labeled “incubation”) does help the learner and problem
solver.” (Tatum 2010). Tatum goes on to tentatively state that “It would appear that for some courses
in some disciplines, an accelerated approach could be a disadvantage for the students because they
are not given sufficient time to ruminate over the course content or the homework problems.”
Another potential threat relates to stress and time pressures. A study by Petrowsky (1996) found that
students taking a two week long intensive course found it particularly stressful.
Davies (2006) when discussing studies by Conrad (1996) and Wlodkowski (2003) stated that certain
key attributes were necessary to achieve successful learning outcomes in an intensive course and
these determined whether the experience was ‘painful and tedious’ or a good learning experience.
These characteristics include high levels of classroom interaction, student input, class discussion and
active learning. Scott (2003) investigated the student experience with a comparison of traditionally
scheduled and intensive formats using a range of qualitative methods. She also found that intensive
courses could provide useful benefits as long as certain key characteristics were present. In terms of
teaching methods, she found that active learning, high levels of classroom interaction/discussion and
experiential/applied learning were valuable characteristics.
The pre-workshop activities which are discussed in this case study may go some way to address
some of the concerns. The provision of a longer time period in which students may engage in the
subject area may aid incubation. It was also anticipated that the resources may reduce stress and
increase confidence due to some exposure to the topics. And the provision of structured introductory
tasks and readings would help students to get to grips with some of the key concepts and ideas
before entering the classroom; thus freeing up more time for higher level discussions, debates and
active learning.
While the EMBA has always put an emphasis on such high level teaching methods, it was anticipated
that the pre-workshop components would enhance teaching and learning by priming students to the
subject matter; thus allowing students and lecturers to ‘hit the ground running’ and allowing such high
level teaching and learning to start from an early stage of the intensive block of classroom based
sessions.
2. The students
All of the students involved were taking the Executive MBA course at the University of the West of
England between January 2010 and March 2011. All had at least three years work experience with
management responsibilities. They were all mature students and most of them were in full time
employment.
Evaluations were conducted with three separate groups of students. For each of these groups
approximately twenty students were enrolled on the module.
3. Pre-workshop activities
Pre-workshop activities and resources were developed for a range of modules on the EMBA
programme. The modules discussed in this case study are called ‘Management in a Complex World’
(mainly Economics), ‘Managing Decisions’ (Finance/Accounting) and ‘Leadership and Change’.
The resources were generally delivered via the Blackboard virtual learning environment (VLE). They
were made available to students one month before their first scheduled classroom based session.
A short video was included for each module in which the module leader would introduce the subject
and the pre-workshop component. The activities themselves comprised a varied set of tasks which
accompanied key readings and aimed to structure the introductory reading, providing an insight into
196

Glenn Duckworth
some of the key concepts involved with the subject matter. There follow a few examples of tasks used
for pre-workshop activities:
Glossary Exercises: A version of this was deployed on all three of the modules with content tailored to
each module. It is an interactive exercise, developed in-house and delivered online. The first part
involves matching key concepts to definitions. The second part requires the students to recall the
definition and attempt to enhance the short definition provided in part one. As well as providing
computer based marking for part one, it encourages self assessment in part two. It also aims to
encourage students to read key sections of relevant texts.
Finance Resources: This was used on the ‘Managing Decisions’ finance module and is made up of
online mini lectures and exercises designed to teach the basics of financial reporting. Students are
encouraged to work through the exercises building up three key financial reports for a fictional
business (statement of cash flows, statement of comprehensive income and statement of financial
position). The original version of the resources comprised mostly interactive exercises with directions
to read key chapters in text books. The resources were modified for the next group of students taking
the module. This version included online mini lectures and more background information which
introduced the key concepts, as well as some tweaking of the exercises based on student feedback.
Online communication, blogs and wikis: Students were encouraged to communicate with each other
online using tools such as discussion boards. A social networking web site was also set up for them
to use. Students were encouraged to post profiles to help familiarise themselves with each other and
to aid social cohesion within the group. Blogs and wikis were deployed on certain modules. A typical
example might be where students were asked to choose a key topic and write a short piece on that
subject.
4. Evaluations
Evaluations were centered around student views on the usefulness of the pre-workshop activities and
resources.
For the three modules identified students were asked to fill in a questionnaire for each module,
comprising likert-type questions and more open ended questions. Informal group discussions were
also conducted to elicit their thoughts.
The student evaluations were concerned with the pre-workshop resources and activities in general,
but also went into some detail about specific components – most notably the glossary exercise and
the finance resources/exercises.
Views of the academic staff teaching on the modules were also sought. Module leaders were
interviewed to assess their views on the usefulness of the pre-workshop activities.
5. Findings
This section will examine the findings from the questionnaires, group discussions and interviews with
lecturers. It will begin by looking at the pre-workshop components as a whole and will then go on to
look at two aspects in more detail – the glossary exercises and the finance resources/exercises. For
each sub-section a set of quotes from students will be included to provide some insight into student
views. Sections from the interview transcripts of the lecturers will also be included.
6. The pre-workshop activities in general
The majority of students on all three modules did engage with the online resources and tasks. A few
students did comment on the potential for these tasks to cause ‘overloading’ but on balance they were
seen as useful preparation for the modules.
A recurring theme was the notion that the activities forced them to prepare. Students on the EMBA
had always been encouraged to read key texts before beginning the module, but with the introduction
of these structured online tasks students were able to assess their understanding of the reading and
learn more actively. From informal group discussions it became clear that they thought the tasks had
helped them to structure their reading and attempt to understand the issues and concepts rather than
197

Glenn Duckworth
merely skim read the texts. The notion of forcing them to prepare was seen as being an extremely
positive factor for these groups of students.
They also stated that the activities helped to ease them into the academic subject areas and helped to
build confidence and reduce stress. This was particularly apparent with the Management in a
Complex World and Managing Decisions modules. Management in a Complex World was the first
module that the cohort enrolled upon in their MBA studies, and a large percentage of them had been
away from higher education for many years. They indicated that the pre-workshop activities helped
them to re-acclimatise to the world of academia and helped to build confidence. Managing Decisions,
a module dealing with finance and accounting has always been treated with trepidation by students
who are not confident with numbers and accounting. The students reported that the resources and
exercises helped to build confidence and reduce their worries about embarking on the module.
For one of the early modules there were some technical/administrative issues relating to enrolment
which led to some students not being able to access Blackboard in advance of the classroom based
sessions. This was overcome by e-mailing the tasks to these students. Subsequently, the majority of
them did engage in the activities. The technical/administrative issues have now been resolved and all
students have access to modules a month in advance.
A sample of student comments:
“Really forces you to do the pre-reading for the module.”
“It forces you to prepare. Therefore the module achieves more.”
“Overall it was good as it forced me to read.”
“Enabled me to get into the zone, focusing on key learning.”
“Very useful - especially for getting you to do pre-reading. Needs to be in Blackboard
plenty of time before module date.”
“Very handy as it gives/offers pre-module insight.”
“I completed the tasks sent by e-mail and they were very helpful.”
The lecturers also found the pre-workshop activities to be very beneficial:
“I found them incredibly useful… we’ve had three cohorts going through using this
system and each time it seems to work extremely well. The students thoroughly enjoy
being exposed to the subject before they get here. In a glossary sense they learn the
terminology which normally takes me weeks to get across to them. And it’s whetted their
appetite to follow through some of these things rather more because there’s a
combination of learning the language of Economics and also being able to start thinking
about the theories and the concepts that we explore early on. And they’re doing that
even before they arrive on the campus. I’m finding in the early seminars and discussion
groups a much greater depth of knowledge than I’ve previously seen in students like that.
So I have found it incredibly useful.” – Module leader for Management in a Complex
World.
The lecturers believed that the activities enabled students to prepare for the classroom based
sessions and therefore more could be achieved in class:
“…I was able to teach from a point of view where I knew they had already been
introduced to basic terms. Very often with this sort of subject you have some students
that actually say ‘I know nothing’ about this and almost sit there with a dazed expression
for quite some time until they … get eased into it. Whereas I think with this they had
already been eased into it.” – Module leader for Managing Decisions.
“They usually tell you something towards the end of the module, not at the beginning. At
the beginning I think they find it an additional burden… at the end they keep coming up
and saying it was so useful being able to understand what you’re talking about without
you having to keep explaining the terminology, the theories or the concepts. So it’s a
really good foundation for me to build on more quickly than I normally would with a
module.” - Module leader for Management in a Complex World.
“I was quite taken aback by the fact that they knew more about the topics I was
discussing with them than previous cohorts… They already had some information at a
198

Glenn Duckworth
very basic level which is crucial for making that first quite big step into the theory… With
Economics being the biggest part (of the module) this helps them enormously. It’s the
first thing they do, it’s sixty percent of the module… it’s a much easier entry than it would
have been without it.” - Module leader for Management in a Complex World.
The lecturers also believed that the exposure to the subject via the pre-workshop activities helped
with student confidence and may have relieved some of the stress and anxiety associated with
beginning the intensive courses:
“I think confidence is crucial. Taking on the MBA is a big challenge for anybody. They are
working as well… so they’ve got continual pressure from that direction. Anything which
eases the stress or burden will build that confidence. And I’ve found a remarkable
difference between the way they used to be before we started this process and the way
they are now. It’s a really valuable initial first step. And for them I think they’ve got a …
safety curtain. They know that if they understand these things then they are well placed
to follow the course in for them, what is a very demanding post-graduate area.” - Module
leader for Management in a Complex World.
“It maybe alleviated some of their concerns about an accounting module. Before, I had to
try and do that in the classroom… so basic accounting terms were introduced to them in
a way that they probably thought ‘this isn’t that complicated really’.“ – Module Leader for
Managing Decisions.
7. The glossary exercise
The glossary exercise was seen by students and staff as being a very useful resource. Students were asked
to indicate whether they found the task to be useful in their learning. With the phrase “I thought the task was
useful”, 95% of respondents indicated that they agreed or strongly agreed (detailed figures are included in
table 1).
The majority of the students on the three modules found the tasks interesting and engaging and most of them
also indicated that the glossary exercise helped them to grasp key concepts.
Table 1: Student views on the glossary exercise (all figures rounded to 1 decimal place)
Phrase Strongly Disagree Neutral Agree Strongly
disagree
agree
“The task was useful” 0% 5.3% 0% 44.7% 50%
“Completing the task helped me grasp key concepts” 0% 3.4% 17.2% 41.3% 37.9%
“The task was interesting and engaging” 2.3% 7.1% 11.9% 42.9% 35.7%
They stated that the glossary exercise gave them a method to assess their understanding of the
reading materials and “forced us to do the reading” – which was seen as a good thing by all who took
part in the exercise.
A sample of student comments:
“Very brilliant introduction to the module and sets the tone for what the module entails.”
“Helps to reflect on one's understanding.”
“Some of the descriptions were so similar that you had to understand not just to have
speed read.”
“It had drag and drop and write your own – very challenging.”
“Helped embed terminology.”
“Very useful as a warm up.”
“Good and welcome initiation to any module. Sets the tone and gives a brief insight into
what the module will entail.”
Students were asked whether they would like the glossary exercise to be incorporated into other preworkshop
components of future modules (with content tailored for that module). Every respondent
said that they would like to use it again, and quite surprisingly, more than half of them (60%) said that
they would like it to be incorporated into every module.
199

8. Finance resources
Glenn Duckworth
The finance resources have been deployed on two iterations of the Managing Decisions module. For
the first iteration an early version of the interactive exercises was used.
The students found the exercises to be generally useful, but did highlight some issues. Many of them
found the exercises to be very difficult and demanding. They suggested some alterations to the
exercises.
“Great resource - thanks. A little tweaking to improve but key ingredients in place.”
“Good and useful.”
“Thanks - a great resource.”
“ Time consuming. Some useful prompts would be good.”
“Got stuck and couldn't move on to next stage. Would be helpful to get answers/hints/tips
after certain number of attempts.”
“Needed additional hints/help as unable to finish and didn't know where I was going
wrong.”
“Get answers, or hints after a certain number of attempts in order to move on to next
step.”
Having discussed the exercises with the students, the module leader also believed that the exercises
were too difficult as first exposure to the subject matter:
“The original group is where we only did the exercises without any introduction and I
think those students really struggled with that. The feedback was that it took a long time
for them to do and they got to a point where they got stuck and couldn’t go forward.”
Based on feedback, the exercises were amended for the next iteration of the module. Extra prompts
were added and hints were incorporated. These appeared when the student provided an incorrect
answer on multiple occasions. Extra resources were provided in the form of online lectures and extra
background information which covered the main points.
The second version of the resources received very positive feedback. They were seen as being very
useful, interesting and engaging, and provided a useful introduction to the module.
Recurring themes from open ended questions were that students liked the fact that the lectures and
exercises were provided online and would like more resources like this.
All respondents indicated that they felt better prepared for the classroom based sessions and that
they found the resources useful.
The quotes below relate to aspects that they particularly liked about the resources:
“Work through of statements was a particularly useful and applicable exercise.”
“Relevance to programme.”
“That they were online and I could access any time.”
“Well structured, good framework.”
“Opportunity to retry.”
“The ability to stop & start.”
“Provided much needed ‘face’ contact.”
“Very useful. Excellent context setter and good insight into what to expect.”
“Make more of them!”
“Generally a very good use of Blackboard.”
The module leader also believed that the second iteration of the finance resources/exercises worked
well and that their incorporation into the pre-workshop component was very useful:
200

Glenn Duckworth
“… I think the second set of exercises that we did where we actually introduced basic
terms - I think that was a lot better for the students. I think the way that the exercises
introduced the module to the students … was to say this is a module which can be very
nerve wracking…. They may not be very au fait with numbers and things like that. And
rather than reading a text book and not really understanding things, I think the
introduction of the basic concepts was very good for them and that really helped them …
(they) got the general basic understanding, were able to do the exercises and this really
started the group off from a very different position. And perhaps that eased them into
reading the textbook.”
9. Other aspects of the pre-workshop components
As well as the glossary exercise and the finance resources, a range of resources and activities were
incorporated.
The introductory videos were generally seen as being useful, as long as they provided useful
information and that information was provided in written form as well.
“Very useful.”
“Some info was only on video. Would have been useful if given written version also to
prevent having to re-run video to pick up elements later.”
“Could use brief repetition of required tasks at close of video.”
“It made it more personable.”
“Could replay to understand.”
“Good. Not too long. Just about right in terms of time.”
Online communication was encouraged through various means. The use of the social
networking/profiles site in advance of the first module was generally seen as useful for group
cohesion. Some students did indicate that it was somewhat time consuming to set up but most
indicated that they found it to be a useful exercise and that they enjoyed learning about their fellow
MBA peers from the profiles.
“Great idea.”
“Helped each other to remember names”
“Took a good couple of hours to set up.”
“Useful to find out about other students - although noticed that not all posted info.”
The use of other communication tools such as discussion boards was somewhat varied. For example,
on the Managing Decisions finance module, the first group of students made use of discussion boards
to discuss issues they were having with completion of the exercises. The second group, however, did
not post any messages at all. When asked about the lack of activity on the discussion board, some
students stated that they did not feel the need to use it (“No need - no questions to ask”) or
commented on the lack of posts from others in the cohort (“No reason to - nobody else was using it”).
Blogs and wikis were used with some success. Most of the students engaged in these activities and
they did produce some online discussion.
10. Conclusions
The process of priming via online pre-workshop activities was seen as being valuable by both
students and lecturers.
There was some concern that students would not engage with the online activities due to time and
work pressures. But this has not been the case, and the majority of students did indeed make use of
the resources and activities before the classroom teaching began. Many of the exercises helped
students to structure their reading and forced them to do the appropriate reading rather than merely
skim read the suggested texts. They also provided a method for students to evaluate their
understanding of the key concepts before even entering the classroom.
It seems that the activities did help students and staff to ‘hit the ground running’ and they could begin
engaging in high level discussion of concepts from day one of the classroom based sessions. This
201

Glenn Duckworth
was seen as being very important on intensive courses. Tutors also perceived higher levels of
knowledge and confidence and less stress than previous cohorts of students who had not been
provided with the same resources and activities.
Whether this type of initiative would be useful to other student groups is up for debate. As Daniel
(2000) points out, students who opt to take intensive courses tend to be older and are more motivated
than traditional students. The composition of cohorts on the EMBA at the University of the West of
England does seem to fit this description. It may be that other intensive courses - particularly those
with cohorts with these characteristics - would benefit from a similar pre-workshop initiative. This may
include other areas of post-graduate studies or continuing professional development (CPD) courses.
The longer term benefits of the initiative are also not yet known. This case study has drawn together
findings from three discreet evaluations for three modules. All of the evaluations were conducted
during the classroom based component, not long after the students had worked through the preworkshop
activities. Planned future research will examine the perceptions of a cohort at the end of
their studies – looking at the their views of the initiative as a whole.
To conclude, from the viewpoint of one of the lecturers…
“It’s a very enjoyable experience. It’s completely unique for me… I’ve never been part of
a process like this. I’ve thoroughly enjoyed doing it. I’ve seen it work and I recommend it
to anybody to use.”
References
Conrad, P. A. (1996) Attributes of high-quality intensive course learning experiences: Students' voices and
experiences. College Student Journal, 30, pp 69-77.
Daniel, E. L. (2000) “A review of time-shortened courses across disciplines”, College Student Journal, 34, pp
298-308.
Davies, W.M. (2006) “Intensive teaching formats: A review”, Issues In Educational Research, Vol. 16, pp 1-20.
Kasworm, C. (2001) Adult learner experiences of an accelerated degree program. Paper presented at the Adult
Education Research Conference, Michigan State University.
Petrowsky, M. C. (1996) The two-week summer macroeconomics course: Success or failure?, Glendale
Community College.
Scott, P.A. (2003) Attributes of High-Quality Intensive Courses, New Directions for Adult and Continuing
Education, 97, pp 29-38.
Tatum, B.C. (2010) Accelerated Education: Learning on the Fast Track, Journal of Research in Innovative
Teaching, Volume 3, Issue 1, pp 34-50.
Wlodkowski, R. J. (2003) Accelerated learning in colleges and universities, New Directions for Adult and
Continuing Education, 97, pp 5-15.
202

Computer-Mediated Reading and its Impact on Learners’
Reading Comprehension Skills
Francisco Perlas Dumanig, Maya Khemlani David and Rodney Jubilado
University of Malaya, Kuala Lumpur, Malaysia
fdumanig@yahoo.com
mayadavid@yahoo.com
rcjubilado@yahoo.com
Abstract: Teaching reading is not an easy skill as most students today easily get bored of reading written texts in
books and other traditional reading materials. With the emergence of technology and its fast development,
educators have integrated the use of computers in teaching reading. Due to technological breakthrough, a study
on computer-mediated reading and its impact on the reading comprehension skills of learners of English as a
Second Language (ESL) is discussed. This study uses an experimental research design to determine the impact
of computer-mediated reading on students’ reading comprehension skills. A total of 140 freshmen engineering
students at Mapua Institute of Technology, Philippines took part in the study. To carry out the study, the
participants were divided into an experimental and a control group. The findings of the study reveal that the use
of computer-mediated reading results in a significant improvement in students’ reading comprehension skills. The
findings are supported by statistical computations using the independent t-test in determining the significant
difference in the pre and posttest results of the two groups. After exposing them to the treatment significant
learning is acquired by those in the experimental group as compared to those in the control group. The study
shows that computer-mediated reading creates a significant impact on students’ reading comprehension skills
specifically in following directions, noting details, sequencing events, getting the main idea, making inferences
and making generalizations. On account of the significant findings of the study, it is recommended that teachers
should use computers in the reading classroom.
Keywords: computer-mediated reading, reading comprehension skills, English as a Second Language (ESL),
eLearning, language teaching
1. Introduction
Reading plays a very crucial role in man’s life as reading influences one’s way of thinking, which
consequently affects one’s total development as a person. Although man’s development is influenced
by many factors, learning is obtained through reading. Consequently, individuals must find ways how
to develop and improve their reading comprehension skills. Integrating new technologies when
teaching reading can help both the teacher and learner. Technology helps to restructure and improve
the classrooms with authentic activities that will enhance collaboration. It is also an effective tool for
interpretative skills, information management and open inquiry. Consequently, it provides an excellent
avenue for student motivation, exploration and instruction in a multi sensory diverse world. However, it
must be emphasized that using technology for the teaching of reading must be complemented with
teachers who have the appropriate pedagogical skills as technology is only a tool. The challenge rests
with the educators to effectively integrate it in appropriate places through out the curriculum (Wright,
Wilson, Gordon, & Stallworth 2002; Barron & Orwig, 1995).
Before integrating technology in teaching, it is essential for educators to examine first the students’
needs. For instance in applying computer-mediated instruction in developing the reading
comprehension skills of students, teachers should be aware primarily of their students’ level of
reading comprehension skills. This will help teachers to use strategies and think of ideas as to how
technology can be integrated in their teaching. Studies show that sending English teachers to
seminars, workshops and to pursue post graduate degrees help to enhance their competence in
teaching the subject (Dumanig, 2005). Attending such training courses expose teachers to new
strategies particularly in computer-mediated teaching. Anisawal (1993) found that difficulties in
different skills of intermediate students were caused by physical factors and teachers’ lack of
knowledge of the computer. She suggested that sending teachers for training and integrating
technology in teaching would help to improve students’ reading comprehension skills. Teachers must
be wise in understanding their role and use of technology when providing a technologically advanced
learning environment.
The integration of technology in teaching brings benefits to students. Traynor (2003) argues that
learning by using electronic communication technology helps students develop and enhance their
knowledge and discover new things they might have not thought of without the aid of technology.
203

Francisco Perlas Dumanig et al.
Students can come to understand technology while technological tools help them tap into real
experiences, fantasies and visions. Min Jung Jee (2011) explains that the use of technology
enhances instruction specifically in the field of language instruction. However, the exploration of new
technologies for teaching and learning requires time as well as a change in attitude among teachers
and students so that they are able to properly use and apply these tools in teaching. Currently, some
teachers who are not computer literate find it difficult to use computers in their respective classes.
Although computers and multimedia instructional technology are now available in many campuses,
the percentage of teachers using them remains low (Cuban, 2003). It is therefore necessary to equip
teachers with skills in using CAI to facilitate classroom learning efficiently and effectively.
Studies on the use of Computer Assisted Instruction (CAI) have identified effective learning outcomes
among college students and adult learners (Rouse, 2007). It is believed that CAI appeals to varied
learning modalities and consequently meets the diverse needs of every student. With CAI students
can learn at a comfortable pace and interact directly and continually with computers that provide
immediate feedback and positive learning effects (Mustafa, Ashhan, and Turgay, 2011). As a result,
many educational institutions today are investing much in technology to enhance students’ learning.
In addition, CAI helps students learn many specific skills at various levels, such as; elementary,
secondary, college and adult education. CAI has positive effects on the achievement and
development of students in different skills (Liao, 2007). In fact, the use of any computer software
helps to enhance language teaching and learning (Sawhill, 2008). Computers play a valuable role in
focusing students’ attention, presenting content and facilitating students’ understanding in a lecture
(Rap, 2006). Computer programs designed to present information on a screen to students using an
electronic blackboard allows an instructor to present similar kind of information as when he uses
handouts or overhead transparencies. This shows that using CAI does not really deviate from the
traditional approach of teaching, rather it modernizes traditional method using modern technology,
and this makes learning easier The use of technology in teaching could be adapted to the type of
activity to be learned .
Cayanan (1998) argues that using computer assisted instruction is significant since it produces a
positive effect on students’ learning capacity. It helps students to progress in accordance with their
level of understanding for they learn at their own pace thus making their learning gradual yet effective.
With the favorable effects of CAI to students’ learning capabilities, Parker (1996) suggests that there
should be improvements to be made in tertiary education regarding the availability and use of
technology in instruction and teacher preparation. It is believed that the use of computer-assisted
instruction will offer a big change and development in improving the quality of education in many
schools around the world.
Due to the fast development of computers the use of computer mediated instruction motivated the
researchers to conduct a study on computer-mediated reading and its impact on learners’ reading
comprehension skills. This study primarily examines the effects of computer mediated reading on
students’ reading comprehension skills such as following simple directions, noting details, sequencing
events, getting the main idea, making inferences and making generalizations.
2. Methodology
This study used a descriptive and experimental research design to determine the effects of computer
assisted instructional devices in teaching reading. These methods were useful in determining the
students’ reading comprehension through the use of computer assisted instructional devices.
The subjects of this study were all freshmen engineering students from Mapua Institute of Technology
(MIT), Manila, Philippines. Four English classes of freshmen engineering students were chosen. The
first two classes were classified as the control group and the other two classes were classified as the
experimental group. The classification of both groups was based on age, gender, computer literacy
and English language proficiency.
The subjects were given two sets of questionnaires, the first set focused on the students’ socio
demographic data to ascertain the students’ age, gender, computer literacy and English proficiency.
The second set of questionnaires focuses on students’ reading comprehension skills. The
questionnaire consists of questions on following simple directions, noting details, sequencing events,
getting the main idea, making inferences and making generalizations. Both sets of questionnaires
204

Francisco Perlas Dumanig,
were distributed to the subjects in the control group and experimental group twice; both in the pre and
posttests. The subjects in the control and experimental groups were given the pretest before they
were exposed to the treatment. After the pretest the subjects in the control group were exposed to the
traditional approach while those in the experimental group were exposed to the use of computer in
teaching reading. The study was carried out in four sessions over four weeks. Each session lasted 1
and a half hours. At the first meeting, the pretest was administered while during the second and third
meetings the actual teaching of reading was conducted and the post test was conducted during the
last meeting.
To validate the findings, an independent t-test was used to determine the significant difference
between the performance of the control group and the experimental group in the pre and post tests. In
addition, a correlated t-test was also used to determine the significant difference between the pre and
posttest scores of both groups.
3. Results
The results of the study are presented in tables to clearly illustrate the reading comprehension skills of
the control and experimental groups in the pre and posttests.
3.1 Pretest reading comprehension level of the control and experimental groups
Table 1 presents a summary of the pretest result of the reading comprehension skills of subjects in
both the control and experimental groups.
Table 1: Pretest level of reading comprehension of the control group and experimental group
Reading Skills No. of Experimental Group
Control Group
Items
Mean Description
Mean Description
Following Simple Directions 5 4.57 Excellent 4.61 Excellent
Noting Details 5 4.29 Good 4.34 Good
Sequencing Events 7 5.77 Very Good 5.36 Meritorious
Getting the Main Idea 5 3.37 Satisfactory 4.33 Very Good
Making Inferences 10 5.59 Satisfactory 4.83 Passing
Making Generalization 3 2.20 Very Satisfactory 2.33 Very
Satisfactory
Table 1 shows the pretest mean of the experimental group’s reading comprehension skill is 4.57 rated
as “excellent” in following directions, 4.29 rated as “good” in noting details, 5.77 rated as “very good”
in sequencing events, 3.37 rated as “satisfactory” in getting the main idea, 5.59 rated as “satisfactory”
in making inferences and 2.20 rated as “very satisfactory” in making generalizations. On the other
hand, the mean of the control group’s reading comprehension skill is 4.61 rated as “excellent” in
following directions, 4.34 rated as “good” in noting details, 5.36 rated as “meritorious” in sequencing
events, 4.33 rated as “good” in getting the main idea, 4.83 rated as “passing” in making inferences
and 2.33 rated as “very satisfactory” in making generalizations.
The results of both groups in different reading comprehension skills show that the subjects have
almost the same level of comprehension skills in the pretest result. The differences in reading
comprehension skills are seen to be minimal and do not create any significance. To measure whether
a significant difference exists in the pretest results, a test of difference is shown In 3.2.
3.2 Test of difference between the pretest scores of the experimental group and
control group
The pretest results of the control and experimental groups are vital in this research. Although, the two
groups of subjects were equated in age, gender, computer literacy and English language proficiency,
the null hypothesis of no significant difference between the score of the experimental group and
control group must be established before the treatment is administered to the experimental group.
Table 2 shows the comparison of the data from the two groups.
The computed t-value is 0.40 which is less than the tabular value of 1.96, therefore, at five percent
level the null hypothesis is accepted. This implies that no significant difference exists between the
pretest mean scores of both groups. Having no significant difference in the scores of the two groups
205

Francisco Perlas Dumanig et al.
in the pretest will help in analyzing if there is a significant difference after exposing the subjects to the
treatment.
Table 2: Comparison of the pretest scores of the experimental group and the control group
Subject Mean S.D. Computed Tabulated Decision Interpretation
t value
t value
Experimental
Group
Control
Group
25.72
25.96
3.51
3.55
0.40
1.96
Accept Ho
Not
Significant
3.3 Level of performance of the experimental and control groups in the posttest
At the end of the experimental period, a posttest was given to the experimental and control groups.
The results are shown in Table 3.
Table 3: Posttest scores of the experimental and control groups
Experimental
Group
Control Group
Score Description f % Score Description f %
32-33 Superior 18 25.71 34-35 Excellent 1 1.43
30-31 Very Good 22 31.43 32-33 Superior 8 11.4
27-29 Good 19 27.14 30-31 Very Good 10 14.2
25-26 Meritorious 9 12.86 27-29 Good 22 31.4
23-24 Very Satisfactory 2 2.86 25-26 Meritorious 17 24.2
23-24 Very satisfactory 10 14.2
21-22 Satisfactory 2 2.86
Total 70 100.00 Total 70 100.00
Mean = 29.49 (Good) Mean= 27.53 (Good)
Standard Deviation = 2.53 Standard Deviation= 3.06
Table 3 shows the percentage scores in the posttest of the experimental and control groups. Twenty
five point seventy one percent (25.71%) of the experimental group is rated as “superior”, in reading
comprehension 31.4 % rated as “very good”, 27.14 % rated as “good”, 12.86 % rated as “meritorious”
and 2.86 % rated as “very satisfactory.” Furthermore, the findings show a standard deviation of 2.53
and a mean of 29.49 rated as “good.” The data tend to show that majority of the subjects are rated
“very good” for they obtain 31.43% and only 2.86 % rated as “very satisfactory”. It is clear then that
the use of computer-mediated instruction is significantly effective in teaching reading particularly in
developing the specific reading comprehension skills. This confirms the findings of (Traynor 2003) that
the process of accumulating knowledge about teaching will be greatly enhanced by technology.
On the other hand, the posttest scores of the control group reveal that 1.43% of the subjects rated as
“excellent”, 11.43% rated as “superior”, 14.29% rated as “very good”, 31.43% rated as “good”,
24.29% rated as “meritorious”, 14.29% rated as “very satisfactory” and 2.86% rated as “satisfactory”,
thus obtaining a mean of 27.53 rated as “good” and a standard deviation of 3.06. Furthermore, the
table shows that the majority of the subjects are rated as “good” which is equivalent to 31.43%. On
the contrary, only 1.43% of the total subjects rated as “excellent.” This means that the improvement in
the reading comprehension skills in the control group is minimal as compared to the experimental
group.
3.4 Posttest reading comprehension level of the control and experimental groups
Table 4 shows the post test result of the experimental group having a mean of 4.93 rated as
“excellent” in following directions, 4.63 rated as “excellent” in noting details, 6.50 rated as “very good”
in sequencing events, 3.94 rated as “good” in getting the main idea, 6.90 rated as “very satisfactory”
in making inferences, 2.79 rated as “excellent” in making generalizations. On the other hand, the
control group obtains a mean of 4.77 rated as “excellent” in following directions, 4.43 rated as “good”
in noting details, 6.44 rated as “very good” in sequencing events, 4.43 rated as “very good” in getting
206

Francisco Perlas Dumanig,
the main idea, 4.84 rated as “passing” in making inferences and 2.54 rated as “excellent” in making
generalizations.
Table 4: The posttest level of reading comprehension of the control and experimental groups
Reading Skills No. of Experimental group
Control Group
Following Simple Directions
Items
Mean Description
Mean Description
5
4.93 Excellent
4.77 Excellent
Noting Details 5 4.63 Excellent 4.43 Good
Sequencing Events
Getting the Main Idea
7 6.50 Very Good 6.44 Very Good
5
3.94 Good
4.43 Very Good
Making Inferences
Making Generalization
10 6.90 Very Satisfactory 4.84 Passing
3
2.79 Excellent
2.54 Excellent
Comparatively, in noting details, the subjects in the experimental group perform better by obtaining a
mean of 4.63 rated as “excellent” whereas the subjects in the control group obtain only a mean of
4.43 rated as “good”. Likewise, in making inferences The experimental group obtains a mean of 6.90
is rated as “very satisfactory” and a control mean of 4.84 rated as “passing”. In getting the main idea,
the control group obtains a mean of 4.43 rated as “very good” whereas the experimental group
obtains 3.94 rated as “good”. The findings reveal that the subjects in the control group perform better
in the pretest but less improvement is shown in the posttest. However, the subjects in the
experimental group show a significant improvement in the posttest for they obtain a mean of 4.33
rated as “very good”. The findings run consistently with the findings of (Mustafa, Ashhan, and Turgay,
2011) that the effectiveness of computer devices, also known as educational technology significantly
enhance student’s learning as compared to conventional instruction.
3.5 Test of difference in the pre and posttest scores of the experimental and control
groups
The experimental group was exposed to the use of computer-assisted instructional devices while the
control group was taught using the traditional method of teaching. The results of the pre and posttests
of both groups were compared to determine if the conventional method of teaching or computermediated
instruction is effective. The data are shown in Table 5.
Table 5: Test difference in the pre and posttest scores of the experimental and control groups
Measurement
Experimental
Group
Pretest Scores
Posttest Scores
Control Group
Pretest Scores
Posttest Scores
Mean
15.72
29.49
25.96
27.53
zd
2.59
92
Computed
t-value
13.95
4.87
Tabular
t-value
2.00
2.00
Decision
Reject Ho
Reject Ho
Interpretation
Significant
Significant
In the experimental group, the computed t-value of 13.95 is higher than the tabular value. Therefore,
the null hypothesis is rejected. This means that a significant difference exists between the pretest
scores and the posttest scores of the experimental group. If interpreted further, significant learning is
acquired by the members of the experimental group after they were exposed to the treatment. This
agrees with the findings of (Rouse, 2007) that using technology in teaching helps in improving the
reading difficulties of students and enhances their reading comprehension skills.
On the other hand, the control group shows the computed t-value of 4.87, which is higher than the
tabular value of 2.00, therefore, the null hypothesis is rejected. This means that there is a difference
between the pre and posttest scores of the control group. This means that significant learning is also
acquired by the members of the control group after exposure to the traditional approach in teaching
reading. However, the difference is less as compared to the experimental group.
207

Francisco Perlas Dumanig et al.
3.6 Test of difference in the posttest results of the experimental and the control
groups
Table 6 shows the comparison of the posttest results of the experimental and control groups. The
level of significance is set at 5 percent and the hypothesis is directional towards the performance of
the experimental group.
Table 6: Comparison of posttests scores of the experimental group and control groups
Subject
Experimental
Group
Control Group
Mean
29.49
27.53
SD
2.53
3.06
Computed
t-value
4.17
Tabular tvalue
(.05)
1.67
Decision
Reject Ho
Interpretation
Significant
From the table, it is seen that the computed value exceeds the tabular value. Therefore the null
hypothesis is rejected hence, the experimental group performs better than the control group in the
posttest. This means that the use of computer assisted instructional devices in teaching reading is
seen to be effective as compared to the traditional approach of teaching reading. CAI improves the
reading comprehension skills of students. We would however like to emphasise that this does not
mean discarding the traditional approach in teaching reading. The traditional or conventional
approach in teaching reading is still effective but the use of computer is seen to be more effective
since students showed improvement after they were exposed to the treatment. The use of computer
mediated instruction still shows an impact on students’ reading comprehension skills after being
exposed to the treatment. Such findings concur with the findings of (Mustafa, Ashhan, and Turgay,
2011) that the use of computer-mediated instruction in teaching produces a positive effect with
respect to the respondents’ learning capacity.
4. Discussion
This study is primarily conducted to determine the effects of computer-mediated reading and its
impact on learners’ reading comprehension skills by considering the different reading comprehension
skills of the freshmen engineering students at the Mapua Institute of Technology in the Philippines.
Two hypotheses were tested and the result shows that there is no significant difference in the pretest
result of the experimental and control groups but a significant difference exists in the posttest of the
experimental and control groups. However, the difference in the posttest of the control group is not as
much as that as compared to the experimental group. This shows that using the computer-mediated
instruction is beneficial to improving students’ reading comprehension skills.
The use of technology is seen to be effective since it enhances the students’ reading comprehension
skills such as following simple directions, noting details, sequencing events, getting the main idea,
making inferences and making generalization. Such skills are seen to be a problem to most readers of
English as a Second Language but when using the computer-mediated instruction such barriers
towards learning have been minimized. This is evident in the results shown in Tables 1-6. All this
information Francis did this come from qualitative interviews? Otherwise how can you vouch for
this??The use of interactive functions as students read the text may have contributed in the retention
of the message. The memory is triggered by the text that is manipulated by the readers themselves.
In addition, the feeling of being comfortable and free to manipulate the reading text allows the
students to minimize reading anxiety since they are in control of the reading text. Consequently, they
easily absorb and understand what they read.
It must be noted in this study that both computer-mediated instruction and the traditional approach in
teaching reading are both effective in teaching reading. However, the findings shown in Tables 5 and
6 reveal that the use of computer-mediated instruction obtain a much higher mean as compared to
the traditional approach in teaching reading. This only shows that integrating technology in teaching
provides more opportunities for students to develop their reading comprehension skills.
5. Conclusion
Based on the findings of the study, it can be concluded that the use of computer-mediated instruction
in teaching reading helps considerably in improving students’ reading comprehension skills such as,
following directions, noting details, sequencing events, getting the main idea, making inferences and
generalizations. The use of computer-mediated instruction is seen to be useful in teaching reading
208

Francisco Perlas Dumanig,
particularly for the new generation of students who are more exposed to the computer when getting
information. Perhaps boredom disappears when retrieving information using the computer.
Consequently, the use of computer mediated instruction in teaching reading provides positive learning
effects to students and creates a significant improvement on their reading comprehension skills.
Therefore, reading teachers should use computers as a strategy in enhancing their reading
comprehension skills. Perhaps, minimizing the use of the traditional approach and emphasizing the
integration of technology in teaching reading may produce more competent and skillful readers.
References
Anisawal, L. (1993) “Reading Difficulties of Intermediate Pupils of Bontoc and Salanga Districts”, Unpublished
Master’s Thesis, Baguio City University, Philippines.
Barron, A.E. and Orwig, G.W. (1995) New Technologies for Education 2 nd ed. Englewood, Colorado, Libraries
Unlimited Inc.
Cayanan, A. T. (1998) “The Efficacy of Using Computer-Assisted Instruction (CAI) as a Teaching Aid for College
Students of Philippine Women’s University”,Unpublished Master’s Thesis, Mapua Institute of Technology,
Manila.
Cuban, L. (2003) “Oversold and Underused: Computers in the Classroom”, Language Learning and Technology
Vol. 7, No. 3, pp 42-45.
Dumanig, F. (2005) “The Effects of Computer Assisted Instructional Devices in Teaching Reading”, In KMUTT
and AIAER, ICLORD 2005 Bangkok, pp 352-364.
Liao, Y. (2007) “Effects of Computer-Assisted Instruction on Students’ Achievement in Taiwan: A Meta-analysis’,
Computers & Education Vol. 48, No. 2, pp 216-233.
Min Jung Jee (2011) “Web 2.0 Technology Meets Mobile Assisted Language Learning”, The IALLT Journal Vol.
41, No 1, pp 161-175.
Mustafa, B., Ashhan, K., and Turgay, A. (2011) “The Effect of Computer Assisted Instruction with Simulation with
Science and Physics Activities on the Success of Students: Electric Current”, Eurasian Journal of Physics
and Chemistry Education Jan (Special Issue), pp 34-42.
Rapp, D. (2006) “The Value of Attention Aware Systems in Educational Settings”, Computers in Human Behavior
Vol. 22, pp 603–614.
Rouse, D. (2007) “Computer-Assisted Instruction: An Effective Instructional Method’, Teaching and Learning in
Nursing Vol. 2, No. 4, pp 138-143.
Sawhill, S. (2008) “The Changing Role of the Language Teacher/Technologist”, The IALLT Journal Vol. 40, No.
1, pp 1-17.
Stigler, J. W., and Hiebert, J. (1999) “The Teaching Gap”, New York, Free Press.
Traynor, P. (2003) “Effects of Computer-Assisted Instruction on Different Learners”, Journal of Instructional
Psychology, FindArticles.com. 20 July 2011.
http://findarticles.com/p/articles/mi_m0FCG/is_2_30/ai_105478983/
Wright, V. H., Wilson, E. K., Gordon, W., & Stallworth, J. B. (2002) “Master Technology Teacher: A Partnership
between Pre-service and Inservice Teachers and Teacher Educators”, Contemporary Issues in Technology
and Teacher Education [Online serial], 2(3).
Available:http://www.citejournal.org/vol2/iss3/currentpractice/article1.cfm
209

Do you see What I see? - Understanding the Challenges of
Colour-Blindness in Online Learning
Colin Egan, Amanda Jefferies, Edmund Dipple and David Smith
School of Computer Science, University of Hertfordshire, Hatfield UK
c.egan@herts.ac.uk
a.l.jefferies@herts.ac.uk
Abstract: In this paper we introduce the 'Hertfordshire Colour-blind Emulator' (HCBE) software application.
Currently, our aim is to raise awareness of the challenges that the colour-blind encounter on their learning
journey. HCBE emulates four major types of colour-blindness: protanopia, deuteranopia, tritanopia and
monochromacy. HCBE accepts an image file and outputs that image in the way that a colour-blind learner would
see the original inputted image. For any inputted images there are four options for outputted images, one for
each colour-blind type. Colour-blindness is often considered to be a mild disability where, on the whole, a colourblind
learner has developed his/her own “coping mechanisms” to avoid, but not to eliminate, problems in their
learning. The problems faced by some colour-blind users of online games have recently been highlighted in the
research, this paper identifies some of the issues for students in their learning. The Equality Act 2010 places the
responsibility of making reasonable adjustments to aid the disabled on the educational practitioner by providing
legal rights for disabled people, irrespective of the severity of the disability. We show that colour-blind learners do
have problems interpreting information when that information is presented as images. In particular with increasing
reliance on using VLEs (Virtual Learning Environments) as a repository for study materials then it is possible that
the challenges which colour-blind users suffer will be exacerbated. Estimates of the frequency of colour-blindness
show that there are approximately 8% males and about 0.4% females who are colour-blind. It is also our
experience that few practitioners are aware of the problems that colour-blindness can cause and even fewer
practitioners that make any reasonable adjustment as required by the UK’s Equality Act 2010.
Keywords: colour-blind, eLearning, online learning, disability, accessibility, Equality Act 2010
1. Introduction
Approximately 8% of males and 0.4% of females in the adult population are typically affected by
colour-blindness (BUPA, 2011). In Further and Higher Education where class sizes of 200 (and
larger) are not unusual, this means that in a class size of about 200 learners there could be up to 16
colour-blind male learners and maybe 1 female learner. As educational practitioners we are typically
using VLEs (Virtual Learning Environments) more and more in the daily delivery of our teaching
materials. Students who suffer from the various forms of colour-blindness may experience problems
when using computers, for example the use of a VLE may cause a colour-blind learner difficulties with
reading text and viewing or interpreting images (including graphs). The focus of this study is to
improve the learning experiences of colour-blind learners.
Colour-blindness is normally hereditary and includes a range of conditions where those conditions are
characterised by an inability to see certain colours in the normal spectrum, currently there is no known
cure. There are four major types of colour-blindness: protanopia, deuteranopia, tritanopia and
monochromacy. Most people are aware of the red-green issues for colour-blind users, but this is only
a more common form of the spectrum of effects. It is generally considered to be a mild disability.
However, in the UK, the Equality Act (2010) (which replaces the Disability Discrimination Acts (1995)
and (2005) provides legal rights for all disabled people irrespective of the severity of the disability. As
educational practitioners it is now our responsibility to comply with the Equality Act 2010 and to
provide reasonable adjustments to our educational materials so that colour-blind learners can be
provided with the same opportunity to learn as those who not colour-blind.
In this paper, we introduce the use of a colour-blind software emulator, the Hertfordshire Colour-blind
Emulator (HCBE) which displays coloured images in the same way that colour-blind users would see
them. The aim of HCBE is to raise awareness of some of the problems that the colour-blind encounter
on their learning journey and to identify how we as practitioners can help such learners by improving
their educational journey. The software allows the passing of an image into the HCBE application and
then the HCBE outputs coloured images which indicate how a colour-blind learner would see the
same image. We then ask the practitioner a simple question “Do the new images convey the same
message to a colour-blind learner as it would to non colour-blind learners?” Where the answer is yes,
then the practitioner does not need to make any changes as the requirements of the learner and the
Equality Act have both been satisfied. In the alternate case, then the image should be amended to
210

Colin Egan et al.
make reasonable adjustment to the needs of the colour-blind learner and to comply with the Equality
Act. The solution may in fact be to redesign the image with a simple separation of colours or to effect
a simple change of colour schema. Recent research into online gaming has already highlighted the
problems that colour-blind gamers have in differentiating quickly enough between for example friends
and foes in some of the more popular wargames (BBC, 2011)
2. Colour-blindness
Colour-blindness has been defined as “a range of conditions that are characterised by an inability to
see certain colours” (BUPA, 2011). However, colour-blindness can be manifested in many different
ways but there is four most prominent types, which we consider in our study, protanopia,
deuteranopia, tritanopia and monochromacy. To understand how colour-blindness affects how a
person perceives colour, it is important to understand the biology of how the human eye normally
perceives colour. A human retina is comprised of rod and cone cells. Rod cells become active in low
light conditions, and determine the brightness of the image that the brain receives. It is the cone cells
that are affected by colour blindness. The “normal” eye contains 3 variants of cone cells: L, M and S.
The L-cone detects long wavelength light (yellow-red), the M-cone detects medium wavelengths
(green) and the S-cone detect short wavelength light (Blue) (de Paula,2006)). The human brain will
determine the colour of an image based on the ratio between the signals from L, M and S. Colourblindness
occurs when one or more of these cone cells types are either damaged or absent (colourblindness.com,
2011).
The outcome of personal discussions with a number of colour-blind learners has revealed that many
learners have developed their own “coping mechanisms” over their learning journey from the early
days of their schooling. “Coping mechanism” may help to reduce the impact of their disability, but it
must be emphasised that such they only reduce and do not eliminate the problems of teaching to the
colour-blind learners. It also highlights that, on the whole, practitioners are not currently making
reasonable adjustments with their teaching material to cater for colour-blind learners.
2.1 Protanopia
Protanopia is manifested as red-green colour-blindness, which is caused by a complete lack of retinal
photoreceptor cells. It has been estimated that approximately 1% of males are affected by this
condition ( Cassin, 1990). Figures 1 a) and b), show a pie-chart that compares normal vision with
protanopian colour-blindness.
Figure 1b) shows a distinct difference to Figure 1a), where for example in red in Figure 1a) appears
as black in Figure 1b), orange in Figure 1a) appears as a green hue in Figure 1b) and so on. If this
pie-chart had been presented as educational material then it is clear that the protanopian colour-blind
person would have difficulty interpreting the data. Furthermore, if the orange and green in Figure 1a)
had been aligned adjacently then the protanopic colour-blind learner would not have been able to
make any differentiation between the two colours. This can also be seen with the blue, and violet of
Figure 1a) and also the indigo and red of Figure 1a). On close examination it can be seen that the red
and green are indistinguishable, and in this example the effect seems to spill over to encompassing
yellow as well.
Figure 1a: Normal vision Figure 1b: Protanopia
211

2.2 Deuteranopia
Colin Egan et al.
Deuteranopia is similar to protoanopia, in that red and green are almost indistinguishable. However,
the cause of the condition is slightly different, in that it is the green receptors in the eye that are
missing [6]. For all intents and purposes the effect is the same as protanopia, although the contrast
between blue and purple seems to be affected to a greater degree by deuteranopia. Figure 2a) is the
same pie-chart as Figure 1a), but Figure 2b) is the same pie-chart but it represents deuteranopia.
Similarly to Figures 1a) and b) Figure 2a) shows a distinct difference from Figure 2b). Following the
same example, red in Figure 2a) appears as black in Figure 2b), the orange in Figure 2a) appears as
a green hue in Figure 2b) and so on. As with Figure 1, if this pie-chart had been presented as
educational material then it is clear that the deuteranopian colour-blind person would have difficulty
interpreting the data.
Figure 2a: Normal vision Figure 2b: Deuteranopia
Furthermore if the orange and green in Figure 2a) had been aligned adjacently then the deuteranopic
colour-blind learner would not have been able to make any differentiation between the two colours.
This can also be seen with the blue, and violet of Figure 2a) and also the indigo and red of Figure 2a).
Again, on close examination it can be seen that that for the colour-blind learners red and green are
indistinguishable.
2.3 Tritanopia
Tritanopia colour-blindness is manifested with a blue-yellow variation due to the lack of blue
photoreceptors. It is not present as frequently as protanopia and deuteranopia [6]. As seen above,
Figure 3a) is the same pie-chart as Figures 1a) and Figure 2a), but Figure 3b) shows the same
coloured pie-chart as those suffering from tritanopia would see it.
Figure 3a: Normal vision Figure 3b: Tritanopia
212

Colin Egan et al.
There are clear differences between normal vision, protanopia vision, deuteranopia vision and
tritanopia vision. Those suffering from tritanopia can distinguish red as shown in Figure 3a) and 3b),
whereas those suffering from protanopia or deuteranopia cannot. However, there are distinct
differences between orange, yellow, green, blue, indigo and violet. On close observation of Figure 3a)
and Figure 3b) it can be seen that blue and green are hard to distinguish, as are purple and
red/orange. Again, as with protanopia and deuteranopia, if this pie-chart had been presented to a
tritanopia colour-blind person there would have been great difficulty in interpreting the presented data.
2.4 Monochromacy
Monochromacy is the complete loss of colour, where colours are visible as shades of grey.
Monochromacy is frequently termed as “total colour-blindness” (Color-blindness.com,2010).
Monochromacy is caused by either defected cones or absence of cones. As discussed above, Figure
4a) is the same pie-chart as Figure 1a), Figure 2a) and Figure 3a), but Figure 4b) shows the same
pie-chart for those suffering from monochromacy.
Figure 4a: Normal vision Figure 4b: Monochromacy
Comparing normal vision with monochromacy vision clearly shows distinct differences to all of the
other varients of colour-blindness. In this case red, orange and green are indistinguishable as are
blue and violet. Only indigo appears distinguishable, but as black. Such a pie-chart would cause a
monchromastic learner a great deal of difficulty interpreting the data.
3. Emulating colour-blindness and HCBE
In this section we discuss how the academic with no colour loss can experience the same view as the
colour-blind user. The simplest method of simulating colour-blindness is by offsetting colours. For
example, in the case of deuteranopia (red-green colour-blindness), in a standard RGB
(red/green/blue) colour value, the red part of the colour could be reduced to 0. However, this method
should not be taken as an accurate portrayal of what a colour-blind person actually sees, since the
cones in the human eye do not map directly to the 3 components of an RGB value. As such, this
method is not a proper representation of colour-blindness.
Alternatively, direct colour mapping can be used where a palette of colours can be referenced directly
against an identically sized palette. However, careful mappings are required to ensure that there are
no errors. A different palette is required for each type of colour-blindness.
A better method is to use linear algorithms where the resultant colour change is computed. Ideally the
result of using direct colour mapping and the usage of linear algorithms should be identical. The
colour values perceived by the eye are mapped out by the LMS (long/medium/short) photoreceptor
activations in the visible spectrum, but these photoreceptor cells do not directly correspond to
standard RGB values. Capilla et al (2004) produced a conversion matrix that converts an RGB value
into an LMS value. In HCBE, we use a matrix based on Capilla's idea where one of the cone values is
modified, depending on which type of colour-blindness is being emulated. Afterwards, HCBE converts
the LMS value back into an RGB value, as shown in Figure 5. The resultant RGB is then outputted.
213

Colin Egan et al.
Figure 5: HCBE's matrix algorithm for RGB conversion to LMS and then LMS conversion back to
RGB
In order to simulate damaged cones or lack of cones, the exact values of all three cones must be
obtained. This can be achieved by using a transformation matrix, such as that proposed by Viénot et
al, (1999) as shown in Figure 6. Viénot's transformation matrix can also be shown as a linear
equation.
Figure 6: HCBE's (Viénot's) transformation matrix algorithm for conversion of RGB to LMS
After modification of either the L, M or S photoreceptor cell value, which depends on the type of colour
blindness, the LMS colour value must then be converted back into RGB to be recognised on the
output display. This is achieved by using the inverse matrix as shown in Figure 7.
To emulate Protoanopia, the L photoreceptor cell values require modification. To emulate
Deuteranopia the M photoreceptor cell value requires modification and to emulate Tritanopia S
photoreceptor cell value requires modification.
214

Colin Egan et al.
Figure 7: HCBE's reverse transformation matrix algorithm for conversion of LMS to RGB
4. Teaching with HCBE
In this section we present an example of the use of a circuit diagram and how this may be seen by
different types of colour-blind users. Figures 8a) 8b) 8c) 8d) and 8e) show a typical circuit block
diagram example used in the teaching of Computer Engineering. The diagram has been generated in
LogicWorks (2010), which is a commonly used interactive tool for the teaching and learning of digital
logic. For this study, it is not necessary to have any knowledge or understanding of Computer
Engineering. In this circuit diagram (Quick, 2010) the input signal is obtained from the left hand side
with 1 being an asserted signal and 0 being a de-asserted signal. In this example, both input signals
are asserted as 1 in the boxes. The signals are then transmitted along the (normal vision) red lines
and they then become inputs into various digital logic components (actual details of these
components are not necessary for this study). The output signals are generated and reflected in the
boxes on the right hand side. In this example, all of the output signals are de-asserted and are shown
in the boxes as 0.
By comparison of all the circuit diagrams in 8b), 8c) and 8d), it is evident that colour-blind learners
would have difficulty in interpreting this circuit diagram. For example, comparing Figure 8a) with
Figure 8b) a learner suffering from protanopia, would not be able to see the input signals as they
appear black and grey. Also, there would be difficulty in seeing at least one of the output signals.
Also, comparing Figure 8a) with Figure 8b) a learner suffering from deuteranopia would have similar
problems. Now by comparing Figure 8a) with Figure 8c) a learner suffering from tritanopia would be
able to see the input signals but would have difficulty with the output signals. Finally, comparing
Figure 8a) with Figure 8d) a learner suffering from monochromacy would be unable to detect both
input and output signals. Furthermore, the LogicWorks background grid-lines can also make the
circuit diagram difficult to be viewed by a colour-blind learner.
5. Some example solutions
The Equality Act 2010 clearly indicates that the responsibility of making reasonable adjustments lies
with the practitioner, and yet in reality few (if any) adjustments are typically being made to help colourblind
learners. Once awareness has been raised there are some quick and simple solutions to
enhance the learning experience of the colour-blind. From the pie-charts shown as Figures 1 – 4, a
simple separation of individual colours, which may appear merged for a colour-blind learner, may be
sufficient to potentially solve the problem. Alternatively, the use of shading such as stripes and/or
dashes may be a potential solution. In other cases such as the circuit diagram shown as Figure 8, the
use of thicker/thinner lines, dashed lines or embolding lines may potentially solve the problem. We
consider that with thought and creativity the practitioner should be able to improve the learning
experience of a colour-blind learner without impacting on the experiences of the non colour-blind user.
215

Figure 8a: Normal vision LogicWorks circuit
diagram
Figure 8c: Deuteranopia LogicWorks circuit
diagram
Figure 8e: Monochromacy LogicWorks circuit
diagram
Colin Egan et al.
216
Figure 8b: Protanopia LogicWorks circuit
diagram
Figure 8d: Tritanopia LogicWorks circuit diagram

Colin Egan et al.
6. Discussion and plans for further work
From this study we conclude that colour-blind learners do have considerable difficulties in interpreting
what those without colour-blindness see as simple and/or detailed information and this has been
borne out by inter alia recent research into colour-blind gamers. From our discussions with colourblind
users they have raised particular areas where their learning has suffered, with one older user
recalling being dismissed from her school chemistry class because she said that a ‘solution was the
wrong colour.’ Frequent anecdotes from other users relayed to the research team have recalled
problems with coloured wiring, as depicted in the example from the teaching of circuit diagrams given
above.
Currently, many colour-blind learners develop their own method(s) to overcome their disability. Even
though colour-blindness is considered to be a mild disability, the Equality Act 2010 clearly places the
responsibility of reasonable adjustments onto the practitioner. In our experience, currently reasonable
adjustments are rarely made to aid the colour-blind learner. In this paper, we have raised awareness
of the detrimental impact that teaching materials can have on the colour-blind learners. We have also
emphasised that the responsibility of making reasonable adjustments lies with the practitioner and not
with the learner. Furthermore, we have shown that reasonable adjustments can be made with some
consideration and deliberation of the teaching material.
HCBE is in the early stages of development and implementation. Currently HCBE only return images
in the way that a colour-blind learner would see the original inputted image. We, therefore, propose to
enhance HCBE to provide example potential solutions to aid the practitioner to make the necessary
reasonable adjustments of their teaching material for the colour-blind learners. Consequently, the
practitioner should be able to enhance the learning experience of the colour-blind learners and
comply with the Equality Act. However, HCBE creating potential solutions will be a non-trivial task.
This is because the solutions HCBE will provide will be generic potential solutions to specific and
detailed teaching material. As HCBE is a software tool, it will have no knowledge or any
understanding of the teaching material. Hence, the ultimate responsibility continues to remain with the
practitioner and HCBE's solutions will serve to identify the problem and support the decision-making
of future suggestions for improving practice.
Acknowledgements
The authors acknowledge with gratitude the support of the University of Hertfordshire’s Proof of
Concept fund for their provision of a generous funding grant to support the HCBE research. We are
also grateful to all those users who have shared their experiences freely to help us understand more
about the difficulties colour-blind users may have with online learning materials.
References
BBC, (2011) http://www.bbc.co.uk/news/technology-13054691 (accessed June 16th 2011)
BUPA. (2008). Colour blindness. [Online] (Updated March 2008) Available at:
http://hcd2.bupa.co.uk/fact_sheets/html/colour_blindness.html [Accessed 30 May 2011].
Capilla, P., Luque M J and D´ıez-Ajenjo M J., Looking for the dichromatic version of a colour vision model Jnl.of
Optics A: Pure and Apploed Optics. 6, pp. 906 – 919, 2004.
Cassin, B. and Solomon, S.,( 1990.) Dictionary of Eye Terminology. Gainsville, Florida: Triad Publishing
Company,
Colour-Blindness.com. (2010). How Color Blindness Works. [Online] (Updated 22 nd February 2010) Available at:
http://www.colour-blindness.com/general/how-it-works-science/ [Accessed 30 May 2011].
De Paula J B., Converting RGB Images to LMS Cone Activations TR-06-49 The University of Texas, Austin,
USA. October 2006.
Disability Discrimination Act (1995). Available at: http://www.legislation.gov.uk/ukpga/1995/50/contents
[Accessed 30 May 2011].
Disability Discrimination Act (2005). Available at http://www.legislation.gov.uk/ukpga/2005/13/contents [Accessed
30 May 2011].
Equality Act (2010. Available at: http://www.equalities.gov.uk/equality_act_2010.aspx [Accessed 30 May 2011].
LogicWorks, Available at:http://www.capilano.com/ [Accessed 30 May 2011]
Quick P., (2010) Computer Systems Architecture,Teaching Material for Level 4 BSc Computer Science module
4COM0051. School of Computer Science, University of Hertfordshire. UK. Semester B Academic Year
2010/11
Viénot F, Brettel H, Mollon J., (1999) Digital Video Colourmaps for Checking the Legibility of Displays by
Dichromats. Color and Research Application, Vol., Issue 4, pp. 243 – 252, August 1999.
217

Researching in the Open: How a Networked Learning Instance
can Challenge Ethical Decision-Making
Antonella Esposito
Universitat Oberta de Catalunya, Milan, Italy
aesposito47@gmail.com
Abstract: This paper focuses on ethics issues implied in a prospective virtual ethnography study aiming to gain
insights on participants’ experience in an emergent context of networked learning, namely a MOOC – Massive
Online Open Course. A MOOC is a popular type of online open course, that provides free content and expertise
to anyone in the world who wishes to enroll. This kind of informal lifelong learning initiative is enabled by a network-based
pedagogy and is enacted in a distributed technology-mediated learning environment. The purpose of
this article is to explore competing views on ethical decision-making when researching in such a globalized,
online and open learning setting. Considering the challenges of this new elearning inquiry context, issues as the
underlying research ethics models, the roles of researcher and participants and the integrity of the research
process are discussed in their interplay with the evolving ethos of the ethnographical methodology being adopted
to investigate participants’ views. Elements drawn from the design of a qualitative study are here utilized to identify
an empirical instance that shapes and is being shaped by research ethics decisions. The study aims to answer
the following question: what are the affordances (opportunities and challenges) of online open courses as
they emerge from the participants’ perspectives? This paper considers the potential operationalization of the
above research question and discusses both theoretical and methodological issues arising from applying research
ethics to this specific case of Internet inquiry. In this sense, ethical approaches in online research contexts
as well as main ethical decisions are discussed and justified, envisioning a submission to an institutional
ethics review board before undertaking the ethnographical study. Topics such as privacy concerns in a public
online setting, choice between overt and covert research, researcher as observer or participant, narrow or loosely
defined application of the informed consent and anonymity are outlined, presenting a range of different options.
This article intends to show that ethical decisions are an iterative procedure and an integral part of the research
design process. Moreover, it endorses the opportunity to produce localized and contextualized ethical decisionmaking.
To this end, it takes into account the guidance available (research ethics literature; narratives of ethics
procedures applied to empirical cases); the ethics debates within the ethnographical tradition and the nature of
the setting being researched (the specific format of the networked learning instance being examined). The discussion
here proposed orientates ethical decision-making towards an overt and participant research approach,
an informed consent intended as a ‘public notice’ and a consideration of participants both as authors in the online
setting and as human subjects embedding unexpected privacy sensitiveness. However such decisions are intended
as many starting points to build a research ethics protocol intended to a degree as a work in progress, in
a problem-solving approach guided by the practical wisdom of participants emerging over time.
Keywords: internet research ethics, massive online open courses, virtual ethnography, situated ethics
1. Introduction
The emerging networked learning environments add unexpected ethical challenges to researchers:
this prompts the exploration of a new vocabulary of research ethics (Beaulieu and Estalella 2009),
and demands a greater effort for “unravelling the intricate tapestry of ethics and method in research
design and process” (Markham 2007: 3). It also seems to suggest an ethics decision-making approach
that strives to combine general rules of the codes of conduct provided by the institutional review
boards with a continuining effort to gain phronesis or ‘practical wisdom’, “which demands understanding
of specific situations and reference to prior experience” (Tracy and Carmichael 2010).
This paper proposes a reflection on an empirical case of networked learning inquiry, as outlined in a
small exploratory research to be preliminarily undertaken within my doctoral thesys focusing on open
education issues in postgraduate online research training. This article deals with ethics issues implied
in this preliminary virtual ethnography study, aiming to gain insights on participants’ experience in an
emergent context of online open education, namely a Massive Online Open Course (MOOC). The
general goal of this paper is to cope with the main issues to be included in an ethical review to be
submitted to an institutional ethics review board, before undertaking a study on this topic. However, in
the current evolving landscape of technology–mediated ethnography (Markham 2003; Beaulieu and
Estalella 2009; Estalella 2007), there is an increasing awareness that the mandatory rules imposed by
the institutional review boards are necessary but not sufficient conditions to illuminate hidden aspects
and suggest practical solutions to researchers in online inquiry settings. So, issues such as different
approaches to ethical issues in an online research context, privacy concerns in a public online setting,
the choice between overt and covert research, the application of the informed consent and issues of
218

Antonella Esposito
anonymity are outlined as many springboards to build a ‘doable’ research ethics protocol. However,
decision-making on ethical issues is intended here to be a continuining reflexive interrogation of one’s
method of inquiry to reveal “hidden ethical practices from inside” (Markham 2007: 3). This engagement
in the interrogation of one’s method is complemented by a recursive work aiming to a “production
of localized, contextualized ethical decision-making” (Whiteman 2009: 65), that needs to be
adapted over time. Given that, the initial elaboration of the research ethics strategy takes into account
the guidance available (research ethics literature; narratives of ethics procedures applied to empirical
cases); the ethics debates grounded in the ethnographical tradition and the nature of the setting being
researched (the specific format of networked learning instance being examined).
2. The research question
I assume as the specific setting of the prospective pilot study a MOOC's edition (Cormier and Siemens
2010), carried out in the area of educational technology and addressing lifelong learners all
over the world. This kind of informal learning experience is enabled by a network-based pedagogy
and enacted in a public, distributed technology-mediated learning environment (Moodle forums and
any social media). Usually a few hundreds of individuals play a role as active learners, against thousands
of ‘lurkers’.
The research question I would like to focus on is the following one: what are the affordances (opportunities
and challenges) of online open courses as they emerge from the participants’ perspectives?
The goals of the proposed study entail: 1) to understand participants’ experience in an emergent context
of online education; 2) to draw recommendations for future course design of online open courses.
A virtual ethnography perspective is being applied as methodology: observations of communication
occurring in forums and social media among active participants and an online, open-ended and
anonymous questionnaire are considered as main data gathering methods.
3. Approaching ethical implications of the online setting
Definition and application of ethical issues to specific research contexts is never straightforward –
both in offline and online contexts - because they are made by a complex blend of social norms, values
and legal issues, are dependent on national and local traditions, and refer to different ethics philosophical
frameworks. Ess (2004) notices that in the EU deontological frameworks are usually applied,
while in the US utilitarian approaches are more common, and elsewhere virtue-laden perspectives
are considered. However, in 2002 the AoIR - Association of Internet Researchers defined specific
recommendations that focus on ethical pluralism and flexiblity of guidelines to be adopted by individual
researchers and institutional review boards.
The case being investigated is positioned in the realm of Internet-based research, in which ethics
mainly refer to a human subject research model (Bruckman 2002b) and focuses – as well as in offline
contexts - on issues such as public versus private ownership, informed consent, anonymity and confidentiality
(Eysenbach and Till 2001; Mann and Stewart 2000). To date, these issues keep on causing
ethical discord and uncertainty in elearning research (Kanuka and Anderson 2007) as well as in a
number of disciplines (Bruckman 2002b). Other views value a model of Internet research that intends
“Internet as production of cultural texts” and online subjects as authors (Basset and O’Riordan 2002),
envisioning an ethics approach which considers issues such as “appropriation, reproduction and removal
of Internet texts from their original context” (ib.).
Moreover, within the ethnographical tradition, Beaulieu and Estalella (2009) have recently led attention
to consider contiguity of settings (blurring distinctions between research fieldwork, the place
where analysis is being carried out and where findings are disseminated and published) and traceability
of data (“the property of inscriptions to be located through search engines and other mechanisms”,
ib.) as many typical tensions and features of the technology-mediated ethnography settings, that urge
a rethinking of the same principles of research ethics, such as anonymization, exposure, authorship
and ownership.
The ethical approach being adopted takes into account the ethical stances of virtual ethnography
(Markham 1998, 2003; Hine 2000), which relies on human subject research model, but also explores
new kinds of text-based settings, such as a MOOC mostly is. So, to a degree this ethical approach
also values participants as authors; and above all considers ethical expectations by participants in
the specific istance being investigated (Eysenbach and Till 2001; Ess and AoIR 2002), as drawn by
219

Antonella Esposito
previous editions of these pilot courses (i.e. PLENK 2010; Siemens 2010; Cormier and Siemens
2010). The aim is to balance the need of preserving research integrity with the provision of advantages
to participants and the effort to minimize related dangers.
4. Public versus private ownership
The public nature of the open learning environment established in a MOOC (Fini 2009; Mak et al.
2010) seems to facilitate the collection of large amounts of observational data, with minor privacy
concerns by researchers. Indeed, unlike formal e-learning environments, a MOOC is likely to be assimilated
to an open web space, since it takes place in multiple, non reserved areas (also Moodle forums
usually have full visibility to non-enrolled readers) and it provides loosely defined constraints of
'enrollment' and 'attendance'. In fact, learners can participate and withdraw at any time and at whatever
level of frequency and engagement, and they can mark their own online presence and interact
with their facilitators and peers while undertaking ordinary social/academic/professional activities using
their preferred social media. Given that, drawing from recent social research studies with popular
technologies, it is worth taking into account that behaviours linked to voluntary choice and use of social
media “suggest a mindful aspiration for publicity” (Vieweg 2010) by participants. Thus boundaries
between public and private ownership tend to blur in the open learning environment being considered.
Indeed, the problem of ownership of messages' transcripts in Net-based spaces is still controversial
(Kanuka and Anderson, 2007): my general view follows Mann and Stewart's position that when one
posts a message "there is an implied license to read, or even archive, the information it contains"
(2000: 46). It is worth noting that the informed consent adopted in a recent MOOC (PLENK 2010) declared
a default research use of all posted messages tagged with the course's name: otherwise, authorization
was asked. Indeed any acknowledgement of authorship of more complex artefacts being
produced during a MOOC is hardly framed within IPR issues. In fact, the setting of such online open
courses is grounded in a culture of sharing that is at the heart of the knowledge production model
suggested by these experimental learning projects. This cultural mood is fed with forms of acknowledgement
of the individual original production – such as the mutual acknowledgement among participants
- that are different from the mere protection of copyright and have more to do with the construction
of one's own digital identity.
Nonetheless, it is also considered that enrolled learners in a MOOC are potentially all over the world
and therefore they are likely to have different cultural and personal sensitiveness about privacy issues
(Vieweg 2010). Moreover, analysis of some specific threads of discussion (e.g. self-evaluation,
learner experiences, etc) might reveal for instance feelings of discomfort by some learners – often
accredited professionals - in a complex networked environment such as a MOOC: these learners
might feel violated if they saw their posts de-contextualized and highlighted in a publication. In this
case the researcher might be at danger to enter learners' private sphere: to mitigate any distress the
researcher should contact authors’ posts to let them the possibility to choose to be anonymized or
credited. Given that, as a general recommendation a debriefing opportunity – usually planned in the
final phase of the study – could be provided as a continuing dialogue between researcher and informants
to be carried out in a devoted forum, in order to monitor if any harm is being perceived and to
provide timely solutions.
Here also issues related to researcher’s sense of ownership and authorship should be considered,
recalling the notion of contiguity of settings discussed by Beaulieu and Estalella: “While fieldwork is
never easy, we felt at time exposed, surveilled and even, on occasions, that actors in the field or colleagues
from ‘home’ were foreclosing on our research” (2009: 8). This challenging perspective questions
traditional conventions to manage distinctions between the role as a fieldworker and that as an
academic, between participants’ and researcher’s voice.
5. Overt vs covert research approach
The ethical attitude of an overt research approach is being endorsed to preserve individual informants,
seen as ‘participants’, and social ecology of the community (Cohen et al. 2007: 156-175). The
negotiation of access to the fieldwork, an “acclimatization process” (Chen et al. 2004: 172), the long
permanence in the field, the acquisition of competence of informants and debriefing procedures are
being used as many cautions to mitigate the disrupting character of the researcher’s intervention. The
disclosure of the researcher’s presence is just among the benchmarks of effective ethnographies
(Splinder and Splinder 1992: 65) and has methodological and ethical relevance in virtual ethnography,
as a by-product of a negotiation of access and self-presence (Hine 2008: 264). However, others hold
that a researcher’s behaviour as a “lurker” is acceptable (Paccagnella 1997; Beaulieu 2004:146), just
220

Antonella Esposito
because online ethnographical observation can be considered as less intrusive than in offline contexts.
On the one hand some official guidelines (Ess and AoIR 2002) seem to authorize a covert role by researchers,
whether participants have chosen to post publicly. On the other hand, Bakardjieva and
Feenberg (2001) warn researchers with respect to ‘technically’ public research settings and refer to a
“non alienation principle”, whereby everyone is welcome to join and use online communities but not to
‘harvest’ or sell information therein.
That said, I think that a covert approach could be even counter-productive for data collection in the
setting being researched, in which participants are encouraged to share critical and creative contributions
within the community. The same disclosure of researcher’s work might constitute a metareflection
in its own right, useful both to community of learners to critically reflect on the collective
knowledge building effort and to course authors to enhance understanding of learners’ perceptions of
the networked learning experience. Therefore, the adoption by the researcher of a role as ‘observer
as participant’ (Cohen et al. 2003:179) is likely to be tolerated by the community of learners, by negotiating
access and self-presence (Hine, 2008: 264) and thus appears to be more functional to the
need to gain insights on challenges and opportunities of an online open course. Moreover, such a
choice seems to be inescapable in the endorsed virtual ethnography perspective:
“lurking online to collect data without participating in culture may not just be less desirable,
but perhaps not possibile if the goal is to explore sense making practices” (Markham,
2003: 5).
6. Informed consent
Informed consent is certainly the key issue to be addressed when building an ethical framework. The
choice of an overt and active approach makes the informed consent an instrument for researcher to
demonstrate credibility and trasparency of conduct. However, there are contrasting views about its
mandatory character and its operationalization.
In my view, the informed consent to be submitted to MOOC’s participants of the proposed study could
assume the form of a mere ‘public notice’ (Ess and AoIR, 2002: 7), before the observational data
collection starts. Therefore, a ‘reverse tecnique’ is being proposed, whereby participants must inform
the researcher if they don’t wish to be investigated as posts’ authors.
Indeed, some scholars (Bakardjieva and Feenberg, 2001; Bruckman, 2002a) state that obtaining
consent (through signed form) from each participant is mandatory, even if it is logistically difficult or
potentially disruptive of the online environment. On the other hand, Fahy and Spencer (2004: 33-34)
maintain that an ethics institutional board can waive informed consent where minimal risks in research
are present, if subjects can be provided with additional information after participation, or there are
serious hurdles in getting it. Indeed, to obtain consent through a signed form from individual
participants can be fairly easy in a voluntary online questionnaire survey, for instance by including a
‘check box’ in the online form. However, this might be problematic for observations, given the
‘rhizomatic’ environment of a MOOC, as characterized by high numbers of enrolled people,
discontinuity in learners’ participation and uncertainty to retrieve reliable contact information in
participants’ profiles. Given that, some hold that in a public arena to ask individual participants for
signing and sending back a form seems to be inefficient and time consuming, whilst posting a general
message to inform the community appears to be intrusive (Eysenbach and Till, 2001). Following
Moreno et al. (2008), I set out firstly to obtain permission by the course’s authors, who can act as
many gatekeepers to the research setting, and secondly to widely inform the community of learners
as a whole about the study’s details. In fact, given the non sensitive nature of the topic and the
peculiar research-focused attitude of the setting, I think that to set up an informative web space within
the course’s forum area could be an acceptable solution for course participants to become aware of
research aim and design, gathering and data protection methods, types of dissemination outputs and
planned levels of commitment. In this line, Fahy and Spencer maintain that, under conditions given
above, the “rights of the majority to participate in research are protected over the objections of those
who may not wish to do so” (2004: 33). Nonetheless, in the proposed study participants would be
involved in an iterative debriefing process (occurring in the same forum area), would be allowed to
withdraw consent at any time in questionnaires (through a devoted ‘exit’ button) and would be able to
communicate to researcher if they have any objections to be directly quoted in research accounts or
if they prefer to withdraw.
221

Antonella Esposito
However, beyond the formal fulfilments, a further step could be considered, reflecting on the
participatory nature of the learning community being investigated and on the varied cultural and
professional backgrounds of research participants. The same informed consent could be developed to
a degree as a work in progress, in a problem-solving approach guided by the practical wisdom of
participants emerging over time: “the different disciplinary perspectives and varied experience
represented offer not only creative solutions to dilemmas but are also a source of critique of the
ethical framework itself, that continues to evolve as a result” (Tracy and Carmichael 2010: 254).
7. Anonymity
One of the risks of naturalistic research is the over-exposure of individuals and groups, which can be
just avoided preserving their anonymity or at least, as I set out to do, allowing informants to choose
anonymity or disclosure of their personal data.
As regards to observation, taking cue from the adoption of a ‘public notice’ to inform research
participants, I opt for a ‘no disguise’ approach (Bruckman, 2002b), that is a use of pseudonyms or real
names of the posts’ authors. In fact, I consider the low level of risk of the research to be undertaken
but also that “Anonymity may not always be preferred as default, especially in a participatory culture,
where people want to be attributed to the stories they publicly share” (Liu, 2010: 2). Indeed,
participants in a MOOC are invited to comment each other and disseminate blog entries and to
experience a role as producers of “remixed” content and various digital artefacts. So, to a degree,
anonymizing material such as videos, diagrams or blog posts could be even perceived as a harm by
active participants in such a context. This is also in line with the idea of the Internet users as “amateur
artists” (Bruckman, 2002b) to whom it seems appropriate to give credit for their work if they desire it.
So, if on the one hand disclosure of participants as authors can appear as a concession (justified by
the setting’s features) to an Internet research model focusing on textuality, on the other hand the use
of an anonymous online questionnaire can help to give voice to the numerous lurkers – who otherwise
would be unreachable and unheeded - without disrupting their privacy.
Moreover, once again taking cue from debates on technology-mediated ethnography, a further issue
is considered, that sheds a new light on the traditional concept of anonymity as a ‘protection’ bulwark
of subjects: “Being traceable could actually mean greater, and more diverse accountability” (Beaulieu
and Estalella 2009). Just because the traces of researcher’s activities can be found online and are
potentially disclosed to all research participants and stakeholders, a “more subtle and modulated
approach to human subject protection” can be envisioned as an object for new discussions and
formulation of future solutions.
8. Conclusions
This paper stated and justified main ethical decisions to be undertaken in a potential
operationalization of a research question focusing on opportunities and challenges of online open
courses, as emerging from participants’ perspectives. The discussion here proposed orientates the
ethical decision-making towards an overt and participant research approach, an informed consent
intended as a ‘public notice’ and a consideration of participants both as authors in the online setting
and as human subjects embedding unexpected privacy sensitiveness. Such choices are highlighted
as many issues to be submitted to an institutional ethics review board for further negotiation and
approval. However, following Markham’s (2007) recommendations for a ‘reflexive ethics’ that
recursively intertwines ethical and methodological decisions, such decisions are intended as many
starting points to build a research ethics protocol intended as a work in progress. In fact, an open
networked learning environment encourages a participatory research approach and therefore fosters
creative suggestions and shared solutions from participants, in an evolving landscape of ethical
opportunities and challenges. This entails for the researcher to devise and assume new kinds of
responsibility and accountability, to research participants and to the same role as a researcher.
Acknowledgements
An early version of this paper was written as a final essay of a module within the MRes in Educational
and Social Research, Institute of Education, University of London. I would like to acknowledge Dr.
William Gibson and Dr. Soh-young Chung for their invaluable support and encouragement.
222

References
Antonella Esposito
Bakardjieva M. and Feenberg A. (2001) “Involving the virtual subject”, Ethics and Information Technology, vol. 2,
n.4, pp. 233-240.
Bassett, E. H. and O’Riordan, K. (2002) “Ethics of Internet Research: Contesting the Human Subjects Research
Model”, Ethics and Information Technology, vol. 4, pp.233-247,
http://www.nyu.edu/projects/nissenbaum/ethics_bas_full.html (accessed on 17/01/2011).
Beaulieu, A. (2004) “Mediating ethnography: objectivity and the making of ethnographies of the internet”, Social
Epistemology, Apr-Sep2004, Vol. 18 Issue 2/3, pp. 139-163.
Beaulieu, A. and Estalella, A. (2009) "Rethinking research ethics for mediated settings" Cologne, Germany. The
5th International Conference on e-Social Science, session 'Ethics and e-Research’, 2009, June 24-26,
http://www.estalella.eu/eventos-academicos/ethics-and-research-practices-in-mediated-ethnography (accessed
on 20/01/2011).
Bruckman, A. (2002a) Ethical guidelines for research online [online], from http://www.cc.gatech.edu/~asb/ethics/
(accessed on 15/01/2011).
Bruckman, A. (2002b) “Studying the Amateur Artist: A Perspective on Disguising Data Collected in Human Subjects
Research on the Internet”, Ethics and Information Technology, vol. 4, n.3, pp. 217-231,
http://www.nyu.edu/projects/nissenbaum/ethics_bru_full.html (accessed on 12/01/2011).
Chen, S. L., Hall, G. J. and Johns, M. D. (2004) “Research paparazzi in cyberspace: The voices of the researched”,
Chen, S. L., Hall, G. J. and Johns, M. D. (2004) Online social research: Methods, issues and
ethics, New York, Peter Lang, pp. 157-175.
Cohen, L., Manion, L. and Morrison, K. (2007) “The ethics of educational and social research”, Cohen et al.
(2007), Research Methods in Education, Routledge, pp. 51-77.
Cormier, D. and Siemens, G. (2010) “Through the Open Door: Open Courses as Research, Learning, and Engagement”
[online], EDUCAUSE Review, vol. 45, no. 4 (July/August 2010): 30-39,
http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume45/ThroughtheOpenD
oorOpenCoursesa/209320 (accessed on 10/01/2011).
Ess, C. and AoIR Ethics Working Committee (2002) Ethical decision-making and internet research: Recommendations
from the AoIR ethics working committee [online] http://www.aoir.org/reports/ethics.pdf (accessed on
13/01/2011).
Estalella, A. (2007) “Field Ethics: Towards Situated Ethics for Ethnographic Research on the Internet”, Forum
Qualitative Social Research, Vol. 8, No. 3. (2007), http://www.qualitativeresearch.net/index.php/fqs/article/view/277
(accessed on 31/05/2011).
Eysenbach, G. and Till, J. E. (2001) “Ethical issues in qualitative research on internet communities”, in BMJ, 10
November 2001, http://www.bmj.com/content/323/7321/1103.full (accessed on 12/01/2011).
Fahy, P., & Spencer, B. (2004) “Research experience and agreement with selected ethics principles from Canada’s
Tri-Council Policy Statement: Ethical conduct for research involving humans”, Journal of Distance
Education, vol. 19, n.2, pp. 28-58.
Fini, A. (2009) “The technological dimension of a massive open online course: The case of CCK08 course tools”,
in IRRODL, vol. 10, n. 5 (2009), http://www.irrodl.org/index.php/irrodl/article/viewArticle/643/1402 (accessed
on 09/01/2011).
Hine, C. (2000) Virtual Ethnography, London, Sage.
Hine, C. (2008) “Overview: Virtual ethnography: modes, varieties, affordances”, N. G. Fielding, R.M. Lee and G.
Blank (eds.) Handbook of Online Research Methods, London, Sage.
Kanuka, H., & Anderson, T. (2007) “Ethical issues in qualitative e-learning research”, in International Journal of
Qualitative Methods, vol. 6, n.2, http://www.ualberta.ca/~iiqm/backissues/6_2/kanuka.htm (accessed on
10/01/2011).
Kiegelmann, M. (1996) The Subject Writes Back, Reflections on Ethics in Qualitative Research, conference paper
at American Educational Research Association, New York, Harvard University.
Liu, S. B. (2010) “The Emerging Ethics of Studying Social Media Use with a Heritage Twist”, presented at workshop
Revisiting Research Ethics in the Facebook Era: Challenges in Emerging CSCW Research, The ACM
Conference, Savannah, Georgia, US, http://www.cc.gatech.edu/~yardi/ethicscscw2010_files/AcceptedPapers.htm
(accessed on 10/01/2011).
Mann, C. and Stewart, F. (2000) Internet Communication and Qualitative Research, London, Sage.
Mak, F. J., Williams, R. and Mackness, J. (2010) “Blogs and forums as communication and learning tools in a
MOOC”, in (eds) , Proceedings of the 7 th International Conference on Networked learning, pp. 275-284,
http://www.lancs.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/Mak.html (accessed on 18/01/2011).
Markham, A.N. (1998) Life online. Researching real experiences in virtual spaces, Walnut Creek, CA, Altamira
Press.
Markham, A. N. (2003) “Representation in online ethnographies: A matter of context sensitivity” [online], Chen,
S., Hall, J. and Johns, M. (eds) (2003) Online social research: Theory, Methods and Ethics, Peter Lang
Publishers, http://collections.lib.uwm.edu/cipr/image/104.pdf (accessed on 17/01/2011).
Markham, A. N. (2007) “Method as ethic, ethic as method. A case for reflexivity in qualitative ICT research”,
Journal of Information Ethics, vol. 15, n. 2, pp.37-55, http://markham.internetinquiry.org/writing/jie.pdf (accessed
on 20/05/2011).
223

Antonella Esposito
Moreno, M. A., Fost, N. C. and Christakis, D. A. (2008) “Research Ethics in the MySpace Era”, Pediatrics, Vol.
121 No. 1 January 2008, pp. 157-161, http://pediatrics.aappublications.org/cgi/content/abstract/121/1/157
(accessed on 15/01/2011).
Paccagnella, L. (1997) “Getting the seats of your pants dirty: strategies for ethnographic resarch on virtual communities”
[online] Journal of Computer Mediated Communication vol. 3, n.1,
http://jcmc.indiana.edu/vol3/issue1/paccagnella.html#bio (accessed on 15/01/2011).
PLENK 2010 – Personal Learning Environments and Knowledge, ‘How this course works’,
http://connect.downes.ca/how.htm (accessed on 15/01/2011).
Siemens, G. (2010) “Reflections on open courses”, blog post August 19 th 2010, in Connectivism. Networked and
social learning, http://www.connectivism.ca/?p=267 (accessed on 10/01/2011).
Tracy, F. and Carmichael, P. (2010) “Research ethics and participatory research in an interdisciplinary technology-enhanced
learning project”, International Journal in Research & Method in Education, vol. 33, n. 3, pp.
245-257.
Vieweg, S. (2010) “The Ethics of Twitter Research”, presented at workshop Revisiting Research Ethics in the
Facebook Era:Challenges in Emerging CSCW Research, The ACM Conference, Savannah, Georgia, US,
http://www.cc.gatech.edu/~yardi/ethics-cscw2010_files/AcceptedPapers.htm (accessed on 09/01/2011).
Whiteman, N. (2009) “Internet Research”, Research Methods Lecture Pack, Mres in Educational and Social Research,
Institute of Education, pp. 57-67.
224

Making Constraints and Decisions Explicit to Support
Project-Based Collaborative Learning
Gert Faustmann
Department of Cooperative Studies, Berlin School of Economics and Law,
Germany
gert.faustmann@hwr-berlin.de
Abstract: Learning projects serve as motivational instruments to transfer knowledge and competences relating to
real world problems. Yet, the learning effect varies among participants. The anticipated results of a project and
their relation to real life questions are formed by complex requirements and, while the project advances, the
complexity of the results emerging increases. Furthermore, project groups often consist of participants with little
experience in the subject of the project. While the project progresses, the overview of requirements and results
already developed is lost. The learning effect can now be raised by making the requirements on the expected
project results as well as the decisions for existing results explicit. In this way, participants understand the
decisions of others engaged in the ongoing project and they are able to align the existing results with the
requirements imposed on them. Additionally, new ways of collaborative learning are possible when, for instance,
the task is to develop constraints on or between the respective project results. Former research developed a
concept to keep less experienced project members in the productive process while forcing other project members
to give arguments for their results. The current work describes an approach which aims to integrate the
requirements on results in the learning process. While arguments were to include digital documents or specific
parts of these documents for clarification purposes, the incorporation of explicit requirements is now also
manifested in digital documents. The requirements can then be related not only to single result documents, but
also to whole sets of expected documents. The paper gives an example of a software development project where
some design documents have to be developed. It shows how constraints on these documents are presented and
how these constraints affect the collaborative learning process.
Keywords: project-based learning, cscl, arguments, digital documents, learning process
1. Introduction
The integration of learning projects into teaching is a common instrument and improves the ability of
students to solve real world problems (Doppelt 2003). Yet, certain problems arise when practicing
project based learning:First, the participants’ knowledge of the project in relation to the project task
often differs. While this is a key reason for implementing a “collaborative” approach in learning, for
some learners the learning effect may stagnate, because the project advances and the results
become more complex making it more difficult for them to understand the new results and their
relationship. Without any explicit help the students will not be able to contribute to the project.
Second, in nearly all domains the expected results share complex requirements and dependencies,
which themselves can be seen as knowledge about the domain space. We can distinguish between
requirements that are related to just a single project result (e.g. the amount of pages for one specific
text document) and overall requirements, that are related to more than one result (e.g. use UML
diagrams for designing a software system). Especially the latter are difficult to learn because they
often contain implicit knowledge of a domain (as in the above example, which diagram types have to
be used and how they interact).
We take a generic approach in order to tackle both sides of the problem. While the participants work
in a learning project and contribute result documents, they are committed to add further documents
that explain these results. These documents can be arbitrary digital documents allowing to add
information of any form. So other project members better understand how the existing results were
developed. To manage the complexity of the project constraints, the teacher can choose to add
documents that influence the later development of the result documents. These constraint documents
support the learners in constructing the overall results. With the introduction of supporting documents
for expected and existing results of a project, it is possible to think of new learning processes that
integrate these documents in different learning phases.
The paper starts with an account of existing approaches that incorporate any kind of metadata into
learning processes for reasons of supporting the learners in understanding the project’s results. Then
an example for a learning project from the field of software programming is introduced. This will be the
basis for defining the different document types and the way they are used in this concept. Afterwards
some new approaches on how to organize the learning process will be presented. The paper ends
225

Gert Faustmann
with an evaluation of the described concepts by presenting a learning platform implementing the
described conceptual aspects. Finally, an outlook will give some ideas on evaluating the concept and
on future research.
2. Related work
Today’s learning theory favors a constructivistic learning approach where learning is an active
process and the learner uses her/his knowledge to construct and solve a problem. The teacher acts
as coach, supporting the learners in tackling the complex problem or situation (Baumgartner 2003).
(Gilbert & Driscoll 2002) define the principles of a constructivistic learning approach as
A group objective,
Cooperating groups with interactions,
An individual selection of materials and decisions,
The use of integrated tools to store results and to communicate.
Especially wikis as tools play an important role in learning environments to support the distributed
construction of documents (Augar et al. 2006) and they are used in different domains as learning
environments (e.g. journalism (Ma & Yuen 2008)). Moreover, they were investigated in terms of their
ability to support communities (Johnson & Bartolino 2009) and their impact on social and cognitive
processes (Cress & Kimmerle 2008). There is further research about extending wiki systems for
specific learning scenarios (Larusson & Alterman 2009) (Pusey & Meiselwitz 2009) (Twu 2009). One
can claim that wikis are a constitutive component of today’s learning environments. But there are also
problems when wikis are used in a learning environment. (Karasavvidis 2010) reports problems like
the students’ resistance to use a wiki or overwhelming task requirements.
One can find different approaches to implement collaborative learning with computer support. Some
use the idea of arguments combined with a learning process. The visualization of arguments and of
their overall structure is also investigated in the cscl literature. There are research projects focusing at
computer-supported learning where the use of arguments is investigated (Cho & Jonassen 2002)
(González et al. 2007)(Jonassen & Kim 2009). While arguments in these projects are observed as
part of an argumentation process, (Scheuer et al. 2010) gives an overview of the project supporting
use of arguments. The main questions addressed are the visualization of argument structures,
interaction between project participants, the combination of arguments with ontologies and the
analyzation of arguments for automatic feedback. Visualization of the constructed knowledge as well
as of the arguments leading to this knowledge is described in other research (Muller Mirza et al.
2007)(Perit Çakır et al. 2009)(Schwarz & De Groot 2007). A software tool for visualizing argument
structures is described in (Sbarski et al. 2008).
Another interesting approach can be found in the software system SenseMaker (Bell 1997) (Bell &
Linn 2000). SenseMakers gathers web resources to build arguments for statements. The document
structure is a container that holds arguments. These arguments then may themselves contain further
arguments. A knowledge repository based on a semantic web ontology and a resource description
framework is built in (Lubliner et al. 2009). The repository contains course materials. There was a
clear increase in exam result points compared to the group not using the repository. This shows that
knowledge repositories with a internal semantic structure (e.g. hierarchies, ontologies) assist learning.
3. Modelling decisions and constraints
3.1 An example of a learning project
We will use an example of a learning project to explain the basic ideas of arguments, constraints and
digital documents in the following section. The example is an assignment in software programming.
Aside from the program code, some design documents are requested as results from the project
participants. The task is to design and program a flexible menu system which can be used by other
programmers within their projects. The menu system should work on the command line interface, so it
is just a character-based interface to the user. The task document defines
How a menu works (a menu option leads to a sub menu or contains an action that is carried out if
the user chooses this option),
That additional menu options can be integrated without changing the original source code,
226

Gert Faustmann
That results must comprise a class diagram, a sequence diagram and the program code.
Figure 1 shows the task document and possible results. The result documents are related to the subtasks
in the overall task document. In most cases, this relation of the results to the task is clear to the
learners.
Figure 1: Example of a project task with expected results
Usually this is the perspective a student has on a learning project of this kind. But, as one can see in
the example, it is not clear why the design is done this way. Questions are: why are there different
classes for the menu options, why is there an interface class for a menu option and why does the
“Menu” class use the getPath() method (as it is modelled in the sequence diagram) to display the
menu options. From a detailed analysis of the program code, these questions could be answered. But
the design principles behind these documents remain unclear.
3.2 Arguments explaining decisions for project results
A first step in clarifying the ideas behind the development of the specific results in a project is to give
explanations for them. We call these explanations ‘arguments’ because often a project participant has
to make one or more decisions when developing a project result. But because arguments support
other project members in understanding results, they need not be formal logical statements. They can
be of very different information types such as lecture slides or parts of a textbook. One can also think
of multimedia documents explaining something, e.g. a procedure with a movie. Figure 2 shows the
basic approach of combining tasks with results by using argument documents.
As depicted in figure 2, only parts of the documents are used for building this “argument network”.
This originates from the fact that often only small parts of a document are important as an explanation
The same applies to task and result documents: only parts of the task document may be solved by
specific result documents (“sub tasks”), and only parts of a result document may need an explanation.
Figure 3 shows the relationship between tasks, results and arguments. Here, they all are parts of
documents, that each belong to exactly one document. The implementation of document parts is
described in detail in (Faustmann 2011).
The class diagram also shows that every result belongs to a task, but must also be associated with at
least one argument. The idea is to force the project participants to provide corresponding arguments
for new results. From the other side an argument only belongs to one result. Different arguments may
227

Gert Faustmann
come from the same source document (and may nearly be the same), but they must be different as
an explanation for different results.
Figure 2: Arguments combining tasks and results
Figure 3: UML class diagram for the integration of document artefacts as arguments
When applied to our example we need to identify a document (or part of a bigger document) that can
serve as an argument for one of the shown results. Let us take a look at the combination of the sub
task “design a class diagram” and the resulting document being a class diagram. Here an argument
document is used to explain the specific design decision implemented in the class diagram. A strategy
pattern allows encapsulating an algorithm with an object. In this way, an algorithm can be easily
exchanged. The introduced argument document is a slide explaining the strategy pattern with a class
diagram and with an additional explanation. In figure 4 it is easy to see, that the structure of both
diagrams is nearly the same. So it is obvious that the MenuItem class encapsulates its action. The
menu system can now manage its options in different objects with each having its specific action.
228

Gert Faustmann
Figure 4: Argument document introducing the strategy pattern
3.3 Constraints revealing requirements on project results
While arguments explain results or show why results are formed the way they are, it is still difficult to
understand the context of project results. When looking at our example, we have two perspectives on
the design documents: On the one hand, every document has to be consistent in itself. This means
that the correct symbols are used, the symbols are correctly assembled and so on. On the other hand,
the diagrams also have to be consistent among each other. One can speak of consistency in relation
to the whole set of diagrams (and also all other documents that can be a result). For example, if a
class is described in the class diagram it should also appear in the source code. While there is no
sense in modelling a class without implementing it, there are also situations that are definitely
erroneous. If an object of a certain class type appears in a sequence diagram, this class must also
appear in the class diagram.
Figure 5 shows the integration of constraint documents and corresponding annotations into the former
system of tasks, results and arguments. Constraints as well as annotations are part of documents. If
an annotation is only text-based the whole ascii document can be the source for this annotation (on
the other hand it is then also possible that e.g. a handwritten document can be used as an
annotation). Constraints belong to results. Usually constraints exist before concrete results are
created. Then we can think of results as containers that have to be filled while the project is
advancing. A constraint has to be related to at least one result document. The function of an
annotation is to match the related constraint document to the special situation of the learning project.
Constraints can be very general and so match all situations (“a class in a sequence diagram must
appear in the class diagram”). But they can also be more specific (e.g. “program components against
an interface”) with an annotation giving a hint how to instantiate this constraint (e.g. “use an interface
for the menu actions”).
Again we take a look at the project example now, with the goal to introduce constraint and annotation
documents. Let us assume that there was an initial constraint document that asked for implementing
independent classes representing menu options on the different levels of a menu system. This indeed
is a constraint that affects the class diagram as well as the expected sequence diagram. The menu
system has to test every class-related object that is a menu option, if it is currently visible.
Additionally, the constraint demands in the example that these classes may be loaded while the whole
system is already running. In java there is a reflection mechanism that gives exactly this kind of
functionality. So the annotation to this second (sub-)constraint is a document describing the java
reflection mechanism. If this constraint is bound to the program code, it is easier to understand how
the java code works.
229

Gert Faustmann
Figure 5: UML class diagram for the integration of annotated constraint documents
Figure 6: Constraint demands independent classes, annotation explains loading classes
4. Learning process
The main goal for a learning project is to create a set of result documents (or one document that
contains consistent document parts, but according to our approach we stay in the perspective of
multiple documents). Therefore, the main process is to provide new result documents and enrich them
with argument documents thereby making decisions and the results better understandable. But the
explicit modelling of supporting information allows to change the learning process in its perspective.
Thus is becomes possible to ask the learners to find relationships between results or give further
context information. Both approaches are presented in the following:
4.1 Result-based project work
Incorporating new results will always be the main work in a project asking for constructive work. The
lerners will work at solutions that are influenced by the project’s assignment and additional
230

Gert Faustmann
constraining documents. One can think of three possible alternatives for providing new results (see
also figure 7):
The learner provides a new result that may be influenced by an already existing constraint
document. She/he is obliged to relate this new result to an argument.
The learner provides a new result, but uses a predefined argument out of a set of argument
documents that was prepared by the teacher. The learner seeks for an applicable argument and
integrates this argument.
Finally, the learner can provide a result where an argument already exists and gives support how
the result ist to be constructed. This would be the highest level of support a teacher can give by
preparing documents.
Here we see that the teacher can provide very different kinds of support in solving a problem. Not only
can she/he provide varying amounts of supporting documents and hints about where to use them, the
teacher may also provide further documents while the project is already advancing. So it is possible to
control not only the general advancement of the project but also the technical elaboration of the given
task (e.g. give a hint to use another design pattern to plan the software system).
Figure 7: Options to provide new results
Again we refer back to our example. According to the three options, possible activities and example
documents are defined here:
The constraint forcing the learner to think about loading classes while the program is running will
surely point to parts of the program code. Perhaps a sample solution for the reflection mechanism
could demonstrate the idea of the project specific program code.
If sample solutions (also for the reflection mechanism) were prepared by the teacher, the learner
can seek for them and use one as an argument for her/his program code.
If there is more than one document for the program code (think of different packages or modules)
then a sample solution could point to a specific module or package.
4.2 Learning context information
Additionally, there are further options for providing documents in the learning process (see also figure
8): The learner can provide additional arguments, constraints and annotations to already existing
result documents in the project.
This could mean the following activities in our example:
The learner provides an additional sample solution to the program code already being in the
system.
The learner provides a specific constraint that was asked for by the teacher. The learner knows
that there is a constraint sought-after for specific results.
231

Gert Faustmann
The learner her/himself formulates a constraint, e.g. for the source code in the system. This could
be, for example, a hint how to use the different classes of the menu system in another software
application.
Finally, it may be necessary to adapt a constraint to a specific learning situation by providing an
annotation.
Figure 8: Options to provide documents different from results
There are other alternatives, but they all follow the same structure: the teacher may ask for a new
document or the student may her/himself provide a new document without being asked to. When
asked for a document, the teacher can provide a set of possible documents or encourage the student
to search for appropriate documents on his/her own.
5. Arguments and constraints in the WeCoLAr system
Implementing the described conceptual approach is difficult because one of the core requirements is
the use of arbitrary digital documents for results and supporting documents. Additionally, these
documents have to be visualized in the system (also as links with the possibility to use a third party
application for opening) and they must be loadable in a native document format so that the learner
can work on them. If a document is repeatedly used in a learning project, it should nevertheless not
be multiply stored in the system.
The web-based learning platform WeCoLAr (Web-based Collaborative Learning by Arguments)
implements these requirements by using markings in picture documents (see figure 9). All documents
are stored in their native document format (e.g. pdf or html), as well as in a universal picture format
(here jpg). The graphical user interface only shows the pictures of the documents. But pictures of
bigger documents (e.g. pdf files with many pages) would be too bulky to be handled and viewed on a
graphical desktop. Furthermore, only parts of the documents are needed for referencing. So
WeCoLAr uses markings defining a specific area in a document. A marking is stored when a learner
loads a new document into WeCoLAr. This marking can be defined interactively and is specific for
every context.
232

Figure 9: Implementing document parts
Gert Faustmann
When handling these “documents” on the graphical desktop the quality of the picture is satisfying.
They can be zoomed in by using the browser’s zoom functionality. Also, it is possible to change to a
single document view. WeCoLAr also has the functionality to filter the viewed documents on the
desktop, so for instance, one can choose to see only those documents that belong to a certain sub
task (all design documents for the menu system). Figure 10 exemplifies the user interface of
WeCoLAr. In the viewed modus it is possible to drag documents within the argument net and change
the structure of the net.
Figure 10: WeCoLAr user interface
6. Outlook
WeCoLAr is now being tested in real learning processes at university level. The example used in this
paper is one of the domains where the WeCoLAr system is used (i.e. introduction to programming,
object oriented programming). But the system will also be put into effect in other domains that are
affected by business processes. One application field will be courses in Macroeconomics.
Future work on the concept of arguments and constraints in learning processes will focus on the
control of the process. (Dillenbourg & Tchounikine 2007) describe macro scripts that implement
didactic concepts. This would match the idea of offering different construction activities in WeCoLAr.
Of special interest could be the question of flexibility in controlled process systems. For now WeCoLAr
is controlled by the learners themselves. A learning script defines some constraints for this process
233

Gert Faustmann
(i.e. who has to provide what douments, timeline of documents etc.). In (Konig & Paramythis 2010) a
control mechanism is defined that can be adapted according to the situation at hand.
Acknowledgements
The project WeCoLAr is funded by the European Social Fund (ESF 2009000038) as a sub project
within „E(r)lernen: Kompetenzvermittlung zum Einsatz neuer Medien“. I like to thank Liangliang Gu for
his contribution to the WeCoLAr system.
References
Augar, N., Raitman, R. & Zhou, W. (2006) Wikis: Collaborative Virtual Learning Environments. In The
International Handbook of Virtual Learning Environments. pp. 1251-1269..
Baumgartner, P. (2003) Lerntheorien und Lernwerkzeuge. ÖZB, p.5.
Bell, P. (1997) Using argument representations to make thinking visible for individuals and groups. UNIVERSITY
OF TORONTO, p.10-19.
Bell, P. & Linn, M. (2000) Scientific arguments as learning artifacts: Designing for learning from the web with KIE.
International Journal of Science Education, 22(8), pp.797–817.
Cho, K.-L. & Jonassen, D. (2002) The effects of argumentation scaffolds on argumentation and problem solving.
Educational Technology Research and Development, 50(3), pp.5-22.
Cress, U. & Kimmerle, J. (2008) A systemic and cognitive view on collaborative knowledge building with wikis.
International Journal of Computer-Supported Collaborative Learning, 3(2), pp.105-122.
Dillenbourg, P. & Tchounikine, P. (2007) Flexibility in macro-scripts for computer-supported collaborative
learning. Journal of Computer Assisted Learning, 23(1), pp.1–13.
Doppelt, Y. (2003) Implementation and assessment of project-based learning in a flexible environment.
International Journal of Technology and Design Education, 13(3), pp.255–272.
Faustmann, G. (2011) PROJECT-BASED COLLABORATIVE LEARNING WITH ARGUMENTS - Flexible
Integration of Digital Resources. In 3rd International Conference on Computer-Supported Education.
Noordwijkerhout, The Netherlands.
Gilbert, N. & Driscoll, M. (2002) Collaborative knowledge building: A case study. Educational Technology
Research and Development, 50(1), pp.59-79.
González, M. et al. (2007) Modelling Shared Knowledge and Shared Knowledge Awareness in CSCL Scenarios
Through Automated Argumentation Systems. In Groupware: Design, Implementation, and Use. pp. 207-
222.
Johnson, K. & Bartolino, J. (2009) Creating Community through the Use of a Class Wiki. In Online Communities
and Social Computing. pp. 471-478.
Jonassen, D. & Kim, B. (2009) Arguing to learn and learning to argue: design justifications and guidelines.
Educational Technology Research and Development.
Karasavvidis, I. (2010) Wiki uses in higher education: exploring barriers to successful implementation. Interactive
Learning Environments, 18(3), pp.219-231.
Konig, F. & Paramythis, A. (2010) Closing the Circle: IMS LD Extensions for Advanced Adaptive Collaboration
Support. In 2010 International Conference on Intelligent Networking and Collaborative Systems. 2010 2nd
International Conference on Intelligent Networking and Collaborative Systems (INCoS). Thessaloniki,
Greece, pp. 421-426.
Larusson, J. & Alterman, R. (2009) Wikis to support the “collaborative” part of collaborative learning. International
Journal of Computer-Supported Collaborative Learning, 4(4), pp.371-402.
Lubliner, D., Widmeyer, G. & Deek, F.P. (2009) Collaborative learning utilizing a domain-based shared data
repository to enhance learning outcomes. Interactive Learning Environments, 17(4), pp.351-366.
Ma, W. & Yuen, A. (2008) A Qualitative Analysis on Collaborative Learning Experience of Student Journalists
Using Wiki. In Hybrid Learning and Education. pp. 103-114.
Muller Mirza, N. et al. (2007) Using graphical tools in a phased activity for enhancing dialogical skills: An example
with Digalo. International Journal of Computer-Supported Collaborative Learning, 2(2), pp.247-272.
Perit Çakır, M., Zemel, A. & Stahl, G. (2009) The joint organization of interaction within a multimodal CSCL
medium. International Journal of Computer-Supported Collaborative Learning, 4(2), pp.115-149.
Pusey, P. & Meiselwitz, G. (2009) Heuristics for Implementation of Wiki Technology in Higher Education
Learning. In Online Communities and Social Computing. pp. 507-514.
Sbarski, P. et al. (2008) Visualizing Argument Structure. In Advances in Visual Computing. pp. 129-138.
Scheuer, O. et al. (2010) Computer-supported argumentation: A review of the state of the art. International
Journal of Computer-Supported Collaborative Learning, 5(1), pp.43-102.
Schwarz, B. & De Groot, R. (2007) Argumentation in a changing world. International Journal of Computer-
Supported Collaborative Learning, 2(2), pp.297-313.
Twu, H.-L. (2009) Effective Wiki Strategies to Support High-Context Culture Learners. TechTrends, 53(5), pp.16-
22.
234

A Strategy for the Inductive Generation of Learning Objects
in Low-Tech Contexts
Ana Mª Fernández-Pampillón, Elena Domínguez, José Mª Lahoz, Dolores
Romero, Isabel de Armas, Susana Palmaz and Jorge Arús
Universidad Complutense de Madrid, Spain
apampi@filol.ucm.es
elenadominguez@filol.ucm.es
josemarialahoz@yahoo.com
doloresromerolopez@gmail.com
iarmas@buc.ucm.es
spalmaz@filol.ucm.es
jarus@filol.ucm.es
Abstract: This article proposes a strategy to create inductive and incremental LOs in tertiary education low-tech
contexts such as the field of Humanities. We take a teaching context to be low-tech when the teachers are not
computer specialists and also have poor IT support. This is the case with many university schools in Spain, such
as the Faculty of Philology at the Universidad Complutense de Madrid (UCM), where the work reported here has
been carried out. In a low-tech context, the use of Educational ICT is considerably expensive, if not impossible.
However, university faculty feels more and more compelled to use ICT tools in their teaching, e.g. e-learning
platforms. In the process of adaptation and integration of ICT into teaching, a basic issue is the transformation of
good teaching materials into digital format in the simplest and most cost-effective way. Good results can be
obtained by applying the Learning Object (LO) model when creating digital teaching materials, but the models
and tools to build LOs are not easy to understand or use without prior knowledge, and their application requires
computer support beyond what is usually available in low-tech contexts. The results of the research carried out
during the last decade thanks to a number of research grants show that, by using the appropriate models, tools
and strategies, it is possible to bring ICT to IT-illiterate teachers and get excellent results regarding the
educational use of ICT by these teachers. In the present work a solution has been designed and tested to
disseminate teaching materials through the generation of LO collections in low-tech contexts. This solution is
based on applying a novel strategy to inductively construct LOs from the original materials. This is done by
applying a model and a repository of LOs, developed and tested in previous works, and a new quality model for
LOs collaboratively developed with the beneficiaries of this strategy, i.e. IT-illiterate Humanities teachers. The
authors of this paper are part of a mixed team of IT-specialist and IT-illiterate Humanities teachers who
developed and applied this strategy. Our paper aims to provide a real and complete picture of the problem and
the solution developed.
Keywords: learning objects, repositories, authoring, digital learning resources, educational ICT
1. Introduction
Suppose you are a University Humanities teacher. In the current teaching context, class notes and
textbooks are no longer the only necessary materials. Audiovisual media, even just a simple
slideshow, are de rigueur. The increasing use of e-learning platforms requires that teachers convert
all their precious analogical resources to a digital format. Furthermore, they need to enlarge their
collections with new materials, as those virtual platforms are intended to provide extra learning
resources beyond the classroom. However, those materials are not usually readily available, and
instructors have to design their own material, despite their limited knowledge of the required
technology and the lack of technical support.
The development of content-quality materials involves a clear commitment in terms of research, and,
more often than not, the process also requires a considerable investment in time and effort to learn to
handle the tools and computer programs used for the creation of those digital objects.
As a collection grows, a further concern is to find an efficient way to store and manage it. Whether it is
for the author’s own use or with the purpose of sharing materials with others, digital repositories of
Learning Objects (henceforth LOs) are proposed as a solution (Neven and Duval 2002). However,
new problems arise such as the need to comply with the quality requirements of the repository, mostly
directed toward ensuring reusability (Bennett et al. 2006). This proves especially difficult when the
mentioned requirements are expressed in terms that are not intelligible to IT-illiterate users. On the
other hand, some technical knowledge is necessary to understand the quality, metadata and content
models.
235

Ana Mª Fernández-Pampillón et al.
In this paper we present an experiment conducted at the UCM School of Philology. A Project for
Educational Innovation and Improvement of Quality Teaching was undertaken for the collaborative
creation of a Learning Object Repository (LOR) targeted at our faculty members. The paper is
structured as follows. Section 2 presents the strategy, which is based on (2.1) an open Learning
Object Model, (2.2) an accessible quality model for Learning Objects (LOs), (2.3) an easy-to-use
LOR, and (2.4) an inductive methodology to create LO collections. Following this, section 3 sketches
the experimentation and results. Finally, section 4 presents the main conclusions and outlines future
lines of work.
2. Supporting university IT-illiterate teachers to create Learning Objects
The idea of improving the profitability of digital materials appears in the field of Educational
Technology and the related industry with the notion of LO reuse (Wiley 2000). The goal is to increase
profitability by creating educational materials as pieces (LOs) that can be assembled together
(Gibbons et al. 2000). It can also be applied to the case of university teachers who are, at the same
time, developers and users of their educative material. However, the theoretical and technological
knowledge to build LOs is beyond the reach of many faculty members. There are two main ways in
which researchers tackle the quest for the efficient creation and use of LOs by instructors: one of
them is the development of guidelines for the creation of LOs (e.g. Smith 2004; Jesukiewicz 2009);
the other is the creation of devices such as templates and toolkits to guide instructors in the design of
new LOs (e.g. Adorni et al. 2008; Watson 2010a). In spite of the unquestionable help that these
resources represent, in low tech academic contexts there is a main limitation: IT-illiterate teachers can
build LOs only with sufficient staff and infrastructure support. Unfortunately, at the moment this is not
the case in some Spanish Universities and Schools (Uceda and Barro 2010). Therefore, with the aim
of combining the effectiveness of the LO model with the needs of producing specialized digital
academic materials, we have developed and implemented a new, straightforward strategy for the
creation of LOs with minimal or no IT support. It is based on:
Using already-tested existing materials, digital or not, rather than creating new ones.
Applying an inductive methodology for the incremental construction of LOs from these materials
based on applying: (a) an open LO model; (b) an accessible quality model; (c) an easy-to-use LO
repository.
Collaboratively developing the metadata and LO evaluation models best suited to philology.
Providing basic ICT support, on site and long-term, by involving the Philology School’s IT staff in
this kind of projects.
This strategy has enabled the creation of two specialized LO collections: a historical collection with
old multimedia language teaching material , and a recent collection
of philological LOs: (Figure1).
Figure 1: Language labs’ audiovisual LO historical collection
236

2.1 An open Learning Object model
Ana Mª Fernández-Pampillón et al.
To facilitate the construction of reusable LOs, some international metadata standards (Duval 2004) as
well as multiple content models (Verbert and Duval 2008) have been developed. However, the
theoretical and technical knowledge necessary to understand and use LOs is beyond the reach of
many teachers, primarily because: (1) specifications are difficult to understand and use without
assistance, and also, it is unclear which one to use among the multiple content models and metadata
schema, (2) specifications of standards are too general to concisely describe the semantic richness of
specialized LO collections, such as Linguistics, Literature, Archeology, etc. (Boot 2005: 76,77), in
spite of the extension mechanism the standards provide. Finally, (3) the LO support technology –
authoring tools, tools for building LORs and LORs – are unknown and inaccessible to IT-illiterate
teachers.
To deal with the first two problems, an open and flexible LO model has been developed: the Virtual
Object Model (VOM) (Sierra and Fernández-Valmayor 2006). This model allows users (teachers and
researchers) to: (1) easily define specialized metadata and vocabularies as well as using metadata
standards; and (2) freely build LO content by using any resource: other LOs or even resources from
other LOs. From a conceptual point of view, a LO has three parts (Figure 2a):
Object Data. A set of specialized attribute-value pairs to describe the specific features of the
object in its scientific or humanistic domain.
Object Metadata. A set of attribute-value pairs to document and classify the LO. It can be taken
from any standard proposals such as LOM or Dublin Core, or can be user-defined.
Content. A set of resources (digital files) with an individual set of attribute-value pairs describing
each of those resources. Resources and resource metadata are freely added by teachers.
Resources can be: (a) own resources, (b) foreign resources, which are references to resources
owned by other LOs; and (c) LO’s resources, which are references to other LOs. Foreign and LO
resources allow the establishment of basic relationships between LOs and the construction of
more complex LOs by adding simpler ones (Figure 2b).
(a) (b)
Data
…….
Metadata
Resources
…
VO
resource
…
Foreign
resource
Figure 2: (a) VO model; (b) VOs can be linked together by using Foreign and VO resources
At this point, it is worth mentioning that the VOM is compatible with the standard IMS Content
Packaging specification to package and distribute LOs (IMS 2003). This functionality is already
implemented in the OdA Repository (see section 2.3). However, at this moment there is no support for
the most general and recent distribution specification IMS Common Cartridge (IMS 2011).
Using the VOM, teachers who have no knowledge of either LO models or authorizing tools can easily
build LOs tailored to their needs (Figures 3, 4 and 5). However, from a design standpoint the model
does not ensure content modularity. For this reason, we developed a tool for assessing the quality of
materials which, applied as self-evaluation during the construction process, guides content design
toward modularization, accessibility and interoperability.
237
…

Ana Mª Fernández-Pampillón et al.
Figure 3: Modular content of a LO from the Philology collection
Figure 4: Example of domain-specific metadata (Second Language Teaching)
2.2 An accessible quality model for Learning Objects
A LO Quality Model is useful not only as an evaluation tool itself but also to address the construction
of LOs if applied during the construction of digital learning materials. This type of evaluation, called
process evaluation can be used to formatively assess the planning, design, development and
implementation of LOs (Williams 2000). Thus, if a quality evaluation tool is applied as part of the task
of creating digital teaching materials in order to improve their quality, it can guide teachers in the
creation of LOs. However, the quality evaluation models available for instructors (Williams 2000;
Taveira and Azevedo 2010; Watson 2010b), are not always easy to understand or apply by the mostly
238

Ana Mª Fernández-Pampillón et al.
IT-illiterate Humanities teachers nowadays. Firstly, many educational software quality models focus
on educational aspects at the expense of technical aspects (Vidal et al. 2010: 343). Secondly, these
models are not user-friendly evaluation tools, convenient for instructors with no theoretical knowledge
or expertise in the creation of LOs. Thirdly, there is a lack of consensus regarding the concept of
quality applied to teaching materials (Dondi and Moretti 2007).
Figure 5: General metadata in the IEEE-LOM standard
To overcome this problem in low-tech contexts, we have developed an accessible Quality LO tool,
called COdA (Fernández-Pampillón et al. 2011a). Under COdA criteria, a LO is good quality if it helps
instructors and students to learn more and in less time, and if it is used frequently ― which ultimately
means that it is reusable. To apply COdA, teachers have to fill in a questionnaire of ten quality criteria
with the help of a short manual with instructions and examples. COdA is based on LORI (E-Learning
Research and Assessment Network 2010), and educational content quality evaluation models
developed in Spanish Universities, notably the UNED (Universidad Nacional de Educación a
Distancia 2011), UCM Virtual Campus (UCM-OCW, 2011), HEDOAR (Morales et al. 2009) and the
University of Murcia (DIGITUM 2011). It also includes basic content accessibility recommendations
from W3C (Chisholm and Vanderheiden 1999) and the IMS Global Learning Consortium (IMS
guidelines 2002).
2.3 An easy-to-use Learning Object Repository for specialized LO collections
To organize and storage the specialized collections of LOs, LORs have been used. A LOR stores
both LOs’ content and metadata, either by storing them physically together or separately in an on-line
database. Typically, A LOR uses fixed metadata scheme (usually IEEE-LOM or Dublin Core profiles)
to support simple and advanced queries as well as browsing through the material by subject discipline
(Neven and Duval 2002). However, LORs do not permit the custom and dynamic definition of domainspecific
databases for the authoring of specialized LO collections or their searching according to their
specific characteristics (Tzikopoulos et al. 2009). Furthermore, in Spain there is a lack of university
LORs (REBIUN 2007). Therefore, teachers end up installing and maintaining their own educational
and scientific collections of digital materials, which is far too demanding for the IT-illiterate teacher.
OdA repository is an easy-to-use LOR for the management and definition of specialized LOs. OdA is
a UCM software tool in whose development some of the authors of this paper participated. Its
development was funded through national research projects and its purpose is to provide teachers,
researches and students with a simple and flexible computer system to disseminate their educational
and research materials (Sierra and Fernández-Valmayor 2006). Apart from the typical LOR’s basic
functionalities, OdA also provides:
239

Ana Mª Fernández-Pampillón et al.
The possibility to dynamically define customized data, metadata and content for LO collections
according to the knowledge domain and the specific didactic needs of users (Figure 6);
Because of (1), an intuitive mechanism for domain experts (teachers and researches) for the
location and selection of LOs based on their discipline’s specialized attributes and terminology.
Hence, when exploring and searching LOs in OdA, students use language and conceptual
domain classifications of each discipline, which helps to improve the learning of terminology and
conceptual structure of that domain.
The possibility to export LOs to an IMS Content Package to publish them in other e-learning tools
such as Learning Management Systems (e.g. Moodle).
Figure 6: Metadata schema management interface
Finally, a minimal set of technical requirements is needed, concerning both IT equipment and
skills. OdA can be installed in a personal computer with a self-installing version.
Therefore, OdA is a software solution to easily create and manage specialized LO collections in
tertiary education low-tech contexts (where teachers are not computer specialists and usually have
poor IT support). It offers an alternative to international repositories, with greater functionality and
interoperability but more complex to use, less flexible to precisely store and manage specialized LOs
and requiring larger equipment and staff infrastructures.
2.4 An inductive methodology to create Philology LO collections
The incremental construction methodology of LO collections is based on the above-presented
elements – an open LO model, an accessible quality tool and an easy-to-use LOR – to inductively and
systematically generate tertiary LO collections in six stages:
Creation/reuse. The teacher or team of teachers create or digitize their teaching materials
according to the usual procedures.
Inductive creation of metadata schema. Each faculty member involved in the collection defines a
metadata record for their materials. A minimum set of metadata to describe and classify all the
initial materials is selected from all the metadata records.
Evaluation. Teachers evaluate the quality of the materials they have created, or are creating, by
means of the COdA tool. Once evaluated, they can continue with this step, or stop if the
evaluation has been satisfactory.
Fine-tuning. If during the evaluation the material does not meet some criteria, it is modified so as
to obtain a higher score. After fine-tuning the material, the teacher usually returns to step 3 to reevaluate
the new version.
240

Ana Mª Fernández-Pampillón et al.
Storage and Documentation. Instructors store LOs in the repository. This action involves the use
of the set of metadata created in step 2 to describe the LO collection. If new metadata are
needed, teachers add them. Changes or deletion are agreed on by the different teachers working
on the same collection.
Teachers can edit or delete their LOs and metadata schema at any time. They can also include
the result of the evaluation as one more LO resource.
3. Experimentation and results
The proposed strategy was tested in the UCM School of Philology because: (i) it is a low-tech context,
(ii) we had institutional support, and (iii) there was a demand from faculty members for help to reliably
create good digital teaching materials for use in the UCM’s Virtual Campus and the new digital
language labs (Fernández-Pampillón et. al. 2011b).
3.1 Participants
Experiments were conducted with a group of 33 participants belonging to three different types: (i)
faculty from nine different philology departments; (ii) the School IT Technicians and (iii) UCM eeducation
researchers.
Participants were organized into three working groups:
1) Teaching Resources Team (TRT), with 20 faculty members in charge of testing the strategy.
Only one of the team members had advanced IT skills; two of them had average IT skills; the
other seventeen had low to intermediate IT skills. None of the members had LO expertise.
2) Modelling Team (MT), responsible for developing the quality guidelines tool (COdA), and also
in charge of inducing the general descriptive model of teaching resources prepared by the TRT.
All group members had low to intermediate IT skills. None of the members has LO expertise.
3) Technical Team (TT), responsible for providing the necessary support to use the LO repository,
advising the other teams on technical matters and training the School’s technician in the
maintenance of the repository (backup and possible failures). Only one of the members had LO
expertise.
3.2 Methodology
The experiment was carried out in five phases:
Phase 1. Preparation of a quality evaluation tool draft by the MT.
Phase 2. a) Preparation of two samples of digital teaching materials and validation of quality
guidelines by the TRT (COdA guidelines). By the end of this phase, the existing version of the COdA
tool was considered to be final.
Phase 3. Writing up the documentation and classification of materials by the MT, done inductively. By
the end of this stage, the existing version of the basic documentation model (metamodel) was
considered to be final.
Phase 4. Building the LO collection in the OdA repository. By the end of this phase the preparation of
the first LO collection of the Philology School was considered to be complete and ready to be open to
the rest of the faculty.
Phase 5. Strategy evaluation. The following results are collected and analyzed: a) the LO collection
content models; b) the LO metamodel and surveys on the usability of OdA and COdA with feedback
from the TRT team’s teachers. Conclusions are then drawn.
3.3 Analysis of results
LO collection Content Model. Concerning the content model, this has been observed to evolve as
teachers gain experience in the use of the repository regardless of their initial IT skills. They are
very simple models at first: a list of own-resources (Figure 3, above); more complex LOs are then
created which include other related LOs forming small networks of LO resources (Figure 7).
241

Ana Mª Fernández-Pampillón et al.
Figure 7: Example of a LO representing a language course consisting of other LOs (book and audio)
With respect to the quality evaluation tool (COdA), beyond our initial expectations teachers have
reported that, through evaluation, they have managed to differentiate learning object from
teaching material, which was not always clear before. The implementation of some minor
changes ― i.e. the incorporation of a template to grade an exercise and thus give feedback to the
student, or a clearer statement of the didactic goals in the LO to raise the punctuation of some
criteria ― appeared to improve the reusability of the final product (Figure 8).
However, not all the COdA criteria were easily applicable. For instance, the specification of
accessibility has been reported at times not to be suitable for audiovisual objects targeted at auditory
perception tasks (see Arús et al. 2011).
Figure 8: COdA criteria evaluation results (from 1 to 5)
Concerning the quality of the LOs (Figure 9), around 84% of the teaching materials that have
been built according to the Quality Learning Object (QLO) building procedure can actually be
considered QLOs (3 out of 35 do not meet the reusability criterion).
242

Ana Mª Fernández-Pampillón et al.
Figure 9: LO quality evaluation results (from 1 to 5)
Finally, with respect to the use of the flexible LO metadata model, it has been observed that
teachers prefer developing specialized classification vocabularies and attributes rather than using
the more general ones provided by the standards. Furthermore, data and metadata schema have
been dynamically changed as the collection grows.
4. Conclusions and future work
This work is a contribution to the efficient creation of good digital teaching resources in tertiary
education by IT-illiterate or semi-illiterate teachers with few IT and technical staff resources. It is a
strategy for the inductive construction of LOs which has been created and tested in collaboration with
IT-illiterate Humanities teachers.
In the generation of our LOR, faculty members were not given any predetermined content model or
metamodel. Instead, they devised their own by means of an inductive method to obtain the minimal
most specific data and metadata scheme that suited the needs of all the participants in the project. By
so doing, we ensured that every scheme element was really necessary and that it could be
understood by end-users. Furthermore, this methodology implies that the LOs and the models are
scalable and can be expanded to suit future needs.
As seen, a quality-assessment tool was also designed to outline certain criteria for teachers to selfevaluate
their LOs and thus enhance their quality and, ultimately, their usefulness. Since the wording
comes from the instructors themselves, the usability of the tool is guaranteed. Interestingly enough,
the need of self-assessment fosters the reflection on one’s own LOs, which not only helps to improve
the objects but also facilitates the intuition of the mere difference between a LO and a more traditional
teaching material, which many instructors did not grasp at the beginning. It encourages the habit of
sharing teaching materials, something not very frequent among instructors in Spanish universities. In
this regard, we suggest that the institutions and committees working on the quality of education
consider and value quality self-evaluation as a further indication of quality, very economical and with
an important positive effect on the quality of teaching.
The LO-building methodology presented allows a high degree of flexibility, which makes it especially
well suited for supporting author-driven, incremental and open approaches to LO production. It is
particularly appropriate in a university environment where teachers are responsible for the creation of
their specialized teaching material with little or no IT support. Indeed, in these scenarios the first
stages of the LO building process were devoted to produce very simple LO counterparts of the preexisting
learning materials. Later stages produced higher-order LOs of more abstract nature (e.g.
foreign language courses containing texts, audios, presentation and exercises related to different
aspects of the learning process). During the successive stages the metadata model was also
extended, fine-tuned and corrected to solve inconsistencies and redundancies.
243

Ana Mª Fernández-Pampillón et al.
Compared to other approaches, our proposal provides a solution accessible to any instructor or team
of instructors with limited computer skills as well as human and technological support. The
experimental results are satisfactory to the extent that 1) the feasibility of the procedure has been
proven by means of a realistic project to build a LO collection in a non-technological field such as that
of Philology; and 2) the procedure has succeeded in improving the quality of teaching materials in
both the pedagogical and technological aspects.
It is not possible, however, to ensure that by applying this approach 100% of the materials in a
collection created by a group of teachers will be LOs, since, as shown by the first experimental
results, the success (84% in our case) depends on 1) the degree of involvement of instructors in
improving the quality of their materials; 2) having some sort of technical support so as to know how to
apply IT quality criteria such as accessibility and interoperability.
The next step in this research will be the study of the use (and reuse) of LO collections in the new
electronic learning environments in non-technological fields of knowledge. In this sense, we will test
the integration of the LO repository OdA into the UCM Virtual Campus with the aim of using it in virtual
sessions, in the classroom or in the language lab. If IT-illiterate faculty incorporate LOs both into their
classroom and virtual teaching in a natural way, and if the LOR keeps growing by the addition of new
LO collections, then the LO creation strategy will have proven its usefulness in the creation, use and
dissemination of digital research and teaching materials in non-technological domains.
Acknowledgments
The research presented here has been funded by the Universidad Complutense de Madrid through
the Proyecto de Innovación Educativa y Calidad de la Docencia 268-2010 UCM; and by the Spanish
Ministry of Science and Innovation through the following two research projects: Arquitecturas
Avanzadas en Campus Virtuales (TIN2009-14317-C03-01/TSI), and Un Enfoque Generativo para la
Construcción de Herramientas de Producción y Despliegue de Objetos Educativos en el Campus
Virtual (TIN2010-21288-C02-01).
References
Arús, J., Fernández-Pampillón, A. M., Lahoz, J. M., Domínguez, E. and De Armas, A. I. (2011) “Learning Object
Management for IT-Illiterate Instructors”, EDULEARN10 Proceedings, [online],
http://library.iated.org/view/ARUSHITA2011LEA.
Adorni, G., Brondo, D. and Coccoli, M. (2008) "Design and Implementation of a User Friendly Environment for
Learning Objects Creation”, IFIP 20 th World Computer Congress, IFIP TC 3 ED-L2L Conference, Milano,
[online], http://www.springerlink.com/content/a8763767281k2220/fulltext.pdf.
Boot, E. (2005) “Novice and Experienced Instructional Software Developers: Effects on Materials Created with
Instructional Software Templates”, Building-block Solutions for Developing Instructional Software, PhD,
Open Universiteit Nederland Heerlen, [online],
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.134.6936&rep=rep1&type=pdf.
Chisholm, W.; Vanderheiden, G. (1999) Tabla de Puntos de Verificación para las Pautas de Accesibilidad al
Contenido en la Web 1.0 [Spanish version]. W3C, [online],
http://www.discapnet.es/web_accesible/wcag10/full-checklist.html.
DIGITUM (2011) Depósito Digital Institucional de la Universidad de Murcia, [online], http://digitum.um.es/xmlui/.
Dondi, C. and Moretti, M. (2007) ELearning Quality in European Universities. UNIQUe, [online],
http://unique.europace.org/pdf/WP1-report-v5_FINAL.pdf.
Dublin Core Education Application Profile (2007) Working Draft of v0.4, [online],
https://docs.google.com/Doc?id=dn8z3gs_38cgwkvv.
Duval, E. (2004) “Learning Technology Standardization: Making Sense of it All”, Vol 1, No. 1, Computer Science
and Information Systems, [online], http://www.comsis.org/pdf.php?id=nnn-1102.
E-Learning Research and Assessment Network (2010) “About LORI”, [online],
http://www.elera.net/eLera/Home/About%20%20LORI/.
Fernández-Pampillón Cesteros, A. M., Domínguez Romero, E., Cazorla Vivas, M. C., De Armas Ranero, I. and
López-Varela Azcárate, A. (2011a) “Herramienta para la revisión de la calidad de Objetos de Aprendizaje
universitarios (COdA): Guía del Usuario. v.1.0. Documento de trabajo”, [on-line],
http://eprints.ucm.es/12533/.
Fernández-Pampillón, A. Sevilla, J. and Poves, A. (2011b) “Conclusiones finales y acciones futuras”, El
Laboratorio de Idiomas y la Enseñanza-Aprendizaje de Lenguas, Editorial Complutense, Madrid.
Gibbons, A. S.; Nelson, J. and Richards, R. (2000) “The Nature and Origin of Instructional Objects”, The
Instructional Use of Learning Objects, ed. D. A. Wiley, [online],
http://reusability.org/read/chapters/gibbons.doc.
IEEE (2002) IEEE Standard for Learning Object Metadata. IEEE Standard 1484.12.1-2002, [online],
http://standards.ieee.org/findstds/standard/1484.12.1-2002.html.
244

Ana Mª Fernández-Pampillón et al.
IMS (2003) “Content Packaging Public Draft 2. IMS”, [online], http://www.imsglobal.org/content/packaging/.
IMS (2011) “Common Cartridge. Version 1.1 Final”, [online], http://www.imsglobal.org/cc/index.html.
Jesukiewicz, P., dir. (2009) 14-08-2009-last update, SCORM, 2004. Version 1.1. Testing requirements. Sharable
Content Object, [online],
http://www.adlnet.gov/Technologies/scorm/SCORMSDocuments/SCORM%202004%204th%20Ed%20V1.1/
Documentation%20Suite/SCORM_2004_4ED_v1_1_TR_20090814.pdf.
LOM-ES (2010) “LOM-ES application profile for standardized Digital Learning Objects metadata” UNE-
71361:2010 AENOR, [online],
http://www.aenor.es/aenor/normas/normas/fichanorma.asp?tipo=N&codigo=N0044685&PDF=Si.
Morales, E., Garcia, F., Rego, H. and Moreira, T. (2009) “Learning Objects Quality Management for e-Learning
Systems”, International Journal of Technology Enhanced Learning, nº 1, pp. 193–204.
Neven, F. and Duval, E. (2002) "Reusable Learning Objects: A Survey of LOM-Based Repositories", Proceedings
of the tenth ACM international conference on Multimedia (MULTIMEDIA '02) ACM, New York, pp. 291.
REBIUN (2007) Encuesta sobre la Situación de los Repositorios de Materiales Didácticos y Objetos de
Aprendizaje en las Universidades Españolas, [online], http://www.rebiun.org/doc/RepositoriosE.doc.
Sierra, J.L. and Fernández Valmayor, A. (2006) "A Heritage Dissemination Approach for the Production and
Maintenance of Repositories of Learning Objects", [online], http://www.e-ucm.es/drafts/e-UCM_draft_57.pdf.
Smith, R.S. (2004) “Guidelines for Authors of Learning Objects (White Paper)”, [online],
http://archive.nmc.org/guidelines/NMC%20LO%20Guidelines.pdf.
Taveira, P. and Azevedo, I. (2010) "Supporting the Reuse of Learning Objects through the Use of IMS LD
Templates", Proceedings of the 9th European Conference on eLearning, eds. P. Escudeiro and C. Vaz de
Carvalho, [online], http://www.academic-conferences.org/pdfs/ECEL_2010-Booklet.pdf.
Universidad Nacional de Educación a Distancia (2011) Portal de Cursos Abiertos de la UNED. Open course
Ware, [online], http://ocw.innova.uned.es/ocwuniversia.
UCM abierta, [online], https://www.ucm.es/campusvirtual/ucmabierta/index.php.
Uceda, J.; and Barro, S. (dirs) (2010) “Evolución de las TIC en el Sistema Universitario Español 2006-2010”.
Conferencia de Rectores de las Universidades Españolas. Comisión Sectorial TIC, [online],
http://www.crue.org/export/sites/Crue/Publicaciones/Documentos/Universitic/UNIVERSITIC2010b.pdf.
UCM-OCW (2011) UCM OpenCourseWare, [online],
https://www.ucm.es/campusvirtual/ucmabierta/index.php?ac=inicio.
Verbert, K. and Duval, E. (2008) “ALOCOM: A Generic Content Model for Learning Objects, International Journal
on Digital Libraries, Vol. 9, Num 1, pp 41-63. SpringerLink, [online],
http://www.springerlink.com/content/j57176h118312386/.
Vidal, C., Alejandra Segura, N., Pedro Campos, S. and Sánchez-Alonso, S. (2010) “Quality in Learning Objects:
Evaluating Compliance with Metadata Standards”, eds. S. Sánchez-Alonso and I. N. Athanasiadis,
Metadata and Semantic Research. Proceedings of the 4th International Conference, MTSR 2010,
Springer, Berlin, [online], http://www.springerlink.com/content/x512216255073457/fulltext.pdf.
Tzikopoulos, A., Manouselis, N., and Vuorikari, R. (2009) “An Overview of Learning Object Repositories”, ed. T.
Halpin, Selected Readings on Database Technologies and Applications, pp. 85-94.
Watson, J. (2010a) "A Case Study: Developing Learning Objects with an Explicit Learning Design, Electronic
Journal of e-Learning, vol. 8, pp. 41-41-50, [online], http://www.ejel.org/Volume-8/v8-i1/v8-i1-art-5.htm.
Watson, J. (2010b) "Many from One: Repurposing Learning Objects to Meet Discipline-Specific Needs", IADIS
Journal on WWW/Internet, vol. 8, no. 1, pp. 16-16-26, [online], http://eprints.soton.ac.uk/170457/.
Wiley, D.A. (2000) "The Nature and Origin of Instructional Objects. " in The Instructional Use of Learning Objects:
Online Version., ed. D.A. Wiley, [online], http://reusability.org/read/chapters/wiley.doc.
Williams, D.D. (2000) "Evaluation of Learning Objects and Instruction Using Learning Objects", The Instructional
Use of Learning Objects: Online Version, ed. D.A. Wiley, [online],
http://reusability.org/read/chapters/williams.doc.
245

Cognitive Communication 2.0 in the Classroom –
Resonance of an Experience in Higher Education
Sérgio André Ferreira 1 , Cornélia Castro 1 and António Andrade 2
1
School of Education and Psychology, Portuguese Catholic University, Oporto,
Portugal
2
School of Economics and Management, Portuguese Catholic University,
Oporto, Portugal
sergioandreferreira@gmail.com
corneliacastro@gmail.com
aandrade@porto.ucp.pt
Abstract: The communication in the classroom is often carried out on a one-to-many basis, with the teacher,
before an audience of dozens of students, playing a traditional role. In this context, still commonly practiced, it is
pertinent to introduce mechanisms of interaction mediated by technology, since research shows a significant
correlation between the use of technology, the time spent in social media and the engagement of students. In
fact, literature shows that social media has attracted the interest of academics more likely to use technology in
education, who thereby seek new ways to motivate their students to a more active learning. The adoption of
cognitive communication 2.0 morphology, in traditional contexts such in communication one-to-many, is a
challenge to be overcome. We consider so of great importance creating and evaluating resources and
pedagogical practices that are aligned with this new paradigm. In this study, we intend to make a contribution to
understanding the problematic of the morphology of cognitive communication in the context of the classroom in
Higher Education, with the integration of web 2.0 tools. On the approach to the problem, we have explored a
PowerPoint presentation with the integration of the micro blogging tool Twitter, as a basis for addressing the
characteristics of cognitive communication 2.0. For data collection a questionnaire was designed, based on
literature, and intended to evaluate several dimensions of the resource used, namely: i) pedagogical issues, ii)
technological aspects, iii) cognitive learning; iv) interactions in the classroom; v) positive behaviour in the
classroom; vi) negative behaviour in the classroom. The goals of this study are: i) to validate the instrument for
data collection, ii) to assess the perceptions of students regarding the effects of the resource and pedagogical
practices used in the classroom dynamics iii) to set in context and to relate the cognitive communication 2.0 in
the classroom with other components of the Hybrid Institutional Personal Learning Environment. This is an
exploratory type research, since it seeks to provide a greater familiarity with the problem and to identify
dimensions and items to be included in the questionnaire. The data collected will be processed under a
quantitative perspective. Considering, therefore, the nature of the study we did not seek to establish correlations
between variables, but only to identify trends, using descriptive statistics. It is expected that the results obtained
will contribute to the articulation of web 2.0 tools with traditional cognitive communication in the classroom, in
such a manner that positive impacts will result in pedagogical and technological effectiveness and thus in
students learning achievements.
Keywords: classroom; cognitive communication; learning; micro blogging; Twitter; web 2.0
1. Introduction
Information Technologies (IT), specially the “web phenomena”, have contributed to changing the way
people work together, share resources, co-produce, co-act and get involved in activities that benefit
all (Fuchs et al., 2010). Nowadays, expressions such as “collaborative learning”, “learning
communities”, “media in education”, “social media” and other similar ones, are essential in educational
investigation. However, research on these topics focuses on online environments or face-to-face
groups of limited size. Studies in which these principles are applied to a classroom with dozens of
students are rare.
In fact, research shows that the classroom has been losing its historic centrality in favour of new
agglutinating poles such as the Personal Learning Environment (PLE) and the Social Learning
Network (SLN), usually associated with spaces outside the classroom – Cloud Learning Environment.
However, despite the development of on-line learning systems and b-learning, classroom learning is
still largely dominant, and the organization of activities continues to have the classroom learning as
the nucleus. Hence the importance of designing activities and creating resources that, in conjunction
with this new paradigm, promote the exploration of the potential of the new ways of learning that
dominate the Cloud Learning Environment.
246

Sérgio André Ferreira et al.
This article aims to contribute to the study of the issues related with the cognitive communication
morphology in classroom lectures, as part of Higher Education, where often the communication is
done from one-to-many. To do so we started with a PowerPoint presentation with the integration of
micro blogging tool Twitter, as a basis for addressing the characteristics of communication 2.0 in
classroom in lectures of a more expository nature.
The methodology of data collection used was a questionnaire built based on literature review, with
which we intended to assess various dimensions of the resource used in classroom lectures and to
verify if the integration of Twitter in the presentation contributes to the upgrading of a cognitive
communication 1.0 (one-way communication, one-to-many, low or non-existent interaction) to
cognitive communication 2.0 (many-to-many, interaction between all the participants). The results of
this study are preliminary and intended to serve as a preliminary approach to the subject.
2. Related work: Change challenges and pedagogical innovation in
institutions of higher education through technology
New ways of communicating and interacting in a society immersed in technology are intrinsically
linked to the imposition of new ways of teaching and learning, which results in the redefinition of
political and pedagogical models. Social and economic factors call for the use of technology as
pedagogical support. Mark Prensky (2001) was the first to use the term "digital natives", which is
associated with expressions such as "residents", "Generation Y" or "Net Generation”. These students
grow up in environments immersed in technology and have different preferences and skills in key
areas related to education, particularly in making use of the immense potential of web 2.0 (Castañeda
& Soto, 2010; Kennedy et al., 2009), especially social software tools (blogs, micro blogs, sites of
video sharing, social media, wikis or podcasts), which facilitate not only the emergence of
communities of users, but also the involvement in social media.
Despite the characteristics of this new generation of students, we cannot assume that all who reach
higher education already possess the necessary skills to use web 2.0 technologies such as learning
tools (Castañeda & Soto, 2010). On the other hand, it is also observed that the Higher Education
Institutions (HEIs) are still inadequately prepared to work with students who have completely different
technical skills and learning preferences (Bennett, Maton, & Kevin, 2008).
To address this complex situation, the HEIs must create programmes and define methodologies that
enhance the use of the enormous educational potential of web 2.0. Thus, it is expected that students
will develop learning skills in this context and increase their motivation and, as a result, will increase
the chances of achieving good academic results.
In the current scenario, it matters that HEIs reconcile their conception of the educational process with
new ways of learning and student expectations. The HEIs should not overlook the fact that IT offers
the students the opportunity to control and manage their own learning beyond the institutional vision.
The globalization of the sources of knowledge, that brings the world to the school and the growing
importance of social media and collaborative work of smart mobs (Tapscott & Williams, 2008)
emphasize the value of the Social Learning Network (SLN), which creates the need for the HEIs to
evolve to a Hybrid Institutional Personal Learning Environment (HIPLE) architecture, as a bridge
between the vision of the institution and the Personal Learning Environment (PLE) of the student.
The introduction of technology in the facilitation of cognitive function in education means a challenge
for applied research with very complex and slow progress. Among the changes that have been
introduced in training models under e-learning, b-learning and face-to-face, it is in the latter where the
pace of change is slower. A. D. Figueiredo (2009) states that the HEIs’ face-to-face model is also the
most traditional one and, in essence, it is based on four functions: i) transmission of content provided
by lectures, often masterful, ii) application of concepts, iii) group work and iv) evaluation. These
functions are shown in table 1.
Therefore, the classroom continues to be a place of excellence in the communication of knowledge
that includes multiple educational concepts. In particular, there is, above all, the transmission of
information from one-to-many, dialogue, teamwork and role-playing. When technology is used it is
explored in the context of the laboratory. Personal computers, tablet PCs or smart phones are poorly
explored. In this context, highly practiced, it is pertinent to introduce mechanisms of interaction
mediated by technology, since research shows a significant correlation between the use of technology
247

Sérgio André Ferreira et al.
and time spent with social media and students engagement (Chen P., 2010; Junco, Heiberger, &
Loken, 2010).
Table 1: Pedagogical face-to-face dominant model in the HEIs (Figueiredo, 2009)
Pedagogical face-to-face model
Transmission of content Lectures
Lectures
Application of concepts
Lectures and practical lessons
Practical lessons
Group work
Laboratories
Projects
Tests/exams
Evaluation
Projects
Essays and Presentations
In fact, it has been observed that the social media – a collection of Internet web sites, services and
practices that support collaboration, community building, participation and sharing – has attracted the
interest of academics more likely to use technology in education and who seek new ways to motivate
their students to a more active way of learning (Junco, et al., 2010).
In this study we focus on the context of the lectures, where teachers are faced with numerous
students and have the need to activate mechanisms of direct instruction, either to save time, or
because this is the teaching model that they are comfortable with. Electronic presentations as a way
to transmit knowledge are a means commonly used in these classes (James, Burke, & Hutchins,
2006).This is a reusable resource, a facilitator of discourse organization and an integrator of multiple
media which can serve different learning styles, as well as becoming a stepping stone of motivation
and it is also conducive to note taking by students.
These presentations are massively materialized on technologies such as PowerPoint and Prezi which
have mechanisms to support text, video, image, flash animation and sound, but they also have the
ability to interact with the so-called web 2.0 systems, such as the micro blogs. A PowerPoint or a
Prezi presentation can therefore be linked with the micro blogging application Twitter, allowing the
teacher the opportunity to speak to his students.
Recent research about the use of Twitter in academic work shows that although 85 % of
undergraduates have a Facebook account, teachers prefer to integrate Twitter into the process of
teaching and learning (Junco, et al., 2010). In the category of micro blogs, Twitter, designed in 2006
by Jack Dorsey, allows users to share messages up to 140 characters. This system also allows
sending messages to a direct channel specifically created to exchange information (# hashtags) and
the vote for alternative options which are placed under review (by vote tweet @ x key_word).
Individual or business initiatives (Elliot, 2011) have developed ADD-INS for Prezi and PowerPoint,
allowing the following dynamics for those who have a Twitter account (teachers and students):
Creation of a channel (not compulsory) for comments on the presentation that is being made;
Students can comment directly on Twitter what they see and listen in the classroom as well as
what they read from sharing with peers (virtual classroom);
The teacher may have prepared additional comments for each slide, that hidden in "notes", can
be sent to Twitter whenever it is projected;
The teacher can capture and project in all, or in some of the screens, what is being shared on
Twitter;
The teacher can ask multiple-choice questions that are answered on Twitter, and the percentages
of the responses for each option can be projected in a slide.
This model of communication in the classroom will increase the level of participation by: i) providing
voice and turn to all the students, ii) facilitating and asking for the participation of more reserved
students in oral participation iii) engaging the learning community in discussions about the theme and
iv) exploring the acuity of young people to use IT.
248

3. Evaluation methodology
Sérgio André Ferreira et al.
As a methodological approach to the problematic of the morphology of cognitive communication 2.0 in
the context of the classroom, we used a PowerPoint presentation with the integration of Twitter. The
exploitation of this resource was made during lectures to several masters classes, at the Portuguese
Catholic University – Regional Center of Porto (Catholic – Porto). In table 2 the demographic
characteristics of the participants in this study are shown.
Table 2: Demographic aspects of the investigation
Total 122
Gender Male: 29% Female: 71%
Age
Group
Training
Undergone
20 to 29 years old: 37 %
30 to 39 years old: 26 %
40 to 49 years old: 20 %
50 to 59 years old: 17%
Marketing: 3 %
Services: 3 %
Education and Music: 15 %
Computing and Education: 3 %
Pedagogical Supervision: 22 %
Religious Sciences: 4 %
Wounds and Tissue Viability: 4 %
Infection and Health Care: 8 %
School Management and Organization: 15 %
Childhood Education and Special Education: 23 %
The way the operationalization of the resource was made is represented, in schematic form, in Figure
1: the integration of Twitter in PowerPoint, with reference to a hashtag #, through which students
could ask and answer questions, vote on matters presented and answer multiple choice questions.
Thus, the electronic presentation, not only had the traditional function of transmitting information from
one to many, but it was also intended to foster interaction content-students, teacher-students,
students-students. The integration of Twitter in an electronic presentation, enables, potentially, a
paradigm shift in teaching: to the one-way communication teacher-class is added the value of the
interaction teacher-student-content. The feedback given by students is an important item because it
allows the teacher to suit his speech to the class and answer students' questions and comments that
appear in real-time presentation.
Figure 1: Integrating cognitive technologies in communication
At the end of the sessions, the participants were asked to fill a questionnaire to evaluate the resource.
The convenience sampling technique was our choice. Although not representative of the population ,
this sampling technique had the advantage of being a fast and simple one and, therefore, suitable for
preliminary studies, as in this case.
The questionnaire was made based on literature review (Hu, 2011; James, et al., 2006; Kurilovas,
2007; Nesbit, 2007; Nokelainen, 2006) and consisted of 41 items spread across six dimensions: i)
pedagogical aspects, ii) technological aspects, iii) cognitive learning iv) interactions in the classroom,
v) positive behaviour in the classroom; vi) negative behaviour in the classroom. The six evaluated
dimensions include: i) issues that students identify as central in the quality of digital learning
resources associated to more direct teaching, ii) the perceived effects by students in the field of
learning, behaviour and attitudes. A Likert scale of five points was used.
4. Presentation of results
The overall results of the questionnaire indicate that the respondents recognize the pedagogical and
technological potential of the resource in the six assessed dimensions, as well as its positive effects
on the quality of learning and type of interaction. In Figure 2 the assessment on pedagogical aspects
249

Sérgio André Ferreira et al.
is presented. In the seven analyzed items, the large majority of respondents evaluated the
pedagogical aspects of the course with level four, "agree" and level 5 "strongly agree." The global
average of the seven items corresponds to 55 % of responses at level 4 and 22 % at level 5, and the
importance of level 1 and 2 is negligible (0 % and 4 %, respectively).
Scale
1. Strongly disagree to 5. Strongly agree
NOP. No opinion
Legend
1. Scientifically rigorous
2. Pedagogically appropriate
3. It has the correct sequence
4. It facilitates the explanation of abstract concepts
5. It is presented clearly (spelling, grammar, ...)
6. It is easy to understand the goals
7. Appropriate level of detail
Figure 2: Pedagogical aspects
As far as the "technological aspects" is concerned (Figure 3), the respondents commented on the
appropriateness of the use of technology, design, usability, interface, added value compared to
printed material, the potential of technology in facilitating learning, building concepts and skills
development. The average of the eight items of this dimension indicates that 52 % of students "agree"
and 32 % "strongly agree" that the technology used was appropriate and that potentiated learning.
Like in the previous dimension, the number of respondents that gave unfavourable levels of answers
(levels 1 and 2) is negligible.
Scale
1. Strongly disagree to 5. Strongly agree
NOP. No opinion
Legend
1. Use of technology appropriate to the content
2. The design is appealing
3. The design facilitates the clarification of the
content
4. The resource is easy to use
5. The interface facilitates navigation
6. The resource is an added value compared to
printed material
7. The used technology facilitates learning
8. The technology helps building concepts and
developing skills
Figure 3: Technological aspects
Regarding "cognitive learning" (Figure 4), if one considers the average of the 11 items on this
dimension: 82 % of the respondents said that they "agree" (47 %) or "strongly agree" (35 %) that the
resource has positive effects. In the average of the 11 items, the value of the terms "strongly
disagree" and disagree" is located at 2 %. However, 7% of the respondents "disagree" and 1 %
"strongly disagree" that the resource facilitates taking notes (item 3).
The potential of the evaluated resource in the "interactions in the classroom" are also recognized by
the students. The average of the five presented items in Figure 5 indicates that 70 % of respondents
"agree" (40 %) or "strongly agree" (30 %) with the positive effects of the resource on this dimension.
The item 1 is the one that meets the highest percentage of negative and neutral answers: 2 %
"strongly disagree" 11 % "disagree" and 35 % "do not agree nor disagree" that the resource leads to a
better knowledge of the classmates in the classroom.
250

Figure 4: Cognitive learning
Sérgio André Ferreira et al.
Scale
1. Strongly disagree to 5. Strongly agree
NOP. No opinion
Legend
1. It helps to improve understanding
2. It facilitates memorizing information
3. It facilitates note taking
4. It favours a more appropriate pace of
presentation
5. It facilitates fluency when transferring
information in the classroom
6. The examples presented become clearer
7. It fosters a better coordination between the
classroom activity and the texts
8. It makes learning more motivator
9. It allows the definition of several ways of
learning
10. It favours the immersion in learning and
encourages further learning of contents
11. Mobilizes prior knowledge
Scale
1. Strongly disagree to 5. Strongly agree
NOP. No opinion
Legend
1. It leads to a better knowledge of classmates in
the classroom
2. Promotes the will to participate in class
3. It facilitates a closer relationship with the
teacher and classmates
4. It facilitates discussion in the classroom
5. It allows participation via a technological
channel
Figure 5: Interaction in the classroom
The results in the dimension "positive behaviour in the classroom" (Figure 6) are in agreement with
those of the other dimensions. Considering the average of the 7 items, 75% of respondents "agree"
(49 %) or "strongly agree" (26 %) that the resource has positive effects on behaviour in the classroom.
The item 1, which states that the resource helps taking better notes in class, holds the highest
number of negative evaluations (1 % "disagree" and 15 % "strongly disagree"). This result is aligned
with item 7 "it stimulates coming to class to take notes (15 % of respondents answered "disagree") as
well as with item 3 in Figure 4.
Scale
1. Strongly disagree to 5. Strongly agree
NOP. No opinion
Legend
1. It helps taking better notes during the lectures
2. The visual images presented in the resource
help to remember the contents during exams
3. It allows a better understanding of the keypoints
highlighted during lessons
4. It enhances attention in the classroom
5. This resource is used as a support to group
work, negotiation and dialogue in classroom
6. It helps keeping interest alive during lectures
7. Encourages coming to class to take notes on
important topics
Figure 6: Positive behaviour in the classroom
In the dimension "negative behaviours in the classroom", the scale should be read in reverse: the
more favourable views about the resource are located on level 1 and the less favourable in level 5.
251

Sérgio André Ferreira et al.
Looking at Figure 7 we can conclude that: i) 36 % "agree" and 19 % "strongly agree" that the resource
increases the possibility of keeping side conversations while the teacher presents the subject, ii) 8 %
"agree" and 8 % "strongly agree" that the resource reduces the motivation to be present in class. This
figure (16 %), although low in percentage terms, assumes an important meaning and is not aligned
with the very positive evaluation made to other dimensions and iii) 24 % "agree" and 7 % "strongly
agree" that they are more likely to skip school if they know that the resource will be available on the
web.
Scale
1. Strongly disagree to 5. Strongly agree
NOP. No opinion
Legend
1. It increases the possibility of keeping side
conversations while the teacher presents the
subject
2. It reduces the motivation to be present in class
3. The probability to miss class is higher if the
students know that the resource will be available
on the web
Figure 7: Negative behaviour in the classroom
Table 3 shows the average of responses in each dimension for each course attended. Only courses
with 10 or more students were taken into consideration as it was assumed that courses with fewer
students didn’t have any statistical significance. However, it is important to state that the same class
could integrate students from various courses, which increases the dimension of students in each
session. What was meant by this analysis was to identify possible differences in opinions according to
the type of training. The students from School Management and Organization and Childhood
Education and Special Education courses are the ones that, overall, make a more positive
assessment of the resource used. On the same scale of 1 to 5, the average score for items 1 to 5
(item 6 was not considered in this average as it presents a reverse scale) was 4,3. The less
favourable reviews were noted in the course Pedagogical Supervision, with an average of 3,7.
Although the type and size of the sample do not make it possible to establish correlations between
variables, the results of this preliminary study indicate that there is material for further analysis in this
field.
Table 3: Average of results in each dimension per course
Training
Undergone
Pedagogy
Technology
Cognitive Learning
Education and Music 3,8 4,1 3,9 3,8 3,8 3,2 3,9
Pedagogical
Supervision
3,8 3,9 3,9 3,4 3,6 2,8 3,7
Infection and Health
Care
4,1 4,0 4,4 4,2 4,0 2,1 4,1
School Management
and Organization
4,0 4,4 4,5 4,3 4,3 2,5 4,3
Childhood Education
and Special Education
4,2 4,4 4,3 4,0 4,2 2,4 4,3
An analysis of differences in various dimensions, by gender, was made in this study. In dimensions 1
to 5 no significant differences are visible. Gender disparities are only visible in dimension 6, which
refers to negative behaviour in the classroom (Table 4). As shown in the three items considered, male
respondents considered that the resource has more negative effects.
252
Interaction in class.
Positive behaviour in
class,
Negative behaviour
in class,
Average
items 1 to 5

Table 4: Significant differences per gender
Sérgio André Ferreira et al.
Men Women
1. It increases the probability to keep side conversations while the teacher
presents the subject
3,7 3,2
2. It reduces motivation to be present in class 2,7 2,2
3. The probability to miss classes is higher if it is known that the resource
will be available on the web
3,1 2,5
5. Results discussion and conclusions
Constructivism as a philosophy and pedagogy is now widely accepted and is close to most
contemporary theories of teaching and learning: "constructivism", "situated learning", "social
cognition", "activity theory", "distributed cognition", "ecological psychology", and "case-based
reasoning" (Jonassen, Howland, Moore, & Marra, 2003). Education policies, including the Bologna
process, give expression to these pedagogical models of constructivist nature. In line with this
perspective is the way new generations of students learn: in a more flexible way, not just playing the
role of passive consumers of information, but as active builders in their learning process. The social
media provided by web 2.0 support this new philosophy of learning based on community building,
participation and sharing.
There are several studies about the use of the web 2.0 tools potential as an approach to more
constructivist philosophies. However, few are the studies focused on classroom lectures, where the
teacher faces a class-group, consisting of several dozen of students with the need for direct
instruction, to be able to present a large volume of information in the minimum time. In this context,
which occurs often in higher education, dialogue and interaction are limited.
Figure 8 depicts the morphology of the communication process in a lecture to large groups, in which
the electronic presentation only allows one-way communication, the interaction is not favored and the
student is sent to a more passive role.
Figure 8: Cognitive communication morphology 1.0 in the context of a lecture
Mayer (Pennsylvania State University, 2010) admits the possibility of using resources more
associated with direct instruction to achieve more constructivist approaches. In this study we sought
to determine whether the integration of the web 2.0 tool Twitter micro blogging in a PowerPoint
presentation contributed to the upgrade of a cognitive communication 1.0 to a cognitive
communication 2.0 in the context of a lecture (Hu, 2011). The applied questionnaire allowed to assess
various dimensions of the resource used in the classroom and to verify the changes in the
morphology of cognitive communication. The results indicate that students recognize the potential of
the resource used in the dynamics and dimensions of a cognitive communication situation 2.0, in
particular with regard to: i) pedagogical aspects, ii) technological aspects, iii) cognitive learning; iv)
interactions in the classroom and v) positive behaviour in the classroom.
253

Sérgio André Ferreira et al.
In Figure 9 the results of the resource at interaction level and the direct effects on the student’s
activity are summarized. Twitter integration favours a multidirectional communication and an increase
of the interaction between teacher-student, student-student and student-content. This leads to an
upgrade of the level of cognitive communication from version 1.0 to 2.0. The findings corroborate the
results obtained when applying this questionnaire to a smaller sample (Ferreira, Castro, & Andrade,
2011) and are aligned with other studies, namely: i) the positive effects on learning (Balanskat,
Blamire, & Kefala, 2006; James, et al., 2006; Junco, et al., 2010) and ii) the relation between the use
of technology and the student involvement in school activities (Balanskat, et al., 2006; Chen P., 2010;
Junco, et al., 2010).
Figure 9: Morphology of cognitive communication 2.0 in the context of a lecture
For future studies, it is suggested i) to extend the sample in order to validate the questionnaire and to
allow the study of correlations between variables and ii) to develop a more systemic way of research,
integrating the issue of cognitive communication 2.0 in the context of the classroom in HIPLE context.
References
Balanskat, A., Blamire, R., and Kefala, S. (2006) The ICT Impact Report: A review of studies of ICT impact on
schools in Europe, European Schoolnet in the framework of the European Commission’s ICT Cluster,
Brussels, Belgium.
Bennett, S., Maton, K., and Kevin, L. (2008) "The ‘digital natives’ debate: A critical review of the evidence", British
Journal of Educational Technology, Vol. 39, No.5, pp. 775-786.
Castañeda, L., and Soto, J. (2010) "Building Personal Learning Environments by using and mixing ICT tools in a
professional way", Digital Education Review, No.18, pp. 9-25.
Chen P., L. A. D. G. K. (2010) "Engaging online learners: the impact of web-based learning technology on
college student engagement", Computers and Education, Vol 54, No.4, pp. 1222--1232.
Elliot, T. (2011) "SAP web 2.0", [online], http://www.sapweb20.com/ (acessed on 20/05/2011).
Ferreira, S. A., Castro, C., and Andrade, A. (2011). "Morfologia da Comunicação Cognitiva 2.0 em sala de aula
no âmbito do Ensino Superior", In Proceedings of CISTI 2011 – 6ª Conferência Ibérica de Sistemas e
Tecnologias de Informação, pp. 294-300, Chaves, Portugal, June.
Figueiredo, A. D. (2009) "Estratégias e Modelos para a Educação Online", In G. L. Miranda (Ed.), Ensino Online
e Aprendizagem, Lisboa, Relógio d´Água, pp. 33-55.
Fuchs, C., Hofkirchner, W., Schafranek, M., Raffl, C., Sandoval, M., and Bichler, R. (2010) "Theoretical
Foundations of the web: Cognition, Communication, and Co-Operation. Towards an Understanding of web
1.0, 2.0, 3.0", Future Internet, Vol. 2, No.1, pp. 41-59.
Hu, W. (2011) "Mayer's SOI model" [online], http://www.personal.psu.edu/wxh139/SOI.htm (acessed on
20/05/2011).
James, K. E., Burke, L. A., and Hutchins, H. M. (2006) "Powerful or Pointless? Faculty Versus Perceptions of
PowerPoint Use in Business Education", Business Communication Quarterly, Vol. 69, No.4, pp. 374-396.
Johnson, L., Smith, L. R., Willis, H., Levine, A., and Haywood, K. (2011) The 2011 Horizon Report, The New
Media Consortium, Austin, Texas, USA.
Jonassen, D., Howland, J., Moore, J., and Marra, R. (2003) Learning to solve problems with technology, Merril
Prentice Hall, New Jersey.
254

Sérgio André Ferreira et al.
Junco, R., Heiberger, G., and Loken, E. (2010) "The effect of Twitter on college student engagement and
grades", Journal of Computer Assisted Learning, Vol. 27, No.2, pp. 1-14.
Kennedy, G., Dalgarno, B., Bennett, S., Gray, K., Waycott, J., Judd, T., Chang, R. (2009) Educating the Net
Generation – a handbook of findings for practice and policy, Australian Learning and Teaching Council,
Melbourne.
Kurilovas, E. (2007) "Digital library of educational resources and services: evaluation of components", [online],
http://bit.ly/mz7UH1 (acessed on 20/05/2011).
Nesbit, J. K. B. T. L. (2007) "Learning Object Review Instrument (LORI) Version 1.5", [online],http://bit.ly/laXZHE
(acessed on 20/05/2011).
Nokelainen, P. (2006) "An empirical assessment of pedagogical usability criteria for digital learning material with
elementary school students", Educational Technology and Society, Vol 9, No. 2, pp. 178-197.
Pennsylvania State University (2010). Mayer's SOI model”, Available:
http://www.personal.psu.edu/wxh139/SOI.htm (acessed on 20/05/2011).
Prensky, M. (2001) "Digital Natives, Digital Immigrants", On the Horizon, Vol 9, Nº 5, pp.1-6.
Tapscott, D., and Williams, A. D. (2008) Wikinomics: A Nova Economia das Multidões Inteligentes, Quidnovi,
Lisboa.
Vilelas, J. (2009). Investigação – O processo de construção o conhecimento, Edições Sílabo, Lisboa
255

To What Extent Does a Digital Audio Feedback Strategy
Support Large Cohorts?
Rachel Fitzgerald
University of Northampton, Northampton, UK
Rachel.Fitzgerald@northampton.ac.uk
Abstract: The UK National Student Survey (NSS) regularly highlights student dissatisfaction with feedback,
identifying factors such as timeliness; personalisation; specific information on how to improve in a clear and
understandable manner and the level of detail given to students related to learning outcomes. For a large cohort,
achieving these targets can be difficult and students are more likely to be at risk of receiving rushed and vague
feedback as lecturers strive to return results as quickly as possible. In his research into the use of digital audio for
feedback in the JISC Sounds Good Project, Bob Rotherham suggests that audio may be a way to assist
“lecturers looking for a way of giving students good quality feedback on their work whilst saving time” Rotherham
(2008, p1). Saunders et al (2005) suggests that a good teacher is one will take advantage of ICT opportunities in
order to enrich the students experience, and this research evaluates the potential to provide meaningful, quality
feedback to a large group of first year students on a Information Management module at Northampton Business
School, via digital audio files. Using an action research methodology, this first cycle of research evaluates the
process from the creation of the digital audio files right through to the personalised approach of returning the files
to the students via the virtual learning environment. This paper analyses the effect of the experience on the
lecturers involved and through subsequent group discussion and questionnaires, this research also evaluates the
thoughts of the students involved and considers the overall impact on both home and international students.
Initial results indicate that for large cohorts there is no simple answer but electronic feedback is certainly seen as
more personal, more beneficial and digital audio may have some unexpected benefits for stakeholders. This
paper will identify how this method will be reviewed for a further action research cycle.
Keywords: digital audio feedback, feedback and assessment, action research
1. Introduction
In his research into the use of digital audio for feedback in the Sounds Good Project, Bob Rotherham
asks “are Lecturers looking for a way of giving students good quality feedback on their work whilst
saving time? Of course they are” Rotherham (2008, p1).This statement led to critical reflection on the
student experience of feedback on an Information Management module of over 100 first year
students, where assessment feedback is provided solely by the module leader. This is a 20 CAT
Credit (approx 10 ECTS), Level 4 long thin module taken by students from range of disciplines
including Business, Information Science, Accounting, Marketing and Human Resource Management.
The module embeds core study skills and acts as a transition to develop first year understanding of
academic skills in context. Delivery is face to face and assessment is online. The University Virtual
Learning Environment (VLE) is used to support the students throughout the module via seminar
activities, additional reading and assessment; the students become quite familiar with the VLE
throughout the programme. The assessment in question requires the students to submit a research
project proposal that is graded on independent research, appropriate reference skills, academic
writing and presentation skills. It is a first step towards an overall project in the module so feedback is
critical to developing a successful project.
Rotherham’s (2009a) Sounds Good funded project was successful on a small-scale and students
participating commented on how personal the approach was and that they felt that they understood
their feedback more than with a traditional hand-written approach. Rotherham found that the process
had taken him less time than if he had written feedback scripts. These results suggested win-win, so
this research was undertaken to evaluate if a similar approach could work for a large cohort and if it
was a solution that might work for other situations for example international distance learning (DL).
The module leader has the autonomy to change practice for this module and is very much part of the
research, from engaging students with the process, marking all the assessments, creating audio
feedback, observing the impact and reflecting on the experience. Where the researcher wants to
improve their practice, increase their student achievements and is completely engaged in the process,
Ferrence (2000) suggests that action research is the most appropriate methodology.
256

2. Literature review
Rachel Fitzgerald
The concept of using technology to support the process of feedback is not new, Harvey & Mogey
(1999) mooted the use of technology as a way for students to access feedback as and when it was
required and Rust (2001) advocated the use of audiotape to comment on students’ work, suggesting
that this would take less time than written feedback and the students would gain more from the tone
of their tutor than they would from written comments, which Rotherham (2009b) found to be the case
using digital audio for the JISC Sounds Good project.
The potential for audio to impact on international students is something that Copley (2007) suggests
is related to having access to ‘teachers’ voice. Kaplan-Leiserson (2005 p2) identifies opportunities for
time efficient feedback via podcasts and similarly mentions the benefit of access to the “professor’s
voice”. Sounds Good provided students with digital audio feedback to evaluate if audio feedback
would provide richer feedback to students and save assessors time, Rotherham (2009b). Results
from this project indicate that students were positive about the personal nature of audio feedback and
that it did save time but only if the assessor was comfortable using the technology. Fish and Lumadue
(2010) suggest that audio feedback has such a significant impact on the student experience, that
tutors engaged in distance learning programmes should consider it as a way to support students.
This is all very positive, however audio feedback is “not a magic bullet for assessment woes”
Rotherham (2009a, p9) and unless the student has been directed to have their original paper in front
of them, they cannot see what the marker is referring to, during the one way conversation, Kerr and
McLaughlin (2008). More fundamentally, feedback, whether audio or not, must be purposeful.
Feedback is considered effective when it is timely, relevant, encouraging and offers suggestions for
improvement, Brown (2001); Rotherham (2009b). Gibbs (1981) suggests that even if the feedback
tells students what to do there is no real reason that they will follow that advice, no matter how many
times they listen to it. Improving feedback for students is an ongoing study, in the Assessment series
for LTSN, Brown (2001) considers the impact of assessment and notes that the purpose of feedback
is to motivate students, to inform how well they have done and how to improve. Rani and Yahya
(2009) advise that the process of feedback should develop the student experience, and suggest that
lecturers should provide feedback that will stimulate learning. Rust (2001) suggests that the tutor
should relate the feedback to the learning outcomes of the assessment and the feedback should
show students their strengths and weaknesses in relation to outcomes, feedback should also indicate
how they can improve in the future, without this level of detail, Rust implies, students will just become
weaker. Although Price (2007, p9) warns of the dangers of using too much detail without clarity as it is
more likely to mislead or confuse students and is a “guarantee that they will be less likely to engage
with feedback”
Not engaging with feedback is an issue, there is a general perception that students are failing to
understand the feedback they receive, and in some cases are just not reading it, Holmes and
Papagergiou (2009); Rani & Yahya (2009). Bols (2010) in a nod to the National Student Survey
results (NSS) suggests that this is a result of lack of timeliness from the tutor and that students have
generally moved on by the time they receive feedback, as it is no longer relevant. Holmes and
Papageorgiou (2009) researched student perceptions and expectations of feedback and pick up on
the requirement for timely feedback, they also suggest that students perceive feedback as particularly
crucial to their first year experience. The NUS (2010), Charter of Feedback and Assessment
encourages more timeliness in the return of feedback and make a suggestion that students should
have access to more personalised feedback, particularly for their first assessment of each year and
that the students should be able to state their choice on what form this should take.
3. Background of the research
The National Student Survey regularly highlights student dissatisfaction with feedback, Lipsett (2007),
this usually relates to timeliness, although other factors have been identified such as the level of detail
given to students related to learning outcomes and specific information on how to improve in a clear
and understandable manner. Being aware of the issues with feedback is important to this research as
the assignment at the heart of the process is part one of two in a first year module in Information
Management. Feedback from the initial assignment is intended to be used by the students to support
and enhance the second assignment, so it is important that they read and understand the feedback
and apply it to their next assignment. There are over a hundred students enrolled on the module, so
257

Rachel Fitzgerald
consideration of the most appropriate way to provide meaningful and memorable feedback is an
issue, particularly as the students are first years and generally in need of academic skills guidance.
Ensuring that the students can access the feedback easily is the first hurdle, Gibbs & Simpson (2004)
suggest that if the students cannot then see the relevance of feedback to any of their other work, they
will just check the grade and chuck the rest in the bin. Other studies demonstrate that students do not
register feedback as feedback unless it is made absolutely apparent, Rani & Yahya (2009).
Assessment is more effective when the criteria is absolutely clear to students, Gibbs (1981), Rani &
Yahya (2009), Holmes & Papageorgiou (2009), therefore feedback should relate to the criteria to
make it understandable and relevant to each individual student but with large student numbers this is
difficult to achieve, and has the potential to lead to perfunctory feedback, Brown (2001). Even with
high student numbers, there is an institutional expectation that turnaround from assignment
submission to receiving feedback will be three weeks, which sometimes leads to rushed, ineffective
feedback. Against this context, it seems a good time to try something new in an attempt to avoid the
pitfalls of repetition whilst maintaining some aspect of individuality for students. As Saunders et al
(2005) suggest, a good teacher is one will take advantage of ICT opportunities in order to enrich the
students experience.
4. Action Research as a methodology
Action Research (AR) is a way for practitioners to engage in meaningful research, that develops the
researcher professionally, Koshy (2005). AR is a cyclical process, O’Leary (2004), as knowledge
emerges from research, so do new questions or new practices to try and the cycle begins all over
again but with ongoing developing awareness.
Figure 1: The Action Research cycle adapted from Kemmis & McTaggart (2005 p564)
For this research, the researcher is interested in how to make feedback more effective for students
with a view to transforming practice and for informing practice for colleagues. The initial cycle plan is
to use audio feedback as an alternative to written feedback and to observe the impact for both myself
as tutor and for the students and to evaluate and critically reflect on the process to inform how to
approach feedback during the next cycle of research.
5. Research method and data collection
For this research the tutor and students were engaged in the process from the outset, students were
informed in advance that personal feedback would be online via a digital audio file and that transcripts
of the audio would be available to anyone who required it. During initial discussions students
confirmed that they had never tried this method before and normally received traditional written
feedback. The turnaround for the project was swift, the assessment was due in mid-Nov and the
audio feedback needed to be available to students early in December. Due to time limitations, data
gathering was done via classroom discussion, an online survey and through evaluation of the number
of times feedback was accessed on the VLE. O’Leary (2005) recommends undertaking a pilot before
258

Rachel Fitzgerald
running full data collection, but with the time factor and a Christmas holiday in the midst of the data
gathering, it was decided to run the full survey. Although not ideal, as the first stage of an AR cycle it
was considered that all data would be of interest and any limitations would serve to improve the next
cycle of research. Survey, as a method was chosen specifically because it suited the time constraints
and students with erratic attendance patterns. An online survey was deployed in the virtual learning
environment where students access all module materials (and assessment feedback), this seemed
ideal for gathering many responses and opinions about the process. Awareness of the students also
informed the researcher to add a number of response categories to the survey to encourage them to
get involved. This allowed the survey to be uncomplicated and swiftly completed with an option to add
qualitative opinions for general analysis. The type of survey questions affect the quality of data
received, O’Leary (2005), but as this data was gathered to get a general consensus, it was expected
to generate questions for further cycles of research.
To participate, students submitted their assignments electronically. Prior to recording the audio, the esubmissions
were read and graded with some rough notes made about the work, then a selection
were sent (via email) to be second marked. Typed comments from the second marker were
incorporated into the rough notes and then the feedback recording for all students took place over two
days. Seven survey questions were put together to measure the students’ perception of these areas
and to understand if they found the experience more personal. In addition, the survey was designed
to collect data about the impact of the process on international students, with a view to informing
practice in the DL area. In terms of the feedback that was to be recorded, Gibbs & Simpson (2004)
suggests that poor feedback is feedback that is backward looking and addresses issues that will not
be required again. With the feed-forward aspect of the assessment in mind, feedback addressed the
student personally and commented on meeting the learning outcomes and included clear guidance
about academic writing, referencing and points to expand on for the next assignment and then they
received the grade.
6. Critical reflection on the process
Recording feedback required little technical knowledge, just the use of a headset and Audacity
software. After a trial recording to test, each recording took less than two minutes, the feedback was
quite specific and therefore time efficient. To keep a personal touch, feedback was in a natural tone
and intonation, and spoken as if talking directly to the student. In a study of 15 students, Merry and
Orsmond (2007) highlight the importance of tone for audio feedback, but what may have been easy
with just 15 students was not the case with over 100 and it was difficult maintaining a natural or even
enthused tone throughout the process. The feedback was similar for many students so it became
perfunctory and repetitious. Rotherham (2008, 2009a) suggests that audio feedback reduces the time
spent on feedback and this was also the case in this research, however while it is easier and quicker
to verbally explain a concept, when one uses typed feedback there is an opportunity to cut and paste–
with large student numbers, this can be a useful tool. In addition, while the recordings saved time,
there was a lot more time spent uploading each individual file to a secure area on the VLE, this
administrative process added at least two days to the process and meant that the feedback was a day
late being returned. This would be a barrier to using audio feedback for these students in the next
cycle.
When the feedback was available to the students, a notice was posted to the VLE informing students
and offering a transcript if required. To get a transcript students had to email me, in all around fifteen
students requested a transcript of their feedback. Using the report tool in the VLE, the graph in Fig 2
shows significantly increased activity on the day the feedback was available (7 th Dec 2010) and in the
subsequent days, this activity tails off during the Christmas vacation and then returns to what can be
considered normal access during Jan 2011. A classroom discussion with the students indicated that
they had no trouble accessing the audio file and most found it to be a novel approach and were happy
to accept this as a new method of feedback. All indicated they were aware that they could have a
transcript on request, but generally didn’t feel the need to request one.
Through the VLE we can see how often each student on the module accessed their feedback, the
VLE report indicated two interesting points that will need further investigation:
The majority of students on the module accessed the file at least twice, with two students
accessing the folder over twenty times. Rotherham (2009b) suggests that students find it harder
to skim through audio feedback and therefore need to go back to listen again to clarify their
understanding. However it could also mean that the students found my voice difficult to
259

Rachel Fitzgerald
understand, although the classroom discussion and the small number of requests for transcripts
indicate otherwise –this requires further evaluation.
A small number of students (that submitted work) did not access their feedback, of these just two
requested a transcript –this is curious as there is no other way to access the grade. I suspect that
these students will access their feedback closer to submission date for the next assignment and
will include this as part of my observation in the next cycle of research.
Figure 2: Graph of hits on feedback on VLE over December 2010
7. Evaluation of the survey
The survey was completed by 51 students, less than half of those enrolled on the module but a
reasonable number for a general consensus, the survey returned both quantitative data as well as
qualitative.
QUESTION 1: DID YOU LISTEN TO THE FEEDBACK ON YOUR FIRST ASSIGNMENT?
Figure 3: Representative of responses to question 1
Although the report indicates that a number of students did not access their feedback, in the survey
there was only one negative response. This is a limitation of this cycle that needs to be looked at in
further detail, a further cycle should consider whether the students just do not engage ever or if they
are just waiting until it is required for the next assignment.
QUESTION 2: IN COMPARISON TO OTHER FEEDBACK THIS METHOD IS DIFFERENT; DID IT
FEEL MORE PERSONAL TO YOU?
The students on this module normally receive traditional written feedback from other tutors so
answers to this question indicate high approval from the students for the personal aspect of the
feedback. They receive feedback in a personal folder; the audio was personally addressed to them
260

Rachel Fitzgerald
too. When comparing those that replied yes to this against the qualitative data, it is notable that the
students repeat the word personal to describe their general thoughts on receiving audio feedback.
Providing a personal touch appears to be a positive outcome of using this form of feedback
Figure 4: Representative of responses to question 2
Comments from Students who found the method more personal
“I found it was more personal to me and gave more information on how I could make my
assignments better. I think that it is a very good method”
“it was so clear and personal and helped me understand where I went wrong with my
assignment!!.”
QUESTION 3: DID THE FEEDBACK SUGGEST WAYS OF IMPROVING YOUR WORK THAT YOU
PLAN TO USE FOR YOUR NEXT ASSIGNMENT
Figure 5: Representative of responses to question 3
The response mechanism used for Q3 is a Likert scale. Most of the responses to this question were in
the positive range, students indicated that the feedback had offered ideas to improve their work Not
many of the students that responded negatively to this question added qualitative feedback, however
one student that did, indicated that they were not fully aware of the purpose of the feedback.
Comment from Student about being suggested ways of improving work
Would be good to be told what can be done to get a higher grade on the next
assignment”
261

Rachel Fitzgerald
Although this is only one comment, this may be something to look at in terms of the language used in
feedback. Perhaps there is a need to be absolutely specific in the feedback.. eg to get a higher grade
you will need to … but then again as Gibbs (1981) acknowledges, students often don’t follow advice
even when it is given with the best intentions.
QUESTION 4: DID YOU RE-READ YOUR ASSIGNMENT HAVING LISTENED TO THE
FEEDBACK?
Figure 6: Representative of responses to question 4
Responses to Q4 suggest that many students did not re-read their assignment either along with or
after hearing the feedback. It would be interesting to ask a similar question to students who had
received written feedback to see if the responses are similar, Gibbs and Simpson (2004) suggest that
students are not interested in feedback if they perceive there is requirement for them to take action
but it was hoped that use of audio would change this. As the audio file and the original submission are
separate from each other, this has proven to be ineffective at encouraging (all) students to read their
work with new understanding. It remains to be seen if the feedback has an impact on the second
assignment.
QUESTION 5: DID YOU FEEL YOU HAD GUIDANCE THAT WILL HELP YOU WITH YOUR NEXT
ASSIGNMENT FOR THIS MODULE?
Figure 7: Representative of responses to question 5
This question was actually similar to Q3, so unsurprisingly, the answers are also similar (a pilot survey
would have highlighted this) however the corresponding qualitative comments are more detailed.
Comments from Student when asked if they had guidance to help with next assignment
262

Rachel Fitzgerald
“Was rather unpleasant as I had to wait a while before you stated my overall grade.
Perhaps you should have stated my grade and then move on to explain why the piece of
work deserved it.”
This was an interesting observation; the grade was deliberately left until the end so that the student
had to listen to the feedback in order to get a grade to avoid the scenario of just accessing the grade
and ignoring everything else.
Comments from Student when asked if they had guidance to help with next assignment
“Audio files are all very well and good, however simply take up such amount of space
and time and are not as beneficial as say yourself passing back the assignment with
comments (too many lecturers with handwriting that cannot be read). If students require
further help, they can request for your time during a seminar. Doing all of the audio files
must have taken you hours.”
While delighted with the concern, this highlights a crucial point – would students be as happy with
written feedback as long as it was typed? Merry and Orsmond (2007) found the majority of students
engaged on their project were happy with just audio, only two from fifteen students mentioned wanting
both written and audio together but this research is showing that the use of audio is not having any
transformative effect.
Comments from Student when asked if they had guidance to help with next assignment
“I consider that his method is very informative and personal. The comments are brief but
however concise and do highlight areas of improvement. I personally took more notice of
this than sometimes assignments being handed back in a written format - not always
digestible and often too many comments to take on board”
If all comments matched this, audio feedback would be the winner! Kerr & McLaughlin (2008) had
similar feedback from their students, receiving comments like “I properly listened to what the marker
was saying” but this is not a common response from all participants in the study.
QUESTION 6: WOULD YOU CONSIDER YOURSELF TO BE AN INTERNATIONAL STUDENT?
Thirteen students identified themselves as an international student; their corresponding qualitative
comments included:
Comments from International Students about receiving Audio Feedback
“Because I am poor in English, I still have some words cannot understand immediately
and clearly. However, I think that this make me feel more personal to me and I am happy
in the radio.”
“I prefer my feedback with hard copy”
“it was all right”
“i really liked it and i think it was much better than any other feedback so far.”
“I like the idea of giving our feedback on an audio file. The feedback was clear and
satisfying.”
“For me, there is no difference between audio and text feedback.”
Overall there are mixed responses from the international students about audio feedback, but the
personal aspect comes up again here. This is a very small sample of students so research is required
before recommending this method as for DL students.
QUESTION 7: DO YOU HAVE ANY OTHER COMMENTS ABOUT AUDIO FEEDBACK?
Other General Qualitative Comments about Audio Feedback
“I like the way feedback was given”
“I would rather prefer a written evaluation than and oral evaluation in form of the audio
file”
263

Rachel Fitzgerald
“The survey was helpful in a way but it didn't make things clear in depth.. I'd prefer a
written feedback instead for the same reason that my mistakes were not justified in a
clear manner.. overall satisfied with it!”
“it felt a bit brief, I felt that had it been written there would have been comments
throughout the assignment and therefore more feedback.”
“For me, there is no difference between audio and text feedback”
Of the rest of general qualitative comments the response is again not particularly in favour of audio, if
anything it suggests that written feedback is more purposeful for these students. It would be
interesting to have measured whether these responses came from students that received the initial
recordings whether tone may have been more enthused or the end of the recordings when the tone
may have been more perfunctory.
8. Conclusions and general reflection
The JISC guide to effective practice lists digital audio as a tool for “improving the quality of feedback”
JISC (2009 p19) but as Rust (2001) advises, when class sizes go up there is likely to be more
“superficial feedback” returned to the student. This research established that this is also the case
when using audio feedback although it is packaged in a more personal way. Feedback from the
students through the various data collection methods suggests that they found the experience more
personal and therefore valued the approach over the more traditional scrawl of handwritten feedback.
Kerr & McLaughlin (2008) suggest that regardless of feedback style, quality personal feedback makes
students feel cared for and that they are thought of as individuals, and as this is difficult to achieve in
large cohorts, this must be something that needs to be considered, when looking at method. The next
assignment on this module will receive traditional feedback as the numbers are too great to go with
audio feedback again. If I were to use audio again, I think I would prefer to work with a smaller cohort
and evaluate the impact across different assignments for the same module as I think the quality of the
recordings would be better and less repetitive and it would offer opportunity to consider the impact of
tone on how the student receives the feedback. Considering a student learning styles and their
reception to method of feedback may also warrant further study. In a future cycle I would look for an
alternative to the VLE, possibly Dropbox or another Web 2.0 shared space, and find a method that
incorporated audio and text. This research suggests that audio feedback alone would not support an
international DL student but needs a combination of audio and written feedback, so a future project
may consider the Fish & Lumadue (2010) project that uses Jing, software that enables a tutor to
merge a text response with audio, something they recommend as a 21 st century yet simple approach.
Kerr and McLaughlin (2008) say that the tutors involved in their research would not repeat their
attempt at video feedback. The general view, they report, is that the exercise was pointless in terms of
time and benefit. From this cycle, I would say that Audio Feedback is a simple method but more
suited to smaller groups of students. It can be adopted with ease and requires very little technical
skills. The process of recording feedback is faster than writing it and it offers a personal response to
students that is hard to achieve with any written method. Brown (2001) says one of the problems with
feedback is that it is not always there when students need it; digital audio offers a solution to this, as it
can be listened to anytime, although this could also be the case with typed electronic feedback. What
does set Audio Feedback apart is the voice and tone of the tutor talking to the student, making
feedback less ambiguous and providing a level of personalisation that fits in with NUS
recommendations and gained the approval of the students involved in this research and even if the
only benefit is that the student feels the marking is personal, then the technique is worth further
investigation, Kerr & McLaughlan (2008).
References
Bols, A. (2010) What Students Want: feedback on assessment; Educational Developments; SEDA; 11:4; 7
Brown, G. (2001) Assessment: A Guide for Lecturers. LTSN Assessment Series: No 3; York, LTSN Generic
Centre.
Copley, J. (2007) Audio and Video Podcasts of lectures for campus-based students: production and evaluation of
student use; Innovations in Education and Teaching International; 44:4; 387-399
Ferrance, E. (2000). Action research. Providence, RI: Brown University, Northeast and Islands Regional
Educational Laboratory.
Fish, W., Lumadue, R. (2010) A Technologically Based Approach to Providing Quality Feedback to Students: A
Paradigm Shift for the 21 st Century; Academic Leadership The Online Journal [online]
http://www.academicleadership.org/article/A_Technologically_Based_Approach_to_Providing_Quality_Feed
264

Rachel Fitzgerald
back_to_Students_A_Paradigm_Shift_for_the_21st_Century
Accessed (28/1/11)
Gibbs, G. (1981) Teaching Students to Learn: A student centred approach. Milton Keynes: Open University
Press.
Gibbs, G., & Simpson, C. (2004) Conditions under which Assessment Supports Students’ Learning: Learning and
Teaching in Higher Education; 1:1
Harvey, J. & Mogey, N. (1999). Pragmatic issues when integrating technology into the assessment of students: In
Brown, S., Race, P. & Bull, J. (1999) (Eds), Computer-assisted assessment in higher education; London;
Kogan-Page.
Holmes, K., Papageorgiou, G. (2009) Good Bad and Sufficient: Students’ expectations, perceptions and uses of
feedback; Journal of Hospitality, Leisure, Sport & Tourism Education; 8:1: 85-96
JISC (2009) Effective Practice in a Digital Age; Bristol; JISC Innovation Group
Kaplan-Leiserson, E. (2005) Podcasting in Academic and Corporate Learning. Learning Circuits [online]
https://www.astd.org/LC/2005/0605_kaplan.html, Accessed (8/1/11)
Kemmis, S; MacTaggart, R. (2005) Participatory Action Research: Communicative action and the public sphere
in Denzin, N. and Lincoln, Y. (Eds) The Sage Handbook of Qualitative Research, London, Sage.
Kerr, W., McLaughlin, P. (2008) The Benefit of Screen Recorded Summaries in Feedback for Work Submitted
Electronically; Loughborough University [online] http://hdl.handle.net/2134/5363
Accessed (16/1/11)
Koshy, V. (2005) Action research for improving practice: A practical guide. London: Paul Chapman Publishing.
Merry, S., Orsmond, P. (2007) Students’ Attitudes to and Usage of Academic Feedback Provided Via Audio Files;
Bioscience Education eJournal [online] http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.pdf,
Accessed (25/1/11)
NUS (2010) Charter of Feedback and Assessment [online]
http://www.nusconnect.org.uk/asset/news/6010/FeedbackCharter-toview.pdf, Accessed (30/1/11)
O’Leary, Z. (2004) The Essential Guide to Doing Research. London: Sage
O’Leary, Z., (2005) Researching Real World Problems. London: Sage
Price, M. (2007) Should we be giving less feedback; Centre for Bioscience Bulletin [online]
http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf Accessed (20/1/11)
Rani, N., Yahya, A. (2009) Engaging Learners in their studies via feedback; Teaching and Learning Open Forum
[online] http://www.curtin.edu.my/TLForum2009/images/TLOF09-18.pdf Accessed (20/12/10)
Rotherham, B. (2008) Using an MP3 to give feedback on student assignments; Educational Developments;
SEDA; 8:2 [online] http://sites.google.com/site/soundsgooduk/downloads/MP3_recorder_for_feedback.pdf?
Accessed (10/1/11)
Rotherham, B. (2009a) It wasn’t me, guv!; Educational Developments. SEDA; 10:1
Rotherham, B. (2009b) Sounds Good: Quicker, better assessment using audio feedback; JISC Final
http://www.jisc.ac.uk/media/documents/programmes/usersandinnovation/sounds%20good%20final%20repo
rt.doc, Accessed (15/1/11)
Rust, C. (2001) A Briefing on Assessment of Large Groups, Assessment Series No 12; York; LTSN Generic
Centre.
Saunders, M., Charlier, B., Bonamy, J. (2005) Using Evaluation to create ‘Provisional Stabilities’; Evaluation the
International Journal of Theory, Research and Practice; 11:1; 37-54
265

Messages of Support: Using Mobile Technologies to
Support the Transition of Students on Articulation Routes
From Higher National Level to Degree
Julia Fotheringham and Emily Alder
Edinburgh Napier University, Edinburgh, UK
j.fotheringham@napier.ac.uk
em.alder@napier.ac.uk
Abstract: This paper explores the role mobile technologies can play in supporting students’ transition to second
and third year of university degree study along articulation routes from Higher National Certificate (HNC) and
Higher National Diploma (HND) study at college. Articulating students face particular challenges associated with,
typically, adjusting to a step up in the level of their academic studies, acclimatising to an unfamiliar academic
culture, and integrating into an existing cohort of students. ‘Message of Support’, a project funded by the
Edinburgh, Lothians, Fife and Borders Regional Articulation Hub (ELRAH), developed a range of SMS, podcasts,
and DVD resources, drawing on the voice and experience of existing students, in order to support new students
and staff in their respective parts in the articulation journey. Through a process of action research, it was found
such resources can aid the transition process by offering timely reassurance and information to students as well
as valuable development materials for staff. Responses to the challenges of using mobile technologies as
support mechanisms for articulating students were also identified.
Keywords: articulation; student support; SMS; mobile technologies; podcast; ELRAH
1. Introduction
The creation of articulation pathways into university has become a strategic priority for many Higher
Education Institutions (HEIs). Articulation describes a particular route through higher education by
which an initial qualification (in Scotland, usually HNC or HND qualifications at Scottish Curriculum
Quality Framework (SCQF) levels 7 and 8 respectively ([SCQF 2009]) gains a student access to a
university degree with advanced standing, directly into second or third year. Such progression often
reflects a formal agreement or partnership between institutions (Gallacher 2006). A successful
articulation route relies not only on careful curricular alignment between the courses in the two
institutions but also depends upon staff at both ends of the articulation journey ensuring that students
are as well prepared as possible to respond to the academic and personal challenges that await them
upon arrival at university (MacAskill 2010).
This paper explores the role of mobile technologies in helping to address some of the challenges
faced by articulating students. The Message of Support project, funded by the Edinburgh, Lothians,
Fife and Borders Regional Articulation Hub [ELRAH] (see www.elrah.ac.uk/Elrah.htm) developed
podcasts, SMS texts and a DVD to provide point-of-need support for articulating students from
Scottish colleges, and professional development for academic staff at Edinburgh Napier University.
While ELRAH is a partnership of many institutions, the Message of Support was focussed on
Edinburgh Napier University.
In this paper we describe how interviews with students who had made successful articulation journeys
to Edinburgh Napier University were used to create a series of podcasts for new students and a DVD
resource for professional development for academic staff. We also discuss our experience of using
SMS texts to encourage students to engage with various strands of academic support provision. For
many students, the first few weeks in a new University environment can result in a severe loss of
confidence (Christie et al 2008) which may result in withdrawal from their programme. Our experience
at Edinburgh Napier confirms that students welcome advice and encouragement from other students
in whose footsteps they follow, recognize the value of structured support such as academic skills
workshops, but also feel strongly about the contribution of academic staff to restoring and building
their confidence and helping them to adjust to the unfamiliar academic and social demands of
university life. Our project confirmed that the deployment of a range of different technologies at
different stages in the articulation journey and amongst the different partners in the articulation
process can enhance a student’s experience of joining a degree programme as a direct entrant.
266

2. Background and context
Julia Fotheringham and Emily Alder
2.1 Supporting student transitions to university study
New university students of all kinds face challenges as they make the transition to university study
(Lowe and Cook 2003). For Scottish undergraduates, the traditional ‘first year’ (SCQF Level 7) is a
period during which students build social networks, familiarise themselves with institutional systems
and learn how to learn as a university student (Tait and Godfrey 2001). Research has shown that
direct entry students articulating to year 2 or year 3 (SCQF Level 8 or 9) face particular challenges.
These include integration into a cohort of students who have already learned together for one or two
years, the step up to a more advanced level of study, and acclimatisation to a different academic
culture.
Cree et al (2009: 896) describe a picture of ‘dislocation and loss’ amongst students struggling to
adjust to degree study. Barron and d’Annunzio-Green (2009) point to a catch-up phase in which
students not only must adjust to these differences but familiarise themselves with the one or two
previous years of learning undertaken by their peers; they highlight the importance of maintaining
student support and building students’ academic self-confidence during the early months of their
degree study. However, since articulating students progress from an HNC or HND, their academic
career begins long before they join the university, and thus developing academic skills and confidence
must also begin well before that point (MacAskill 2010). Pike and Harrison (2011) emphasise the
importance of a ‘smooth’ transition, to which good communication between institutional partners is
essential. Continuity of support between levels is thus one of the keys to successful transition.
Amongst the difficulties identified by articulating students in this research was lack of awareness of
what to expect from university study. New students attach considerable value to hearing from
previous articulating students about their experiences (MacAskill 2010; Kivlichan and Chirnside 2011).
Further, Pike and Harrison (2011) suggest that raising awareness among new direct entrants that
their concerns are shared by peers could reduce feelings of isolation.
Research points to the key role that academic staff play in supporting the student transition. This
indicates a need for staff development so that academic staff are aware of articulating students in
their classes and equipped to address their needs. Bingham and O’Hara (2007) found that students
felt some of their Higher Education (HE) lecturers were unaware of the prior learning they had
experienced; Pike and Harrison (2011) recommend that university teaching teams should be aware of
new direct entrants in their classes and be prepared to respond to their needs.
2.2 Institutional context
At Edinburgh Napier University, student feedback has indicated a similar range of challenges
associated with direct entry to university. While the majority of direct entry students settle in well to
university in the first trimester, a recent survey found that direct entrants were much more (46%) likely
to have experienced difficulties with the transition to university than year one students (18%)
(Kivlichan and Chirnside 2011). In particular, issues were encountered with use of IT systems,
challenges with particular modules, producing lab reports, time management, and personal or
financial matters. When asked what further help the university could have provided, responses
included more information (e.g. timetables, course content, and explanations of the amount of work
required) provided while students were still at college, examples of assignments, and hearing from
existing direct entry students about their experiences (Kivlichan and Chirnside 2011: 14).
Research by Howieson and Croxford (2011) into the experience of direct entry students across four
universities (Edinburgh Napier, Heriot-Watt, Queen Margaret and Stirling) in the ELRAH partnership
supports these findings. After three months of degree study, direct entry students to year two or three
were less likely (47%) to feel very or quite well prepared for their course than college students
entering year one (72%). Direct entrants felt that advance preparation before starting university was
key, including awareness raising about available support, experience of aspects of degree study, and
integration with existing students. Online information and materials were also resources for many
students (Howieson and Croxford 2011). Mobile technology, therefore, becomes a crucial tool in
enabling models of transitional support to be consistently available to students passing to a new stage
of their education.
267

Julia Fotheringham and Emily Alder
2.3 Role of mobile technology in student support
Traxler and Kukulska-Hulme (2005: 1) acknowledge the potential for mobile learning to address the
needs of ‘the new constituency of learners’, given the flexibility and near ubiquity of mobile
technologies. Data from a recent Edinburgh Napier survey of student usage of mobile technologies
(2010, unpublished) confirms this; 99.5% of our survey respondents own a mobile phone/device.
Recent mobile student support pilot studies in universities and colleges reveal three main consistent
themes; first, students value the use of mobile technologies in their interactions with the University
(Riordan and Traxler 2005,); secondly they feel socially connected by these interactions (Lunsford
2010; Mentor 2011) and thirdly, institutional strategies for mobile student support are likely to feature
different mobile devices and materials (Lunsford 2010).
Riordan and Traxler (2005) highlight technical and pedagogical issues around effective and
appropriate use of SMS texting. Student acceptance and engagement is enhanced where SMS
interventions are short, personalised and focussed (Riordan and Traxler 2005). Elsewhere, SMS
student-support initiatives are motivated by differing administrative and pedagogical priorities. The UK
Council for International Student Affairs (http://www.ukcisa.org.uk/pmi/case_studies_support.php)
provide a collection of best practice case studies which describe projects where podcasts and SMS
are used to provide support for international students. For example, the Loughborough College
initiative is designed to assist its International Office to keep touch with students, whereas Sheffield
Hallam’s initiative responds to international student demand for support materials on arrival in the UK.
While these case studies provide examples of mobile student support practice, the students’
experiences of these initiatives are not described in any detail.
Mentor (2011) explores the potential for SMS texts to offer a sense of social connectedness to
students experiencing isolation from their college community. Although connections between the
perception of emotional well-being and improved academic performance still require further
exploration (Mentor 2011), Tinto’s seminal model of student integration (1975) continues to provide a
frame of reference for understanding the connections between social interaction, integration with
academic systems and student retention. Models of student retention and engagement have evolved
since then (Tinto 2007) to reflect a broader range of institutional contexts and student circumstances
including online and distance learning students (Rovai 2002), but the concept of involvement, (or
engagement) remains a key element for almost all mobile-mediated student support initiatives.
The benefits of mobile accessible materials for enhancing social connectedness for distance learning
students at the Open University (OU) (Lunsford 2010) appears to be broadly consistent with Mentor’s
(2011) findings. The OU investigation comprised several strands of activity each making different use
of mobile technologies and learning materials. Lunsford (2010) observes the important affective
impact of the interactions mediated by mobile technologies, where students report feeling more
connected and involved with the organisation as a result. An important outcome of that project is a
‘model of student support using handheld technologies’ (Lunsford 2010). This model offers a flexible
but coherent framework recognizing that different parts of the organisation will make use of mobile
technologies in different ways. A particular mobile device may be more useable in one context than in
another (Kukulska-Hulme 2005).
2.4 Action research
A commitment to improvement is of paramount importance in any action research project (McNiff and
Whitehead 2010). Despite the differences amongst the various typologies of action research offered
in the literature, most share a common way of thinking about the role of the researcher as practitioner
(Cohen 2000), the value of collaboration in offering differing perspectives (Reil 2011), the cyclical and
systematic nature of enquiry that is undertaken in the name of action research (Norton 2009) and the
inherent contribution that transparent reflection makes to every part of the research and its outcomes
(McNiff and Whitehead 2010).
3. Methodology
Messages of Support explored the potential of mobile technologies to improve the transition
experience of students coming to university as direct entrants from Scotland’s colleges. Messages of
support, information and encouragement were embedded in a suite of podcasts and in a timeline of
SMS texts messages. The target audience for podcasts and SMS were students in Lothians, Fife and
Borders colleges, as well as students just arrived at Edinburgh Napier University. Locating the
268

Julia Fotheringham and Emily Alder
podcasts within the University student portal would have made them inaccessible to college students
given the need for a matriculation identity. As a result, we hosted the podcasts on Podbean, an openaccess
podcast hosting service, and distributed the text messages from an SMS server based in
Edinburgh Napier University, accessible online from various college locations. Action research,
carried out by practitioners on their own practice, presented a strong methodological framework with
which to underpin our collaborative exploration of the potential benefits of mobile accessible materials
for transition support. We established a project team of practitioners from colleges and University and
although the membership of the team varied as the project developed, the mixture of expertise and
professional focus ensured that at every stage the reflective process was collaborative and
participative. Norton’s (2009) five-step action research process provided the most useful way of
sharing the practicalities of our different but related reflective cycles of activity. We started by
producing a suite of podcasts for students (Table 1) and generating a timeline of SMS texts (Table 2,)
then out of the experience of Reflective Cycles 1 & 2 we produced a DVD for staff (Table 3).
Table 1: Reflective cycle 1 (podcasts)
Step 1 Identifying the issue Research provided evidence of the challenges that students face joining
university as direct entrants from college.
Step 2 Thinking of ways to Interest in mobile technologies and the rise of mobile learning initiatives
tackle the issue across the university and the HE sector provided a catalyst to develop a
series of podcasts for direct entrants.
A project team of academics and student support staff in colleges and the
university established the podcast topics (see Figure 1), which
complemented other existing university student support podcasts. This
project sought to recognize the value of student voice in providing
reassurance and authentic material for the podcasts.
Step 3 Doing it Students who had joined second or third year of undergraduate degree
programmes were interviewed. The interview questions were informed by
Tinto’s model of student integration (1975).
We asked questions about our interviewees’ experience of academic
integration during their first trimester (whether or not they were enjoying
their subject and coping with academic assignments) and in relation to
social integration (whether or not they had friends and the nature and
extent of the contacts that they had with university staff). We asked them
what advice they would give to students who would follow them from
college to university.
Student interviews were recorded, edited and developed into a suite of
podcasts.
Podbean site (http://college2uni.podbean.com) was developed and
podcasts were launched on a time-released basis.
Publicity posters helped to publicize the initiative.
Step 4 Evaluating it We counted the number of visits to the site, and the number of ‘hits’ on
each of the podcasts.
Each podcast has an online ‘comments’ form on which participants were
invited to submit feedback or to suggest other podcasts.
Invitation to participate in interviews or focus group discussions
Informal comment and feedback from members of the project team
Step 5 Modifying practice We address this in more detail in the Discussion. There is no doubt that
when using open access platforms such as Podbean, it is essential for
evaluation purposes to find ways of making contact with the students who
are likely to be accessing the site.
Table 2: Reflective cycle 2 (SMS texts)
Step 1 Identifying the
issue
Step 2 Thinking of ways to
tackle the issue
As for podcasts in Table 1.
As part of other projects, ELRAH staff produced a timeline of interventions to
facilitate successful articulation. A series of SMS text messages could be
deployed across the span of that timeline, from the process of application to
university until the end of the first university trimester.
The working group drew upon the expertise of the University’s Lead IT
Developer. The University’s SMS interface system enables bulk text
messages to be sent directly to individual students’ personal mobile
numbers. The SMS system is web-enabled, and access is thus not restricted
269

Julia Fotheringham and Emily Alder
to the university campus. This enables both college and university staff to
operate the system to reach students at all stages.
Step 3 Doing it A series of SMS text messages was designed according to the timeline
described in Step 2. The SMS messages aimed to support students’ sense
of belonging to the university and to reinforce their induction programme with
reminders of activities such as academic skills workshops or where to find
advice on exam techniques.
Students in the target group at college or direct entrants to university were
invited to register by sending a text message to the SMS system. Students
would receive no more than 2 SMS messages each month, between July
and December.
Eleven messages were issued between July and December, variously
targeted to students while at college, and then after matriculation to
university
Step 4 Evaluating it Analysing the number and range of students taking part in the initiative.
Evaluations from project team on the method and management of the SMS
interface.
Step 5 Modifying practice We explain in the discussion how we have adapted our approach for the
academic session 2011/12.
Table 3: Reflective cycle 3 (DVD)
Step 1 Identifying the
issue
Step 2 Thinking of ways to
tackle the issue
The first round of interviews (for the podcasts) told us that some articulating
students reported serious difficulties during their first year as direct entrants,
including social isolation and problems with assimilating the HE academic
culture.
We recognized that the student narratives could make a powerful impact on
academic staff. A DVD was selected as a suitable format for a professional
development resource.
We anticipated that the material on the DVD was likely to be sensitive and
not appropriate for student use, so in order to limit access, the resource has
not been web-enabled.
Step 3 Doing it Both student and staff accounts were used in order to maximise the impact
of the DVD. We re-interviewed and filmed students from the podcast
interviews. We also interviewed one new member of academic staff who had
valuable experience of working with groups of students that included a
significant proportion of direct entrants.
The DVD has been used for the purpose of Academic Induction and as a
resource for CPD for staff in the university.
Step 4 Evaluating it Feedback from participants at Academic Induction and from members of the
project team
Step 5 Modifying practice Use of the DVD with new groups of staff and with programme teams will be
monitored and re-evaluated throughout the forthcoming academic session.
We explore this further in the discussion.
4. Results
The evaluation of each of the three cycles of activity represented the most challenging aspect of our
project. The table below takes Step 4 (Evaluating it) from each of the reflective cycles documented
above and illustrates the results associated with each of the evaluation methods.
Table 4: Results for reflective cycle 1 (podcasts)
Evaluating it Results
Counting visits to the site and
hits on each podcast
Comments forms for each
podcast
Invitation to participate in
interviews or focus groups to
Between January and July 2011 the College2uni Podbean site
(http://college2uni.podbean.com/) received 1618 visits. The number of
‘hits’ on the podcasts vary from the least popular (Staying on for
Honours) with 13 hits and the most popular (Personal Development Tutor
[PDT]) with 337 hits.
None were completed.
In May an email was issued to all 2 nd and 3 rd year students who came to
Edinburgh Napier University from College in 2010, inviting them to offer
270

discuss their experience of the
podcasts or to provide feedback
by way of email.
Julia Fotheringham and Emily Alder
feedback about their experience of the ‘college2uni’ podcasts. We
received no responses.
Evaluating it Results
Feedback from articulation
support staff and from others
working in Scotland’s colleges
Three articulation support advisors who had actively used the podcasts
responded to the invitation to provide feedback. A consistent theme
emerges from their responses around the importance attached by
students to the student-voice perspective of the podcasts:
‘I think it’s beneficial to hear this information first hand, as in my
experience students will listen to staff, but tend to believe other students.’
The podcasts were well-received by staff working in central support roles
in ELRAH partner colleges. We received requests from all of the colleges
for the publicity posters and several colleges sought permission to
upload the podcasts on their own student portals.
Figure 1: Number of hits per podcast
Table 5: Results for reflective cycle 2 (SMS)
Evaluating it Results
Participants 9 students registered with the SMS system. Of these, 3 articulated to
Edinburgh Napier (plus 1 to year one), 1 to Heriot-Watt, and 2 to an
Edinburgh Napier degree in Carnegie College. The remaining 2 did not
go on to university. This uptake rate was disappointingly low, and
participation will be more actively promoted in the future.
The range of student destinations, however, demonstrated the system’s
potential to reach a diverse and dispersed student population.
A closing message requesting feedback on the messages elicited no
responses.
Administration Participation relied on students contacting the SMS system. In future,
student mobile numbers will be collected and entered, with their
permission, in advance.
Interface Administrators found the interface to be straightforward to use. It allowed
students to be variously grouped (e.g. by year of study, college of origin,
or university course) for ease of targeting messages.
271

Table 6: Results for reflective cycle 3 (DVD)
Julia Fotheringham and Emily Alder
Evaluating it Results
Feedback from academic
induction
5. Discussion
5.1 Podcasts
Feedback from induction participants was positive. Comments were
made about the powerful impact of hearing actual student stories. Other
participants had not previously considered their role in supporting direct
entry students prior to viewing the DVD.
Figure 1 displays the distribution of hits on each podcast. One podcast, ‘What is a PDT?’, received
nearly 5 times as many hits as the next most popular. We speculate this may have been caused by
the acronym and the podcast’s title being posed in the form of a question. If so, this may point to the
importance of demystifying jargon such as PDT (personal development tutor). The other podcasts
average 38 hits. Allowing for the uncertainty over how many users were students, this may suggest
the need of more active and directed promotion of the podcasts to the target group in the future.
As noted in Table 4, the lack of qualitative data from students means we are unable to analyse how
and why students (if indeed they were students) accessed the podcasts, whether or not they
subscribed to the whole series, why there was such a variation in the number of hits on each podcast
(from 13 to 337), nor, perhaps most importantly, how valuable they found the form and content of the
series. We were pleased to support colleges to host the podcasts within their own institutional virtual
environments. However, given the dispersed location of the podcasts across various student portals,
and the anonymity of the College2uni podbean site users, evaluating the usage and value of the
podcast series became unachievable.
5.2 SMS
The SMS project was successful in testing the bulk mailing system, which its administrators found
reliable and straightforward to use. Student recruitment to the SMS system proved a challenge.
Although 6 of the 9 students participating in the initiative progressed to enrol as undergraduates at
Edinburgh Napier University, the sample size is too small to draw any conclusions about its impact.
Students were required to register their mobile number by sending a text message to the system, but
very few students actually took this step despite having expressed interest in taking part. As a result,
opportunities to assess the impact of the SMS support in 2010/11 were limited. Consequently, in
preparation for 2011/12, mobile phone numbers are being collected in advance for entry by the
system’s administrator. The effect of this change is immediately evident: at the time of writing (July
2011), around 80 participants have provided mobile numbers. This indicates that the success of this
strand of the project is dependent not on the technology but on its management.
5.3 DVD
We expected to make regular use of the DVD at academic induction, but the small numbers of new
academic staff and the lack of induction events mitigated against this. However, at the start of
academic session 2012/13 we expect to respond to requests from programme teams who will be
using the resource in preparation for the new intake of direct entrants in September 2012. This should
provide us with ongoing evaluation data.
6. Conclusions
In the Message of Support project, the use of podcasts made the voice and experience of existing
students readily and widely available online to all, circumventing the restrictions of access often
associated with institutional logins.. The podcasts and SMS messages can also promote a sense of
belonging and connectivity to the institution, as well as raising awareness of key topics known to
present challenges to students. Student feedback has highlighted the importance of staff awareness
of the articulating student experience, and the DVD promotes a wider understanding amongst
academic staff of the needs of this student group.
The project offers support that is consistent, inclusive, and longitudinal, spanning the college to
university transition. Mobile technologies offer real advantages by making it as easy as possible for
272

Julia Fotheringham and Emily Alder
students to engage on their own terms with open access resources. However, as use of open access
resources grows more widespread, familiar monitoring and evaluation techniques may increasingly
prove to be insufficient. In the rapidly evolving world of mobile technologies, new approaches to
managing these initiatives are required to ensure that their potential for student support can be fully
exploited.
References
Barron, P. and d’Annunzio-Green, N. (2009) ‘A smooth transition? Education and social expectations of direct
entry students’, Active Learning in Higher Education, Vol. 10, No. 7, pp. 7-25.
Bingham, R. and O’Hara, M. (2007) ‘Widening participation on early years degrees: “I realised I could, and would,
do this – and I have!”’, Journal of Further and Higher Education, Vol. 31, No. 4, pp. 311-321.
Christie, H., Tett, L., Cree, V., Hounsell, J., and McCune, V. (2008) ‘“A real rollercoaster of confidence and
emotions”: learning to be a university student’, Studies in Higher Education, Vol. 33, No. 5, pp. 567-581.
Cohen. L., Manion, L. & Morrison, K. (2000). Research Methods in Education, 5 th edition, Oxon:
RoutledgeFalmer.
Cree, V., Hounsell, J., Christie, H., McCune, V., and Tett, L. (2009) ‘From Further Education to Higher Education:
Social Work Students’ Experience of Transition to an Ancient, Research-Led University’, Social Work
Education, Vol. 28, No. 8, pp. 887-901.
Edinburgh Napier University (2010) Survey of Mobile Technology Use at Edinburgh Napier University,
unpublished.
Gallacher, J. (2006) ‘Blurring the boundaries or creating diversity? The contribution of the further education
colleges to higher education in Scotland’, Journal of Further and Higher Education, Vol. 30, No. 1, pp. 43-
58.
Howieson, C. and Croxford, L. (2011) Students’ Experience of the Transition from HN to Degree Study. Final
Report to ELRAH. University of Edinburgh.
Kivlichan, N. and Chirnside, A. (2011) Students Calling Students: Trimester 1 2010/11. Final Report, Edinburgh
Napier University.
Kukulska, A. (2005) ‘Mobile usability and user experience’, in A. Kukulska-Hulme and J. Traxler (eds.) Mobile
Learning; A Handbook for Educators and Trainers, Oxon: Routledge .
Lowe, H. and Cook, A. (2003) ‘Mind the Gap: Are students prepared for higher education?’, Journal of Further
and Higher Education, Vol. 27, No. 1, pp. 53-76.
Lunsford, J. (2010) ‘Using handheld technologies for student support’, Journal of the Research Center for
Educational Technology (RCET) Vol. 6, No.1. [online]
http://www.rcetj.org/index.php/rcetj/article/view/82/181 (accessed 7th August 2011)
Macaskill, K. (2010) ‘Student Evaluations of Articulation: A Guide to Institutional Practice’, Paper presented at
Hitting the Ground Running Symposium, Glasgow Caledonian University, Glasgow, June.
McNiff, J. and Whitehead, J. (2010) You and Your Action Research Project. Oxon: Routledge.
Mentor, D. (2011) Supporting students’ connectedness via texting. Educause Quarterly Magazine, Vol. 34, No.1.
[online]
http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/SupportingStudent
sConnectednes/225851 (accessed 5th August 2011).
Norton, L. (2009) Action Research in Teaching & Learning, Oxon: Routledge.
Pike, A. and Harrison, J. (2011) ‘Crossing the FE/ HE divide: the transition experience of direct entrants at Level
6’, Journal of Further and Higher Education, Vol. 35, No. 1, pp. 55-67.
Reil, M. (2011) Understanding Action Research, Centre for Collaborative Action Research, [online],
http://cadres.pepperdine.edu/ccar/define.html (accessed 7th August 2011).
Rovai, A. (2002) Sense of community, perceived cognitive learning, and persistence in asynchronous learning
networks, Internet and Higher Education, Vol. 5. pp. 319-332.
Riordan, B. and Traxler, J. (2005) ‘Supporting computing students at risk using blended technologies’.
Proceedings of 4th Annual Conference. Galway, Ireland: LTSN Centre for Information and Computer
Science, pp. 174-175.
Scottish Credit and Qualifications Framework. (2009). SCQF Handbook. [online]
http://www.scqf.org.uk/Resources/ (accessed 28 July 2011).
Tait, H. and Godfrey, H. (2001) ‘Enhancing the Student Experience for Direct Entrants to the Penultimate Year of
Undergraduate Degree Programmes’, Journal of Further and Higher Education, Vol. 25, No. 2, pp. 259-265.
Tinto, V. (1975) ‘Dropout from Higher Education: A Theoretical Synthesis of Recent Research’, Review of
Educational Research, Vol. 45, pp. 89-125.
Tinto, V. (2006) ‘Research and Practice of Student Retention: What next?’, Journal of College Student Retention,
Vol. 8, No. 1, pp. 1-19.
Traxler, J., & Kukulska-Hulme, A. (2005). ‘Evaluating Mobile Learning: Reflections on Current Practice’.
Proceedings of mLearn 2005, [online], http://www.mlearn.org.za/papers-full.html (accessed June 2011)
UK Council for International Student Affairs (UKCISA), ‘Best Practice Case Studies’, [online]
http://www.ukcisa.org.uk/pmi/case_studies_support.php (accessed 7th August 2011)
273

Blended Learning at the Alpen-Adria-Universität Klagenfurt
Gabriele Frankl and Sofie Bitter
Alpen-Adria-Universität, Klagenfurt, Austria
gabriele.frankl@aau.at
sbitter@edu.uni-klu.ac.at
Abstract: The application of blended learning, i.e. the combination of face-to-face and eLearning elements, is
one of the university’s declared goals and seen as a chance to cope with the challenges educational institutions
are facing today. After several years of implementing eLearning and blended learning at the Alpen-Adria-
Universität Klagenfurt (AAUK), a survey has been conducted to detect the status quo. The survey shed light on
the potential of eLearning and blended learning as well as risks, fields of application and contentedness.
Furthermore, the survey focused on the benefits of blended learning as well as on reusability. Data were
collected with online and paper questionnaires and 1154 valid answers (collected over a period of three
semesters) were obtained. Students and lecturers were asked to judge, how studying is supported by the various
forms blended learning offers. Furthermore, it was investigated how common different forms of blended learning
are and what the most frequented tools are. Students see great potentials of eLearning and blended learning in
flexible learning (24 hours a day 7 days a week), in using their study time more efficiently and in being able to
learn at their own pace. In addition, students were asked about their attitude towards mobile learning, as this form
of learning is planned to be further expanded in the near future. The eLearning service department has
additionally collected suggestions for improvements regarding the eLearning and blended learning environment
at the AAUK. Based on these results our already existing service of video-recording lectures has been further
expanded. Apart from that, we have started to offer a “secure exam environment” where online exams can take
place on a grand scale. With this special software students can use their own devices but have no access to any
of their files or to non-specified Internet resources.
Keywords: blended learning, potentials, risks, benefits, video recording of lectures, secure exam environment,
online testing
1. Introduction
The Alpen-Adria-Universität Klagenfurt (AAUK), situated on the campus in the Wörthersee region of
Carinthia, Austria, hosts about 10,000 students. It is divided into four faculties: (1) Humanities and
Social Sciences, (2) Interdisciplinary Studies (with additional centres in Vienna and Graz), (3)
Management and Economics, and (4) Technology. The AAUK seeks high quality of teaching
combined with excellent support of individual students. Due to the small number of students, the
personal coaching of students can be guaranteed. (e.g., CHE University Ranking 2009/10) However,
coaching already suffers in highly frequented study programs. Thus, offering up-to-date learning and
teaching methods, up-to-date didactic skills of the lecturers and adequate infrastructure is an
important aim of the AAUK. ELearning and blended learning are integral parts of the university’s
strategy to offer attractive study conditions. On the one hand-side, there are necessary challenges
required due to financial shortenings of the ministry. Also other universities face challenges that
require rethinking and changes in teaching methods taking advantage of blended learning (e.g.
McNaught 2011). However, on the other hand-side, “digital natives“, which enter universities in
increasing numbers, cannot be neglected, since they definitely need the application of new teaching
methods (Prensky 2008).
The university pursues a bottom-up approach concerning the spread of eLearning and blended
learning. This means that lecturers can decide autonomously which facets of eLearning and blended
learning they regard as useful for their individual teaching contexts. This is especially for the blended
learning concept an advantageous precondition. Lecturers can individually and step-by-step convert
their usual teachings to the eLearning environment. As already highlighted by Bakhtiar (2011), the
lecturer is the critical element in distance teaching and learning. Students can only profit, if the
lecturer is convinced about his/her teaching methods.
Blended learning does not only mix media and methods but integrates a number of ideas for teaching
and learning. Thus, it enlarges established educational concepts with the digital flexibility and does
not completely condemn existing structures, attitudes and routines. This process does not overburden
lecturers and students since it offers the possibility to slowly integrate the new methods and
procedures (Reinmann-Rothmeier 2003, p. 30 and pp.42 ff.).
274

Gabriele Frankl and Sofie Bitter
As the results of study by Thompson and MacDonald (2005) suggest, structure and design are critical
quality features for eLearning courses. In transforming their courses, the eLearning service
department at the AAUK offers lecturers assistance for the conceptual design and the planning of
didactic blended learning courses, as well as support regarding the technical and organizational
implementation. Additionally, lecturers can apply at the eLearning service department for specially
trained e-tutors who accompany and support the lecturer for one or two terms.
Face-to-face teaching at the AAUK is still very important and should remain as significant as it is.
Results of recent brain research show that the strongest and the best ‘drug’ for people are other
people (Bauer 2008). Therefore, efforts target to optimally combine the benefits of face-to-face
teaching with the advantages of eLearning. Thus, a broad range of blended learning activities is
covered at the AAUK, starting from minor additions and extensions of face-to-face teaching to a few
teaching scenarios that are nearly completely based on eLearning. It has to be pinpointed that a vast
majority does use tailor-made concepts of blended learning with great contentedness.
1.1 The theoretical potential of eLearning and blended learning
One potential that we can derive from theory is that eLearning can bridge the technology gap in
education, but in the context of methodology (e.g. Luhmann and Schorr 1979). According to Luhmann
and Schorr (1979), education was based on the communication between teachers and students.
Traditionally, the lecture hall - under the control of the chair holding professor - is the only relevant
organizational unit for the production in higher education. This focus on the traditional classroom
setting is frequently reproduced in the eLearning literature while it might be helpful to consider other
educational contexts and settings as well (Frankl, Pfeffer, Zederbauer, Steinberger 2008). Learning
can only happen actively. However, in a traditional classroom setting, the active part was/is mainly the
lecturer. The learners play/ed the passive part. Due to costs, written homework was exclusively
addressed to the teacher and was not available to students for feedback. In the digital environment
these obstacles can easily be overcome through means of online peer-reviews. Additionally, it is also
possible to jointly work on wikis, ePortfolios or tools like Etherpad. The teacher was not only active
and autonomous, but also a lonesome fighter. The predominant oral culture of the traditional course
resulted in the interaction of the people present. Additionally, technology transfer (the transfer of the
‘how to’ of teaching) and hence the improvement of teaching was very difficult. Teachers normally did
not learn their profession from their peers or from formal training, but rather from past experiences as
students. (Frankl, Pfeffer, Zederbauer, Steinberger 2008)
The situation changed with the evolvement of new digital media in education. These new technologies
allow new forms of communication and cooperation fostering knowledge generation, exchange and
usage. In a blended learning environment, it is now possible that teachers share experience and invite
each other for a skype-conference to participate as co-lecturers or guest-lecturers. Further, it
becomes feasible to visit colleagues’ online learning courses and give feedback or to share literature,
methods or discuss ideas.
Web 2.0 does indeed show great potential also for higher education (Grosseck 2009). Furthermore,
as highlighted in a study by Lee and Tsai (2011), technology supported learning shows better
outcomes than traditional face-to-face learning. This holds true for the fields of collaboration, selfregulated
learning and information seeking. A detailed picture is provided by Dabbagh (2011),
focusing on the influences of online media tools and on self-regulated learning in higher education.
Another study by Cabero et al. (2010) also pinpoints the positive influence of blended learning on
students’ learning and lecturers’ teachings. Empirical results also reveal that more experienced
students (with online tools) were much more positive towards eLearning. However, the study also
reports that the overall trend was openness towards new tools and/or applications (Lam et al. 2011).
The aim at the AAUK is to establish “learning without walls”. To evaluate the status quo, to initiate
future developments and to design the future, we started a survey about eLearning and blended
learning at our university.
2. Survey methodology
In the summer term 2009, winter term 2009/10 and summer term 2010, we conducted online surveys
on eLearning and blended learning at the AAUK. The aim was to identify the status quo and recent
developments in the eLearning and blended learning sector at the university as well as potentials for
improvement. With the data gained through these surveys the eLearning department is in the position
275

Gabriele Frankl and Sofie Bitter
to react to new needs and tailor its services offered to the actual requirements of the university
members. All students and lecturers at the AAUK were invited via email to take part in the online
survey. In addition, during the second and third survey, paper questionnaires were also available at
the secretariats of the various institutes. Therefore, we have also motivated those lecturers and
students of the university who prefer paper-questionnaires. Summing up all three surveys, 1154
people completed the questionnaire. In addition, five qualitative guided interviews with experienced
and didactic well-trained lecturers were conducted. However, they are not included in this study since
it would go beyond the scope of this paper.
As highlighted in table 1, the participation in the survey slightly increased over the three terms (this
might be due to raffles offered with the second and third survey), whereas only the valid responses
have been considered for the evaluations and analyses.
Table 1: Sample characteristics
Summer term 2009 Winter term 2009 Summer term 2010
N % N % N %
Total 432 - 488 - 504 -
Valid 327 100,0 412 100,0 415 100,0
Missing 105 - 76 - 89 -
Male (valid) 136 41,6 197 47,8 189 45,5
Female (valid) 191 58,4 215 52,2 226 54,5
Student 216 66,1 273 66,3 314 75,7
Lecturer 90 27,5 112 27,2 76 18,3
Non-lecturing staff 21 6,4 27 6,6 25 6,0
3. Survey results
3.1 eLearning, blended learning and Moodle in general
As Figure 1 shows, an increase among those members of the university who actively use eLearning
and blended learning can be recognized at the AAUK. Especially from the summer term 2009 to the
winter term 2009/10 a substantial augmentation can be recorded. However, this increase in using
eLearning and blended learning is due to a sponsorship which the eLearning department received
and which was invested to expand the use of the eLearning and blended learning activities at the
AAUK, e.g. through the assignment of more e-tutors and through expanding the varieties of the fields
of applications.
Figure 1: Usage of eLearning and blended learning among AAUK lecturers and students
* 384 valid answers and one missing answer for this question (total of lecturers and students 385)
** 389 valid answers and one missing answer for this question (total of lecturers and students 390)
276

Gabriele Frankl and Sofie Bitter
In the summer term 2009, 770 (out of 1493; 51,6 %) university courses implemented Moodle (cf.
https://elearning.aau.at/moodle or http://moodle.org) in their course outline, in the winter term 2009
this number increased to 823 (out of 1460; 56,4 %) courses and finally in the summer term 2010, 842
Moodle courses (out of 1510 university courses; 55,8 %) were employed, as highlighted in Figure 2.
This statistic goes in line with those respondents (lecturers and students) who answered that they use
Moodle at the university.
Figure 2 Overview of Moodle usage compared to Moodle courses
A more detailed picture provides graph 3 by highlighting the distribution among lectures and students
at the AAUK. Especially among students Moodle become very popular over the three terms. Although
it looks like that lecturers reduced their Moodle use, we cannot confirm this, since the ratio of Moodle
courses to normal courses gives us a clearly different picture. Due to the fact that approximately 56 %
of the university courses use Moodle in the summer and winter term 2010, we assume that the
sample of lecturers is not representative and does not very well reflect the actual situation. In addition
to the online survey (for the last two surveys) paper questionnaires have been distributed reaching
also those lecturers who might be not that technology affine as compared to the summer term 2009,
where solely online questionnaire were distributed.
Generally, we do not know whether the same people completed all three surveys. It has to be
pinpointed that the analyses highlighted in the forthcoming pages do not necessarily reflect a trend
but rather reflect the mood of the respondent group.
Figure 3: Usage of Moodle, the university’s online learning platform
*241 out of 306 lecturers and students completed the question
**342 out of 385 lecturers and students completed the question
***342 out of 390 lecturers and students completed the question
Taking a glance at the different faculties and their employment of Moodle it gives a very well defined
picture that humanities and social sciences followed by management and economics are the clear
leaders using this eLearning application as highlighted in Figure 4. It has to be added that Moodle at
the faculty for humanities and social sciences is especially popular amongst the language institutes as
well as at the institute for media and communication. Unfortunately the institute for educational
sciences forms the taillight. The interdisciplinary studies have the smallest number of Moodle users
but also the lowest number of lecturers and students.
277

Figure 4 Moodle usage among the AAUK faculties
Gabriele Frankl and Sofie Bitter
3.2 Usage of eLearning and blended learning
Exchange of information and materials, time scheduling and messaging, as well as online
assignments and students communicating via means of eLearning are the most popular features
among the users. Less frequently used are learning logs and discussions in an online environment.
However, features like video recording of courses and offering the materials via the eLearning
platform Moodle to the students as well as giving feedback and to do online tests is starting to
become more and more popular. For the graphical illustration see Figure 5.
Figure 5 eLearning methods employed
*This question was not included in the survey of the summer term 2009
In the Moodle courses of the three terms, students mostly worked with forums, assignments,
calendars, feedbacks and surveys. They did online group works, online practice activities and tests
and worked collaboratively in wikis. The features least provided were e-portfolios, glossaries, SCORM
learning packages, lessons and peer reviews.
Slightly different compared to our results, a study by Miyazoe and Anderson (2010) investigating an
English blended learning course at a Tokyo university, highlights that wikis are the preferred blended
learning applications before blogs and forums. A possible reason could be the different usability of the
applied wikis. The wiki application in Moodle shows indeed technical weaknesses and a lack of
usability.
3.3 Rating of potentials and risks of eLearning and blended learning
In our survey, we presented typical potentials of eLearning to students and lecturers (cf. Figure 7).
Overall, students and lecturers see great prospects in eLearning and blended learning. Additionally, a
positive trend amongst students and lecturers over the period of three terms can be recognized.
278

Gabriele Frankl and Sofie Bitter
Figure 6: Moodle tools used
*“Materials” were not included in the survey of the summer term 2009
*The figure in the winter term 2009/10 reflects only those users who solely uploaded materials (no
other Moodle tools used); whereas in the summer term 2010 figure also includes users who used
other Moodle tools in addition to providing materials
Students indeed see great potential of eLearning and blended learning in flexible learning 24 hours a
day 7 days a week, in more efficient use of their study time and in being able to learn at their own
pace. They appreciate the possibility to take over responsibility for their own progress and avoiding
cramming at the end of the semester thanks to the opportunities of continuous assessment. As a
contrast, only few of the lecturers, who were also asked to rate these potentials for students, saw the
latter two potentials. The students see the smallest potential in the support of collaboration through
eLearning and blended learning. Whereas lecturers, see the greatest potential of eLearning and
blended learning in the exchange of information and materials but do not think that learning time can
be used more efficiently. A detailed evaluation of the potentials of eLearning and blended learning for
students and lectures gives the following picture:
Figure 7: eLearning and blended learning potentials (lecturers vs. students)
Measurement: 4-point frequency scale (1 disagree to 4 agree)
°“Exchange” was not included in the survey of the summer term 2009
279

Gabriele Frankl and Sofie Bitter
Note: Not every respondent gave answer to all items of the question. Therefore, the response rate
varied as follows:
*summer term 2009 (199-223 respondents out of a total 306 students and teachers)
**winter term 2009/10 (232-328 out of 385)
***summer term 2010 (322-333 out of 390)
In the first study (summer term 2009) we explicitly asked to rate predefined potential risks of
eLearning and blended learning. The major concern was that there might not be enough opportunities
to get into personal contact with fellow students and lecturers. Additionally, users thought that
eLearning might increase the workload for students. Hardly any users believed that lecturers would
become unnecessary because of eLearning. In the consecutive two studies (winter term 2009/10 and
summer term 2010) only one open question relating to risks was included in the survey to discover
which risks users see without having predefined suggestions. Indeed, among the most commonly
named risks were lacking social contact and anonymity, reduced development of social competences,
suffering from quality of teaching as well as further increasing the distance between students and
lecturers. At this point, the authors would like to pinpoint that the survey reveals that no qualitative
short cuts, due to eLearning respectively blended learning, have to be feared. ELearning and blended
learning have to be regarded as a chance to enhance quality in teaching and learning methods (see
Frankl & Bitter 2011).
3.4 Benefits thanks to eLearning and blended learning
In addition, we asked lectures to what extend they think that they benefit from eLearning and blended
learning. We asked the lectures whether eLearning and blended learning supports better structured
courses, helps them to turn down the work load, increased the flexibility of learning regarding time
and/or space, as well as reusability and transparency of the courses. We used a 4-point frequency
scale from totally agree to totally disagree. Especially having better overview and structure of the
courses as well as reusability and work load relief (after a certain period of time) were the top benefits
reported by lectures. Figure 8 gives a detailed overview of the different factors and highlights the
mean of the frequency results.
Figure 8: Benefits from the teachers’ perspective
Measurement: 4-point frequency scale (1 totally disagree to 4 totally agree)
We also asked the students to estimate to what extent eLearning and blended learning support them
in reaching their learning aims. The data revealed that eLearning and blended learning had the
greatest impact on successful learning thanks to access to important study materials, communication
with and feedback from lecturers as well as fellow students and online self-tests and opportunities of
collaborative work. The study also reveals huge potentials regarding re-academization processes and
improving the quality in learning and teaching methods (Frankl & Bitter 2011). However, the smallest
benefit from eLearning and blended learning for students is the consideration of individual interests,
individual strengths and weaknesses. Figure 9 gives a overview of the students’ ratings:
280

Gabriele Frankl and Sofie Bitter
A study in the field of eLearning in an accounting class that partially reflects our results, revealed that
student satisfaction was associated inter alia with usefulness and availability of lecture notes, online
assessment, model answers, and online chat (Basioudis & de Lange 2009).
Figure 9: Benefits from the students' perspective
Measurement: 4-point frequency scale (1 disagree to 4 agree)
“time-delayed communication” was not included in the survey of the summer term 2009
4. Implications and consequences of the survey results
The results of our survey have several implications for further activities in the blended learning sector
at the Alpen-Adria-Universität Klagenfurt. As a long-term goal we would like to spread blended
learning further across the various disciplines. The provision of materials, time scheduling and
messaging, assignments, communication among students and feedback from lecturers to students
are relatively widespread, whereas the opportunities for collaborative and reflexive tasks are relatively
rare, although those taking advantage of this possibilities use it very well and in a very innovative way.
To point out the potentials of working with wikis, keeping learning diaries, doing peer-reviews or
working with ePortfolios, we need to collect good-practice examples, develop practical suggestions
and instruction manuals and share and discuss them with the lecturers to explore the potentials of
these forms of learning in more courses. Likewise, we have to communicate the differences in the
assessment of particular potentials of eLearning and blended learning between students and
lecturers: if lecturers knew that students do see great potentials in continuous assessment and in
taking over more responsibility for their learning outcomes, lecturers might dare to modify their
teaching strategies. It is appropriate to source out learning and teaching methods to the online
environment in order to foster flexibility and autonomous accessibility of online courses and online
learning. However, this also means to use the reduced face-to-face times more efficiently and
advantageously. For example, to get to know each other, to build up trust, to get introduction into new
knowledge areas, to conduct in-depth discussions or to foster reflexion processes. For example,
forums on frequently-asked-questions or video-recorded lectures can be sourced out to support faceto-face
lectures and to gain valuable time.
In the survey of the summer term 2009, students could also express their blended learning wishes for
the future. Two thirds said they would appreciate video recordings of lectures so that they can revise
the topics covered in class whenever they want. Almost two thirds said that they would like to get
guest access to online courses of previous semesters so that they can select courses better. About
one third wanted to have an online room independent of concrete courses for communication and
exchange with their fellow students. We have already started to put these suggestions into practice.
Thus, an increasingly popular service that the eLearning service department offers and which has
been expanded is video-recording lectures. The lecturer’s speech as well as the presentation slides
are recorded and provided on Moodle to the students. This allows students to revise content
281

Gabriele Frankl and Sofie Bitter
whenever they want: instead of having to watch an entire 90 minute video, the presentation slides can
be used to navigate exactly to the section students want to watch again.
OpenCourseWare Klagenfurt, inspired by the MIT OpenCourseWare Initiative, is a collection of free
educational resources organized in courses. Since 2005 we are part of this initiative and still the only
university in the German-speaking area participating in it. Finally, we have set up platforms where
students can exchange anything from their personal experiences to lecture notes in a self-organized
way. At the moment we are sustaining such platforms for beginner students in the most popular
subjects at the AAUK as well as for incoming/outgoing students. Additionally, we provide information
and communication platforms for lectures to exchange study programme relevant issues e.g.
methods, literature. Lectures are no lonesome fighters anymore but support each other enhancing the
teaching quality. Additionally, we are planning to expand our learning services to support mobile
learning.
5. Conclusion
The survey results have given us a first picture of the status quo of eLearning and blended learning at
the AAUK. The surveys have revealed the scope of eLearning and blended learning as well as its
most common forms. At the same time it helped us to identify facets of eLearning potentials which
have not been fully explored or which have been underestimated. These forms of eLearning can thus
be put into the centre of interest for university-internal training programmes. The students’ ratings of
benefits and risks of eLearning and blended learning as well as their wishes have shown the direction
into which to promote further eLearning activities.
Further, we are in the starting phase of offering a “secure exam environment” (SEE) where online
exams can be conducted without being restricted to computer labs. The purpose of this environment
is to enable students to conduct tests online on their own devices without having access to any of
their own files and to the Internet (except if explicitly stipulated by the lecturer). Thus also no other
form of communication among the students is possible e.g. chats, mails. Students who do not have
own mobile devices receive loan devices. This insular testing is possible thanks to special software.
Students have to boot their devices from an USB stick or from a DVD and immediately are directed to
the Moodle log-in screen. They can then navigate to the respective course and open the exam. The
SEE uses the save-exam-browser hindern access to any materials/resources that are not permitted in
the test. Thanks to various Moodle features, answers and questions can be mixed hindering cheating.
The development of the secure exam environment is a major leap forward and constitutes an
enormous workload relief for lecturers and students. Lecturers can decided whether to prompt
automatic feedback (e.g. in case of standardized questions like multiple-choice) or to provide
individual feedback. With standardized questions, students also immediately receive the test scores
and – if applicable – feedback. For a detailed overview on the development and technical as well as
organizational implementation of the secure exam environment, confer to Frankl, Schartner and
Zebedin (2011). A survey, after the first exams, reveals a very positive feedback of students and
lectures about this new way of testing. The negative critique is the unfamiliar handling of the loan
devices. Furthermore, in the next development phase of the SEE, Excel will be integrated. Hence, this
enables certain study programs (e.g. information systems, mathematics) to test common calculation
methods. Up-to-date learning also requires up-to-date testing.
Concluding, we would like to add that online collaboration still needs time to settle and get more
advanced. Especially lecturers need time to accustom to these new ways of teaching and to
restructure their previous teaching methods. However, according to Prensky (2008), lecturers do not
need to be scared of implementing new eLearning and blended learning methods since there is no
need for them to fully understand the digital technology. The new generation of students, namely the
“digital natives” are already experts for these new learning technologies. The role of the lecturers is to
foster curiosity and to activate learning processes.
Each terms new students are joining the university being much more connected and feeling part of
the digital era we face. It is of utmost importance to foster, enhance and further develop our services
to optimally support these future students. The mood towards eLearning and blended learning at the
AAUK makes us confident to move forward making most efficient use of the potentials eLearning and
blended learning offers.
282

References
Gabriele Frankl and Sofie Bitter
Bakhtiar, F. (2011) “The Role of Information and Communication Technologies (Ict) in Distance Education”,
Journal of American Science 7.6, 318-322.
Basioudis, I. G. and de Lange, P. A. (2009) “An assessment of the learning benefits of using a Web-based
Learning Environment when teaching accounting”, Advances in Accounting, 25:1, 13-19.
Bauer, J. (2008) Prinzip Menschlichkeit: Warum wir von Natur aus kooperieren, Heyne, München.
Cabero, J., C. Llorente and Puentes, A. (2010) “Online Students' Satisfaction with Blended Learning”.
Comunicar.35, 149-157.
Dabbagh, N. and Kitsantas, A. (2011) “Personal Learning Environments, Social Media, and Self-Regulated
Learning: A Natural Formula for Connecting Formal and Informal Learning”, The Internet and Higher
Education Article, in Press.
Frankl, G., Pfeffer, T., Zederbauer, S. and Steinberger, C. (2008) “e-Collaboration at the Alpen-Adria-Universität
Klagenfurt”, Proceedings of the Apera 2008.
Frankl, G. and Bitter, S. (2011): “Blended Learning at the Alpen-Adria-Universität Klagenfurt. Improving quality
and fostering re-academization”, handed in at the ICL 2011, Pieštany, Slovakia, submitted.
Frankl, G., Schartner P. and Zebedin, G. (2011) „The ‘Secure Exam Environment’ for Online Testing at the
Alpen-Adria-Universität Klagenfurt / Austria”, handed in at the E-LEARN 2011, Honolulu/Hawaii, submitted.
Grosseck, G. (2009) “To use or not to use web 2.0 in higher education?”, Procedia - Social and Behavioral
Sciences, 1 (1), 478-482.
Lam, P., Lee, J., Chan, M. and McNaught, C. (2011) “Students’ Use of Elearning Strategies and Their
Perceptions of Elearning Usefulness”, in Global Learn Asia Pacific 2011, Barton, S.-M., Hedberg J. and
Suzuki, K. (eds.), Melbourne, Australia, AACE, 1379-1388.
Lee, S. W. and Tsai, C.-C. (2011) “Students' perceptions of collaboration, self-regulated learning, and information
seeking in the context of Internet-based learning and traditional learning”, Computers in Human Behavior,
27 (2), 905-914.
Luhmann, N. and Schorr, K. E. (1979) „Das Technologiedefizit der Erziehung und der Pädagogik“, Zeitschrift für
Pädagogik, Vol. 25, No. 3, pp. 345-365.
Luhmann, N. (1975) Interaktion, Organisation, Gesellschaft. Anwendungen der Systemtheorie, In Niklas, L. (ed.),
Soziologische Aufklärung 2. Aufsätze zur Theorie der Gesellschaft, Westdeutscher Verlag, Opladen.
McNaught, C. (2011) "The Best of Both Worlds: Effective Hybrid Learning Designs in Higher Education in Hong
Kong" in Hybrid Learning, Kwan, R., Fong, J., Kwok, L.-F. and Lam, J. (eds.), Springer Berlin / Heidelberg,
1-9.
Miyazoe, T. and Anderson, T. (2010) “Learning outcomes and students' perceptions of online writing:
Simultaneous implementation of a forum, blog, and wiki in an EFL blended learning setting”, System, 38,
185-199.
Prensky, M. (2010) Teaching Digital Natives. Partnering for Real Learning. Corwin, Thousand Oaks.
Reinmann-Rothmeier, G. (2003) Didaktische Innovation durch Blended Learning: Leitlinien anhand eines
Beispiels aus der Hochschule, Huber, Bern [et al.].
Thompson, T. L. and MacDonald, C. J. (2005), "Community Building, Emergent Design and Expecting the
Unexpected: Creating a Quality Elearning Experience." The Internet and Higher Education 8.3, 233-249.
Zeit-Online (n.d.) “CHE University Ranking 2009/10”, [online], http://ranking.zeit.de/che10/CHE_en
283

Evaluating the use of Social Networking Sites as a Tool for
Knowledge Sharing for Developing Higher Education in
Developing Countries: An Exploratory Study of Egypt and
Iraq
Elaine Garcia, Ibrahim Elbeltagi, Sawasn Al-Husseini and Ahmed Abdelkader
University of Plymouth, UK
elaine.garcia@plymouth.ac.uk
i.elbeltagi@plymouth.ac.uk
sawasn.al-husseini@plymouth.ac.uk
a.kader006@gmail.com
Abstract: Educational institutions are today facing increasing pressures due to economic, political and social
upheaval. This is only exacerbated by the nature of education as an intangible good which relies upon the
intellectual assets of the organisation in terms of its staff. It is within this context that Social Networking Sites
(SNS) offer an important potential alternative method to manage and share knowledge within educational
institutions. The focus of this research therefore explores the role that SNS could play in relation to the
development of Higher Education (HE) within developing countries with a focus on the countries of Egypt and
Iraq as having one of the highest and lowest rates of internet usage within the Middle East respectively. In order
to gain a better understanding of the potential drivers and barriers to the use of SNS as a knowledge sharing
(KS) tool within educational institutions, within the developing world, a case study approach has been used. This
approach consisted of semi-structured interviews with forty members of staff, both senior managers and
academic staff, within Mansoura University’s Faculty of Commerce in Egypt and Al-Mustansirya University’s
Faculty of Economy and Administration in Iraq. The results show there are both areas of agreement and
difference in the views of staff within both institutions. In both cases issues of culture appear to affect the
participation of female staff and additionally age appears to be a predictor of use. However between the
institutions there appears to be wider differences relating to the existence of KS strategies, the extent to which
staff understand the nature of SNS and technology in general and the extent to which Senior Managers and other
staff views are shared. Overall the results suggest that Mansoura University in Egypt is far more prepared for the
use of SNS as a KS tool and some usage appears to already be occurring. Within Al-Mustansirya University, Iraq
however the usage of SNS as an organisational tool appears to be unlikely to occur within considerable
promotion and changes to strategic drivers.
Keywords: social networking sites, developing countries, case study, higher education, knowledge sharing
1. Introduction
The education sector is today facing an increasingly challenging period. Global recession, increased
competition, globalisation, technological advances and demand for increasing educational quality
(Mathew, 2010) are all leading to a need for education across the world to become increasingly
competitive, efficient and innovative.
Within this context Knowledge Management (KM) represents an imperative area of consideration in
terms of capturing, organizing and sharing organisational knowledge (Liao et al., 2004) to maximise
and fully exploit the intellectual assets institutions hold.
The use of Social Networking Sites (SNS) offers potential to share organisational knowledge within
educational institutions as they allow for the capturing of tacit knowledge and allowing Knowledge
Sharing (KS) to occur which in turn provides greater opportunities for creating new knowledge and
innovative practices.
SNS are also currently appearing to have a substantial impact specifically within developing countries
where a number of political events have recently been reportedly influenced by SNS. These include
Egypt, (Idle & Nunns; 2011; Alexander, 2011; Nabi, 2011) Tunisia, (Delany, 2011), Libya (Beaumont,
2011) and Pakistan. (Sutter, 2011; Hill, 2011, Blair, 2011)
Whilst the growth of Facebook, the largest and most well known SNS has slowed and user numbers
have decreased in the countries in which it was first launched, this has been tempered by growth in
developing countries within the last few months (Arthur, 2011).
284

Elaine Garcia et al.
According to the Spot on PR Report (2010), shown in Figure 1, the number of Arab internet users has
reached more than 65 million whilst Arabic Facebook users are now predicted to total more than 15
million.
Figure 1: Middle East and North Africa’s (MENA’s) top five Facebook communities source: Spot on
PR (2010)
Based on this report the largest number of Facebook users exist within Egypt followed by Saudi
Arabia and in both cases SNS have been used to mobilize collective action toward breakthroughs in
social, educational, economic and political contexts as discussed above.
By comparison in Iraq, internet penetration is amongst the lowest in the Middle East with just over 1%
of the population being able to access the Internet. Therefore the Iraqi use of SNS is relatively low
although there is growth of membership within the country.
It is important to note also that whilst there is an increasing number of Muslim based SNS these
appear to have relatively low membership and are not a preferred means of networking for the
majority of the Arabic population. (Dashan, 2011) The majority of Arabic users appear to prefer sites
such as Facebook which have promoted international use through marketing and the provision of 43
translations of the site. (Facebook, 2011)
The reasons for lower usage in Iraq whilst numerous are likely to include political instability and the
Iraq War which led to the destruction of the ICT infrastructure of the country which is only now more
thoroughly restored.
Due to the clear differentiation between these two countries this paper shall focus on Egypt and Iraq
as the highest and lowest users of SNS within the Middle East.
2. Knowledge Management and Knowledge Sharing
KM and KS have been the subject of much research and scrutiny in recent years. KM can largely be
defined from two perspectives. The first has its roots in the concepts of Artificial Intelligence (AI) and
subsequent Information Systems (IS) developments.
This perspective relies on the ability to codify knowledge in order to share it (Barnes, 2007; Snowden,
2005; Schutt, 2003) and therefore relies on knowledge in explicit form, not dealing with tacit forms.
(Hildreth & Kimble, 2002)
285

Elaine Garcia et al.
The process by which tacit knowledge can be shared is therefore the subject of the second
generation of KM which is particularly relevant where products are largely intangible and often shared
through a process of socialisation. (Nonaka & Takeuchi, 1995).#
It is therefore the nature of SNS as a popular web 2.0 tool which encourages and supports
socialisation that makes systems such as Facebook a relevant proposition for KM.
3. Social Networking Sites
In terms of KS the main benefit from SNS is the manner in which they ‘would let us see who knows
what, who does what and even who controls what and update us continuously on all of these fronts.’
(Donath, 2007)
Additionally, the popularity of existing sites suggests that, the use of SNS would be relatively straight
forward. However the motivation for existing use may not always be compatible with organisational
KM and may lead to conflicted motivation for continued use ultimately affecting the ability of KS to
occur. (OFCOM, 2008; Ardichvilli, Page & Wentling 2003)
The main issue is ‘what makes a social network function isn’t so much the tools as it is the attitude
you’ve got to have to “want to” and that isn’t something you can get out of a box.’ (Suarez, 2007) In
terms of existing users the motivation to use SNS as a tool for KS may not necessarily be compatible
with primary usage. (Ardichivilli, Page & Wentling, 2003)
Further important factors affecting the motivation for SNS use will relate to the desire for individuals to
have ownership, trust and privacy of their information and knowledge.
For non-users privacy concerns will largely relate to issues of personal safety (OFCOM, 2008; Hewitt
& Forte, 2006) and a desire not to share personal information with a wide group, particularly if they
are not chosen but due to shared organisational connections.
For existing users privacy concerns are likely to relate to the combining of personal and public use
particularly as others can inform what is posted and therefore visible about an individual. (Dwyer et al,
2007; Barnes, 2007)
Furthermore privacy concerns could relate to the sharing of proprietary information or even libellous
or defamatory comments across the internet which could prove to be a problem both individually and
organisationally. (APQC, n.d; Go, 2007)
In terms of KS, barriers may also exist from the fundamental way in which these sites operate.
Information posted can become the property of the software company and therefore its unauthorised
reuse cannot be guaranteed. (Jones & Soltren, 2005; Schonfeld, 2009)
Any information posted is also persistent and accumulative making it searchable, replicable and
available to any number of invisible audiences for an indeterminate period of time. (Cain, 2007;
Barnes, 2007, Dwyer et al., 2007)
McConnell (2008) however considers that there is a great deal of “paranoia” concerning the privacy
and inappropriate use of personal data on the internet and a balanced view of these issues is
required.
Whilst a number of barriers are considered perhaps the most prevalent area for consideration must be
the nature of relationships and specifically the manner in which weak ties can be utilised and
strengthened through the use of a SNS. (Donath, 2007; Boyd & Ellison, 2007)
The strength of weak ties within an SNS are likely to lead to a large network especially where existing
users already hold a personal network of friends and family. However it is important also to consider
that networks can become too large and unmanageable.
Donath (2007) places the maximum number of relationships an individual can have at 150. In a large
organisation this could be easily exceeded when considering that the average Facebook user will
already have 120 confirmed friends. (Sandberg, 2009)
286

Elaine Garcia et al.
The nature of organisations and the limited maximum number of relationships could lead to the
creation of silos as homogenous groups remain separated and offline relationships are simply
replicated online, no further enhancing KS or communication across the organisation. (Boyd & Ellison,
2009)
In addition there are likely to be a number of individuals who are less familiar with technology and who
may become segregated from the KS activities of the organisation resulting in a lack of completeness
across the organisation.
The nature of technology acceptance and use represents a fundamental barrier to the use of SNS as
a KM tool and indicates the need for adequate training, technical support and senior management
support to ensure that individuals who may not traditionally use SNS will be encouraged to trial use
within an organisational context.
4. Methodology
Generally previous studies have used a qualitative approach to identify the barriers to the use of SNS
as a KM tool. This approach is most appropriate due to the assumption that access to the reality of a
situation is only possible through social construction. (Smith et al, 2008)
A qualitative approach seeks to answer questions posed by studying different social settings. As
noted by Berg (2009), qualitative techniques make it possible for researchers to participate in
understanding and perceiving others, as well as permitting them to discover how people structure
their daily lives to make them more meaningful.
According to Creswell and Clark (2011) the advantages of utilising this approach are numerous:
It provides a suitable means of studying complex phenomena.
It attempts to interpret and describe people’s personal experiences of phenomena.
It tries to understand the process of interaction between individuals.
It allows participants to share their views.
Therefore the validity of the results of this approach is high (Creswell, 2009).
4.1 Case study
Case studies are increasingly used as research tools (Creswell, 2007) and according to Yin (2009)
investigate holistically and feature real-life events. Moreover, case studies are uniquely strong as they
are able to deal with a variety of evidence, i.e. documents, interviews, etc (Saunders et al., 2009),
thus providing a more detailed view of the issue under consideration (Yin, 2009). In addition
Siggelkow (2007) indicates that case studies are beneficial in order to understand the significance of
the research questions posed and to encourage new ideas.
In order to investigate the drivers and barriers that affect the use of SNS as a KS tool in order to
develop HE in Egypt and Iraq, case studies will be undertaken, one institution in Egypt and one in
Iraq.
4.2 Interviews
In this case study interviews, which Gray (2009) defines as a conversation between people, whereby
one party assumes the role of the researcher, will be used. According to Hair et al. (2007) this method
is helpful where issues are complex as it enables the researcher to receive feedback.
In both institutions semi-structured interviews were conducted with participants enabling the
researchers to explore and investigate opinions from respondents, giving them the opportunity to both
analyse in-depth and expand upon their answer (Gray, 2009)
This method appeared to be suitable as it elicits the assumptions and stories participants hold about
the issues under consideration.
287

4.3 Data analysis
Elaine Garcia et al.
A total of 40 managers within Al-Mustansirya University in Iraq and Mansoura University in Egypt
were interviewed using open-ended questions and consisted of Deputy Deans (DD), Heads of
Department (HD), Academic Heads (AH) and academic staff (AS).
The interview results were analysed using narrative analysis. This method is defined by Saunders et
al. (2009) as ‘an account of experience that is told in a sequential way, indicating a flow of related
events that, taken together, are significant for the narrator and which convey meaning to the
researcher.’ This method allows the stories people employ in recounting events to be emphasized
and analysed.
The advantages of this method include factors such as its expedient nature and the way it allows an
examination of the social background of the interviewee, the relationship between individuals and the
interviewee’s views of the organisation (Smith et al., 2008).
5. Case study institutions
5.1 Mansoura University in Egypt – Faculty of Commerce
Mansoura University was established in 1972 under the name East Delta University but had existed
as a branch of Cairo University since 1962 and is now one of 17 State owned Universities within
Egypt.
The Faculty of Commerce was established in 1973/1974 and currently has 22,000 students enrolled.
After the civil unrest within Egypt in January 2011 a small number of academic staff at the institution
established a Facebook group for the purpose of networking and KS within the faculty with an aim to
assist in the development of HE within the institution. Currently this group has 123 members.
5.2 Al-Mustansirya University in Iraq - Faculty of Economy and Administration
Al-Mustansirya University is affiliated to the Ministry of HE and Scientific Research and was
established in 1963.
The Faculty of Economy and Administration was founded in 1963. The Faculty has had a relatively
variable enrolment level in the last few years and current enrolments stand at 733 students.
The institute faces a number of challenges and changes, in terms of its structure, curriculum,
approaches and resources particularly due to political instability over the last two decades.
6. Findings
6.1 Existence of KS strategy
From the interviews, it appears a contradiction exists regarding the existence of a KS strategy within
Al-Mustansirya. Some interviewees for example, claim there is no strategy for KS within the University
whilst others consider a strategy does exist under an alternative name.
Largely those further down the organisation consider a strategy does exist indicating a lack of clarity,
a lack of communication or a desire for interviewees to appear to be aware of potential strategies of
which they have no knowledge.
The Deputy Dean did however state:
“There is no particular strategy, but I think there are recognizable priorities for KS.”
(DD.2)
This may therefore lead to others to believe that these priorities represent the KS strategy within the
organisation. The Deputy Dean also stated that such a strategy was imperative in the future, reflecting
the acceptance of the need for a KS strategy.
Within Mansoura similar results were found with general staff considering a KS strategy exists and
only one senior manager agreeing that it does. Two senior managers did however identify the need to
focus on informal not formal KS providing an indication of some awareness of KM approaches and
strategies.
288

Elaine Garcia et al.
In comparison it is clear there is greater clarity within Mansoura regarding the existence of a strategy
for KS in addition to the need to focus on both formal and informal KS although in both institutions
confusion existed as to the existence of a KS strategy reflecting different results from senior
managers and other staff.
6.2 Acceptance of use
In relation to the acceptance of use, the findings from Al-Mustansirya clearly indicated that the
University does not encourage the use of SNS and resistance to use exists across the organisation.
Staff across the University acknowledged that SNS were becoming widespread amongst student
communities but this led to an expectation that SNS can solely be used for social purposes and are
therefore inappropriate for work use.
Some managers could comprehend that SNS use for exchanging and disseminating knowledge was
useful and thus SNS use could benefit the institution but on the whole inappropriate comments,
issues of trust and privacy in addition to concerns about the low levels of technological capabilities of
staff all act as barriers to use.
It should be noted however that technological capabilities appear to be influenced by age as a Head
of Department within Al-Mustansirya stated:
“Some of the staff do not like to enter these sites due to their age and they lack
understanding about how to use technology...... By contrast, these sites are used
frequently by young staff” (HD1).
In Mansoura University however all interviewees agreed that SNS could have a large role in future KS
and particularly for the dissemination of knowledge, between staff or between staff and students to
the extent that:
“People feel free and relax when they express their views through this medium more than
face to face communication” (SM.1)
The key differences between the two institutions appear to relate to technological capabilities which in
Mansoura ranges between good and acceptable although again there are higher skills levels seen
amongst younger staff that tend to be more familiar with the technology.
In terms of other barriers to use the two institutions are relatively similar with issues of bullying,
security, privacy and trust all prevalent within Mansoura in addition to Al-Mustansirya.
Within Mansoura some concerns were often expressed by staff relating to the triviality of SNS:
“There are a lot of issues raised on SNS that are trivial and to read or comment on it is
just wasting time”. (AS5).
This may, as in Al-Mustansirya, reflect concerns relating to the social nature of SNS and the types of
communications and information that is shared on these sites.
Through the interviews it became clear that the use of SNS is associated with negative images and
culture in the minds of some people generally as a reflection of religious views within the region:
“The negative culture of the users of SNS is a big barrier for use” (AS1)
In practice this means that if you are using SNS you are immoral and this is especially the case for
female users. This view is reflected by the manner in which most female academic users do not put
their real pictures on the sites and are currently underrepresented on SNS.
Within the Mansoura University, Faculty of Commerce group on the facebook which contain 123
members of staff, only five of the forty female members put their photo on the site and of these three
are with their husbands. This was expressed by one of the female members of academic staff who
said that:
“.... One of the most serious issues for female users is the privacy”. (AS9).
Thus it can be concluded that the lack of acceptance of SNS within both institutions is the result of a
lack of technological skill (linked to age), concerns relating to bullying or inappropriate comments,
289

Elaine Garcia et al.
issues of privacy, trust and security, and, the ambiguity of perceived benefits aside from social
relationships.
The issues of age and gender are important considerations within the context of SNS in both
institutions and can be difficult to measure due to the nature of these groups where females and
elderly members of staff are underrepresented within both institutions either due to the familiarity with
the technology or due to the threat of privacy.
Issues of gender are likely to be directly linked to religious, cultural and social norms within Arab
states and therefore may continue to exist or take time to change as political and social change
occurs within the region.
Issues of gender may also be linked to age as due to these sites only recently entering the Arab
world, younger generations are more conversant and comfortable with them and this may result in
greater acceptance as the technology and population matures.
6.3 Motivation for use
In terms of motivation for use within Al-Mustansirya there appears to be a contradiction regarding the
extent to which KS is encouraged within the institution for whilst the Deputy Dean and Departmental
Heads indicated KS is open and encouraged, academic staff claim it is constrained:
“We always hear the term “open culture” at work but if you try to share ideas with
somebody higher than you in the organisational structure you will find a wall” (AS.8).
However, all interviewees agreed motivations for use will include benefits such as networking, selfexpression
and KS enhancement particularly in informal environments.
There was additionally however some resistance to motivation for use due to some of the issues
previously found such as the perception of sites as solely social:
“I consider these sites social. It is custom only to use them only for communicating with
friends and relatives” (AS.7).
However within Mansoura, where some use of SNS is already occurring and common as indicated by
one interviewees who did consider the social aspect of SNS to be useful for KS:
“SNS are useful in increasing the social bond of the staff which increases job satisfaction
and makes the work environment more likable” (AS3).
Furthermore within Mansoura the existing use of SNS appears to a greater understanding of potential
benefits as all interviewees agree that Mansoura is already benefiting from SNS use. These benefits
vary from networking, global KS, speed of communication, constructive feedback, supporting
innovation, generating new ideas, mass opinion polling cheaply, developing more social/collaborative
views of learning, improved motivation, engagement , creating a connection to real-life learning and
finally flexibility of use as one of the interviewees at Mansoura commented:
“SNS technology is already available free of charge to many people either at home, work
or even while they are on move. This makes it very easy and quick to use” (AS1).
The existing use within Mansoura also highlights the role that senior managers play in motivating staff
to use such sites as whilst senior managers consider that SNS can act as a sensor to evaluate the
popularity of their decisions and as a way to disseminate information to staff:
“SNS are a good way to know the problems within the school and respond to it quickly”
(SM5).
However from observation of the Mansoura Faculty group within Facebook and the interviews
undertaken it would appear that the contributions from senior managers are relatively limited and this
is noticed by staff:
I felt that the use of SNS by senior management within the school is just as a publicrelations
tool rather than a means of communication and solving people problems”
(AS11).
290

Elaine Garcia et al.
It would appear that this lack of contribution is considered to also indicate a lack of commitment from
senior managers and this is considered by academic staff to result in confusion and a lack of clear
direction:
“Lack of purpose and clarity of common goals of using these websites in terms of the
conflict of the nature of use .......will continue in the future until we have more
commitment from both senior management and academic staff” (AS6).
To conclude it is clear that there are no clear policies in place within either institution to adopt and use
SNS as a KS tool however the organic growth of a community of this nature within Mansoura would
appear to suggest that the institution is in the early stages of adoption and may be of significance in
the future.
It is also clear from the results that Arabic culture and social norms play important roles in the use of
SNS within these countries. Females in both countries tended to be more concerned about online
privacy than males. There is, nonetheless, a general acceptance and recognition that SNS encourage
and enable discussion, communication and KS.
7. Discussion
From the findings it is clear that these institutions are at very differing points of development in terms
of SNS usage, KS strategy and technology usage.
To some extent the differences appear to highlight the importance of senior management support for
whilst Mansoura appears to have a KS strategy and SNS policy in place and thereby some
acceptance of SNS use, Al-Mustansirya has neither. This therefore appears to highlight the
importance of senior management support, as identified in the literature, for achieving the use of SNS
for KS.
From both institutions issues of privacy, trust and openness seem to be prevalent. This interestingly
relates to concerns identified in the previous literature but there is also a cultural dimension relating to
issues of gender that are evident within Arab nations as a wider cultural issue. This consideration is
not one which has been given a great deal of attention within the area of SNS literature as the
majority of previous studies focus upon the role of SNS within the developed world; largely within the
USA and UK.
In relation to privacy and trust, as within previous studies, there appear to be a number of concerns
relating to the informal nature of SNS and the manner in which users are required to combine
personal and professional profiles. In addition it would appear that the inclusion of “social” within the
name of such tools gives users a sense that the sites are not serious and cannot be used for work.
This view is supported in both institutions. This therefore suggests that the name of the site is value
laden and represents a barrier to use.
Technological barriers appear to be present within both institutions but are to some extent related to
age. Increased barriers to use relating to technology within Al-Mustansirya may exist more due to the
nature of the organisation, its culture and strategy than necessarily in relation to individual usage
although previous literature has suggested that age may be a factor.
Within Al-Mustansirya for example there is no clear KS strategy and this perhaps reflects that the
institution does not know how knowledge transfer can be accomplished effectively and certainly how
SNS could assist in enabling this to occur.
The two institutions appear to be at different stages of cultural acceptance of SNS as would be
predicted from the number of existing users of Facebook within both countries. Whilst Egypt appears
to be reporting benefits already from using such tools and support comes from the possibilities of
social bonding, free software and easily accessible sites, within Iraq benefits relate to networking,
self-expression and knowledge enhancement but its role outside that of a communication tool for
friends and family is difficult for interviewees to comprehend.
Overall the findings appear to suggest that the higher usage of SNS within Egypt does result in a
greater acceptance and understanding of the role that SNS could play in KS whilst the slower gain in
usage within Iraq seems to result in lower acceptance of formal use within the workplace.
291

8. Conclusions and recommendations
Elaine Garcia et al.
This research has examined the drivers and barriers to using SNS for KS to develop HE in developing
countries. The study found that there are contradictions and differences in viewpoints concerning the
application of SNS as a KS tool and therefore factors which act as both barriers and drivers to usage
within developing countries.
Within both institutions studied there is clear evidence of the need to ensure that KS strategies are put
in place and for this to be achieved greater senior manager support will be vital. Within Mansoura the
manner in which SNS adoption is already occurring and SNS are considered to be valuable for KS
particularly demonstrates the advantages that can be gained. Potentially the sharing of this best
practice within and between the two institutions considered here and more widely than this may be
one way in which the promotion of KS strategies and SNS could be achieved. This would be
particularly effective if SNS was the vehicle used to promote such activities as it would demonstrate
the manner in which SNS enable institutions to create global communities conveniently and cheaply.
Using SNS as the vehicle to share best practice would not however overcome the barriers that exist
due to lack of technological and IT skills particularly seen within the Iraqi University. It is expected that
as the technological infrastructure of Iraq is more fully developed the country will become more
technologically capable however it will be important that technology is highlighted within the institution
and where possible training, assistance and the promotion of technology as a tool to assist in KS will
be undertaken. This, to some extent will depend upon budget constraints and wider investment within
the country. It may however be possible for the institution to make use of mobile technologies, which
are usually more reliable and available throughout the country, to demonstrate the use of SNS and
the ease of use of such technology, particularly to older staff that it appeared were less accepting of
technology use than younger staff.
Whilst senior management support and technological barriers should be relatively easy to overcome
the issues of privacy and trust, particularly relating to cultural and social norms within the region will
be harder to overcome.
Again the development of best practice groups and exposure to SNS will undoubtedly alleviate some
of these concerns, particularly if privacy settings and user controls are explained and a
recommendation for appropriately sensitive privacy settings can be provided. Furthermore if an
institution wide approach were to be taken and agreement concerning pictures and personal details to
be included were agreed by all affected groups, but to be implemented by all staff, this may assist in
both exposing individuals to and determining the appropriateness of SNS use within the institution.
Once again the sharing of best practice between institutions may be useful in enabling issues such
cultural and social norms to be discussed and previous experience by some may be promote future
usage by others through discussion of concerns and the manner in which these may have already
been overcome within another institution.
It is important to note particularly that the use of sites that have been developed specifically for
Muslim communities do not appear to date to have been popular however this may be due to the lack
of interest in such sites due to the technological barriers discussion above. It may therefore be of
interest for institutions with specific social and cultural issues to investigate the use of such sites and
more successful sites such as Facebook as either may represent a potential SNS solution for the
institution and an integration of both may provide an optimum solution if it can be achieved.
Overall there was broad agreement that there were advantages to employing such sites, given that
SNS aim to facilitate KS amongst users. The research therefore concludes that the Iraqi University
should adopt SNS and draw upon their benefits as a KS tool. Additionally, for effective change the
institution should work to achieve clear communication amongst staff and implement strategic plans to
encourage the KS during working hours. The time has come for the institutions not only to dispel the
ignorance of SNS but also to make academic staff aware of their necessity. The adoption of new
strategies to seize SNS is an absolute must – to the irrefutable benefit of senior staff in their bid to
accomplish an elevated echelon of communication with all other affiliates of their institutions.
292

Acknowledgements
Elaine Garcia et al.
This project has been partially funded by Plymouth College of Art
References
Alexander, A. (2011) Internet role in Egypt’s protests, [Online], http://www.bbc.co.uk/news/world-middle-east-
12400319 [Accessed 10/06/11]
APQC. (n.d.) The Growing popularity of Social Networking and Expertise Location, [Online],
http://kmedge.org/homepage.html [Accessed 10/05/08]
Ardichvili, A., Page, V. & Wentling, T. (2003) Motivation and barriers to participation in virtual knowledge-sharing
communities of practice, Journal of Knowledge Management, 7, pp164-177
Arthur C. (2011) Facebook growth slows for the second month in a row, [Online],
http://www.guardian.co.uk/technology/2011/jun/13/facebook-growth-slows-for-second-month [Accessed
13/6/11]
Barnes, S. (2007) A privacy paradox: Social networking in the United States, [Online],
http://www.firstmonday.org/issues/issue11_9/barnes/index.html [Accessed 01/06/08]
Beaumont, P. (2011) The truth about Twitter, Facebook and the uprisings in the Arab world, [Online],
http://www.guardian.co.uk/world/2011/feb/25/twitter-facebook-uprisings-arab-libya [Accessed 10/06/11]
Berg, B. (2009) Qualitative Research Methods For the Social Sciences, 7th edition, Pearson Education
Company: USA
Blair, K. (2011) Behind Bin Laden: The Role of Social Media, [Online], http://socialtimes.com/behind-bin-ladenthe-role-of-social-media_b60489
[Accessed 10/06/11]
Boyd, D. & Ellison, N. (2007) Social Network Sites: Definition, History, and Scholarship, Journal of Computer-
Mediated Communication, No. 13 [Online], http://jcmc.indiana.edu/vol13/issue1/boyd.ellison.html [Accessed
17/05/10]
Cain, J. (2007) Online Social Networking Issues within Academia and Pharmacy Education, American Journal of
Pharmaceutical Education, No. 72 [Online],
http://findarticles.com/p/articles/mi_qa3833/is_200801/ai_n24394683/pg_1?tag=artBody;col1 [Accessed
15/05/08]
Creswell, J. (2007) Qualitative inquiry research design- choosing among five approaches, SAGE Publication
LTD: London
Creswell, J. (2009) Research Design: qualitative, quantitative, and mixed methods approaches, SAGE
Publications, Inc: USA
Creswell, J. & Clark, V. (2011) Designing and conducting mixed methods research, SAGE Publication, Inc: USA
Dashen, M. (2011) Facebook has little to fear from Muslim social networking sites, [Online],
http://www.guardian.co.uk/commentisfree/belief/2011/may/31/muslim-social-networking [Accessed 10/6/11]
Delany, C. (2011) How Social Media Accelerate Tunisia’s Revolution: An Inside View, [Online],
http://www.huffingtonpost.com/colin-delany/how-social-media-accelera_b_821497.html [Accessed 10/06/11]
Donath, J. (2007) Signals in Social Supernets, Journal of Computer-Mediated Communication, No. 13, pp231-
251
Dwyer, C., Hiltz, S. & Passerini, K. (2007) Trust and Privacy concerns within social networking sites: A
comparison of Facebook and MySpace, [Online],
http://csis.pace.edu/~dwyer/research/DwyerAMCIS2007.pdf [Accessed 01/06/08]
Facebook. (2011) Facebook: My Account – Primary Language, [Online],
https://www.facebook.com/#!/editaccount.php?language [Accessed 13/06/11]
Go, A. (2007) The Paper Trail, [Online], http//usnews.com/blogs/paper-trail/2007/2/27/columnist-says-he-wasfired-for-facebook=comments.html
[Accessed 21/07/08]
Gray, D. (2009) Doing research in the real world, SAGE Publication Ltd: London
Hewitt, A. & Forte, A. (2006) Crossing Boundaries: Identity Management and Student/Faculty Relationships on
the Facebook, [Online], http://www-static.cc.gatech.edu/~aforte/HewittForteCSCWPoster2006.pdf
[Accessed 01/06/08]
Hildreth, P & Kimble, C. (2002) The duality of knowledge, Information Research, Vol. 8, No. 1
Hill, A. (2011) Osama bin Laden killed – how a live blogger captured the raid, [Online],
http://www.guardian.co.uk/world/2011/may/02/osama-bin-laden-live-blogger [Accessed 10/06/11]
Idle, N & Nunns, A. (2011) Tahrir Square tweet by tweet, [Online],
http://www.guardian.co.uk/world/2011/apr/14/tahrir-square-tweet-egyptian-uprising [Accessed 10/06/11]
Jones, H & Soltren, H. (2005) Facebook: Threats to Privacy, [Online],
http://groups.csail.mit.edu/mac/classes/6.805/student-papers/fall05-papers/facebook.pdf [Accessed
19/05/08]
Liao, S., Chang, J., Cheng, S. & Kuo, C. (2004) Employee relationship and knowledge sharing: A case study of a
Taiwanes finance and securities firm, Knowledge management and practice, vol.2, pp24 - 34
Mathew V. (2010) Service delivery through knowledge management in higher education, Journal of knowledge
Management practice, vol.11. no. 3, pp1-14
McConnell, B. (2008) Social Networks, from 80s to the 00s, [Online], http://gigaom.com/2008/01/20/socialnetworks-from-the-80s-to-the-00s/
[Accessed 15/05/08]
293

Elaine Garcia et al.
Nabi, A. (2011) Facebook protester: ‘A lot of people love Mubarak and want to defend him, [Online],
http://www.guardian.co.uk/world/2011/may/19/egypt-100-days-on-facebook-protester-mubarak [Accessed
10/6/11]
Nonaka I. O. & Takeuchi, H. (1995) The knowledge-creating company: how Japanese companies create the
dynamics of innovation, Oxford University Press: New York
OFCOM. (2008) A quantitative and qualitative research report into attitudes, behaviours and use, [Online],
http://www.ofcom.org.uk/advice/media_literacy/medlitpub/medlitpubrss/socialnetworking/report.pdf
[Accessed 03/05/08]
Sandberg, S. (2009) How many friends can you have? [Online],
https://www.facebook.com/blog.php?post=72975227130 [Accessed 13/06/11]
Saunders, M., Lewis, P. & Thornhill, A. (2009) Research Methods for Business Students, Pearson Education
Limited: England
Schonfeld, E. (2009) Zuckerberg on who owns user Data on Facebook: it’s Complicated, [Online],
http://technews.am/conversations/techcrunch/zuckerberg_on_who_owns_user_data_on_facebook_it_s_co
mplicated [Accessed 13/06/11]
Siggelkow, N. (2007) Persuasion with case studies, Academy of management Journal, vol.50 no.1, pp20-24
Smith, M., Thorpe, R. & Jackson, P. (2008) Management research, Sage Publication Ltd: London
SPOT ON PR. (2010) Middle East & North Africa Facebook Demographics – May 2010, [Online],
http://www.spotonpr.com/wp-content/uploads/2010/05/FacebookMENA_24May10.pdf [Accessed 11/06/11]
Sutter, J. (2011) How bin Laden news spread on Twitter, [Online], http://articles.cnn.com/2011-05-
02/tech/osama.bin.laden.twitter_1_bin-tweet-twitter-user?_s=PM:TECH [Accessed 11/06/11]
Yin, R. (2009) Case study Research- Design and Methods, 4th Edition, SAGE Publications, Inc: Los Angeles,
294

The Relationship Between Mindful Learning Processes and
Course Outcomes in Web-Based Learning
Danny Glick and Roni Aviram
Ben-Gurion University, Beer-Sheva, Israel
glickd@bgu.ac.il
roniav@zahav.net.il
Abstract: Online education has been growing rapidly within all levels of education. Reports published in 2007-
2009 by the Sloan Consortium based on responses from over 2,200 colleges and universities in the US reveal
that institutions report record online enrollment growth on both a numeric and a percentage basis. Nearly 96% of
the very largest institutions, schools with 15,000 enrollments, have some online offerings, and about two-thirds of
the very largest institutions have fully online programs. The Association of International Educators predicts that
by 2020, global higher education demand for seats will reach 200 million. Much of this growth will be in distance
education. While online education is growing rapidly, the impact of online learning on course outcomes has been
subject to considerable debate. In 1983, Richard E. Clark famously argued that media have no more effect on
learning than a grocery truck has on the nutritional value of the produce it brings to market. Recent research
findings support Clark’s statement revealing that students taking online courses score lower than students in
face-to-face or blended courses. The research hypothesis is that the main reason for the little impact online
learning has on learning outcomes is lack of Mindful Learning Processes (MLP), a new term coined by the
authors of this study. The authors define MLP as an online course that is based on four interconnected
components: learning theory, learning environment, learning process, and assessment. The goal of this paper is
to check whether the research hypothesis may be accepted. To achieve that, the authors have reviewed case
studies describing online courses published in peer-reviewed journals in the area of educational technology and
e-learning. They authors have identified three case studies that fulfilled the MLP requirements stated above, and
analyzed them in terms of learning theory, learning processes, learning environment and assessment. To
measure the impact of the MLP on course outcomes, the authors analyzed these case studies in terms of test
scores and students’ perception of the e-learning experience. Analysis of the three case studies reveals that the
outcome of the three case studies was positive in terms of achievement outcomes and students’ opinion about
course. The authors conclude that a Mindful Learning Process may lead to a meaningful and pedagogically
sound course where theory, design, learning and assessment harmonize with each other. These findings support
earlier work on instructional technology that found that the reason for the learning benefits of computer media is
not the medium of instruction, but the instructional strategies built into the learning materials.
Keywords: Mindful Learning Process, web-based learning, impact on learning, text analysis, review of research,
learning outcomes
1. Introduction
Online education is growing rapidly at all levels of education (Allen & Seaman, 2010; International
Association for K-12 Online Learning, 2010; Watson, 2007). A report published by Allen & Seaman
(2010) on Web-based learning in higher education, based on responses from over 2,500 colleges and
universities in the United States, reveals that institutions report record online enrollment growth on
both a numeric and a percentage basis. Nearly 5.6 million students were taking at least one online
course during the fall 2009 term, which is an increase of nearly one million students over the number
reported in the report from the previous year. The 21% growth rate for online enrollments far exceeds
the less than 2% growth of the overall higher education student population. Nearly 30% of highereducation
students now take at least one course online (Allen & Seaman, 2010).
Research indicates that educators and decision-makers have embraced online learning. Allen and
Seaman (2010) report reveals that 63% of all reporting institutions said that online learning was a
critical part of their institution’s long-term strategy. According to Rovai & Downey (2010), many
universities have increased their recruiting efforts to reach a larger and more diverse audience. Some
universities also extend their reach with cross-border initiatives and seek international students in
order to promote enrollment growth and global learning. The economic potential of distance education
and academic globalization has attracted numerous higher education providers, many of which
operate on a for-profit basis (Rovai & Downey, 2010).
The Association of International Educators predicts that by 2020, from a year 2000 base, global
higher education demand for seats will double to 200 million, suggesting a growth market in
international higher education. Much of this growth will be in distance education. Consequently, an
increasing number of universities are transforming themselves from single-mode on-campus
295

Danny Glick and Roni Aviram
universities to dual-mode universities as they acknowledge the importance of distance education and
the opportunities it provides (Redden, 2009).
A recent report published by The International Association for K-12 Online Learning (2010) reveals
that online learning in K-12 schools is growing explosively. Supplemental or full-time learning
opportunities are available to students in 48 states across the USA. 27 states, as well as Washington,
DC, have state-wide full-time online schools. Moreover, many virtual schools in the US show annual
growth rates between 20% and 45%.
The number of virtual universities offering full-time academic degrees is growing rapidly. Universities
in Latin America (www.unad.edu.co), North America (www.phoenix.edu), Europe (www.uned.es), Asia
(www.english.cyberkorea.ac.kr) and Oceania (www.curtin.edu.au) offer numerous online academic
courses in a wide range of departments and fields such as languages, education, finance, heath and
engineering.
The impact of technology and online learning on education is great. Hutti (2007) claims that
technology has impacted on learning at an extraordinary pace, perhaps like no other innovation in
previous years, decades or centuries. Distance education, specifically virtual classrooms, may have a
greater impact on the nature of higher education than any innovation since the invention of the
printing press (Hutti, 2007).
In light of the strategic role online learning plays at all levels of education, one would expect webbased
environments to offer quality instruction. However, recent research indicates that students
taking online courses score lower than students in face-to-face or blended courses. Moreover, recent
meta-analyses comparing students’ attitude towards online and classroom instruction reveal that there
is significant effect on overall attitude outcomes in favor of classroom instruction.
2. The little impact online learning has on learning outcomes and student
satisfaction – review of relevant literature
A meta-analysis published by the US Department of Education (2009) reviewing online learning
studies from 1996 through 2008, identified 51 independent effect sizes (EF) that could be subjected to
meta-analysis. Meta-analysis is a technique for combining the results of multiple experiments or
quasi-experiments to obtain a composite estimate of the size of the effect. The result of each
experiment is expressed as an effect size, which is the difference between the mean for the treatment
group and the mean for the control group, divided by the pooled standard deviation (Bernard et al.,
2004; Shachar, 2008; US Department of Education, 2009). The meta-analysis found that instruction
combining online and face-to-face elements had a larger advantage relative to purely face-to-face
instruction than did purely online instruction. The mean effect size in studies comparing blended with
face-to-face instruction was +0.35, p < .001. This effect size is larger than that for studies comparing
purely online and purely face-to-face conditions, which had an average effect size of +0.14, p < .05
(US Department of Education, 2009). This finding suggests that blended learning has greater impact
on learning than does virtual learning. Furthermore, the report indentified 28 research papers
comparing the courses outcomes of purely online courses with face-to-face instruction. The findings
reveal that significant difference in favor of online learning was found in only 2 research papers. The
remaining research papers (92.8%) found no significant difference in favor of online learning. This
finding suggests that, statistically wise, online learning is not better than face-to-face instruction.
In another meta-analysis reviewing online and distance learning studies from 1985 through 2002
(Bernard et al., 2004), 688 independent effect sizes that could be subjected to meta-analysis were
identified. The meta-analysis found that in terms of achievement outcomes, the mean effect size is
near zero (standard deviation of 0.439). Bernard et al. point out that it is clear from the range of effect
sizes (−1.31 to +1.41) that in almost half of the cases, classroom instruction is far better than distance
education. Furthermore, a comparison of student attitude towards distance and classroom instruction
reveals that there is a small negative but significant effect on overall attitude outcomes in favor of
classroom instruction (g+ = −0.0812). The researchers also found a significant effect in favor of
classroom instruction (g+ = −0.0573) on retention outcomes. The research findings suggest that
distance learning is not better than face-to-face instruction in terms of achievement outcomes.
Moreover, students appear to perceive traditional instruction more favorably than online instruction.
296

Danny Glick and Roni Aviram
In another meta-analysis comparing online and distance learning studies from 2000 through 2002
(Ungerleider and Burns, 2003), 12 studies that have enough statistical information for inclusion into
the meta-analysis were found. In this meta-analysis the researchers looked at the effectiveness of
online versus traditional forms of instruction by focusing on two measures: achievement measures,
and satisfaction ratings. The researchers found no difference between forms of instruction on grades,
and a significantly negative effect (-0.509) of online instruction on student satisfaction (p < .0001).
Ungerleider and Burns point out that students appear to perceive online instruction less favorably than
traditional instruction, regardless of their test scores grades. In addition, the researchers also
conducted in-depth narrative reviews of 25 studies that did not include enough statistical information
for a meta-analysis. The 25 studies in the narrative reviews present a mixed picture of the
effectiveness of online learning. While some studies reported significantly higher levels of
achievements and student satisfaction with the traditional format, other studies showed a reversed
difference. Ungerleider and Burns claim that the number of studies purporting to show an effect on
one method over the other is small and there is nothing present in the studies reviewed to allow the
researchers to make strong statements one way or the other.
Table 1 presents a summary of the meta-analyses discussed in this chapter.
Table 1: Summary of the meta-analyses
Meta-Analysis Years What was Number of Effect Mean
Measured Sizes Identified Effect Size
US Department of Education
(2009)
1996-2008 Learning
Outcomes
51 +0.35 t
+0.14 t
Bernard et al. (2004) 1985-2002 Learning
Outcomes
318 -0.0100
Bernard et al. (2004) 1985-2002 Attitude 262 -0.0812*
Ungerleider and Burns
(2003)
2000-2002 Attitude 4 -0.509**
Ungerleider and Burns 2000-2002 Learning
12 0.000
(2003)
Outcomes
* p < .001; **p < .0001; t Instruction combining online and face-to-face elements had a larger
advantage relative to purely face-to-face instruction (+0.35) than did purely online instruction (+0.14).
Thomas Russell’s No Significant Difference Website (www.nosignificantdifference.org)
provides access to hundreds of research reports that document no significant difference as well as
reports that do document significant difference in student outcomes between technology-enhanced
teaching and face-to-face instruction. Analysis of the reports published on Russell’s Website found 42
research papers comparing online learning with face-to-face instruction. Analysis of these research
reports reveals that significant difference in favor of online learning was found in 13 research papers
(30.9%). The remaining 29 research papers (69.1%) found no significant difference in favor of neither
of the instruction modes.
To sum up, the literature review suggests that in terms of achievement outcomes, online learning is
no better than face-to-face instruction, and is some cases, inferior to classroom instruction. In terms of
student satisfaction, a comparison of student satisfaction towards distance and classroom instruction
reveals that there is a small negative but significant effect on overall attitude outcomes in favor of
classroom instruction. Finally, the meta-analyses found a significant effect in favor of classroom
instruction on retention outcomes.
3. Research hypothesis
The research hypothesis is that the main reason for the little impact online learning has on learning
outcomes and student satisfaction is lack of Mindful Learning Processes (MLP), a new term coined by
the authors of this study. The authors define MLP as an online course that is based on four
interconnected components (see Figure 1):
1) Learning Theory: Learning theories play a key role in any learning process. Anderson (2008)
claims that theory allows and even forces educators to see the big picture and makes it possible to
view their practice and their research from a broader perspective than envisioned from the murky
trenches of their practice. This broader perspective helps educators make connections with the work
of others, facilitates coherent frameworks and deeper understanding of their actions, and perhaps
297

Danny Glick and Roni Aviram
most importantly, allows them to transfer the experience gained in one context to new experiences
and contexts.
2) Leaning Environment: A well-designed Web-based learning environment and a user interface that
is based on a theory are an integral part of a Mindful Learning Process. Plass (1998) defines user
interface as the part of an application in charge of communication with the learner. Plass argues that
the user interface has a key function because it conveys the functionality of a computer application to
the user, and translates the user's input into a machine-specific format.
3) Learning Process: The learning process is a stage in which the learner realizes the course
objectives through interaction with the course platform, content, and communication tools. Anderson
(2003) argues that interaction is a critical component of the educational process, and should include
three types of interaction: student-student; student-teacher; student-content.
4) Assessment: Assessment is the forth component of a Mindful Learning Process. Using traditional
and alterative assessment tools, educators should assess their students’ progress and provide
personalized feedback. Birenbaum (2003) claims that the course assessment tools should reflect the
course goals and objectives.
An online course that wishes to fulfill the MLP requirements must, first and foremost,
be based on a learning theory. Bigge and Shermis (1999) claim that action, whether a part of
teaching or any other activity in life, either is linked to a theory or is blind and purposeless. A teacher
who does not make use of a systematic body of theory in his day-by-day decisions is behaving
blindly. Consequently, any purposeful action is governed by theory. Furthermore, the learning
environment, learning process and course assessment criteria should reflect the learning theory the
course is based on and harmonize with each other.
Figure 1: The four components of a Mindful Learning Process
The goal of this paper is to check whether the research hypothesis outlined in this chapter may be
accepted. To achieve that, the authors have reviewed case studies describing online courses
published in peer-reviewed journals in the area of educational technology and e-learning. They
authors have identified three case studies that fulfill the MLP requirements stated above, and
analyzed them in terms of learning theory, learning processes, learning environment and assessment
tools. To measure the impact of the MLP on course outcomes, the authors analyzed these case
studies in terms of test scores and students’ perception of the e-learning experience. The next
chapter provides a description of the three case studies together with the impact of the MLP-based
learning environment on course outcomes.
298

Danny Glick and Roni Aviram
4. The impact of MLP on course outcomes – case study analysis
4.1 A constructivist approach to online college learning (Rovai, 2004)
This paper evaluates the effectiveness of an online graduate course in education research design
(N=82). This online course, which fulfills the four MLP requirements outlined in chapter 3, was
designed and delivered based on constructivist epistemology. The author of this paper fulfills the first
MLP requirement (Learning Theory) by linking the course to a learning theory. The author provides a
detailed description of a constructivist approach to online learning and outlines the differences
between traditional and constructivist learning environments. With regard to the second MLP
requirement (Learning Environment), the paper fulfills this requirement by designing a learning
environment that reflects the constructivist learning theory the course is based on. The course
learning environment includes a wide range of constructivist-based tools such as tools for
collaborative group work, Instructor-Student and Student-Student tools for interactions, tools for
guided discussions, discussion boards for posting the assessment tasks, etc. The course also fulfills
the third MLP requirement (Learning Process) by describing the course constructivist-based learning
processes which require learners to participate in a problem-oriented discussion forum, and to solve
authentic tasks requiring the evaluation of research-based case study. Finally, the course fulfills the
fourth MLP requirement (Assessment) by assessing students’ progress in the course using
constructivist-based assessment tools such as authentic assessment tasks, analysis of researchbased
case studies, use of rubrics to assess forum discussions and collaborative work, etc.
The research findings reveal that the course outcomes were positive in terms of course grades and
student level of satisfaction. With regard to course grades, course letter grades were distributed as
follows: F, 3 students; C, 3 students; C+, 0 students; B−, 7 students; B, 23 students; B+, 15 students;
A−18 students; and A, 3 students. In other words, 69 students (81.9%) have earned an A or a B
grade. To measure perceived learning effectiveness, students were asked to respond to the following
item via an online survey: On a scale of 0 to 9, how much did you learn in this course, with 0 meaning
you learned nothing and 9 meaning you learned more than in any other course you’ve had? The
mean and standard deviation for perceived learning among this group of participants were 7.17 (on a
9-point scale) and 1.23, respectively. When comparing this course with other courses (both traditional
and Web-based), students report that they learn more than in most other courses that they have
learned.
4.2 Community building, emergent design and expecting the unexpected: Creating a
quality eLearning experience (Thompson & MacDonald, 2005)
This paper evaluates the effectiveness and factors that influence the quality of an online Synthesis
Seminar, the last of ten courses required to obtain a M.Ed. at the University of Ottawa. This online
course, which was based on a constructivist learning theory, fulfills the four MLP requirements. In this
Synthesis Seminar, learners (N=19) reflect on their professional development throughout the M.Ed.
program and demonstrate that they can apply new learnings by writing a 25-page research paper.
The knowledge built throughout this learning experience was directly related to the learners’
professional and personal interests and constructed as they wrote their paper and shared insights in
the discussion groups. The only content provided were brief online documents designed to support
learners in this constructivist learning environment. As mandated by the University, this pass/fail
course was based on assessment of the research paper. To help learners produce the best possible
paper, a series of five cumulative assignments provided opportunities for formative assessment and
enabled the learners to build their papers in stages. Several strategies were employed to encourage
online participation: (a) online participation was included in the list of course requirements to
emphasize its importance; (b) online forums were intended to be useful to the learners and designed
to have the potential to engage a diverse group of people in a meaningful dialogue that would
enhance their research papers; and (c) the focus of each discussion group was highlighted in the
course outline and the forum moderator posted frequent reminders and encouragement.
Constructivists acknowledge that learning is active, situated, and social. Consequently several
strategies were implemented to facilitate community in the course. First, learners were grouped into
triads; triad members were expected to give each other feedback on course assignments. Second,
dialogue in the discussion groups was emphasized. Eight one-week discussion groups were
strategically placed throughout the course. Learners were asked to share: aspects of their paper,
299

Danny Glick and Roni Aviram
opinions and insights, their journey as e-learners, and how their research related to challenges in their
professional work.
The research findings reveal that the course outcomes were positive in terms of perception of the elearning
experience. Students were asked to respond to the following questions: On a scale of 1 to 5,
(a) as a result of this course I have acquired personal and professional growth and (b), the course met
my learning objectives, with 1 being Strongly Disagree and 5 being Strongly Agree? The mean for
questions (a) and (b) was 4.3 and 4 (on a 5-point scale), respectively. With regard to achievement
outcomes, all 19 students passed the course (100%).
4.3 Instructional design strategies for intensive online courses: an objectivistconstructivist
blended approach (Chen, 2007)
This paper evaluates the effectiveness of a three-credit online course for graduate education majors
(N=12) regarding the integration of technology in the K-12 curriculum. This online course, which fulfills
the four MLP requirements outlined in chapter 3, was designed and delivered based on a mix of
objectivist and constructivist theories and design strategies. With regard to the first MLP requirement
(Learning Theory), the author provides a detailed description of objectivist and constructivist
approaches to online learning and outlines the differences between these two learning theories. With
regard to the second, third and forth MLP requirements (Course Design, Learning Process and
Assessment), this course assigned the students a problem-based capstone learning task with a real
world context. The learning task was designed to provide opportunities for collaboration and social
interaction during learning and reflection after learning. To scaffold the complexity of the capstone
task and define its structure, objectivist approach was used to break the capstone task into several
subtasks and provide a set of clear and detailed guidance for each subtask to direct group
interactions for completing the problem/task. The culminating task required students to develop a
course plan integrating technology into teaching and learning. The culminating task was broken down
into subtasks such as creating a website evaluation instrument, evaluating web resources for
instruction, and developing a web-enhanced lesson plan. Each subtask was designed as a unit. A
project was assigned for each unit. To support collaboration and interaction among members of the
community, a number of strategies were adopted for the design of a social support system such as
forming heterogeneous groups, enhancing effective peer review, opening forums for social
interaction, etc.
The research findings reveal that the course outcomes were positive in terms of students’ opinion
about the online course. Students were asked to respond to the following questions: On a scale of 1
to 4, (a) I find the course learning tasks, activities and instruction effective; (b) the course met my
learning objectives in terms of personal and professional development, with 1 being Strongly Disagree
and 4 being Strongly Agree. The mean for questions (a) and (b) was 3.68 (SD=.26) and 3.60
(SD=.13), respectively. With regard to the course grades, the average score on the final project was
93.75.
Table 2 below presents a summary of how the above case studies fulfill the MLP requirements. Table
2 also presents the outcomes of each of the three courses in terms of test scores and student level of
satisfaction.
Table 2: Case study summary in accordance with the MLP requirements
Learning
Theory
Learning
Environment
Case Study 1
(Rovai, 2004)
Constructivism
-Tools for collaborative
group work
- Tools for guided
discussions
- Discussion boards
- Tools for Instructor-
Students and Student-
Student interaction
300
Case Study 2
(Thompson &
MacDonald, 2005)
Constructivism
-Tools for guided
discussions
-Discussion boards
- Tools for peer feedback
Case Study 3
(Chen, 2007)
Constructivism &
Objectivism
-Tools for collaboration
-Tools for social
interaction
- Forums
- Tools for peer review

Learning
Process
Assessment
Test
Scores
Student Level of
Satisfaction
5. Discussion
Danny Glick and Roni Aviram
Case Study 1
(Rovai, 2004)
-Problem-oriented
discussion forum
- Research-based case
studies
-Authentic assessment
tasks
- Analysis of researchbased
case studies
- Rubrics
81.9% of students have
earned an A or a B grade
Case Study 2
(Thompson &
MacDonald, 2005)
-Research paper
-Knowledge application
- Group discussions
- Peer feedback
-Research paper built in
stages
All students passed the
course.
Case Study 3
(Chen, 2007)
-Problem-based tasks
- Authentic tasks
- Reflection
- Breaking task into
subtasks
- Peer review
- Project Work
- Developing a course
plan
- Evaluating Web
resources
- Developing lesson plans
93.75
7.17 on a 9-point scale 4.3 on a 5-point scale 3.6 on a 4-point scale
Online education is growing rapidly within all levels of education (Allen & Seaman, 2010; Watson,
2007) and is expected to grow during the coming years (Redden, 2009; Rovai & Downey, 2010). In
addition, Chief Academic Officers have made online learning part of their long-term strategy (Allen &
Seaman, 2007; Rovai & Downey, 2010). Research, however, indicates that online learning has little
impact on course outcomes. The authors of this paper hypothesize that this paradox is a result of lack
of Mindful Learning Process in online education. The hypothesis of this paper is that online courses
that fulfill the MLP requirements as defined in chapter 3 are likely to have positive outcomes.
Analysis of three case studies (Rovai, 2004; Thompson & MacDonald, 2005; Chen, 2007) suggests
that online courses that fulfill the MLP requirements may lead to positive outcomes in terms of test
scores and student level of satisfaction. These case studies fulfill the first MLP requirement – Learning
Theory – by linking the course to a learning theory and providing a detailed description of the theory
the course is based on. They also fulfill the second and third MLP requirements – Learning
Environment and Learning Process – by using technologies and applying e-pedagogies which reflect
the course learning theory. And finally, the authors of these papers use a variety of alternative
assessment methods to reflect the course learning theory.
The research findings suggest that a Mindful Learning Process may lead to a meaningful and
pedagogically sound course where theory, design, learning and assessment harmonize with each
other. These findings support earlier work on learning theories and online learning (Ally, 2008; Clark,
2001; Rovai, 2002) that found that the reason for the learning benefits of computer media is not the
medium of instruction, but the instructional strategies built into the learning materials, and that the
development of effective online learning materials should be based on proven and sound learning
theories.
References
Allen, E. I. and Seaman, J. (2007). Making the Grade: Online Education in the United States, 2006, The Sloan
Consortium, Needham.
Allen, E. I. and Seaman, J. (2010). Class Differences: Online Education in the United States, 2010, The Sloan
Consortium, Needham.
Ally, M. (2008). “Foundations of Educational Theory for Online Learning”. In T. Anderson (Ed.), Theory and
Practice of Online Learning, 2 nd ed. (pp 15-44), AU Press, Edmonton.
Anderson, T. (2003). “Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction”,
International Review of Research in Open and Distance Learning, Vol 4, No. 2.
Anderson, T. (2008). “Towards a Theory of Online Learning”. In T. Anderson (Ed.), The Theory and Practice of
Online Learning (pp 45-74), AU Press, Edmonton.
301

Danny Glick and Roni Aviram
Bernard, R. M., Abrami, P. C., Lou, Y, Borokhovski, E., Wade, A., Wozney, L., Wallet, P. A., Fiset, M. and Huang,
B. (2004). “How Does Distance Education Compare with Classroom Instruction? A Meta-Analysis of the
Empirical Literature”, Review of Educational Research, Vol 74, No. 3, p. 379-439.
Bigge, M. L. and Shermis, S. S. (1999). Learning Theories for Teachers (6th Ed.), Longman, New York.
Birenbaum, M. (2003). “New Insights into Learning and Teaching and their Implications for Assessment”. In
M. Segers, F. Dochy and E. Cascallar (Eds.) Optimizing New Methods of Assessment: In Search of
Qualities and Standards. (pp 13-36), Kluwer, Boston.
Chen, S. J. (2007). “Instructional Design Strategies for Intensive Online Courses: An Objectivist-Constructivist
Blended Approach”, Journal of Interactive Online Learning, Vol 6, No. 1, pp 72-86.
Clark, R. E. (2001). “A Summary of Disagreements with the ‘Mere Vehicles’ Argument. In R. E. Clark (Ed.),
Learning from Media: Arguments, Analysis, and Evidence (pp 125–136), Information Age Publishing,
Greenwich.
Hutti, D. (2007). “Online Learning, Quality, and Illinois Community Colleges”, MERLOT Journal of Online
Learning and Teaching, Vol 3, No. 1, pp 18-29.
iNACOL. (2010). “Fast Facts about Online Learning”, [online], International Association for K-12 Online Learning,
http://www.inacol.org/press/nacol_fast_facts.pdf.
Plass, J. L. (1998). “Design and Evaluation of the User Interface of Foreign Language Multimedia Software: A
Cognitive Approach”, Language Learning & Technology, Vol 2, No. 1, pp 40-53.
Redden, E. (2009). “In Global Recession, Global Ed Still Growing”, [online], Inside Higher Ed.,
http://www.insidehighered.com/news/2009/05/29/international. .
Rovai, A. P. (2002). “Building Sense of Community at a Distance”, International Review of Research in Open and
Distance Learning, Vol 3, No. 1.
Rovai, A. P. (2004). “A Constructivist Approach to Online College Learning”, Internet and Higher Education, Vol
7, No. 2, pp 79-93.
Rovai, A. P. and Downey, J. (2010). “Why Some Distance Education Programs Fail while Others Succeed in a
Global Environment”, The Internet and Higher Education, Vol 13, No. 3, pp 141-147.
U.S. Department of Education. (2009). Evaluation of Evidence-Based Practices in Online Learning: A Meta-
Analysis and Review of Online Learning Studies, Washington, D.C.
Shachar, M. (2008). “Meta-Analysis: The Preferred Method of Choice for the Assessment of Distance Learning
Quality Factors”, The International Review of Research in Open and Distance Learning, Vol 9, No. 3.
Thompson, T. L. and MacDonald, C. J. (2005). “Community Building, Emergent Design and Expecting the
Unexpected: Creating a Quality eLearning Experience”, Internet and Higher Education, Vol 8, No. 3, pp
233-249.
Ungerleider, C. and Burns, T. (2003). “A Systematic Review of the Effectiveness and Efficiency of Networked ICT
in Education”, A state of the field report to the Council of Ministries of Education, Canada and Industry
Canada, October.
Watson, J. F. (2007). “A National Primer on K-12 Online Learning”, [online], NACOL,
http://www.nacol.org/docs/national_report.pdf.
302

Researching and Sharing – Business School Students
Creating a Wiki Glossary
Andrea Gorra and Ollie Jones
Faculty of Business and Law, Leeds Metropolitan University, UK
gorra@leedsmet.ac.uk
o.jones@leedsmet.ac.uk
Abstract: Often students find that the terminology involved in a new subject can be a barrier to learning, Some
theorists term this 'cognitive load' - a high amount of information that the brain has to process before it can begin
to construct new knowledge. Glossaries have been used in many different educational contexts; however
traditional glossaries are passive and have less capability to promote student engagement. This paper outlines a
small scale study in a UK business school that used a small wiki situated within the Virtual Learning Environment
to encourage students to construct their own glossary at the start of a module. Whilst many studies have looked
at use of wikis in student work, especially collaborative projects, relatively few have investigated the use of wikis
for constructing simple glossary entries created by students. In our study each student was allocated a particular
subject related term. The students were instructed to construct a wiki entry describing what the term meant, citing
at least two appropriate references. The study was evaluated by a variety methods including quantitative analysis
of the Virtual Learning Environment usage and access statistics, alongside qualitative and quantitative survey
data. Generally the quality of the entries was very high and students indicated that writing the wiki glossary
entries helped them to understand the terminology of the new subject. Students stated that they had read a
number of entries over the course of the module and some chose to reference the wiki in their final written
assignment. Our research indicates that students found creating the wiki more useful than the finished resource
itself.
Keywords: wiki, glossary, active learning, VLE
1. Background
Often students find that the terminology involved in a new subject can be a barrier to learning about
concepts (see for example Wandersee, 1988; Lidbury and Zhang, 2008). Theorists such as Sweller
(1994) determine this as ‘cognitive load’ – the higher the amount of information the brain the must
process before it can begin to construct knowledge. Glossaries have been used in many different
educational contexts (Hall, 2002) to enable more effective dialogue between tutor and student.
Laurillard (2002) believes that the use of glossaries can help with the students’ learning process.
However traditional glossaries are passive and have less capability to promote student engagement,
especially at the beginning of studying a subject.
This paper outlines a small scale study in a Business School in the North of England that used wikis
to encourage students to construct their own glossary at the start of a module. A wiki is a website that
can be modified and contributed to by internet users (Encyclopædia Britannica, 2011). Wikis and
other social networking sites are part of a host of applications that are also known as Web 2.0
technologies (O'Reilly, 2005).
The rationale for introducing a glossary task was to foster enquiry based learning (Khan and
O’Rourke, 2005) in order to engage students early on in the 12 week Operations management
module and to help the students to familiarise themselves with the terminology of this new subject.
Whilst a lot of studies have looked at use of wikis in student work , especially collaborative projects,
relatively few have looked at using wikis for constructing simple glossary entries. In those studies that
do, such as Cubric (2010), the glossary element forms a small part of the overall research.
The students taking part in this study were in their second year of study (Level 5), from two separate
undergraduate business courses, namely Business Studies, and Business and Management studies.
The module taken was ‘Operations Management’ (OM) which was taught in the first semester at
Leeds Metropolitan University. The OM module was delivered by 5 tutors over 12 classes and was
delivered by using lectures and tutorials.
The assessment was a multi-media case study accessible the Virtual Learning Environment (VLE)
and via an additional DVD. The case study aimed at developing the students’ employability skills and
303

Andrea Gorra and Ollie Jones
to help them to handle and make sense of complexity (see Jones and Kerr, 2008, for more detail on
the multi-media case study).
Our key research questions were as follows:
Do students find terms, language in unfamiliar subject a barrier to learning?
Can the process of constructing a glossary help learning?
Is the outcome of a student generated glossary a useful/used resource for students?
2. Study details and data collection methods
All students taking the Operations Management module were allocated a particular term or concept
from the subject area of Operations Management. The students were then given two weeks to
research this term and construct a wiki entry describing what the term meant. They had to reference
at least two appropriate sources, which could include websites, books etc.
The wiki entry was a module requirement and completion was necessary in order to gain access to
the assignment multi-media case study early on in the semester.
Operations Management concept allocated to the students were terms such as the following: “Agile
Supply Chains”, “Buffer Inventory”, “Kanban” and “Lean”. See Figure 1, below, for an example of a
wiki entry written by one of the second year students.
Sample wiki entry
Lean production has three main components; just-in-time, total quality management and time-based
management (Marcousé, 2003). First adopted by Toyota motor corporation lean production aims to minimise
waste, reduces costs, increases productivity and expand margins in order to make business more efficient in its
use of resources (Biz/ed n.d.)Bibliography: Marcousé et al (2003) Business Studies. 2nd ed. Hodder
Arnold.Biz/ed (n.d.) Glossary description. [internet], Available from: < http://www.bized.co.uk/cgibin/glossarydb/browse.pl?glostopic=1&glosid=1346>
[Accessed 8th October 2010].
Figure 1: An example of a wiki glossary term created by a student
After the students had submitted their term descriptions in form of a wiki glossary entry via the VLE,
the module tutors marked the entries as either pass or fail. Students who ‘failed’ were permitted to
submit a revised entry.
Following this, the wiki was available for students to use throughout the rest of the module and the
approach was embedded into the overall enquiry based learning strategy. The students were able to
use the wiki to aid them later in the module and were encouraged to utilise the definitions in their
assignment. In addition, some of the students’ wiki glossary definitions were incorporated into the
lectures.
See Figure 2 for an overview of the steps involved in the creation of the wiki glossary for the
Operations Management module.
The wiki task attracted a good 95 percent participation rate from the 260 student strong cohort.
However, this is not surprising as the creation of the allocated two terms for the wiki glossary was
needed for the students to gain access to the multi-media assignment material early on in the
semester.
Our small scale study was evaluated using a variety methods including quantitative analysis of the
VLE usage and access statistics, alongside qualitative and quantitative survey data.
3. Results from the survey
Empirical data was collected by using an online survey and the link to the survey was distributed to
the students via their student email address. 56 out of the 260 students responded to the survey
which equates to a 22% participation rate. In addition, we used statistics from the VLE to ascertain
how many students accessed the finished wiki glossary.
304

Tutor finds and allocates
Glossary Terms
Andrea Gorra and Ollie Jones
Wiki -Glossary Design Story Board
Students gain Access
Student researches
Terms from two sources
Students read each others entries
Students can access
Assessment Resources Faculty at of later Business date & Law
Figure 2: Process of creating the wiki glossary
3.1 New subject terminology and preferred learning materials
Upload to the VLE, include
references
Tutor(s) validates all entries
Tutor uses selection
Of terms
In some lectures
The survey confirmed findings from the literature in that the terminology of a new subject can be
perceived by students as an initial barrier, with 70% of the 56 students either agreeing or strongly
agreeing with this statement (see Figure 3, below).
Figure 3: Students perceptions of new subject terminology
305

Andrea Gorra and Ollie Jones
However, it appears that our students would not typically access a glossary when approaching a new
subject. Survey findings (see Figure 4, below) indicate that the majority of our second year students
would only “sometimes” use a traditional glossary when starting a new subject.
Figure 4: Students' use of 'traditional' glossaries
3.2 The wiki glossary as a constructive learning experience and as a resource
The majority of our students perceived that constructing the glossary had helped them to understand
the subject area of operations management (see Figure 5,). In addition, 52% of our 260 students
agreed with the statement “I found the wiki glossary task helpful in getting me engaged and interested
in the operations management subject”, while 29% disagreed and a fifth neither agreed, nor
disagreed.
Figure 5: Wiki glossary aiding understanding
When asked whether the wiki glossary created by fellow students helped them with the written
assignment, the students’ responses were rather ambivalent. About a third of the students agreed
306

Andrea Gorra and Ollie Jones
with this statement, while as much as 40% disagreed and about a quarter neither agreed or disagreed
(see Figure 6, below).
Figure 6: Use of the wiki glossary for the OM assignment
The majority of students perceived the completed wiki glossary was a useful learning resource (see
Figure 7), while an even greater majority believed that ‘constructing’ the glossary has helped their
learning process in this new subject area (Figure 8).
Figure 7: Perceptions of the wiki glossary as a resource
307

Andrea Gorra and Ollie Jones
Figure 8: Has constructing the wiki glossary helped learning?
Finally we asked the students’ opinion about their wiki glossary entries being used in the Operations
Management lectures. While the majority of students (59%) agreed with statement “I liked the tutor’s
use of student created definitions in subsequent lectures”, a small minority (18%) were less happy
with this (see Figure 9, below).
Figure 9: Students’ views on wiki glossary use in lectures
Some qualitative comments about the wiki use in lectures helped us to understand the students’
views further. Some positive comments were expressed as follows:
“Showed that the input from students mattered.”
“It made them easier to understand”
308

Andrea Gorra and Ollie Jones
“We could see that are efforts weren't going in vain. and it's always good to see your
name somewhere”
While some less positive ones indicated that some students would have preferred to be only
presented with materials created by tutors (“I would prefer the tutors more”), while others felt the
inclusion of the wiki comments did not fit into the flow of the Operations Management lecture:
“because it was never clear and on point to the questions”
Students are not a homogenous group and some found the use of their own wiki glossary entries
embarrassing while others thought it built their confidence as the following comments show:
“it was embarrassing for those who had written them and a lot of the time it broke the
flow of the lecture”
“It helped to build students confidence”.
4. Conclusions and implications for future research
The module tutors read and marked every wiki entry and the overall tutor impression was that the
quality of the entries was generally high.
By evaluating the empirical data, we identified some answers to our initial three research questions:
1. Do students find terms, language in unfamiliar subject a barrier to learning?
Our survey confirmed our initial suspicion and findings from literature review (e.g. Lidbury and Zhang,
2008) in that students do perceive the language and definitions of a new subject area as a barrier to
both understanding and learning.
2. Can the process of constructing a glossary help learning?
The results from the survey show that students found the process of construction useful as well as the
resource itself. Qualitative comments acknowledged that constructing a wiki glossary engaged the
students. In addition, the students perceived it as positive to be able to read other students’ definition
of a key term as opposed to only having the tutor’s or text book definitions available as for example
the following comment illustrates: “Because it engaged students and gave us another students
perspective on the definitions“.
As discussed earlier, the process of constructing a glossary by the students can be seen as an
enquiry-based learning approach which expected students to do their own research on given glossary
terms. Through this process of constructing their own understanding of these terms, we believe,
similar to Laurillard (2002), that this process can help students’ learning.
The social component of learning was less pronounced in our chosen approach, as enquiry-based
learning is often carried out in small groups (see Khan and O’Rourke, 2005). However, the fact that
each wiki glossary term had several visits from other students may be an indication of an added
(small) social dimension to learning when using glossaries. The aim to increase student motivation
through enquiry-based learning (Khan and O’Rourke, 2005) appears also to have been fulfilled as the
student quote above indicates.
3. Is the outcome of a student generated glossary a useful/used resource for students?
Our students perceived the final glossary to be a useful resource, as the survey findings show. For
example Figure 7 indicates that the majority of students agree with the statement “I found the wiki
glossary a useful learning resource for the module”. However, a significant number of students did not
feel that the wiki glossary entries helped them with the assessment (for detail see Figure 6).
Overall it can be concluded, that the actual creation of the wiki resource, i.e. the researching and
writing about a subject-specific term, was perceived as more useful as the final resource in form of a
wiki itself. This ties in with some of the students commenting that they appreciated that “[..] that all
definitions were correct and proof checked [by the tutors]”.
309

Andrea Gorra and Ollie Jones
Students appeared to read a number of entries over the course of the module, with our VLE statistics
showing that on average every wiki term had seven visits from other students.
There was mixed feedback regarding the use of wiki glossary entries in the lectures. Some students
commented rather negatively using comments such as “it was embarrassing for those who had written
them and a lot of the time it broke the flow of the lecture” or “I would prefer the tutors more”. While
other students commented “It's good to see work we've done being appreciated” and “It is an
acknowledgement that Student's contributions are valued”.
Overall, we can conclude that the creation of wiki glossary terms had been a successful addition to
the module. Students appeared to engage with the specialist terms of the subject area from an early
stage of the 12 week module and appreciated having access to definitions created by their fellow
students. The final glossary resource based on the collaborative writing platform of a wiki can be seen
as the outcome of ‘collaborative group work’, which also has familiarised the students with this Web
2.0 technology (Shailey, 2009).
However, we feel that this approach will only work if there are sufficient incentives for students to
create these short wiki glossaries. For example, we used the provision of early access to the multimedia
assignment case study as an incentive.
We are planning to use this approach in future deliveries of this Operations Management module as
well as other modules. Comments about the study to the authors are very welcome.
References
Cubric, M. (2010) Wiki-based Process Framework for Blended Learning. Proceedings of the 2007 international
symposium on Wikis, p.11-24, October 21-25, 2007, Montreal, Quebec, Canada.
Encyclopædia Britannica (2011) [Internet]. Available from: [Accessed 27 June
2011].
Hall, R. (2002) Aligning learning, teaching and assessment using the web: an evaluation of pedagogic
approaches, British Journal of Educational Technology, 33.2, pp 149-158
Kerr, M., Jones, O. (2008) ‘e-invigorating operations management case assessments using a multi-media
approach ’, Proceedings of the 15th Annual EurOMAConference,Tradition and Innovation in Operations
Management, University of Groningen, The Netherlands, June 23 rd 2008.
Khan, P., & O’Rourke, K. (2005). Understanding enquiry-based learning. in Barrett, T., MacLabhrainn, I., Fallon,
H. (eds), Handbook of enquiry and problem based learning. Galway: CELT.
Lidbury, B. A. & Zhang, F. (2008). Comprehension of Scientific Language as a Strategy to Enhance Learning and
Engagement for Molecular Biology Students. Australian Biochemist: The Magazine of the Australian Society
for Biochemistry and Molecular Biology, 39, 10-13.
O'Reilly, T. (2005) What Is Web 2.0. Design Patterns and Business Models for the Next Generation of Software
[Internet]. Available from: [Accessed 27 June 2011].
Shailey, M. (2009) "Role of social software tools in education: a literature review", Education + Training, Vol. 51
Iss: 5/6, pp.353 - 369.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4,
295-312.
Wandersee, J. H. (1988). Ways students read texts. Journal of Research in Science Teaching, 25, 69-84.
310

A Qualitative Evaluation of Academic Staff’s Perceptions of
Second Life as a Teaching Tool
Rose Heaney and Megan Anne Arroll
University of East London, School of Psychology, London, UK
r.heaney@uel.ac.uk
m.a.arroll@uel.ac.uk
Abstract: The aim of the study described in this paper was to investigate the potential of Second Life® (SL) as a
teaching tool on an undergraduate Psychology programme at University of East London (UEL). A qualitative
methodology (Interpretative Phenomenological Analysis; Smith, 1996) was chosen as we wanted to allow the
participants to express openly their first hand experience of using SL in their teaching at the same time as
bringing the researcher’s interpretation to bear. We interviewed four lecturers, with varied prior experience of
online teaching, before and after they conducted a tutorial-type session within SL (although one participant was
interviewed once only - after his session). From the interview transcriptions, we identified three superordinate
themes, namely ‘comfort’, ‘investment’ and ‘clear rationale’. Participants stated that the virtual and anonymous
nature of SL appeared to allow students who might have had difficulty voicing their questions/concerns in a
traditional, physical environment to engage actively in the session. It was also seen as a useful adjunct to the
physical campus for students studying at a distance for reasons of geography, illness, bad weather and the like.
However, these positives had to be offset against drawbacks such as the lack of non-verbal cues so important to
staff in the physical classroom and the potential for staff embarrassment in front of students in what is a
challenging and unpredictable environment. (Unlike students, staff could not hide behind their avatars.) Given the
unique characteristics of SL and the not insignificant investment of both time and resources by both staff and
students to take full advantage of the environment, they all emphasised the need for a clear rationale for its use.
All felt it had obvious potential for activities such as role play exercises, problem based learning scenarios and
student presentations but were less certain about its suitability for tutorials (the main activity experienced during
the study) and other forms of group teaching. In sum, this study illustrated that with adequate training of both staff
and students, clear strategies for use and a supportive and encouraging institutional environment, SL can be a
beneficial addition to the teaching and learning repertoire of higher education.
Keywords: second life; staff perceptions; interpretative phenomenological analysis
1. Background
Second Life® (SL) is a web-based, 3D virtual world, where users interact with each other via their
avatars (3D representations of themselves). As one of the largest and best known virtual words with
millions of users and tens of millions of square metres of virtual land, SL has been put to many uses,
not least educational. As a Multi User Virtual Environment (MUVE) rather than a Massively Multiplayer
Online Role-Playing Game (MMPORG) such as World of Warcraft (WoW), SL does not come with
pre-defined structures and goal oriented game-play though activities of this sort may be created.
Users or residents of SL create content - 98% of all content in SL is user-generated (Boellstorff, 2008)
- or use resources developed by others, make choices about how and with whom to interact and
whether to restrict themselves to their own (or their institution’s) private spaces or visit the many
public arenas. Many universities in the UK and elsewhere have been active in SL for several years
and continue to be, though recent increases in land charges and its continued lack of adoption as a
mainstream learning platform cast some doubts on its use in the longer term. However, the unique
ability of SL (and Virtual Worlds in general) to give users a sense of being present in an environment
other than the one they are actually in, and to interact within it (Schroeder, 2008), make for a rich
educational experience that can be expected to prevail in one form or the other and, as such, a fruitful
area for research and exploration. Additionally, SL’s text and voice communication facilities, in
combination with its 3D features, make it a viable and potentially more rewarding setting for online
meetings, tutorials etc., than those in more common use.
2. Teaching and learning in SL
The most documented educational uses of SL are those that benefit from its immersive and
visualising qualities such as: displays and exhibits; role play; simulations; historical recreations;
language learning; problem based learning (Warburton, 2009). Butler and White (2008) categorise
activities into three distinct types of educational models: real-time interactions such as role play
exercises or teaching sessions; machinima or the viewing of virtual interactions including orientations
and/or knowledge transfer; asynchronous interactions, for instance the creation of virtual objects for
assessment or the response to scripted, existing objects/environments.
311

Rose Heaney and Megan Anne Arroll
A published example (Walker, 2009) of a real-time interaction model concerns the use of SL to teach
counselling skills to distance learning students. When assessing students’ and staff’s experiences,
Walker’s findings are mixed; students’ perceptions of the benefits of SL were only reported as slightly
higher than those of more traditional learning environments, whereas the instructor found the
environment very useful and reported that it offered potential in terms of pedagogy. This is at odds
with other studies evaluating online learning environments (OLEs) (Jones and Jones 2005; Palmer
and Holt; 2009), where students’ ratings were higher than staff’s. However this mixed picture of SL is
typical of other such studies and only serves to underline the potential barriers to effective use of this
complex and demanding environment summarised by Warburton (2009) under the headings of
technical, identity, culture, collaboration and time amongst others. The present study aims to focus
exclusively on academic staff’s experiences and perceptions in an attempt to uncover in greater depth
both the barriers and advantages of using SL in a higher education context and thereby hope to
contribute to greater pedagogical efficacy in future.
3. Context of the present study
The University of East London (UEL) has had a presence on SL since early 2009 in the form of two
islands. The school of Psychology is housed on UEL island where it has some large lecture rooms,
break-out rooms and tutorial huts outside as well as space for social interaction and information
boards and notices. The second island UEL HABitat belongs to the school of Health and Bioscience
and hosts a range of simulations and problem based learning resources including a laboratory for
molecular biology experiments, a multi-disciplinary polyclinic with virtual patient scenarios and a crime
scene house for forensic investigations as well as formal and informal meeting spaces. The HAB
activities are designed in the main to be used by students on a self-directed, asynchronous basis with
minimal use to date for synchronous or group learning activities. This contrasts with the use of SL by
the school of Psychology as a virtual conference and teaching space to complement or replace
equivalent activities and resources on the physical campus.
Having commissioned a specialist SL developer to build conferencing and tutorial facilities on UEL
Island, a group of staff in the school of Psychology have been running a range of activities for
students from all levels of the BSc Hons Psychology programme since early 2010. The students on
this programme are all part-time or full-time campus based students i.e. none are distance learners.
This paper focuses on staff experience of running group tutorials in the specially designed huts
outside the main building (Figure 1).
Figure 1: Small tutorial hut on the UEL Island, outside the main Psychology building
Tutorials on this programme are normally offered face to face and in the rare event of offering
anything online, the primary mechanism would be the VLE (Blackboard) discussion forums or
chatrooms. The main drivers for using SL were a perceived need to find alternatives to physical
tutorial spaces which are at a premium on UEL’s compact Stratford campus and to establish the
312

Rose Heaney and Megan Anne Arroll
viability of SL as a meeting place in general when staff and / or students might be prevented from
attendance at a campus location through health issues, transport difficulties, bad weather and the like.
The SL tutorials were all optional and, apart from one which was integrated with a particular module,
were on topics of general interest and advertised to students at all levels of the programme. They
were held outside normal office hours either in the evening or at weekends meaning that the majority
of participants (staff and students) were working remotely from the campus when they accessed
them.
In terms of preparation for the SL tutorials some of the students had attended face to face orientation
sessions run by the developer earlier in the semester when the facilities had first been built. They also
had access to self-help guides on Blackboard. Staff participants had attended the orientation sessions
or already had some familiarity with SL. Overall the majority of participants in the tutorials were
unfamiliar with SL, this being their first real use of it for teaching and learning purposes.
According to the pre-session interviews, apart from attendance at the orientation sessions mentioned
above, preparation by staff for the tutorials ranged from doing nothing specific, to linking the content
with a previously delivered lecture and ensuring the students were adequately prepared. In the event
attendance at the different tutorials ranged from 0 to 20+, the highest numbers being seen at the
session that was linked to a specific module and where the member of staff knew which students
were likely to attend.
4. Methodology
4.1 Epistemological and analytical framework
The goal of this study was to explore the experience of teaching via SL; therefore an inductive, rather
than deductive, approach was taken. This desire to uncover experiential data led to the decision to
attain verbal, as opposed to numeric, data in a naturalistic setting. Thus, the approach of this study
was one of phenomenology, which is concerned with meanings and the individual’s personal
perception of events and experiences rather than trying to manufacture an objective statement of an
event (in this case, conducting a tutorial in SL) or experiences (Giorgi, 1995). Hence, the most
appropriate analytical technique under these circumstances and with this epistemological perspective
was deemed Interpretative Phenomenological Analysis (IPA; Smith, 1996; Smith & Osborn, 2003) as
it was able to take into account both the individuals’ experience of using SL and the researcher’s
interpretations rather than simply stating the participants’ opinions of this novel tool.
4.2 Participants
Four full-time lecturing staff from the Psychology department of UEL were interviewed. All but one
respondent participated in a pre- and post-teaching interview; the member of staff who was only
involved in a post-teaching interview was asked to answer pre-teaching questions in retrospect as
best he could. The sample consisted of 3 female and one male participant, all of whom were involved
in undergraduate teaching duties. Three of the interviewees had some previous experience of SL but
none had conducted tutorials prior to this study. One of them had experience of Adobe Connect and
Skype for online tutorials with distance learning students.
4.3 The interviews
A study pack including an information sheet and consent form was given to each participant before
the first interview in line with the ethical approval granted from the UEL Ethics Committee. To allow for
flexibility within the academics’ schedules, the interviews were conducted over the telephone and
recorded via a digital device. The interviews were semi-structured and consisted of a range of openended
questions, including prompts that allowed further elaboration of the topic under discussion. If it
was deemed that the participants had adequately answered any questions remaining on the
schedule, these were omitted from the interview. The structure of the interview schedule complied
with a funnelling approach (Guba & Lincoln, 1981), and started with a question regarding preparation
for teaching within SL. Subsequently, participants were asked about methods of communication,
experience of the environment, and their expectations at pre-teaching interview and their actual
experience of their session within SL at post-teaching interview. The average duration of the
interviews was 32 minutes pre-teaching, 12 minutes post-teaching and the single merged interview
313

Rose Heaney and Megan Anne Arroll
lasted 26 minutes. Pseudonyms were created for each participant to ensure their anonymity and full
permission was obtained to use verbatim quotations in the final write-up of the study.
4.4 Data analysis
The process of IPA used in the study consisted of a number of stages starting with a case-by-case
analysis and followed by comparison across cases. To begin with, one transcript was read thoroughly
and repeatedly, which permitted the researcher to become familiar with the account, as each
additional reading tended to evoke new insights. Initial coding then entailed noting down anything of
interest, interpretations and making summaries of ideas. From this stage recurrent themes were
extracted, with key words or phrases that captured the essence of the content acting as codes. This
procedure was then repeated for each transcript. At this point the researcher attempted to identify
repeated patterns emerging in the subsequent transcripts whilst still allowing additional topics to be
identified (Smith & Osborn, 2003). The next stage involved looking for thematic connections, both
within and across transcripts. These themes were then clustered and developed into a consolidated
list of master or superordinate themes. The transcripts were reread to ensure that the themes and
sub-themes could undoubtedly be recognised in the verbatim transcripts. Finally, transcript quotations
were noted for each theme and a file created.
5. Findings
Three superordinate themes emerged from the data; comfort, investment and clear rationale. Each of
these themes contain sub-themes which include both positive and negative views of using SL as an
effective teaching environment and will be discussed in turn.
5.1 Comfort
The theme ‘comfort’ is based on the participants’ feeling and thoughts regarding the environment as a
whole and how this impacts upon teaching and learning. The academics felt that the anonymity that
the virtual environment provided was beneficial to students because ‘perhaps they’ll feel more
comfortable in saying “I don’t understand it” because you’re not a person’ (Jane). Similarly, Greta
thought that SL gave students ‘a way of asking anything they want without people knowing who they
are’. This level of anonymity could be very important to students who find it difficult to vocalise
problems and ask questions in real life for fear of embarrassment and could be most beneficial to
those that do not have experience in presenting work, for instance students in their first year of higher
education. Interviewed after his teaching session and after he had discussed it with students (this
academic asked for direct feedback from his students following the SL session), Tom noted that:
‘those that are… more reticent or nervous about contributing in class … were very vocal
and explicit’ (Tom)
Although students could benefit from the anonymity of the environment, staff on the other hand could
feel too exposed. Jane in particular appeared concerned by her lack of experience:
‘I do feel like it’s that I’m too young and inexperienced rather than a competent and
that’s not a nice feeling I think…oh gosh, I must be really stupid because I can’t do x, y, z
…’ (Jane)
In addition to the psychological comfort that SL afforded students, there were practical advantages as
well:
‘I think it could be a useful way of… getting round … difficulties … in London, noncampus
places…that there is actually some sort of social cohesion to what they’re doing.’
(Tom)
As Tom notes, there is often difficulty in engendering a community feeling when a university is split
into separate buildings and campuses, not all of which may be accessed by all students, and
therefore SL may provide a way of integrating student experience onto a common platform. Also, by
developing a virtual world, scarce resources could be saved as ‘we’re not using University space’
(Tom).
Another area where participants could see a use for SL was its ability to engage students who could
not attend traditional lectures:
314

Rose Heaney and Megan Anne Arroll
‘Those interested in distance learning, will probably be the ones who really get the most
out of it… for people who are physically unable to get out perhaps, for whatever reason,
whether it’s a physical thing or a childcare issue’ (Jane)
This was not only seen as a benefit for students but also staff, especially in dealing with issues of
transport and illness during the academic year:
‘Programme leader’s meeting on Wednesday ….. a snowy place …. won’t be able to get
in ….brilliant to have it in SL, everybody could be there.’ (Cathy)
‘We thought swine flu was taking over and we’d have to hold meetings there, but then it
didn’t so we didn’t. But that sort of thing would be useful.’ (Cathy)
However, the interviewees also noted the limitations of SL as a teaching tool in terms of the lack of
non-verbal information that is fundamental to effective teaching and learning:
‘If you are teaching something for the first time on there, you would not, you wouldn’t get
the visual feedback from the audience. So, for example, I had probably twenty-three
students and I’d say fifteen, fifteen or sixteen were constantly responding to me…you
assume from sort of radio silence that, that those individuals may not be engaged, but
you would never know’ (Tom).
Additionally, Greta stated that in comparison to other technologies, SL was not as effective:
‘In other systems I’ve used there is a “hands-up” signal and that allows you to have larger
numbers so it hasn’t got the mechanisms you need for successful teaching. As a general
tool for interactions between students and lecturers online, I don’t think it’s very good.
(Greta)
In fact, Greta said that she would not recommend the use of SL for online tutorials. These concerns
were echoed by Tom who was cautious with the proposition that SL should be used to the exclusion
of other, perhaps more suitable, technologies:
‘I think there’s a slight danger that people will just use it for the sake of it and that’s when
you might start to go down avenues where it’s potentially not as useful as it could be.’
(Tom)
In sum, SL was seen to be advantageous in terms of the anonymity it lent to the students and the
convenience of its virtual nature although these benefits have to be weighed against the difficulties in
interaction and the danger of using it as a one-size-fits-all instrument.
5.2 Investment
Participants were unanimous in the belief that staff needed to prepare themselves and students well
for SL activities. For example, Jane, who had not used SL in her teaching activities before the initial
interview, stated:
‘I will give myself plenty of time to make sure I’m there or at least know how to get there
when it is time…but I’m still not very confident…I am sort of a bit nervous…I found it
quite sickening moving, I’m one of these that gets motion sickness [but] it got better in
the second one’ (Jane)
However, for others who were more familiar with SL, this was not an issue and implies that, with
training, early reticence may be overcome:
‘I use SL, so I didn’t have to really get used to just interacting within the environment
because I was already familiar with that’ (Greta)
Tom was more concerned with ensuring that the students were able to navigate the environment as
he had already attended a training session earlier in the year:
‘I mean the big thing was preparing the students [as] I had no problems at all, but that’s
probably a symptom of the fact that it occurred during a training session.’ (Tom)
The strategy that Tom used to prepare the students included directing them to guides held on UEL
Plus (Blackboard), an oral introduction to SL in his face-to-face session which included registering and
setting-up an avatar, instructions on how to get to UEL Island and:
315

Rose Heaney and Megan Anne Arroll
‘The week before I said, “This is what I’m doing,” and I gave them, I pointed them in the,
in the direction of those guides that I was talking about that were available on UEL Plus.
And then I said, “If I see you I see you, if not, don’t worry’ (Tom)
Although it was integrated with a specific module teaching programme, Tom offered his SL tutorial as
an additional resource for the students, i.e. it was not compulsory. The issue of whether the virtual
sessions should be mandatory was viewed differently by the participants; Cathy, who previously had
organised a training session for staff found that only a third of the department attended that training
session because ‘it’s all optional and people didn’t want to volunteer’. Jane stated that ‘if someone
said we had to do it, then I’d have to do it’ but did not see the justification in using SL more
extensively at present. This reluctance was viewed by Cathy as a possible impact on the student
experience:
‘I think they wouldn’t engage with it if they think that, some staff won’t engage with it’
(Cathy)
The importance of a positive first experience, the frequency of use and the need to integrate the
virtual tutorials were also highlighted by the academics:
‘It [first experience] has to be something that they really get a lot out of and think ‘yes,
that was worth it. I just think if people are using it a lot, then they will become familiar with
it and it will feel less daunting…It [SL] needs to be embedded like UEL Plus, if it was an
option, they probably wouldn’t bother to go to it.’ (Jane)
Therefore, from the participant narratives, it is apparent that there is a need for investment and
organisation to reap the benefits of SL as a teaching tool. This can be achieved by holding training
sessions for staff and students so that everyone can feel confident in the environment and able to
appreciate the advantages of the environment for teaching and learning. This may require a paradigm
shift in the mindset of mainstream higher education users but is by no means unreasonable to
achieve; as Cathy points out ‘we’re in same position with SL as I used to be with the VLE’.
5.3 Clear rationale
Participants were divided on SL’s suitability as an effective environment for tutorials and group
teaching generally although all agreed that it could be beneficial for role play exercises, problembased
scenarios, student presentations and conferences:
‘Things like counselling courses where it may give them a way of trying out counselling
sessions in a completely safe, anonymous environment and [it is] useful for practice-type
sessions, simulations, but not for lectures/seminars’ (Greta)
UEL’s School of Health and Bioscience uses the environment for simulations and the interviewees
could see SL as very beneficial for practice sessions with virtual patients and, as above, hypothesised
that counselling trainees could interact through avatars before they were allocated to real world
placements. This type of ‘practice’ was also believed to be of value for student presentations:
‘I think that is a nice idea because I think the anonymity, again, is great and really, it
gives them a very safe environment’ (Greta)
Regarding traditional ‘chalk-and-talk’ lectures where large audiences may contain anti-social
elements, Cathy shared her experience of audience behaviour while attending large talks in SL:
‘I have watched a large set of lectures and things in SL before I got involved in this, I
didn’t see any sort of bad behaviour or anybody rioting or anything’ (Cathy)
Cathy also stated that for staff, a virtual environment would assist in some of the more difficult subject
matter:
‘Statistics is really difficult to teach in big lectures and that’s what I used to do and that’s
what people are used to doing and they talk to 250 people on the t-test and it was very,
very difficult then.’ (Cathy)
Small group discussions and seminars were also seen as possibilities within SL by Cathy:
‘In our conference centre, we’ve actually got a breakout room so that’s certainly possible
to do that, after a lecture, you could ask so many people to go into this, so many, and
you go off and feedback, that’s entirely possible.’ (Cathy)
316

Rose Heaney and Megan Anne Arroll
From the respondents’ experience of using SL for a tutorial, it appears that there is uncertainty if this
type of teaching is the most suited to the environment. Similarly, half of the respondents believed that
large-scale lectures would be effective in SL , although the lack of visual feedback from the students
appears problematic in terms of gauging understanding. Consensus was found for more practical
applications within SL , such as simulations and problem-based learning.
6. Discussion
This study has presented a qualitative analysis of the perceived barriers and advantages of using a
virtual world technology, SL, as a tool in higher education from the perspective of academic staff.
Previous research has shown that students report higher levels of satisfaction with teaching and
learning in online learning environments than staff (Jones and Jones, 2005; Palmer and Holt, 2009)
and that there may be numerous barriers to the effective use of SL in education (Warburton, 2008).
The findings of this study have shown that the sense of comfort derived from being anonymous can
act as both a benefit to students but also as an obstacle to teaching staff in the lack of non-verbal
cues though some of this could be resolved by fuller exploitation of the environment i.e. use of
gestures such as nodding etc. The participants’ narratives also illustrated the need for a clear
rationale for the use of SL expressing the belief that problem-based, practical sessions may be more
suited to this virtual environment than tutorials. There was also the assertion by some that SL might
be an effective medium for large, traditional ‘chalk-and-talk’-type lectures to mitigate anti-social
behaviour amongst other things, though this contradicted other views of not making use of SL
mandatory.
The need for investment, both in terms of time and funds, came through very strongly and accords
with Butler and White (2008) who note that the return on investment that educational establishments
gain from using SL is primarily dependent on making this type of technological innovation an
institutional priority. In the present study, participants highlighted the need for investment in staff and
student training, as without this front-loaded investment staff would not feel confident in using SL,
which in turn would impact on student engagement and, subsequently, their learning experience.
7. In conclusion
The participants in this study perceived both advantages and problems associated with using SL as a
teaching tool. Some of the limitations noted had more to do with staff’s inexperience than the
limitations of the environment per se. However, whether their criticisms of SL were valid or not the
overall sense was that SL is a beneficial addition to the learning environment provided staff feel
comfortable, confident and well-supported in the technology and have sound reasons for using it.
Participants could appreciate the more obvious applications for SL such as role play exercises or
virtual patient interactions but also saw a place for it in traditional teaching such as the online tutorials
featured in this study. In terms of moving forward with pedagogy, this study points towards the need
to embed technology such as SL into module programmes from the outset with specific training for
both staff and students. As to the lack of non-verbal cues frequently cited as a drawback in a SL
tutorial or other type of SL teaching situation, recent initiatives such as that at University of Southern
California to embed technologies such as Microsoft Kinect in SL herald much richer forms of feedback
as SL users have a more natural means of conveying their real emotions and gestures to their
avatars. Finally, this study suggests that when the positive aspects of the three super-ordinate themes
identified are present a worthwhile SL educational experience is achievable whatever form that might
take. In other words it would appear that the type of activity itself is less important than the conditions
under which it takes place.
Acknowledgements
This study was funded by a UEL Teaching Fellowship awarded to Rose Heaney.
References
Boellstorff, T. (2008) Coming of Age in Second Life: An Anthropologist Explores the Virtually Human, Princeton,
Princeton University Press.
Butler, D. and White, J. (2008) “A slice of Second Life: Academics, support staff and students navigating a
changing landscape” [online], Proceedings ascilite Melbourne,
http://www.ascilite.org.au/conferences/melbourne08/procs/butler-d.pdf.
Giorgi, A. (1995) “Phenomenological psychology”, in J.A. Smith, R. Hare, & L. Van Langenhove (Eds.),
Rethinking psychology, London, Sage.
Guba, E. and Lincoln, Y. (1981) Effective evaluation, San Francisco, Josey Bass.
317

Rose Heaney and Megan Anne Arroll
Jones, G. H. and Jones, B. H. (2005) “A comparison of teacher and student attitudes concerning use and
effectiveness of web-based course management software”, Educational Technology & Society, Vol 8, No 2,
pp 125-135 [online] http://www.ifets.info/journals/8_2/12.pdf
Palmer, S. and Holt, D. (2009) “Staff and student perceptions of an online learning environment: Difference and
development”, Australasian Journal of Educational Technology, Vol 24, No 3, pp 366-381.
Schroeder, R. (2008) “Defining virtual worlds and virtual environments”, Journal of Virtual Worlds Research, Vol
1, No 1, [online] http://journals.tdl.org/jvwr/article/view/294/248.
Smith, J. A. (1996) “Beyond the divide between cognition and discourse: using interpretative phenomenological
analysis in health psychology”, Psychology and Health, Vol 11, pp 261-271.
Smith, J. A. & Osborn, M. (2003) “Interpretative phenomenological analysis”, in J.A. Smith (Ed.), Qualitative
psychology: a practical guide to research methods, London, Sage.
University of Southern California (2011) “Using Kinect and OpenNI to Embody an Avatar in Second Life: Gesture
& Emotion Transference”. [online]
http://ict.usc.edu/projects/gesture_emotion_transference_using_microsoft_kinect_and_second_life_avatars/
Walker, V. L. (2009) “3D virtual learning in counselor education: Using Second Life in counselor skill
development”, Journal of Virtual Worlds Research, Vol 2, No 1, [online]
https://journals.tdl.org/jvwr/article/view/423/463.
Warburton, S. (2008) “Six barriers to innovation in learning and teaching in MUVEs”, Liquid Learning, [online]
http://warburton.typepad.com/liquidlearning/2008/07/six-barriers-to.html.
Warburton, S. (2009) Second Life in higher education: Assessing the potential for and the barriers to deploying
virtual worlds in learning and teaching, British Journal of Educational Technology, Vol 40, No 3, pp 414-426.
318

Introducing and Using Electronic Voting Systems in a
Large Scale Project With Undergraduate Students:
Reflecting on the Challenges and Successes
Amanda Jefferies
School of Computer Science, University of Hertfordshire, Hatfield, UK
a.l.jefferies@herts.ac.uk
Abstract: Electronic Voting Systems (EVS) have become a popular medium for encouraging student
engagement in class-based activities and for managing swift feedback in formative and summative assessments.
Since their early days of popularity and introduction some five or more years ago, the author’s UK based
University has been successful in refining strategies for their use across individual academic Schools and
Departments, as previously reported at ECEL (e.g. Lorimer and Hilliard, 2008). The focus of this paper is a
reflection on the introduction of EVS with 300 first year undergraduate students in the School of Computer
Science, within the context of a wider ‘change’ project in teaching and learning affecting the whole institution. The
author examines what lessons can be learnt following this rapid scaling up of EVS activity both at a local level
and more widely across an HE institution and in reflecting on the successes and challenges of this experience
provides key indicators for success and useful support for others considering using EVS. The paper first
considers the landscape of EVS use within the UK and then the specific introduction of EVS at her own
institution, before exploring the issues in her own academic School around the latest phase of their introduction
as part of an institution–wide project to review measures to support assessment and feedback.
Keywords: electronic voting systems, change management, assessment, feedback, staff training, challenges,
successes
1. Introduction to EVS at the University of Hertfordshire
The University of Hertfordshire has supported the use of electronic voting systems (EVS) for some
years since they originally started appearing as a piece of electronic technology to support teaching
and learning around 2003 in the UK. Sometimes known as ‘clickers’, the technology grew in popularity
in the U.S. where EVS had been used in a more routine manner as part of regular classroom activities
in HE as well as across the broader education sector. There are a number of competing EVS
systems available commercially but they all typically comprise four elements for working effectively;
an electronic hand-set per student or group of students, a digital receiver to capture responses from
the individual handsets, lecture presentation software into which questions can be embedded,
typically based around the use of Powerpoint and with the facility to collate and present the
responses in a variety of ways, and the associated hardware to present the questions i.e. a computer
and digital projector.
In our own early days of piloting them, sets of student handsets were available for lecturers to reserve
for classroom use from the central learning support services. This requirement to pre-order the EVS
was seen as rather unwieldy since time was required at the start and end of a class to first of all hand
out the handsets and then collect them in again afterwards. This was not an insignificant commitment
when there might be a class of over 100 students in an hour long lecture. Additionally the software
and receiver were also required to be pre-ordered for each class. The developing practice of EVS and
the design of questions for their in-class use was initially focussed on a small group of learning and
teaching technology ‘enthusiasts’ (Moore, 1982) working across the different schools of the
university. From this early interest in using the EVS for multiple choice questions (MCQ) and in-class
quizzes, their use was taken up in local pockets of interest by those who saw an opportunity to
increase interactivity in the classroom,(JISC 2005). This opportunity to engage more closely with
student learning and focus the in-class activities around MCQs was particularly developed at the
University of Hertfordshire by those teaching in the vocational health related subjects and with smaller
class sizes, (Lorimer and Hilliard 2007). At the same time the pedagogy and practice of the use of
EVS was being researched and reported from a variety of scenarios across UK based HE institutions
(Draper and Brown 2004;Kennedy and Cutts 2005).
Our institution’s initial success might therefore be described as having been built up through a
successful ‘cottage industry’, where local EVS champions crafted their expertise with groups of
students and developed a series of action research projects in the process. The resulting interest in
the pedagogy into the use of EVS and its potential for use in social constructivist approaches to
319

Amanda Jefferies
learning was further developed in line with research outputs both from the University of Hertfordshire
(Lorimer and Hilliard, 2007; Russell, 2008) and further afield, for example: Nichol (Nichol, 2007) at the
University of Strathclyde, and Masikunas (Masikunas, Panayiotidis et al. 2007). All these researchers
have indicated an increase in student engagement with the content of their learning when EVS were
used regularly. The EVS technology was refined over the past five years and their use was no longer
seen as only requiring a series of knowledge-based MCQs, where the expected answers are
designed to be purely testing a student’s knowledge in terms of Bloom’s taxonomy of learning (Bloom
1953).
In 2008 the University’s School of Life Sciences piloted the use of EVS with a complete cohort of
undergraduates who were given a personal handset and retained it for use across their
undergraduate careers until they graduated. As a result of this scheme there was a further increase in
the training of academics and a necessary development of study materials to promote interactivity
and engagement. This in turn led to an archive of local material and expertise which demonstrated
some of the wide variety of ways in which EVS were being used creatively in the classroom by
academics across the curriculum. The following examples indicate situations where EVS use became
commonplace within the wider Faculty; group discussion and decision-making; ethical debates with a
‘before’ and ‘after’ poll; conditional branching, where students follow through the logical
consequences of their initial answer to a question to its conclusion in a case study, (Lorimer and
Hilliard 2009) as well as revision quizzes to test previously taught material.
In response to this increased enthusiasm among academics and students in Life Sciences where the
use of EVS had grown over six years to become a regular feature of the student classroom, a crossuniversity
project was drawn up to introduce the use of EVS across a much broader group of the
university’s campus-based students from September 2010. The aim of this ambitious project was to
promote a culture change in the use of learning technologies by providing an impetus for the use of
inter alia EVS by an extended set of academic schools and to increase student engagement in line
with the University’s own stated agenda for improving the student experience.
The project was designed and grounded in line with much of the previous work of the University’s
Learning and Teaching Institute in the principles for good assessment practice and feedback (Nichol
and Macfarlane-Dick, 2006) and underpinned by the seven principles of ‘Good Teaching and
Learning’ (Chickering and Gamson, 1999).The use of EVS had already become widespread in other
institutions, (see above e.g. Draper and Brown 2004, Kennedy and Cutts 2005) but their take-up was
not always valued beyond the increased student engagement in classes nor did EVS necessarily lead
to an overall reduction in costs for the host School. Student satisfaction and their enjoyment of EVS
had been identified as a useful marker of success, (Lorimer and Hilliard, 2008); however, the broader
aims of this project wanted to delve into the nature of student engagement with EVS and see if their
use could lead to measurable savings overall. These savings were envisaged as being primarily time
savings through a reduction in the assessment burden for staff, reducing their marking load or
replacing an examination with regular EVS tests. For some programmes engaged in the wider project
there was an expectation of higher student retention rates (Nichol, 2007).
Following an invitation to individual Schools from across the university to bid for funding, the project
proceeded with the provision of personal EVS handsets to nearly 4,000 students across nine
academic departments for use throughout the second semester of the academic year for a minimum
period of at least 6 months. Many Schools intended at this stage that undergraduate students should
expect to retain their handsets throughout their university career. In practical terms this ambitious
project required that a number of the academics in participating Schools would have to both undergo
training in the use of EVS and to reflect personally on any necessary pedagogical changes to their
practice to be made, once students were provided with a personal EVS handset. The overall project
also required a necessary commitment to the scaling up of teaching and learning support, to solve the
practicalities of organising the handset distribution to individual students, as well as considering the
provision that needed to be made for the increased number of disabled students, who were now
participants in the project.
2. The local situation: Introducing EVS across a wider user group of
academics
The situation in the School of Computer Science with regard to the use of EVS essentially mirrored
practice across the wider university. There was a small group of enthusiasts who had been reviewing
320

Amanda Jefferies
their pedagogy and developing their own practice continually since the original opportunity for the
provision of handsets. During the previous year a small-scale ‘change management’ project, called
‘Piloting Innovative Practice’ (PIP) was developed in the School in line with Change Academy
principles (Anderson et al, 2008; Doolan, 2010; Saward and Anderson, 2010). One mini-PIP project
explored the potential of EVS in a non-traditional use for peer assessment and feedback for student
presentations, as a new way of managing student engagement in class. This PIP was extremely
successful in terms of engaging students and cutting staff time in administration of mark allocation
and collection. It was subsequently demonstrated to other academics in the School, with the intention
of gaining greater buy-in for the School’s participation in the wider EVS project planned for Autumn
2010. The scene was now set for developing the use of EVS more widely across the School in the
next academic year.
The University of Hertfordshire operates on a semester based system and throughout the first
semester 2010-2011 between October and December 2010, there was much planning required for
the introduction of EVS to a cohort in excess of 4,000 students. The School of Computer Science
planned to introduce EVS with first year undergraduate students on the BSc Computer Science and
Information Technology programmes, having previously worked with Masters students and final year
undergraduates. Each student registered on the programme would be given a handset, which it was
intended they would be able to use through the rest of their undergraduate programme. The
importance of the anonymity of the handset user has been widely discussed in the literature, (e.g.
Banks, 2006) but it was also important to know which handset belonged to which student so that they
could be used regularly in summative testing. The process therefore required each student to have
their handset registered to them and a central database was designed and linked to the managed
learning environment (MLE) to accommodate this instead of individual schools keeping their own
records. This also facilitated the movement of students between schools where their studies
overlapped from one to another, for example students who majored in Business might also be
studying Computer Science but should only receive a single handset from the university.
Academics teaching on the programme who were typically very experienced with teaching first year
undergraduates were offered training and additional support for re-designing their materials as
necessary, as part of their participation in the project. The central university project team offered
seminars and developed online ‘how to’ videos of, for example, downloading student lists and how to
register student handsets. The chosen provider of the EVS technology at the University of
Hertfordshire was Turningpoint; this was based on the prior experience of handsets and software
used by the School of Life Sciences and the Radiography Department and from the comparison of
user experiences recorded since the early days of using locally bought sets of EVS at the university.
The various reporting facilities of the software allowed student scores to be gathered (as they were
linked to their personal handset) and thus the use of EVS for summative testing as well as for
formative work.
2.1 A methodology for capturing the staff and student responses to the EVS
The author’s role within the School was to both lead the project and to provide support for colleagues
on a day-by-day basis as they incorporated more EVS-type activities into their teaching. The
methodology for assessing and evaluating the introduction of the EVS to the cohort has used both
qualitative and quantitative methods and some of the work in assessing the student experience is still
ongoing at the time of writing. In addition the author provides her own reflections on the process and
the outcomes of the project in the conclusion below.
A set of student-focussed questions with a Likert scale for answers was used by Computer Science to
enquire into the student experience. The questions were incorporated into a presentation and used by
staff at the end of a lecture to gather student opinions about using EVS (n= 50). Additionally, because
the case for EVS use with students has been discussed widely in the literature the author was
interested in the challenges that their introduction might bring to established teachers and so
individual face to face interviews were held with a cross-section of colleagues in the School, those
with prior experience of using EVS and those with none. Their comments are reported below.
3. Successes from the EVS project in the School of Computer Science
In this section the author considers four areas where the introduction of EVS could be judged a
success, in terms of student and staff engagement with both the process and the outcomes and the
321

Amanda Jefferies
way in which the outputs from the project have informed ongoing enquiry into assessment and
feedback approaches.
3.1 Student engagement
It has been widely reported by inter alia Lorimer and Hilliard, (2008, 2009), Nichol (2007) that
increasing interactivity in the classroom promotes a greater student engagement with their learning.
The outcome from the student side of the use of EVS has been positive. As technically-literate
students they rather predictably found the EVS ‘easy to learn’ and ‘easy to use’. They particularly
liked the immediacy of the feedback when EVS were used in class (80%) and would recommend their
use for students studying the same module the following year and in other modules. They also widely
agreed (74%) that ‘using EVS helped my understanding of the module’. They did not all agree (54%)
that the use of ‘Weekly EVS questions were the only reason for attending lectures’, but this could be
attributed to the variety of styles of use by academics across modules. A larger group responding to
the same questions in the Business School (n=200) had similar responses in terms of student use of
EVS. This can be deemed a success for the local use of EVS. although the results could have been
predicted from the many other national studies.
3.2 Staff engagement
Five undergraduate module teams committed to using EVS for some of their teaching from February
to April 2011. Prior experience of their use was limited but some academics embraced the new
technology with great enthusiasm, fired up by the examples and opportunities shown through the
training sessions.
“Seeing different examples of experienced colleagues’ use of the EVS inspired me to try
other ways to use them myself.” (Module tutor)
Other colleagues who had previously used EVS were keen to experiment with their use in large
groups and seminars particularly now when the handsets were permanently linked to individuals and
they did not need handing out at the start of a class and their ownership recorded for that
assessment.
“Being able to automatically link each handset with the student’s personal ID really
reduced the time taken for inputting and checking marks.” (Module tutor)
Particularly popular were the use of ‘surprise quizzes’, dropped in randomly to the teaching sessions
but with a small summative assessment element, to ensure regular attendance and high engagement.
These were used by one module team with great success as the gathering in of marks was immediate
and removed much of the burden of previous testing with MCQ style questions, marked by hand.
The ‘drop- quizzes’ proved to be a successful method for encouraging regular student
attendance and engagement throughout the module.” (Module leader)
Other teams explored the use of student discussion with EVS, before a group was required to give
their joint answer, as a means of encouraging the students to articulate their understanding and to
explain the solution to a problem to others.
3.3 Developing the User database
Much of the success of the EVS project and its embedding into the culture of the teaching of this
cohort of first year undergraduates could not have been achieved without the initial design of the user
database and its subsequent use for generating separate attendance lists of students registered for
each module. The process of registering each handset to its student owner once and for all removed
the time-consuming handing out and collection of handsets on a per class basis. Each handset has a
unique barcode identifier and when the handsets were issued a barcode reader was used to register
first the handset and then the student’s ID card number. This was recorded on the university’s MLE
and provided an entry which could be searched either by student ID or by handset ID. In the case of a
lost handset being retrieved, any lecturer in the university could access the database and arrange to
notify the student of the return of the handset. Furthermore if students had their handsets piled onto a
table during a discussion, it was easy to identify which handset belonged to which student. The
student registration lists were downloadable from the central MLE and customisable for each module.
322

3.4 Building on previous work
Amanda Jefferies
The success of the project at a local level was also due to the leadership of the project at an
institutional level. This had built on the success of the introduction of the university’s MLE (Jefferies
and Oliver, 2003) and its Blended Learning Unit (Bullen et al, 2009) which between 2005 and 2010
had encouraged broader use of technologies to enhance learning and teaching practice across the
university and to develop associated pedagogies, to enhance the student experience. Accordingly,
lessons learnt from prior use of EVS across the university were shared, with the more experienced
users from the School of Life Sciences and the Department of Radiography exploring in seminars and
workshops how they had developed their own ‘best practice’ guidelines in the use of EVS to build
constructive formative assessments which served to encourage interactivity in class and greater
student engagement with their learning materials. This enthusiasm was infectious and encouraged
colleagues to take up the challenge.
4. Challenges of the EVS project
While some of the immediate successes of the project have been outlined above, it is however worth
noting the challenges which were faced during the swift introduction of EVS across the nine
participating Schools and in Computer Science in particular. These have been categorised in terms of
the hardware and software issues, and staff time and ongoing training.
4.1 Hardware and software updating
Introducing new IT can traditionally be a challenging time for both project managers and IT users. The
challenges might include the human factors issues faced by the users who are asked to incorporate
new ways of working with IT and in this case to design new ways to present their teaching materials.
Alternatively, issues may arise as a consequence of the timing of the upgrading of the IT hardware
and software and the availability of technical support in a large organisation, where major tasks and
associated planning of their staffing might be scheduled on a three to six month or annual advance
planning basis. In relation to this project the technical challenges related firstly to the timescales of the
EVS introduction and to a previously unanticipated need to install new EVS receivers on some of the
classroom hardware, and secondly to software issues where there was an incompatibility issue with a
previous version of the software, already installed on certain classroom based PCs. An upgrade of the
software from the previous academic year when the Computer Science CABLE project took place
meant that new software had to be downloaded by the academics. This was not in itself a problem as
this was a simple web download onto staff machines, but the problem arose as a new digital receiver
was required and the earlier versions of the software which were already loaded onto some
classroom and staff machines were then found to be incompatible with this new receiver. The
complexity of the project’s roll out of EVS to academic staff was compounded with the nine Schools
being based across two separate but linked campuses, which had different provision for software
support.
The newer campus commissioned and opened originally in 2003 had an integrated software upgrade
system whereby selected software products could be upgraded centrally for each classroom on the
campus overnight. The older campus whose IT systems were historically supported at faculty level
did not at the time offer a centralised software upgrade facility and the necessary upgrading of the
Turningpoint software along with the installation of tamper proof hardware receivers had to be
undertaken on a per room basis. The scheduling of this significant task was soon realised to be too
great to be undertaken during the normal teaching timetable throughout the semester when rooms
were rarely out of use for more than an hour at a time. The temporary solution for the Computer
Science academics based on this campus was to use a dedicated laptop on which was installed the
recent software upgrade which was compatible with the new receivers purchased and to take this
along to their classes with the portable receiver. This then required rather more ‘kit’ to be taken to
classrooms instead of the preferred option of using a memory stick with their ready prepared
materials installed. This challenge will have been removed by September 2011 because a schedule of
hardware and software revision and upgrading has been put into place for the summer months when
the classrooms are less heavily scheduled, in preparation for the new academic year.
4.2 Staff time
Academic staff readily took part where possible in the training sessions offered prior to the start of the
semester. Some of those newer to the lecturing environment willingly and enthusiastically prepared
323

Amanda Jefferies
their teaching materials taking the opportunity to incorporate regular EVS sessions throughout the
semester.
For those with a particularly heavy teaching load which included the cohort of 300 students, through
February to April, there was much less time and inclination to change tried and tested ways of
delivering familiar material to incorporate the newer technology and introduce greater interactivity than
they were already. In some cases it was the smaller tutorial sessions where the use of EVS proved
most successful and where students shared the opportunity to discuss their answers collectively
before giving a group response.
This challenge will be approached by offering more local support for reconsidering the pedagogical
issues related to social constructivist approaches of using EVS and associated course re-design
during the quieter non-teaching periods of the summer. The module teams will thus be encouraged to
plan their regular use of EVS with greater elapsed time for planning than was available in the 2010 -
2011 academic year. Further training opportunities are being offered at both entry and advanced
levels to promote the maintenance of the necessary skills.
5. The way forward: Plans for 2011-2012
The old proverb states that ‘Rome was not built in a day’, neither can such a major change as
introducing EVS across 4,000 students with the resultant impact on pedagogical approaches be fully
achieved within a single academic semester. The lessons learnt in the School of Computer Science
from introducing 300 students and lecturing staff to using EVS include those issues which arose
internally and those which arose through external decision-making.
The enthusiasm of the core staff who have been using EVS regularly has been matched by the
student engagement and enthusiasm for using the handsets. Plans have now been made to offer
ongoing training so that this enthusiasm will encourage others to re-design and remodel their module
delivery. It has become clear that local support is vital for the ongoing success of using EVS
otherwise the use of this or any other newly introduced technology is likely to subside over time until
the users include only those keen and already committed users. Building on the lessons learnt since
the introduction of our university’s MLE from 2001 (Thornton et al,2004), it is therefore planned to
have a network of local EVS champions to support academic colleagues with technical issues and
through the pedagogical redesign of course delivery. Over the next year this will help to embed a
changing culture in the School which will encourage the increased use of EVS, after which each
succeeding cohort of undergraduate students will be offered their own free handset on entry to the
university. There has been a programme of continuing classroom support via our Equalities Unit for
those students whose disabilities mean they do not have the fast reaction motor skills for using EVS
in tests.
6. Reflections and conclusions
The university’s project to make EVS available to a far wider group of staff and students than
previously envisaged has already (June 2011) had some notable successes. Students have
expressed their enthusiasm as users of the EVS, where this has been measured across the
university. Students appreciate knowing their results and receiving feedback immediately through the
use of EVS, with the opportunity to learn from their mistakes and have the correct answers explained
instantly. This outcome has been in line with past research into student experiences of their
introduction and use. Further savings are foreseen at this institution in terms of redesigning
summative assessments for the forthcoming academic year so that EVS can be used to save more
academic time currently taken up with repetitive marking of scripts. At the time of writing not all
results are available but a significant increase in first year retention rates seems likely.
As far as the pedagogy of using EVS is concerned this has been shifting for some time away from a
culture where the traditional lecture dominates as a teaching medium and towards a more socially
constructivist view of gaining knowledge. This process can not happen overnight nor even in a single
academic year but the introduction of the EVS has offered further evidence of greater student
engagement in their learning and of a changing academic culture that embraces this.
324

Acknowledgements:
Amanda Jefferies
The author acknowledges the generous support of colleagues across the University of Hertfordshire
in promoting EVS and sharing their experiences.
References
Anderson, I., P. Bullen, et al. (2008). “CABLE: an approach to embedding blended learning in the curricula and
across the institution”. Reflecting Education 4: 30-41.
Bloom, B. S. (1956). Taxonomy of Educational Objectives, Handbook 1: The Cognitive Domain New York, David
McKay Co Inc.
Bullen,P., Russell, M., Jefferies, A. (2009) "Harnessing technology, transforming higher education for the Future"
Higher Education Academy Conference, University of Manchester,
Chickering, A. W. and Z. F. Gamson (1999). "Development and Adaptations of the Seven Principles for Good
Practice in Undergraduate Education." New Directions for Teaching and Learning 1999 Vol 80 pp 75-81.
Doolan, M. (2010). "Pedagogical and Innovative Practice: Using technologies to Support Assessment and
Assessment Feedback in Real-Time" Proceedings of 9th European Conference on E-Learning, Oporto,
Portugal, ACI.
Draper, S. W. and M. I. Brown (2004). "Increasing interactivity in lectures using an electronic voting system."
Journal of Computer Assisted Learning Vol 20 Issue 2 pp 81-94.
Jefferies, A., and A. Oliver, (2002) "Easing through the Pain Barrier – introducing a campus-wide MLE for staff
and students" 3 rd ILTHE conference, Heriot-Watt University, July 2002
JISC (2004). "Effective Practice with E-Learning " pubd HEFCE at
http://www.jisc.ac.uk/elearning_pedagogy.html
Kennedy, G. E. and Q. I. Cutts (2005). "The association between students' use of an electronic voting system
and their learning outcomes." Journal of Computer Assisted Learning Vol 21 Issue 4 pp 260-268.
Lorimer, J. and A. Hilliard (2007). "Net gen or not gen? Student and staff evaluations of the use of
Podcasts/AudioFiles and an Electronic Voting System (EVS) in a Blended Learning module". 6 th European
Conference on E-Learning, Copenhagen Business School, ACI.
Lorimer, J. and A. Hilliard (2008). "What is your response? It's time to get personal". 7 th European Conference on
E-Learning, Aghia Napa, Cyprus, ACI.
Lorimer, J. and A. Hilliard (2009). "Use of an Electronic Voting System (EVS) to facilitate Teaching and
Assessment of a Decision Making Skills in Undergraduate Radiography Education".8 th European
Conference for E-Learning, Bari, Italy, ACI
Masikunas, G., P. Panayiotidis, et al. (2007). "The use of electronic voting systems in lectures within business
and marketing: a case study of their impact on student learning." ALT-J Research in Learning Technology
15 Issue 1 pp 3-20.
Nichol, D. (2007). "Laying a Foundation for Lifelong Learning: Case Studies of E-assessment in Large First Year
Classes." British Journal of Educational Technology 38 Issue pp 668-678.
Nichol, D. l. and D. Macfarlane-Dick (2006). "Formative assessment and self-regulated learning: a model and
seven principles of good feedback practice." Studies in Higher Education 31 Issue 2 pp 199-218.
Russell, M.B.(2008) "Using an electronic voting system to enhance learning and teaching," Engineering
Education 3, Issue 2
Saward, G. and I. Anderson (2010). “Macario Rock, Paper, Scissors: Overcoming Barriers to Changing Staff
Practice in Technology Enhanced Learning.” 9 th European Conference on E-Learning, Oporto, Portugal,
ACI.
Thornton, M., Jefferies, A. et al. (2004). “How was it for you? An evaluation of Student Learning Experience
Following the Introduction of an MLE in one English University” BERA, UMIST, Manchester.
325

A Methodology for Incorporating Usability and
Accessibility Evaluations in Higher Education
Anne Jelfs and Chetz Colwell
Institute of Educational Technology, Open University, Milton Keynes, UK
a.e.jelfs@open.ac.uk
c.colwell@open.ac.uk
Abstract: Digital academic resources are rapidly growing in number and range of content, including visual as
well as text based materials. To access these resources in a timely and effective way necessitates an easy to
use and easy to learn environment. We consider in this paper the needs and requirements of disabled as well as
non-disabled students who use these resources for study-related activities. We are currently involved in
evidence-based research at the Open University on the uptake of usability, accessibility and learnability in the
resource development cycle. To gain feedback from students we use a number of qualitative research methods
primarily through interviews and observation studies and in our evaluations we unite disabled and non-disabled
student user experience for final reports to faculties and developers The aim is to develop best practice with
guidelines and information to support the institution in its development of web-based provision. At the Open
University we have nearly 12,000 students with a declared disability, so ensuring the accessibility of course
materials and websites is of primary concern as a large proportion of our provision is mediated online. However,
we also need to retain the usability of resources because of the large non-disabled student population (approx
180,000 post graduate and undergraduate students). Therefore we collaborate together to ensure that
differences in usability and accessibility feedback are acknowledged and potential conflicts are part of the
evidence we use when reporting to Module Teams and developers. What we are able to demonstrate in our
paper is the ability to include multiple forms of evaluation including ‘expert’ walk throughs and the use of different
assistive technologies, and end user evaluations. We report on further developments of the methodology
developed by Colwell and Jelfs (2005; 2007) which is thought to be unique in ensuring that potential conflicts
between usability and accessibility are minimised. In our paper we will present recent Open University
developments to illustrate our methodology.
Keywords: accessibility, usability, evaluation
1. Introduction
There has been increased use and development of information technology as a tool to disseminate
teaching and learning across the world. The Internet has become a part of many people’s daily lives,
which in turn has led to the development of educational materials for presentation on the ‘web’ and
increasingly courses and universities are integrating online materials in their face-to-face teaching
(Crowther, Keller & Waddoups 2004; Kelly, Phipps & Swift 2004; HEFCE 2010). For many students
and teaching staff their first experience of eLearning is through their institution’s virtual learning
environment (VLE) which provides online teaching materials, university communities, assessment and
other university specific activities. It is often seen that web-based or university based intranet
resources can be one way to increase student access to materials and potentially to widen access to
a new and increasingly diverse student cohort. More recent studies have shown that technology has
become an ubiquitous and integral part of students’ lives (Jefferies & Hyde, 2009). However, this
potentially changes the emphasis from paper-based course materials and student support to online
provision of these key features of a university.
In this paper we discuss our work at the Open University (OU) which is Europe’s largest educational
establishment, delivering mainly distance learning modules. The OU has provided distance education
since the early 1970s and has a strong commitment to eLearning. The OU currently has 180,000
active students, of which over 11,000 declare a disability (~ 5.5%). Approximately one-half of all
disabled students receive some form of support from the university to enable them to participate in
their studies, for example providing audio files instead of text or software support through the
Disabled Student Allowance. The OU has also made extensive investment in eLearning particularly
since the late1990s for all students and there is a current major development programme for our VLE.
The OU also has a commitment to widening access to higher education, to providing high-quality,
interactive educational materials that meet students’ needs and operating within the mission of
‘openness to all’. The OU is committed to making its online educational content and student services
accessible to disabled students and usable by all; a considerable challenge given the size of the OU
student population (Cooper, Colwell & Jelfs 2007). Although we are reporting here on our work in
326

Anne Jelfs and Chetz Colwell
distance education we realise that all universities face similar issues. That is providing good electronic
support to staff and students and therefore evaluating how students are working with module
materials and online provision is paramount.
2. Evaluating internet based educational resources
As a relatively new form of educational delivery there is no clear definition of how education resources
presented electronically can to be evaluated. To feed into the development of our framework we
looked at current practices and the accepted methods of data collection for evaluating electronic
materials. Our methods are based on valid evaluation practices, but our findings inform our
framework.
A number of recommendations relating to the evaluation of interactive systems have been established
by researchers (see Hix & Hartson, 1993; Nielsen 1994; Newman & Lamming, 1995; Joynes, 2000;
Sheard & Markham 2005; Light, 2006). For example, the notion that participants should be recruited
from the target population of users and that participants should be observed as they complete a set of
pre-planned representative tasks (van Schaik, 1999). In fact, there is an assumption that a
comprehensive evaluation of a web-based environment needs to consider both the technical and the
pedagogical aspects of the system (Sheard & Markham 2005).
A major challenge for designers and human computer interaction researchers is to develop software
tools able to engage novice learners and to support their learning even at a distance, because poorly
designed interface which makes students feel lost, confused, or frustrated will hinder effective
learning and information retention (Ardito, Costabile, DeMarsico et al 2006). Ihamäki & Vilpola (2004)
say that if the interface of a VLE is not usable, the user’s focus on the actual content is diminished,
because using the VLE requires a considerable amount of concentration. Good usability, on the other
hand, allows the user to focus on the content thus improving learning results and making websites
accessible has growing importance as the Intranet becomes the main delivery channel for eLearning
resources (Phipps & Kelly 2006).
Often educational evaluations focus on learning outcomes, however with the growth in online
provisions, our research focuses on making the system easy to use and to improve access to high
quality education. In the next section we consider the recognised forms of usability and accessibility
evaluations before moving to the methods we have adopted in our joint work.
One of the methods at the forefront of software evaluations is verbal protocol which is also know as
think aloud protocol This is where a participant is asked to speak about what they are doing and why.
The think aloud protocol requires the participant to complete a set of tasks using a prototype website
and to ‘think aloud’. To say everything they are thinking while they complete the tasks (Turnbow,
Kasianovitz et al 2005). The participant needs to maintain a commentary about what they are doing
and why they are taking a particular path. This allows the researcher to have some understanding of
the choices the participant has made.
It is also valuable if the participants are observed and video recorded so that the participants’ actions
can be used in further data analysis (Hughes & Parkes 2003). Usually there are two observers where
one observer notes everything the participant says and if and how the participant completes the task.
Another observer directs the participant and answers any questions asked. This type of user test
provides essential real-time feedback on potential problems in the design and organization of a
website. It is useful because it gives the researcher immediate information that may be forgotten by
the participant at the end of the evaluation session. An approach we adopt is to have chunks of tasks
so the students are only talking aloud for short periods within the longer session.
When conducting think aloud sessions Dumas (2001) considers there to be three levels of interaction.
Level 1 of verbalization expects the person conducting the research to sit behind and out of view of
the participants and not to speak to the participant unless the person does not follow the instructions.
If participants stop talking they are prompted to keep on talking. This is usually used in problem
solving tasks. Level 2 is the manipulation of non-verbal information such as geometric shapes and
Level 3 participants are asked to verbalise but also to explain why they are thinking or doing
something. It is at Level 3 we usually work, but have on occasions tried Level 1, but our work is
primarily not problem solving, but understanding how we can improve on the current provision.
327

Anne Jelfs and Chetz Colwell
To try and overcome any problems associated with talking aloud Wright and Monk (1991) told users
to think of themselves as ‘co-evaluators’ of the system. Wright and Monk’s ‘relaxed’ approach allows
the evaluator to ask questions of the participant, which maximizes the possibility of identifying and
correctly interpreting the problems experienced by the participant. Hertzum, Hansen & Andersen
(2009) found that relaxed think aloud makes participants search more comprehensively and in a
different way to the classic think aloud approach.
Another accepted form of evaluation of electronic provision is heuristic evaluation. Heuristic
evaluation is conducted by studying a user interface and then forming estimation about what is good
and bad about it. The evaluation is conducted according to a certain set of rules, such as those listed
in typical guidelines documents (Nielsen 1993). According to Nielsen heuristic evaluation and user
testing should be alternated, because these two usability practices have been shown to find fairly
distinct sets of usability problems. Some evaluators think that heuristic evaluation is valuable as a
testing method because it can easily be conducted with the assistance and equipment generally
available on a college or university campus (Miller-Cochran & Rodrigo 2006). This is important when
there is little or no budget put aside for user testing as many education providers do not allow for the
money needed for usability testing which according to Nielson & Norman (2000) is very short sighted.
We have found that heuristic evaluation is useful when we conduct testing in an ‘expert’ role where
we give initial feedback to developers before bringing in any students for our studies.
3. Usability and accessibility combined
In our work we work together to achieve an understanding of the needs for all groups of students and
to produce reports to the developers without conflicting feedback. As Kelly, Phipps and Swift (2004)
say it might appear desirable to simply include usability alongside accessibility but to do so there is a
need to be aware of potential conflicts due to usability preferences that conflict with accessibility
guidelines. Our design of the evaluation means that such conflicts are minimised and in most cases
we have no differing views.
Accessibility represents an important step toward independence for individuals with disabilities and it
also guarantees broader access for all users of the web (Foley 2003). His claim is that accessible web
design creates pages that are often easier to read, easier to navigate and faster to download. It is
acknowledged by some researchers that little or no consideration is given to the real participants in
the learning experience, such as disabled learners (DeMarsico, Kimani, Mirabella et al 2006). These
researchers see disabled students as a rich source of guidance on how to address accessibility
issues in the development and usage of e-learning material and systems. Whereas they see usability
as a separate, but related concept, and that usability and accessibility experts should work side by
side. However, DeMarsico et al in their publication are not conducting this type of testing.
There is a lot of consideration of the value of working together for usability and accessibility mainly
because it is seen as ‘Sites that are developed to meet accessibility standards are generally, but not
always more usable in their practical application. This may be attributable to the similarity between the
concepts shared by both accessibility and usability, such as the inclusion of appropriately contrasting
colour schemes and the exclusion of non-browser compliant elements such as JavaScript on web
pages’ (Yates 2005 p182). He goes on to say that websites that conform to accessibility and usability
standards ‘will support the creation of web environments which perform their function more effectively
and provide an enhanced experience for the use: thus accessibility and usability standards are central
to good design methodologies.’ (Yates 2005 p182). A different view is that of Byerley and Chambers
(2002) who found that often accessibility does not equate to usability where adherence to the law can
make the website less usable for others.
Although others have worked with disabled and non-disabled users to assess the accessibility of
websites these have been relatively small numbers (Harrison & Petrie 2006). Harrison and Petrie do
however say that ‘Many guidelines focus specifically on either accessibility or usability rather than
evaluating both simultaneously, making them less efficient.’ (p 241). This is something we endorse
and is why we have been conducting these types of evaluations over a number of years and
numerous websites, both educational and commercial and feel in a position to discuss our findings.
Harrison and Petrie go on to say that ‘The fact that participants with disabilities find more problems
reinforces the concept that they make the best test participants.’ (Harrison & Petrie, 2006 p 245).
Although we acknowledge that disabled participants do make good participants, where we differ is
328

Anne Jelfs and Chetz Colwell
that when working together with all types of participants we realise there is a different level of
engagement by them.
4. Putting the study into practice - the framework
Our way of working together is informed in part by the needs of the students and the conformance to
standards such as WAI (2010); BS 8878 (2010); and QAA (2010). A substantial part of this work is
conducted through our ‘expert’ walkthroughs and heuristic evaluations. The third part of our
framework is the user input to the new developing websites. Within the framework we work together,
but separately at the same points within the evaluation:
1. At the wireframe stage we will work independently reviewing the proposed sites and then come
together to report back to the developers. At this stage the Accessibility standards will also be
considered. We will review the wireframes from different student viewpoints.
2. At the ‘click through’ stage we will again work independently and report back individually to our
developer colleagues.
3. At the next stage which may not be a fully functioning site we will bring in students for our
observational studies. This will usually be 5 disabled students and 5 non-disabled students.
It could be argued that observational studies, particularly under laboratory conditions, are not the
same conditions as those the student works in at home. However, to gauge some understanding of
student use and relevant feedback on the web resources, we have found these types of studies
invaluable in our resource development. We acknowledge their limitations, but value the insight they
provide. In this section we discuss how we have attempted to alleviate as many of the problems as
possible.
We have found that student recruitment can often be problematic, especially when trying to identify
people with specific accessibility needs for our studies. Usually students are recruited via notices on
module or university based websites or through existing contacts. In conducting the studies we also
have to make additional allowances, such as how far the disabled toilet facilities are and specific
specialist requirements such as software e.g. JAWS and Dragon Naturally Speaking. There is also a
need to recruit interpreters for deaf students who do not lip read and for one evaluator to be in the
same room facing the student for those students who do lip-read in order to communicate.
The way we work is that students are invited to attend one of a number of sessions for an
observational study which is carried out in the Jennie Lee Research Labs in the Institute of
Educational Technology (IET) at the Open University. This facility has one specifically allocated
Accessibility Lab which contains additional support materials such as an adjustable desk, various
keyboards, and devices such as trackballs. There are also two other Usability Labs, a Games
Research lab, an observation room and reception room. The cameras capture human–computer
interaction, through simultaneous ‘views’ of: the user, the computer they are working at and
interactions with the application on the screen. The resulting video enables the evaluator to
concentrate on the interaction as it occurs and to have a recorded observation for future analysis. A
number of scenarios are set up for the students to complete during the observational study to
simulate some of the activities that students would be completing in their interaction with the virtual
learning environment, such as wikis, forums and assessment. The aim is to work for about 45 minutes
before a break and all participants are free to leave at any point in the evaluation.
At the start of the evaluation session, participants are informed of the purpose of the evaluation, about
the cameras and what would happen during the session. It is explained to participants that they will be
asked to think aloud when they use the website during the task completion. We also ask students to
sign a consent form allowing us to record their activities and permission to use their findings in
publications. We usually make a small payment plus travel expenses in acknowledgment of their
participation. The use of the laboratory facilities can be daunting to participants at first so we try and
create a relaxed environment and offer tea, coffee and biscuits at the start of the sessions, during any
breaks and if necessary we also offer lunch.
329

Anne Jelfs and Chetz Colwell
We work together taking the role of observer and facilitator alternatively depending on whether the
person is disabled when Chetz takes the role of facilitator, or non-disabled where Anne is the
facilitator. This method almost replicates the approach suggested by Craven and Booth (2005)
‘Ideally the session should include a facilitator and an observer.The facilitator sits next to
the participant and directs the testing while the observer watches the process remotely
either through a one-way mirror, or by using a video to record the session which the
observer watches in another room. This method enables the observer to concentrate fully
on what the participant is doing and saying, rather than on the actual running of the
experiment.’ (pp 187-188).
While participants complete the tasks, they are observed and prompted as appropriate. We use
probing questions to uncover reasons for particular actions and perceptions of what the participant
expects to find. Tamler (2001) suggests that a good example of a question is ‘when you were
selecting that file what were you expecting’ rather than ‘why did you select that file’ which makes the
user feel that they need to justify their behaviour. We also give some feedback as the student
progresses because they are working as co-evaluators and not purely in isolation so we might say
something like ‘this is just the sort of thing we need to hear or your information will be very useful to
the developers.
Notes are taken by the observer concerning critical incidents, participants’ comments and relevant
actions. Participants are not given a time limit to complete each task. Finally we use a pre-prepared
interview schedule at the end of the session to find out about their opinions and preferences
concerning the websites and to allow them to give any other views they might have. The interview can
also be seen as a de-brief where we might explain further about the work we are conducting.
4. Analysis and report
We do task based analysis – how users tackled the tasks and where the major difficulties arose. The
observer’s notes are invaluable in guiding the analysis as one hour of video tape has been reported to
take at least 5 hours of analysis, or even a day or more (Preece et al 1994, p620). From both the
observer’s notes and the videos we are then able to write a joint report for the developers.
5. Conclusion
This method of working together has proved invaluable to the developers as it give them a report
where they can apply our recommendations knowing that we have discussed our findings and we are
recommending the optimum solution for all users.
If points remain which are inaccessible then these are highlighted, so that statements can be made on
the module website. It is imperative that disabled students are made aware of any potential areas that
are inaccessible for particular types of student or supporting software.
References
Ardito, C., Costabile, M. De Marsico, R. Lanzilotti, S.,Levialdi, T., Roselli, V. Rossano (2006) An Approach to
Usability Evaluation of e-Learning Applications.
http://www.di.uniba.it/~ivu/papers/2006/UAIS2006_Arditoetal.pdf (accessed 09.06.11)
BS 8878: Web accessibility. Code of Practice
http://shop.bsigroup.com/en/ProductDetail/?pid=000000000030180388 (accessed 09.06.11)
Byerley, S. and Chambers, M.B. (2002) Accessibility and usability of web-based library databases for non-visual
users. Library Hi Tech vol 20: 2, pp169-178
Colwell, C., Jelfs, A. & Mallett, E. (2005) Initial requirements of deaf students for video: lessons learned from an
evaluation of a digital video application. Learning, Media and Technology, Vol. 30, No. 2, July, 201–217
Cooper, M., Colwell, C. and Jelfs, A. (2007) ‘Embedding accessibility and usability: considerations for e-learning
research and development projects', ALT-J, Vol 15, No. 3, 231 – 245
Crowther, M.S., Keller, C.C. and Waddoups G.L. (2004) Improving the quality and effectiveness of computermediated
instruction through usability evaluations. British Journal of Educational Technology Vol 35: 3 pp
289–303
Hertzum, M. Hansen, K.D. and Andersen, H. (2009) Scrutinising usability evaluation: does thinking aloud affect
behaviour and mental workload? Behaviour & Information Technology 28:2 pp 165-181
HEFCE (2010) Student perspectives on technology – demand, perceptions and training needs. Report to HEFCE
by NUS
Hix, D. and Hartson, H.R. (1993) Developing user interfaces: Ensuring usability through product and process
Wiley & sons, New York
330

Anne Jelfs and Chetz Colwell
Hughes & Parkes (2003) Trends in the use of verbal protocol analysis in software engineering research
Behaviour and Information Technology. Vol 22: 2 pp127-140
Ihamäki, H and Vilpola, I. (2004) Usability of a Virtual Learning Environment Concerning Safety at work.
http://www.ejel.org (accessed 09.06.11)
Jefferies, A. and Hyde, R. (2009) Listening to the Learners’ Voices in HE: how do students reflect on their use of
technology for learning? Electronic Journal of e-learning, Vol 7, 2 pp119-126
Joynes, G. (2000) An evaluation model for supporting higher education lecturers in the integration of new
learning technologies, Educational Technology and Society, 3(4) 56–65.
Kelly, B., Phipps, L. and Swift, E. (2004) Developing A Holistic Approach For E-Learning Accessibility, Canadian
Journal of Learning and Technology, Vol. 30, Issue 3.
Light, A. (2006) Adding method to meaning: a technique for exploring peoples’ experience with technology.
Behaviour and Information Technology 25:2 pp175-187
Newman, W. and Lamming, M. (1995) Interactive System Design Addison-Wesley Educational Publishers inc.
Nielsen, J. (1994) Heuristic evaluation, in: J. Nielsen & R. L. Mack (eds) Usability inspection methods (New York,
Wiley), pp 25–62.
Preece, J. (1994) Human Computer Interaction Addison-Wesley Educational Publishers inc.
Quality Assurance Agency for Higher Education (2010) Code of Practice – Section 3 Disabilities
http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section3/Section3Disabilities2010.pdf
(accessed 18.03.11)
Sheard, J. & Markham, S. (2005) Web-based learning environments: developing a framework for evaluation,
Assessment & Evaluation in Higher Education 30(4), 353–368
Tamler, H. M. (2001). High-tech vs. high-touch: Some limits of automation in diagnostic usability testing. In HT
Consulting, http://www.htamler.com/papers/techtouch/ (accessed 09.06.11)
Turnbow, D., Kasianovitz, K. and Snyder, L. (2005) Usability testing for web redesign: a UCLA case study. OCLC
Systems & Services, Vol. 21 Iss: 3, pp.226 - 234
Van Schaik, P. (1999) Involving users in the specification of functionality using scenarios and model-based
evaluations. Behaviour and Information Technology Vol 18, 6 pp 455-466
Web Accessibility Initiative (2010) www.w3.org/WAI (accessed 18.03.11)
Wright, P. C. & Monk, A. F. (1991) A cost effective evaluation method for use by designers, International Journal
of Human-Computer Studies, 35, 891–912.
Yates, R. (2005) ‘Web site accessibility and usability: towards more functional sites for all’ Campus-Wide
Information Systems Vol 22, No. 4 pp180-188
331

The Virtual Learning Environment - Directions for
Development in Secondary Education
John Jessel
Goldsmiths, University of London, UK
j.jessel@gold.ac.uk
Abstract: While the use virtual learning environments (VLEs) in universities and colleges is now relatively
established, this is not the case in many schools today. This paper examines the particular demands made of
VLEs in secondary education in the UK and how effective and innovative use inside and outside the classroom
could be promoted. Work currently being carried out with teachers in five schools as part of a two-year EUfunded
project [1] is reported. Surveys to establish the perceived needs of teachers and senior managers were
followed up by curriculum-led workshop sessions totalling six days with up to four teachers in each school
working with another teacher who is highly conversant with VLE technology. Qualitative data gathered through
field notes, observations, focus groups, recorded interviews and on-site documents were analysed in terms of the
uptake and use of the technology and the quality of learning activities developed. It was found that innovation in
the school setting is a potentially complex process; schools have a variety of responsibilities and demands that
must be concurrently accommodated. With regard to the quality of learning activity and student engagement, a
theoretical framework is introduced within which technology-use can be mapped. In relation to this and the use of
VLE technology, the scope for learning as participation was explored. Principles arising from monitoring the
sessions included teacher ownership of development; embedding the use of the technology to existing learning
objectives or teaching and learning resources; levels of personalisation; external comprehensibility in view of the
availability of a VLE to a variety of users. It is also argued that development of the use of the VLE in the school
context could lead to both curriculum enhancement and curriculum transformation. [1] The work carried out in the
UK along with partner institutions in Spain and Italy forms part of a two-year project ‘Teaching to Teach with
Technology’ funded by the EU Leonardo Da Vinci Life Long Learning Program.
Keywords: VLE, learning perspectives, professional development, secondary education
1. Introduction
The need for the effective integration of technology in the UK school curriculum has been a recurrent
concern of different governments over recent years. In a strategy paper published in 2005 the belief
was stated that future development of ICT in education could ‘transform teaching and learning and
help improve outcomes for children and young people, through shared ideas, more exciting lessons
and online help for professionals’ (DfES, 2005: 4). Although there has since been a change of
government and the current Schools White Paper (DfE, 2010) does not directly specify a role for
technology, the pervasiveness and availability of technology made available through initiatives
following the 2005 Strategy raises key questions about the part technology can continue to play.
As part of an expressed need for a common digital infrastructure to support transformation and
reform, the 2005 Strategy encouraged the use of virtual learning environments (VLEs). The
importance of VLEs arises from their connectivity; as computer-based systems they offer facilities
such as e-mail, bulletin boards and newsgroups together with ways of storing and presenting course
materials. In many respects, and in contrast to the internet of the last decade, they allow exchanges
that are many to many or ‘interactive’ rather the than one-to-many characteristic of a ‘broadcast’
(Selwyn, 2008). VLEs are typically closed to an institution and in that sense regarded as relatively
secure and different rights can be given to users such as pupils, teachers and more remote potential
users such as parents or carers. Well-known systems in the UK university sector are Blackboard and
Moodle while systems such as Frog, Fronter and SIMS Learning Gateway are available in many
schools. Although there are differences between each system most have the facilities noted above
and for the purposes of this article VLEs will be treated generically.
Much current use of VLEs in schools is administrative, with systems for recording and accessing data
on student attainment. VLEs typically allow hand-in and storage of assignments for marking and
feedback. Teachers can devise online tasks that are often marked automatically. While these are
important aspects of use, those more central to learning are relatively unexplored.
Despite major investment, the use of VLEs in schools is relatively patchy; they may work effectively in
one setting but this does not guarantee they will work well in another. While the use of VLEs in
universities and colleges is relatively established (UCISA, 2008; 2010), this is not the case in many
332

John Jessel
schools today (Ofsted, 2009). The current work explores the issues at a time when VLEs are
expected to play a major role in learning and when ways of realising the potential for learning through
use of VLEs in schools are relatively unexplored. The focus in this paper is on the development of
resources and methods at a pre-trial stage by teachers in five London secondary schools. It considers
some possibilities VLE technologies can offer and reports on ideas developed by teachers and some
issues involved in the take-up of innovation.
2. Perspectives on learning
Regardless of innovation, a central concern within education is the potential technology offers for
learning. The relationship between technology and theoretical perspectives on learning is one that is
continually under review, particularly as new technologies emerge. Over the last five decades ideas
linking learning to technology have ranged from an associationist perspective where learning is
viewed in terms of observable behaviours that could be shaped by progressive reinforcement through
a tightly defined sequence of objectives that could be presented to the learner. A learning programme
could be administered with the aid of a teaching machine where elements of information could be
presented, questions asked and immediate feedback to responses given. From this perspective the
brain was a ‘black box’; other than the effects of reinforcement what went on inside was of no interest.
In contrast to viewing the mind as a black box and focusing on the programming of observable
behaviours, a cognitive perspective is concerned with inner mental activities. The focus is on
processes such as thinking and reasoning and representation in memory. According to constructivist
theory learning is an active process. We form mental representations that are based upon our
experience of the world around us and form the basis of our knowledge or understandings.
Constructivist pedagogy is based on creating an environment in which learners can interact and
providing activities that encourage experimentation and discovery (Jonassen, 1994). The use of
technology in accordance with a constructivist framework would include the provision an environment
where learning can take place through experimentation and interaction, as might be achieved with the
use of computer simulations or the control of a screen ‘turtle’ using a language such as Logo.
More recently in educational theory there has been a shift of focus towards work that highlights
learning as a social activity rather than a cognitive and individual one (Crook, 2001). An important
contribution is Vygotsky’s (1978) emphasis on the social context of learning. In a ‘social constructivist’
approach learners can be encouraged by those more experienced in an area to reflect on and
articulate the principles involved in activities in which they are engaged. The role of language is
regarded as important and through such ‘scaffolding’ the idea is that learners would be able to
achieve understandings that they would not have done if unsupported in this way.
Although there are different views on what counts as social constructivism, a general feature is that
meaningful knowledge, as opposed to something that already exists and waiting to be passed on, is
socially and culturally constructed: meanings are created through human interaction and with the
environment (Kukla, 2000). However, knowledge can also be seen to emerge as it is ‘constructed and
reconstructed between participants in specific situated activities, using the cultural artefacts at their
disposal, as they work towards the collaborative achievement of a goal’ (Wells, 1999: 140). This has
been made explicit through Bakhtin’s (1981) notion of dialogism where interaction is viewed on a
more symmetrical basis.
The place of dialogue within learning is also consistent with the concept of a community of practice
(Lave and Wenger, 1991; Wenger, 1998). A community of practice occurs when people jointly engage
socially in a common enterprise or activity, and through this learn together. Rather than being an
individual pursuit or academic exercise largely confined to the classroom, learning is ‘situated’ within
the framework provided by the community of practice through which participation can take place.
Here, participation is central to learning; this may be relatively peripheral at first but through
observation and practice learners take on the identity of the community and become more central
(Wenger 1998). This more ‘situated’ form of learning can, of course, occur without the aid of digital
technology. However, with the emergence of VLEs or Web 2.0 and there is further scope for social
participation.
333

John Jessel
2.1 An outline framework for learning and technology-use
A framework that summarises the main learning perspectives is shown in Figure 1. It was designed to
provide a convenient reference and basis for conceptualising, articulating and developing a discourse
amongst those interested in promoting effective learning using VLE technologies.
Figure 1: A framework for mapping technology-use according to learning perspective
Rather than regarding different technologies as having intrinsic properties in relation to the kind of
learning they may support the framework places emphasis on how a technology is used. Digital
technologies can be mapped into the space delineated by the two axes, but only according to how
they are used. In other words, according to use, a given technology could take up one of many
different positions, or spread over a more inclusive area.
Learning paradigms arranged along the horizontal axis begin with those aligned with an associative
perspective. The remainder of the axis, however, is representative of a cognitive or constructivist
perspective and acknowledges an increasing level of initiative and creativity on behalf of the learner.
Although the kinds of learning are reminiscent to those identified by Bloom (1956) the perspective is
primarily cognitive. The social dimension is marked out on a vertical axis. In its most basic form, social
involvement in learning is characterised by a relatively didactic transmission of information as shown
at the lowest point of the axis. Moving up the axis is the kind of interaction characteristic of scaffolding
within a social constructivist approach where a teacher or more experienced peer engages within an
activity so that learners are helped to develop their own understandings. Further up the axis social
engagement becomes more dialogic and participatory in nature. Participatory approaches can entail
an authentic quality (Brown et al., 1989). This has not been represented on the axes shown and could
be represented on a third orthogonal axis.
2.2 Encouraging participation through digital technologies
Although digital technologies could support learning in the ways outlined it does not automatically
follow that this will occur. While many adult learners may be self-motivated and work independently,
this is not always the case with younger learners and to some extent might account for the noted
334

John Jessel
differences in VLE use in the school and higher and further education sectors. An important factor
could be the role of the learner. For example, to what extent do new technologies allow learners to be
self-directed, to be a resource for their own learning, to learn from peer feedback and apply their
learning to real-world problems? These andragogic principles (Knowles, 1977) have hitherto been
primarily associated with adult learners but could be reviewed in the context of digital technologies. A
further issue, then, is what strategies could lead to the effective use of technology in learning by
younger learners and how can these be established in the school environment?
3. Method
Participants were recruited through professional contact in schools that work in initial teacher training
partnership with the university department coordinating the UK part of the Project. Altogether five
London inner-city secondary schools with pupils from a range of backgrounds became involved.
Three core areas of the UK National Curriculum, namely mathematics, science and English were
focused on. In addition, Personal, Social and Health Education (PSHE), now a compulsory curriculum
component, was explored by one of the schools.
The teachers who participated were subject specialists in English, mathematics and science and were
either in their early or mid career. Senior management staff from the participating schools were also
interviewed. The workshops were run on site with up to four teachers in each school working with one
of the two teachers from other educational institutions who were highly conversant with VLE
technology.
A number of principles underlay the workshop design. Firstly, work was curriculum led: any innovative
methods would sub-serve existing curriculum objectives, the intention being to include the use of the
technology as part of mainstream teaching activity. For similar reasons this remit included firm links
with assessment. The decision to hold workshops in situ in terms of subject department was to
facilitate the development of the use of technology in a way that is less likely to be isolated from the
curriculum or other learning resources. Another important element of the workshop provision was to
encourage use of the VLE that promoted learning through social participation as well as cognitive
interaction. In addition to supporting any needed discrete technology skills, the idea was to allow the
teachers to draw on their own expertise as subject specialists and use the technology as creators in
the development of curriculum-specific methods and resources.
The theoretical perspectives outlined above were introduced briefly with the aid of the diagram similar
to that shown in Figure 1. This marked out the broader learning possibilities and understandings of
the development of technology systems and associated pedagogy. Identification of learning goals and
assessment opportunities were also included in the sessions as were considerations of the role of the
student, the teacher and the technology. Field trials prototyping resources and methods with a subset
of teachers and students were planned in conjunction with the workshops and intra- and interinstitutional
dissemination and methods of sustainability discussed.
Qualitative data gathered through field notes, observations, focus groups, audio recordings of
workshop sessions and recorded interviews and on-site documents were analysed in terms of the
uptake and sustained use of the technology and the quality of learning activities developed.
4. Findings
4.1 Innovation in the school setting
With regard to recruitment of participating schools and teachers, it was found that introducing
innovation in the school setting is a potentially complex process; schools have a variety of
responsibilities and demands that must be concurrently accommodated. Points of entry for innovative
practice that were voiced by senior personnel and teachers ranged from those driven by national or
local policy, by impending crises, pressing social or pastoral needs and changes of staff. In other
words a programme of development does not occur in isolation and often needs to be timed in
relation to other demands, circumstances or initiatives; it needs to be negotiated and flexible. Even
with assured financial resources needed for cover, flexibility was needed in arranging sessions in view
of unforeseen demands that were not unusual in the schools concerned.
An initial small-scale survey was conducted to get an indication in-class and out-of-class use of
different ICT facilities and the perceived needs of teachers. With regard to the VLE, from the 20
335

John Jessel
teachers who agreed to participate, reported out-of-class rates of usage were proportionately higher
(10 using the VLE at least once or twice a week) than reported in-class rates (only 4 using the VLE at
least once or twice a week), interview data suggesting that out-of-class use was largely for
administrative purposes. Although there was a lower rate of in-class usage (16 using the VLE less
than once or twice a month which included 10 never having used it) this did not appear to be because
of lack of interest; responses to a further questionnaire revealed that the technology was regarded as
innovative and that all of the teachers surveyed were interested in finding out more about its use.
4.2 Workshop sessions
Applications of VLE technologies in teaching mathematics, science, English and PSHE were
explored. With mathematics, opportunities for students to develop problem-solving skills were
addressed through the online dimension taking account of a need for inner city students to develop
their extended writing skills. The idea was to use the technology to create project-based lessons that
fitted within both the GCSE and KS5 Curriculum. Work with science teachers focused on developing
resources and a structure for the VLE with a focus on BTEC Science. In relation to the PSHE, the
focus was on a core of underachieving girls and trying to use the technology available to engage both
pupils and parents to increase engagement and communication. The aim was to use the VLE to
communicate with these pupils in a less formal way in an attempt to remove some of the barriers to
their learning. It will also be used to engage with and help parents, many of whom do not have good
memories from their own educational experience. The idea was to set up a chat room for engagement
between students. Older Y10/Y11 girls would be encouraged to take a mentoring role for younger
girls. The possibility was that a forum would be a natural place to start for asking for help. Finding
ways of getting girls to support each other was regarded as crucial. However virtual peer mentoring
was not seen as the sole means of communication; ‘real-world physical mentoring would tie up with
virtual mentoring’. It was thought that a sense of community could be developed through building links
between the year groups.
Two schools were involved in developing the use of the VLE in the English curriculum. The
technology also included handheld audio and video devices within the Level 1 Functional English
Qualification, Reading, Speaking and Listening, with Year 8 (12-13 year old) students. The work,
done in relation to an existing scheme of work for a BBC School Report initiative giving students a
chance to make news reports for a real audience, will now be described in more detail.
The aim was for pupils to use radio or television for their reports. In addition to developing an
understanding of the media they were to use, attention was given to what constituted a news story,
how to go about finding a story and how stories could be put together to form a final sequence. This
would involve editorial meetings and use of the VLE for an online discussion forum.
As with the other schools, the technology-use framework was introduced at the start of the workshop
sessions. This was found to be largely self-explanatory and it was apparent where the different modes
of use of the VLE could be mapped or located within the social and cognitive space. The framework
also had a discourse-setting effect and allowed discussion to move beyond the more prevalent use of
the VLE as a dissemination instrument for tests and factual material. In particular, with regard to the
quality of learning activity and student engagement, the framework encouraged account to be taken of
types of learning paradigm and the social and collaborative dimension afforded by the connectivity
available through the VLE. However, the move towards the more participatory approach to learning
did not occur immediately.
In common with the majority of London schools, the VLE used was Fronter. In response to being
asked about his initial understanding of Fronter one teacher replied ‘I think it is a virtual learning
environment. It’s modelled like a virtual school where there are different rooms and corridors and
pupils can go into rooms and teachers will be in certain rooms. Just like a classroom, there are
resources in there.’ At this stage the potential for learning was in line the more prevalent uses of VLEs
mentioned earlier: ‘I think Fronter is a really good place to store resources so any child who’s got
access to a computer at home can carry on with their work’ ‘Children are allowed to upload work that
they’ve done so they can hand in work for the teacher to mark.’ ‘You can set work, upload resources
and set tasks in Fronter.’
In regard to the potential of the VLE for exploring participation in learning in the development of a
news story, the question was asked ‘Why bother with Fronter?’ Although the scope for participation
336

John Jessel
was not immediately apparent, with continued discussion and working through ideas the
communicative and participative potential began to emerge: ‘Just like a real classroom students and
teachers interact in that room like they would in a real one’. On further discussion particular qualities
of the virtual room were explored: ‘If you build into a scheme of work the idea that sometimes the
teams are discussing live verbally, sometimes they are discussing through an internet forum and
there is some kind of opportunity for them to evaluate and compare the differences in the way they
use language within those two different media’. This led to a teaching strategy that could be explored
with pupils in classroom equipped with computer terminals: ‘It’s a simple rule that’s established; when
you discuss online you don’t speak to each other.’
Later on the discussion had moved on to address aspects of pupils collaboration: ‘This is where the
Fronter thing could come in as a sharing forum … so you could have two areas with Fronter. You
could have, like, a forum discussion room where you generally discuss within your team, ideas, and
you could have like a public forum where you could put the whole class to share. So you could have a
shared forum and a team forum, and they’d have to also make decisions about what is appropriate for
each forum and why some discussions would stay within the team and some things go beyond the
team and can be shared with others, and what would be really nice is if they had to make those
decisions themselves about which forum would be appropriate. This is where I begin to see where
Fronter is a really useful way to do it.’
Through having the opportunity to discuss and explore with minimal prompting from the technologyuse
framework the vision had moved from ‘Why are we bothering to use Fronter?’ to ‘being able to
develop a really interactive use of Fronter’.
4.3 General principles concerning the use of VLE technologies
Analysis of the observational data revealed a number of general principles that emerged across all of
five schools that appeared to play a key role in the development of resources and methods of working
with the VLE and associated technologies. These are now summarised.
Embedding - This led on from the principle that the work should be curriculum led. Through
negotiation, existing areas of the curriculum within which to new practices involving technology could
be identified. This ‘linking-in’ to existing learning objectives also extended to existing teaching
materials or learning resources used within the departments. Embedding was not only in terms of
content but also in terms of structure and style of presentation.
Student engagement - This was expressed in terms such as ‘kids have to take a position and have to
argue’. This emerged in relation to the news editing forums mentioned above. It was also recognised
that this is seldom a straightforward process and considerable discussion centred around topics such
as encouraging pupils to analyse video material rather than simply access it.
Enrichment - It was recognised that easy access to a range of resources in different modalities
through the VLE allowed reading around the topic. Alerts to resources elsewhere such as TV
documentaries to enrich were also built in. Blogs were regarded as enriching and encouraged reading
around a topic.
Time and location specific - The VLEs allowed small items of information such as blogs or newsfeeds
to be connected to each page and could be changed when the page was revisited. The need was
expressed to keep these items up to date and to make them time location specific (e.g., ‘The last
lunchtime club this week is on Thursday - don’t miss it!’) Levels of personalisation were regarded as
important and suggestions for these varied in terms of the institution, the learner group and the
individual. ‘If you have a news feed then make sure it is something you put in – not just a link to any
website so that pupils see it as something specifically for them.’
Externally comprehensible - The availability of access to the VLE by a variety of users means a selfexplanatory
context has to be provided because it may be used in different contexts within the school
and outside the school by, for example, pupils, teachers, senior management, parents and carers.
Ease of use / ergonomics - Many suggestions for this arose. These included avoiding ‘front-end’
writing; ‘teachers and pupils do not want to read it, just want to get on with the work’. Another
suggestion was to make as few clicks as possible for students to get to where they are working:
337

John Jessel
‘“reduce the clicks” is my motto’. Uniformity of pathways was considered as important, colour coding
often helping. When designing pages it was felt that ‘it has to be obvious to the pupils as to how it
should work’ and ‘scrolling horizontally is a no no’, vertical scrolling being more manageable.
Reliability - Rather than this being a matter of the technology breaking down, the concern was with
planning methods of working in sufficient detail so that learning outcomes can be achieved in a
dependable way. Where the methods are innovative and carry a higher level of risk, backup strategies
were included for achieving the objectives.
Inter-institutional comparison - Because the trainers worked across schools that were known to each
other there was interest in how other institutions were approaching particular curriculum areas and
solutions to common problems were shared. In one instance a teacher from one school visited the
school where the trainer taught to see particular methods in action. Although this could have been a
recipe for uniformity, it was felt that one institution was never a blueprint of what happened in another
and so this was regarded as a form of enrichment.
Ownership - The need for teachers to take ownership and negotiate and augment aims to serve their
interests was considered essential at many different levels: ‘rather like a lesson plan innovation is
developed within an institution’ (teacher); [Innovation is] ‘something you develop yourself and have
control over rather than have it thrust upon you’.
5. Summary and conclusion
Although the VLE in secondary education may have the potential to enhance learning, effective
uptake may not occur automatically. One reason for this is that VLE technologies make innovative
demands on practice – both by learners as well as teachers. Innovation within the school setting is
something that does not occur in isolation and has to be considered along with other initiatives,
changes and demands occurring within that context. Innovation has its costs: it became clear that
time was needed to identify and adapt suitable resource, to structure content and develop new ways
of working This can also involve the kind of detail expressed in the form of the general principle that
emerged from the workshops. Much of the work associated with development appears ‘invisible’ in
that it is done in advance of the use of the technology. The use of the VLE has implications for the
role of the teacher and the learner. The work highlighted the fact that the teacher was not solely
instrumental in passing on an established body of knowledge. Teachers acted as facilitators bringing
together curriculum, teaching and technical experience in a jointly creative enterprise. It is also likely
that a certain amount of inertia can be involved in the adoption of new technologies as evidenced by
the move away from assumptions of the VLE as a storage and administrative system towards a more
participative learning medium.
Insofar that a curriculum can be viewed in not only in terms of learning outcomes but also in terms of
the means and wider learning context there are implications regarding the impact of VLE
technologies. The work connected with writing news reports illustrates, for example, how, on the one
hand, technology can be used to enhance existing curriculum objectives, while, on the other,
technology can be used to transform an existing curriculum and introduce new challenges.
References
Bakhtin, M.M. (1981) The Dialogic Imagination: Four Essays by M.M. Bahktin, Ed. M. Holquist, Austin, University
of Texas Press.
Bloom, B.S. (1956) Taxonomy of Educational Objectives, Handbook 1: The Cognitive Domain, New York, David
McKay Co. Inc.
Brown, J.S., Collins, A., and Duguid, P. (1989) “Situated Cognition and the Culture of Learning, Educational
Researcher, Vol. 18, pp 32-42.
Crook, C. (2001) “The Social Character of Knowing and Learning: Implications of Cultural Psychology for
Educational Technology”, Technology, Pedagogy and Education, Vol. 10, No. 1, pp 19-36.
DfES (2005) “Harnessing Technology – Transforming Learning and Children’s Services”, [online], Department for
Education and Skills, www.dscf.gov.uk/publications/e-strategy.
DfE (2010) “The Importance of Teaching: Schools White Paper”, [online], Department for Education,
www.education.gov.uk/b0068570/the-importance-of-teaching/.
Jonassen, D.H. (1994) “Thinking Technology: Toward a Constructivist Design Model”, Educational Technology,
Vol. 34, No. 4, April, pp 34-37.
Knowles, M. S. (1977) The Modern Practice of Adult Education: Andragogy Versus Pedagogy (Eighth ed.), New
York, Association Press.
338

John Jessel
Kukla, A. (2000) Social Constructivism and the Philosophy of Science, London, Routledge.
Lave, J. and Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation, Cambridge, Cambridge
University Press.
Ofsted (2009) “Virtual Learning Environments: An Evaluation of their Development in a Sample of Educational
Settings”, [online], Ofsted, www.ofsted.gov.uk.
Selwyn, N. (2008) “Education 2.0? Designing the Web for Teaching and Learning”, Teaching and Learning
Research Program, London: Institute of Education, [online], www.tlrp.org.tel.
UCISA (2008) “2008 Survey of Technology Enhanced Learning in Higher Education in the UK”, [online],
Universities and Colleges Information Systems Association, www.ucisa.ac.uk/publications/tel_survey.aspx.
UCISA, (2010) “2010 Survey of Technology Enhanced Learning in Higher Education in the UK”, [online],
Universities and Colleges Information Systems Association, www.ucisa.ac.uk/groups/ssg/surveys.aspx.
Vygotsky, L. (1978) Mind in Society, Cambridge MA, Harvard University Press.
Wells, G. (1999) “Language and Education: Reconceptualising Education as Dialogue”, Annual Review of
Applied Linguistics, Vol. 19, pp 135-155.
Wenger, E. (1998) Communities of Practice: Learning, Meaning and Identity, Cambridge, Cambridge University
Press.
339

Mutlimodal Teaching Through ICT Education: An e-
Twinning Program as a Case Study of Intercultural
Exchange
Paraskevi Kanari and Georgios Potamias
National Kapodistrian University of Athens, Greece
viviandeesse@yahoo.fr
giogeo_gr@yahoo.co.uk
Abstract: Multimodality is a key factor in promoting knowledge through multidimensional aspects in an approach,
determined by social environment. In the approach presented, school is a part of an accepted social context. This
paper shows that development of critical visual literacy via ICT education and e-learning combined with
conventional activities can be an innovative multidisciplinary approach for the development of pedagogical
projects. The cultural program in particular presented, which was carried out in the island of Spetses in Greece,
was integrated in this approach. The ultimate goal was to create a film transfer of a local legend by the students
of Spetses’ junior high an area culturally challenged. The program was integrated to an e-twinning project (elearning
European exchange program) concerning revitalizing local legends through e-learning, filmmaking,
teaching local History through French language and intercultural exchange with the cooperation of two European
schools, one in the Greek island of Spetses and one in Chorwovie in Poland. Through this e-learning project the
students carried out a different multimodal approach of French Language and Local Culture achieving the
development of their creativity and imagination through cultural e-interactivity and their cognitive and social
awareness as well.
Keywords: multimodal approach, social semiotics, e-twinning, cinema, local history, public school
1. Multimodality and cinematographic education
In a time of intense semiotic, visual stimulus (Cocula & Peyroutet 1999: 10) (medias, computer,
cinema) an educational system that stays attached to the “ex cathedra”(traditional frontal) method of
teaching is impossible to correspond to the interests and needs of modern day students and it is
inevitably boring in comparison to their every day learning experiences. The case of critical visual
literacy which is defined as the ability of understanding, decoding, producing and criticizing visual
messages is in our times more indispensable than ever (Kress& Van Leeuwen 1996:3; Pleios
2005:238). According to Sankey (2005: 2) a widespread development of critical visual literacy is
necessary in order to cultivate and enhance the ability of reading and semiologically decoding both a
text and an image as a whole or separately. Schirato & Yell (1996: 209) add that in western societies,
a literate person will be considered primary one that can identify, read, analyze and develop a wide
variety of visual signs and interactive stimulus. Researches on imagery and its semiotic sustainability
(Levie & Lentz 1982) prove that images can largely promote teaching and social based knowledge.
The most important advantages of teaching through the use of images are:
The images are stocked easily in long term memory, creating information as connecting parts or
meanings in human memory.
Images provoke a wide range of reactions like the rise and wakening of imagination. They create
connections; they promote and cultivate creative thinking and abstract imagination.
The readers or users of teaching material prefer illustrative material for its higher information
quality.
The combination of imagery and text is more effective in promoting comprehension than the text
alone.
The images can be useful in understanding and learning abstract meanings that can be hardly
expressed with words.
Images can help mentally or physically challenged students in learning efficiently.
The teaching use of images is proved more effective when teaching is connected with the flexible
and combined use of internet; computer enhanced virtual environments and cinema (Kress 2005:
11-12).
The role of the teacher is to create an educational and pedagogical environment that could encourage
students’ critical thinking towards imagery (images of whatever status or quality) as the increasing
number of visual stimulus make students confuse reality with the virtual reality of computer or visual
340

Paraskevi Kanari and Georgios Potamias
empowered artificial environments (Joly 2005:447; Smyrneos 2008:214). Therefore, the development
of critical visual literacy is imperative for modern students and generally citizens.
The teaching via cinema, the cinematographic education, is strongly attached to multi-modal
education (Jacquinot 1977:27; Tardy 1973:27). Cinema and moving pictures, more traditional semiotic
means of education than modern computer empowered visualizations, can be the source of multiple
teaching activities with parallel analysis of images, text and sound. The ultimate goal in this approach
is the promotion of multiple communicative and sensory interactions between students (Régis 2009:
9).
The cinematographic education is necessary for the understanding of modern multimodal media
world. The multimodal cinematographic teaching, based on image, promotes the ability of:
Using an inexhaustible source of visual material.
Reading, criticizing and decoding the multiple diversity, function and temporality of visual material.
Using rationally electronic equipment (computer, video, cinema) and visual means for gathering
and managing critical information.
Using multimedia applications in order to create multimodal contexts and interactive compositions
(combination of image- text- sound).
Promoting and presenting cinematographic creations in an informative sustainable way
(Yakoumatou 2006:74-75; Semoglou 2005:18; The New London Group 1996).
Despite the objections towards developing powerful and sophisticated criteria that can be theoretically
able to promote students’ understanding of visual communication (Barthes 1977:88; Baudinet –
Mondzain 1990:15), the need for promoting multimodal interactive teaching is urgent in modern day
school.
An effort for the creation of a school curriculum that can empower the management of critical
information has been carried out in many countries. Such curriculums theoretically consider very
important the cultivation of metacognitive skills, the ability of managing critically important knowledge
(Greek Institute of Pedagogy, 1998). Since the development of the internet has given students access
to an inexhaustible source of visual information, with the risk of confusion between valid knowledge
and useless information, it is imperative to encourage the integration of a more visually aesthetic
critical education in a modern curriculum (Barnett 2000:22; Gonnet 1997:17).
Although it would be an exaggeration to assert the primacy of visual literacy, it is generally accepted
that many students accept willingly the integration of visual material, in order to support and enrich
textual and non-textual material in a holistically empowered semiotic context (Semoglou 2005:564-
565).
Very recently, the introduction of interactive blackboards, and generally rich visual educational means
(dvd, internet videos, use of youtube or teachtube videos in the classroom) has an innovative and
revolutionary impact in the heart of modern school ethos. The students are excited from the
immediacy, audacity and relevance of illustrations and visual reproductions of all forms. However, the
teachers should be very cautious and concerned towards not only the semiotic informative quality and
pedagogical content of the imagery used but also towards their ability to use, manipulate and
successfully manage the inexhaustible source of visual information (Sankey 2005:10; Van Leeuwen &
Jewitt 2001:107).
2. Application of the theory in an educational context
In the context of multimodal cinematographic education, a cultural program was carried out in the
island of Spetses in Greece during the school year 2007-2008. The ultimate goal of this program was
to create a film transfer of a local legend by the students of Spetses’ junior high (students from 12 to
15 years old), an area culturally and educationally challenged. Spetses is a small island of 5000
inhabitants with long and rich history but with poor cultural activity, due to its isolation, especially in
winter time. The authors took part in the design, the implementation and the evaluation of this
program, mentoring students and recording every step of its development through the e-platforms of
the e-twining program. The results of the program were presented virtually in the e-twinning e-platform
and in a cultural event hosted by the municipal authorities of the town.
341

Paraskevi Kanari and Georgios Potamias
In the case study presented in this paper, multimodality is presented as an educational means of
holistically enhancing intercultural and social awareness in school. Thus multimodality in praxis affects
positively not only educational competencies of students but also the social context as a whole.
The program was integrated to an e-twinning project (e-learning European exchange program)
concerning the revival of local legends through e-learning, ICT education, filmmaking, teaching local
History and intercultural exchange. In this case two European schools cooperated, one in the Greek
island of Spetses and one in Chorwovie of Poland.
The purpose of the program was discovering and revitalizing local legends of each area and
exchanging creatively this cultural experience with the other school. The program was divided in three
parts. In the first part, each school presented itself by writing and uploading an e-twinning portrait of
their school and of themselves. In the second part the students tried to discover various local legends
by asking their parents or the elders of their community. After selecting amongst the amount of
legends discovered, each school presented the first part (the beginning) of one or more legends and
the other school tried to imagine the end of the story. Written activities in French concerning any given
legend were given to students (description of the main character, developing language enhancing
activities such as crosswords with the new vocabulary etc.). The Greek legend turned into a film was
entitled in French language: “La vieille dame et le paon” (“the old lady and the peacock). In the third
part, the end of the story was given to the students. The presentation of schools, the comments and
the e-communication between students was in French. The Polish school chose to present the
legends through the use of Microsoft’s power point programme. The Greek school chose to present its
legend through a short film.
Through this e-learning project the students carried out a different multimodal approach of Culture
and History achieving not only the development of their creativity and imagination through cultural einteractivity,
but their cognitive and social awareness as well. E-learning and visual enhancing tools
promoted the breaking of social and educational barriers through intercultural activities in two
culturally challenged European areas
2.1 General aims of the cultural e-twinning program
Approaching visual literacy through cinema and its techniques
The production of a short film.
Opening school towards other culture and mentalities (Polish legends)
Opening school towards local society.
Developing of communicative skills in a common language, as “lingua franca”, the French
language.
Considering a foreign language (French) as an empowering tool of communication and cultural
exchange.
2.2 Specific purposes
Getting to know Greek and French cinema.
Sharpening the creativity and imagination of students with exercises and games (role playing
games, brainstorming and creativity games).
Exercising creative writing through writing a scenario.
Finding and cultivating students’ talents and developing responsibility towards class and
community.
Creating a short film as a final product of the project
Learning a foreign language (French) through multimodal approach.
Approaching two different cultures and environments (an old miners’ region in Poland and a
summer resort in Greece).
342

2.3 Results
Paraskevi Kanari and Georgios Potamias
Students got to know unknown areas of cinema and developed their critical thinking towards
cinematographic art.
Students got to know legends and fairy tales of their native place.
Development of the creativity and imagination of students and teachers as well.
Constructive use of students’ free time and creation of new educative stimulus.
Development of students’ responsibility with the assignment of specific tasks and keeping a
specific time plan.
Improvement of students’ communicative skills in French. Using a foreign language outside
classroom, in order to fulfill communicative needs was a realistic goal.
Approach of a different culture and mentality.
Creation of a short film based on a local legend.
Presentation of the film in local community as a meta-result of the project in the end of the school
year.
2.4 Final evaluation
The program promoted the integration of social and cognitive challenged students in two different
countries: Socially isolated students had an opportunity to work in group activities and carry out more
interesting and stress-free tasks (isolation due to personal, family or even linguistic problems, e.g.
children of immigrants with inadequate literacy).Through this intercultural pedagogical experience,
students became more “mature” by increasing their social skills. The school, a place of academic
knowledge, usually detached from social context, became an integrated part of society. The research
of local history and mythology led to an active acquisition of knowledge based on the given social
context. Finally through film making, students and educators revaluated their cultural inheritance by
discovering the best sites and traditional abandoned constructions of their island for shooting the film,
in order to choose places for the scenes of the film (old houses turned into museums, old
coffeehouses). E-learning became a social power tool that provoked educational awareness and
cultural interchange. The case study presented in this paper sets an example of how modern
innovative multimodal educational approaches, sponsored by European intercultural programmes and
combined with traditional pedagogy, can enhance cultural literacy in educationally and culturally
challenged areas.
Acknowledgements
We would like to thank the National Scholarship Fund of Greece for its support during our research.
We would also like to address our gratitude to the people of the island of Spetses for its help and
support during the film making.
References
Barnett, R. (2000) Realizing the university in an age of super-complexity, SRHE/OU, Buckingham
Barthes, R. (1977) Rhetoric of the image, Hill and Wang, New York.
Baudinet –Mondzain, M.J. (1990) “Icone”, Les motions philosophiques, t.1, Presses Universitaires de France,
Paris.
Cocula, B. & Peyroutet, C. (1999) Sémantique de l’image, Editions Delagrave, Paris.
Gonnet, G. (1997) Education et Media, Presses Universitaires de France, Paris.
Greek Institute of Pedagogy (1998) Interdisciplinary Curriculum, Greek Pedagogic Institute, Athens.
Jacquinot, G. (1977) Image et pédagogie : Analyse sémiologique du film à intention didactique, Presses
Universitaires de France, Paris.
Joly, M. (2005) A propos de la formation à la lecture de l’image et à son analyse : entre le diable et l’ineffable in
Semoglou, K. (2005) Image and Child, Macedonia University Press, Athens.
Kress, G. (2005) Imagination, the world of images and the new media in Semoglou, K. (2005), Image and Child,
Macedonia University Press, Athens.
Kress, G., Van Leeuwen, T. (1996) Reading Images. The Grammar of Visual Design, Routledge, London.
Levie, W. H. and Lentz, R. (1982) “Effects of text illustrations: a review of research”, Educational Communication
and Technology Journal, No. 30, 195-232, Sage Publications, New York.
Pleios, G. (2005) Civilization of Image and Education: The role of Iconic Ideology, Polytropon, Athens.
Régis, C. (2009) ‘’La lecture de l’image ne s’improvise pas’’, Les français dans le monde, No. 364, pp. 29-31.
343

Paraskevi Kanari and Georgios Potamias
Sankey, M. (2005) ‘’Considering visual literacy when designing instruction’’ [online] University of Southern
Queensland from:www.usq.edu.au/users/sankey/Resources/article0602.pdf (accessed on 15/07/2006)
Schirato, T. and Yell, S. (1996) Communication & Cultural Literacy: An Introduction, Allen & Unwin Pty Ltd,
St.Leonard.
Semoglou, K. (2005) Image and Child, Macedonia University Press, Athens.
Smyrneos, A. (2008) The didactic of History, Grigoris Publications, Athens.
Tardy, M. (1973) Le professeur et les images, Presses Universitaires de France, Paris.
The New London Group (1996) ‘’A pedagogy of Multiliteracies. Designing Social Futures’’, [online], www.
Thenewlondongroup.com (accessed on 07/07/2007)
Van Leeuwen, T. & Jewitt, C. (2001) Handbook of Image Analysis, Sage, London.
344

Effectiveness and Learners’ Evaluation of Combining
Audio and Written Online Formative Feedback for
Language Learning
Rosario Kane-Iturrioz
Department of English and Languages, Faculty of Business Environment and
Society, University of Coventry, UK
r.kane-iturrioz@coventry.ac.uk
Abstract: Within the factors and drivers identified as leading assessment in higher education, feedback often
emerges as ‘the most powerful single influence’ (Gibbs & Simpson, 2004) on students’ engagement and
achievement. Although research on feedback is relatively new, in particular, that relating to audio feedback
technology, the results indicate that such has to be aligned with the assessment criteria and learning outcomes in
addition to being timely in order to respond to students’ expectations (Weaver, 2006). Furthermore, not all types
of tutor comments are equally useful as Walker (2009) reports. Price et al (2010) argue that “the learner is in the
best position to judge the effectiveness of feedback, but may not always recognise the benefits it provides”
leading to the need for an improved student advice on how to understand and use feedback. This paper reports
on the development of a multi-faceted approach to providing formative feedback on a blended e-learning module.
The method combines immediate written feedback on web-based learning materials and formative online
assignments with near-immediate written and audio feedback through audio eMail technology (Wimba voice
eMail) regarding online oral tasks. The study was conducted over a period of four years using a multi-method
approach to qualitative and quantitative data collection and analyses from a total of 100 students studying
Spanish for Business at beginner’s level. The use of both written and audio feedback is welcomed by students
and seems to overcome some of the problems highlighted in previous research on formative assessment
regarding quality, detail and timing of feedback. Nevertheless, a few students reported technical difficulties and/or
misapprehension regarding the use on campus of audio eMail technology for oral assignments. The paper also
presents the results on the correlation between students’ grades and improvements on formative online
assignments and quality of feedback and concludes with reflections and advice on the introduction of audio
feedback in combination with other forms of formative feedback.
Keywords: effectiveness; online formative feedback; language learning; audio and written feedback; students’
evaluation
1. Introduction
The extensive review of literature on assessment performed by Gibbs & Simpson (2004)
demonstrated the strong link between assessment requirements and students’ learning strategies.
Other studies have highlighted the potential of information and communication technology in changing
students’ approaches to learning through appropriately designed assessments, a recent example
being the study by Kirkwood & Price (2008).
Research into technology-enhanced assessment has reported on the positive effects on student
retention, enhanced quality of feedback, flexibility in distance learning, strategies to deal with larger
student numbers, objectivity in marking and more effective use of virtual learning environments.
Within the factors identified as leading assessment in higher education, feedback often emerges as
‘the most powerful single influence’ (Gibbs & Simpson 2004) on students’ engagement and
achievement. Race (2006) postulates that in order to be effective in promoting learning, a formal
feedback strategy should be timely, constructive, motivational, personal, manageable and directly
related to the assessment criteria and learning outcomes.
Although research on feedback is relative new, the results indicate that such has to be aligned with
the assessment criteria and learning outcomes in addition to being timely in order respond to
students’ expectations (Weaver, 2006). Furthermore, not all types of tutor comments are equally
useful as Walker (2009) reports. Price et al (2010) argue that “the learner is in the best position to
judge the effectiveness of feedback, but may not always recognise the benefits it provides”
suggesting a need for improved student advice on how to understand and use feedback. Regarding
the application of audio feedback technology, this is a new development in higher education and the
limited number of research studies published seems to indicate a greater effectiveness over textbased
feedback, leading to positive perceptions from students and increased retentions rates (Lunt &
Curran, 2010).
345

Rosario Kane-Iturrioz
Regarding language learning, there are few studies focusing on the exploration of students’
experiences of web-based formative assessments and feedback and even less dealing with the
specific issues affecting beginner language learners (Furnborough and Truman, 2009).
This paper reports on students’ perceptions on formative online assessment and a multi-faceted
approach to providing formative feedback on a blended e-learning module. The current work draws
upon some of the qualitative and quantitative data collected over a four-year period from a total of 100
students studying Spanish for Business at beginner’s level. Using the annual data gathered, this study
sought to answer three questions, namely,
Did students perceive the formative assessments (Progress Tests) as an effective method of
checking their own learning?
Whether the formative feedback provided online involving text and oral formats had been useful in
supporting students’ learning?
Were students’ perceptions about the improvements in their language skills corroborated by the
summative tests results?
2. Method
2.1 Participants
The study was conducted over a four-year period and all participants were enrolled in a 10 credit
module which focused on Spanish for Business at beginner’s level. The module lasted 10 weeks (2
contact hours per week) in 2007-08, 2008-09 and 2009-10 but was changed to 20 weeks (one hour
contact per week) for all the language modules within the Employability Programme during 2010-11 in
order to accommodate requests from some language tutors and students.
2.2 Research setting
The structure of the module included an extensive e-learning component supported by a limited
number of in-class introductory and revision sessions in order to provide flexibility in time, place and
pace of learning to university-wide undergraduate students and to minimize feelings of anxiety about
learning in what for many was an unfamiliar environment.
The online materials were based on an innovative e-learning template designed and developed in
2006-07 that integrated several learning technologies and incorporated interactive and reusable
learning objects. All the online activities and exercises provided immediate computer feedback and
allowed students multiple attempts (Kane-Iturrioz, 2009). The online workbook was re-mastered using
the authoring programme SoftChalk in 2009-10 in order to incorporate a greater range of interactive
activities. These materials were available to students on the University’s virtual learning environment
(VLE.).
2.3 Formative assessment
Formative assessments called ‘Progress Tests’ were aimed at providing students with the appropriate
means to check their progress. They also offered tutors the opportunity to personalise their feedback
based on students’ strengths and weaknesses in order to guarantee that they had achieved through
their online learning the necessary knowledge, understanding and language skill development which
would facilitate a more productive engagement with the communicative activities to be performed in
the revision sessions.
The Progress Tests were developed in the same computer-based format as those in the two
summative assessments and delivered using the assessment tool provided by the University’s VLE in
order to be consistent with the module’s main learning mode. Consequently, these tests also aimed at
familiarizing students with a test format that was new to the majority of them. The questions covered
all four language skills and reflected real life business scenarios. Multiple attempts were allowed for
both the tests and the questions within each test.
Changes in the organization and content of the Progress Test were implemented throughout the four
years of the study in order to accommodate students’ requests and syllabus modification. In the first
year of the study, Progress Tests containing five questions were developed for Topics 1 & 2, Topics 3
346

Rosario Kane-Iturrioz
& 4 and Topic 5. These were initially scheduled in weeks 3, 6 and 8 as part of students’ preparation
for the in-class revision sessions. In order to emphasise the formative character of these tests and
encourage students to engage in checking their learning, no scores were provided, the tutor issuing
personalised feedback both online and in class during the revision weeks.
The 2007-08 students’ perception questionnaire revealed that the majority liked the immediacy of the
computer generated feedback of the online learning materials and requested that a similar approach
was taken with the Progress Tests and also to have such a test after each of the Topics studied.
Consequently, from 2008-09, Progress Tests were developed for each Topic incorporating immediate
computer feedback for most of the questions and including scoring. Guidelines were given online to
students advising the achievement of a 70% score in each test before moving to the next Topic.
2.4 Formative feedback
Formative feedback was the main means of supporting the teacher’s role in promoting learning
(scaffolding), reflection and motivation both during class contact and online learning sessions. For the
latter, an action research approach was used to develop a multi-faceted method of providing timely
and effective feedback including computer generated text (Figure 1)and written and oral tutor
comments via audio eMail technology.
Figure 1: Example of interactive activities and feedback
Formative feedback given to students for each activity in the Progress Tests evolved throughout the
four years of the study in response to students’ perceptions expressed in the end of year
questionnaires. A transition took place from tutor comments to a combination of computer and tutor
feedback. Figure 2 shows an example of the formative feedback provided via the Progress Tests.
Wimba voice tools were initially integrated into online revision activities in order to provide extra oral
practice and opportunities for collaborative work among students. Students were also given the option
to send their oral recordings via voice eMail in order to receive tutor formative feedback. The
experience was a very positive one for the students who used this tool leading to the inclusion of oral
activities in the Progress Test for the last two topics in the syllabus. Figure 3 provides an example of
feedback provided via voice eMail.
347

Rosario Kane-Iturrioz
Figure 2: Example of progress test and question feedback
Figure 3: Example of voice eMail feedback
348

2.5 Data collection
Rosario Kane-Iturrioz
2.5.1 Survey of students’ perceptions
With the approval of the University Ethics Committee, two data gathering instruments (surveys and
semi-structured interviews) were used in the study. The semi-structured interviews were conducted in
2008-09 with funding received from the Centre for Inter-Professional e-Learning (CIPel).
The initial questionnaire included background information; native language and level of knowledge of
other languages; learning preferences and use of technology and, familiarity with online learning. The
end of module survey was intended to ascertain students’ perceptions of the module’s blended
learning environment and contained 20-26 close-ended questions. Additional questions were inserted
every year in order to accommodate students’ feedback regarding the changes made in response to
the previous year’s comments. To facilitate statistical analysis most of the questions were in a 5-point
Likert scale. Additionally, several open-ended questions attempted to elicit more detailed students’
feedback.
For the purpose of this study, questions relating to the students background information, language,
computer literacy and prior online learning experience were analysed. The data from the survey’s
closed-questions underwent a separate analysis from that involving the open-questions and the
interviews transcripts. The results were cross tabulated and analyzed for statistical differences (p

Rosario Kane-Iturrioz
Tests very useful as a self-assessment facility to check their progress. A breakdown of students’
perception by year can be seen in Table 1.
Table 1: Students’ perceptions on the usefulness of the progress tests
Statement: I have found the Progress Tests useful to keep
track of how much I have learnt online.
2007-08
(N=13)
2008-09
(N= 31)
2009-10
(N=28)
2010-11 (N=17)
Strongly agree 3 (23.08%) 12 (38.71%) 11 (39.29%) 5 (29.41%)
Agree 10 (76.92%) 15 (48.39%) 15 (53.57%) 10 (58.83%)
Neutral 0 (0.0%) 4 (12.90% 1 (3.57%) 1 (5.88%)
Disagree 0 (0.0%) 0 (0.0%) 1 (3.57%) 1 (5.88%)
Strongly disagree 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%)
In 2008-09, students were also asked to select which aspect they liked most about the module’s
online provision from a list of ten items. Progress Tests were the students third favourite preference
(63%) after “work at my own pace” and “multiple attempts allowed”. The open question
(questionnaires) and interviews (2008-09) corroborated these results and many students stated that
they “would rather do the Progress Test than the voice mail or discussion”. Others liked this tool
because it “...help(s) us to revise as well as prepare for the face-to-face sessions.” or, “I could see
how I got to learn more really and you could always go back”. Although many students associated the
Progress Test with an improvement in their independent learning skills: “..this experience was good
and I have got a lot of advantage out of it, like self-discipline and time management.”; they also
indicated that they had faced difficulties: “I ran out of time and so my main aim was to learn the topic
material” or “there was nobody pushing me”. In one case the problem was fear of failure “Didn’t do
most of them (Progress Tests) because wasn’t confident enough”.
One of the aims of the Progress Test was to familiarise students with the module’s summative
assessment format. A computer-based test was chosen in order to be consistent with the module’s
main delivery mode. Students were asked about their perceptions regarding the computer-based
summative assessment at two points during the four years of this study, 2008-09 and 2010-11. The
students’ answers indicate a clear change in perceptions in the intervening year. 52% and 74%
respectively “found the computer-based assessment better than the traditional assessment (including
face-to-face oral tests). The breakdown of the results can be seen in Table 2.
Table 2: Students’ perceptions of computer-based assessments
Statement: “I have found the computer-based assessment better
than the traditional assessment (including face-toface
oral tests).
2008-09 (N = 31) 2010-11 (N =19)
Strongly agree 6 (19.35%) 5 (26.31%)
Agree 10 (32.26%) 9 (47.37%)
Neutral 9 (29.03% 3 (15.79%)
Disagree 4 (12.90%) 2 (10.53%)
Strongly disagree 2 (12.90%) 0 (0.0%)
3.3 Students’ perceptions about formative feedback
In 2008-09 and 2010-11, students were asked to select which aspect they preferred most about the
module’s online coursework from a list of ten items. Immediate computer feedback was the 6 th
favourite preference in 2008-09 and the third in 2010-11 after “work at my own pace” and “variety of
activities”. Many students referred to the feedback provided for activities and exercises linking them to
positive effects on enjoyment: “That is the bit I enjoy the most in the module, the whole
interactiveness of it, it made it seem like fun”. Others reflected on the perceived motivational help
provided by the activities feedback (mainly computer-based): “ It feels you being there with me’’. This
statement seems to indicate that a comprehensive feedback is seen by students as an indication of
‘caring behaviour’ on the part of the tutor.
The Progress Test feedback both computer-based and tutor provided was in general perceived as
highly useful (84.62%) and motivating: “What really got me motivated was getting the results straight
back; to have a look through at what the mistakes were so it helps us”. A few students would have
350

Rosario Kane-Iturrioz
preferred all the feedback computer-based because of the perceived effect of the immediacy of a
response, “with an immediate answer (I) maybe do an extra half and hour”. Other students preferred
the mixture of computer-based and tutor feedback as being very appropriate to prepare them for the
summative assessments.
The use of Wimba voice tools for oral practice was received with mixed reactions. The students more
familiar with technology seemed to welcome the use of state-of-the-art tools, “Also I like the Wimba
voice mail, is quite new technology, I can record my speaking, listen it, if I don’t like it, I can record
again”. Other students found this tool interesting and fun: “The Wimba… I find it really interesting; it
was fun as well because there were things I have never used before”. Although appreciating the
benefits of Wimba’s voice tools, approximately 50% of the students did not use this resource very
often for a number of reasons. Sometimes microphones were unavailable on campus or the students
did not like practising on campus with other people listening to them. A larger proportion of the
students (60% to 70%) submitted the voice eMails linked to the oral activities in the Progress Tests.
Further insights came from the questionnaires’ open questions in which many students stated that
they only engaged in activities that had a direct relevance to the summative assessments.
The audio and text feedback provided via voice mail technology was well received by those students
who submitted the recordings corresponding to the oral activities. There were two aspects highly
valued by students, the first was a greater effectiveness of the feedback due to the synergies
achieved in the integration of audio and text commentary, “The written feedback was helpful but the
voiceMail feedback help me to understand better were I went wrong with my pronunciation”. Students
reported that the audio feedback had reinforced the written feedback because it was more detailed
and personalised. The second benefit referred to the effective delivery of feedback comments, “The
feedback for my voice eMail submissions was great as I received it straight into my mail inbox”.
3.4 Language skills
Students’ perceptions about the module being effective in increasing their language skills in the first
two years and in the final year of the study can be seen in Figure 4. This figure shows that the mean
values for Reading, Listening and Oral skills increased substantially in 2008-09 and again in 2010 in
the case of Reading. The highest values recorded corresponded to Writing and these values
remained high in all three years.
Mean Likert values (1-5)
2.5
2
1.5
1
0.5
0
Reading Listening Writing Oral
Figure 4: Perceptions of improvements in language skills
351
2007-08
2008-09
2010-11

Rosario Kane-Iturrioz
When compared with the mean test scores (summative) for language skills over the same three years
(Figure 5), it is apparent that these show the opposite distribution to the mean Likert values for Writing
in 2008-09.
80
70
60
50
Mean scores 40
30
20
10
0
Reading Listening Writing Oral
Figure 5: Mean test scores (summative) for language skills
4. Discussion
The study set out to explore the effectiveness and learners’ perception of formative assessment and
feedback in the context of a blended learning module where online learning was the main feature.
There were three questions that the study sought to answer, dealing with students’ perceptions
regarding the effectiveness of the formative assessment strategy, their evaluation of the formative
feedback provided and the students’ views relating to their improvement in language skills.
Regarding the first question, the results of the study clearly demonstrated that the Progress Tests
were perceived as an effective strategy. These results corroborated other research findings which
have reported that learners valued the availability of online formative assessments because this
allowed them to take control of their learning (Lyons and Thorpe, 2009) by facilitating reflexion on the
gap between their performance and the level of attainment needed (Gibbs and Simpson, 2004). Some
students reported that the Progress Tests had contributed to the development of better independent
learning skills involving self-discipline which appeared to indicate that they were on their way to
develop a better self-regulation of their learning (Nicol and Mcfarlane-Dick, 2006). Additionally,
students stated in the 2008-09 survey that the Progress Tests were the third most popular element of
the online provision only surpassed by the convenience of working at their own pace and that multiple
attempts were allowed (for both the online exercises and the Progress Test). The latter result confirms
that of Felix (2001) who reported that students felt that having multiple attempts promoted language
learning.
The second question focused on the students’ views regarding the formative feedback received. The
results indicated that the respondents associated the provision of an immediate and comprehensive
feedback with an increase in their motivation and also in their sense of efficacy. Similar findings were
reported by Segarra and Zapata (2008) and Furnborough and Truman (2009).
Computer-based feedback was included in the majority of the Progress Tests’ questions from the
second year of the study onwards in response to students’ requests. In the case of some learners, the
352
2007-08
2008-09
2010-11

Rosario Kane-Iturrioz
instant feedback led to spending “many unanticipated hours in front of the computer in pursuit of the
perfect score” (Segarra and Zapata, 2008). A few of the Progress Tests questions retained tutorbased
feedback in order to provide students with semi-structured activities that were too complex for
automatic computer feedback and required detailed explanation from the tutor. Students found the
combination of computer and tutor feedback very useful in subsequent years because this method
integrated the benefits of timely guidance with constructive feedback and information on how to
improve their performance (Felix, 2003).
The voice tools were integrated into the module’s homepage (VLE) and considered easy to use by
both students and the tutor. They were used by 40% to 50% of the students because it provided
opportunities to practice their oral skills through an online environment and they could also keep their
audio files for future reference by downloading them onto their computers or mobile phones. For the
tutor, the recorded audio files provided opportunities for comprehensive feedback. From a practical
perspective, it also facilitated internal and external moderation of students’ summative oral
assessments.
However, voice tools for oral practice were not universally welcomed. Approximately one third of the
students each year revealed reluctance about using computers to practice their oral skills. The
consistent feedback from these students was that they preferred to practice oral conversation in a
class-based setting with ‘real’ people. A further drawback mentioned by some students was that they
did not like being overheard by other students when using voice tools on-campus. These perceptions
showing a dislike by some students to computers as a medium for language learning and the
discomfort felt whilst recording on campus were also key features in a study of learner drop-out
reported by Stracke (2007). A larger proportion of the students only used the voice eMail tool for the
oral activities contained in the Progress Tests as preparation for the summative assessment.
In an earlier paper regarding learners’ experiences and perceptions of blended learning for languages
(Kane-Iturrioz, 2010), it was reported that the students’ learning experience was still mainly
determined by a traditional class-based model that did not foster independent learning skills. In the
current study, students referred to their difficulties in adjusting to a regular weekly online workload
very different from the learning pattern used in other modules where most of their study was linked to
assessments and exams. This corroborates the results of Murday et al. (2008) who also report
students’ problems regarding ‘self-regulation’ and its effect on timely study.
The final research question sought to determine whether the summative test corroborated students’
perceptions regarding the effectiveness of the module in improving their language skills. The results
reported indicate that students’ perceptions of such improvements in 2008-09 seem to mirror the only
change introduced in that year, that is, the reorganization of the Progress Tests. Those in 2010-11
could be attributed to the enhanced feedback provided in both materials and Progress Tests and also
to the change in the module’s schedule which gave students more time to develop their skills. Which
of these changes contributed more to the students’ perceptions cannot be ascertained from the
available data. The comparison of the trend in the mean scores and perception values seems to
indicate that they closely resemble each other in their incremental direction.
A minority of students expressed doubts regarding the appropriateness of an e-blended approach to
language learning. However, one of these students who had little confidence in using computers
stated, “it was incredibly hard ...but overall I did enjoy it and, you know, I have learnt a lot more
Spanish that I thought I would do”. Some students expressed the opinion that “.. for a language
module I think online is not the way”. Nevertheless, one of these found the module useful in helping
him to become a better independent learner, “I think that I know where I stand as an independent
learner, which really helps”.
5. Conclusion
Students highly appreciated the module’s assessment strategy designed to provide regular
opportunities for reflection on their online learning. The range of formative feedback offered was also
welcomed by students because of the combination of timeless provision of immediate computerbased
comments/hints and detailed commentary from the tutor. The first type of feedback encouraged
many students to engage with the online materials and also improved motivation through a better
understanding of the language and the test requirements. The second type, helped students to
appreciate their strengths and weaknesses and facilitated a deeper knowledge of the subject.
353

Rosario Kane-Iturrioz
Although the voice tools provided were not used by approximately one third of the students, those
who did access them reported that the audio and written feedback received became part of their
learning process. The verbal comments recorded by the tutor complemented and clarified the written
part of the feedback. Voice mail technology made the feedback easily accessible via students’
personal inboxes and the tutor’s comments were received soon after the assessment enabling them
to analyse their performance quickly.
From the tutor’s perspective, voice mail technology does not need complex settings and instructions.
The beneficial effect reported by students was achieved through the tutor dedicating a few minutes to
type and record the feedback as soon as the submitted voice mail messages were received.
Research has shown that students adapt their learning strategies to the summative assessment
requirements (Felix, 2001; Gibbs and Simpson, 2004; Kirkwood and Price, 2008). The results in this
study appear to validate the idea that students’ learning can be shaped by regular, short formative
assessments leading to a steady focus on study, perception of improved language skills development
and the attainment of better grades.
References
Felix, U (2001) “The web’s potential for language learning: the student’s perspective”. ReCALL, Vol 13, No. 1, pp
47-58.
Felix, U. (2003) “Teaching language online: Deconstructing the myths”. Australian Journal of Educational
Technology, Vol 19, No. 1, pp 118-138.
Furnborough, C. Truman, M. (2009) “Adult beginner distance language learner perceptions and use of
assignment feedback”. Distance Education, Vol 30, No. 3, pp 399-418.
Gibbs, G., and Simpson, C. (2004) “Conditions under which assessment supports students’ learning”. Learning
and Teaching in Higher Education, Vol 1, No.1, pp 3–31.
Kane-Iturrioz, R. (2009) “Enhancing Blended Language Learning: Development of a new eLearning Template”,
[online], EUROCALL Review 15, March-September. Special Issue, pp 36-55, http://www.eurocalllanguages.org/news/newsletter/15/index.html
Kane-Iturrioz, R. (2010) “Effective Blended Learning for Languages: Learners’ Experiences and Perceptions”.
Proceedings to the Third International Conference on Innovation in Learning for the Future 2010; e-
Learning, Istanbul, May, 10-14, 2010. TC Istanbul Kultur University Publication No: 125, pp 340- 350.
Kirkwood, A. and Price, L. (2008) “Assessment and student learning: a fundamental relationship and the role of
information and communication technologies”. Open Learning: The Journal of Open, Distance and e-
Learning, Vol 23, No. 1, pp 5-16.
Lunt, T. and Curran, J,. (2010) “Are you listening please? The advantages of electronic audio feedback compared
to written feedback”. Assessment & Evaluation in Higher Education, Vol 35, No.7, pp 759-769.
Lyons, H. and Thorpe, L. (2009) “Investigating students experiences of e-learning using the Diary Interview
Approach”, [online], Brookes eJournal of Learning and Teaching, Vol 2, No. 4. http://bejlt.brooks.ac.uk/
(accessed on 03.06.2009).
Murday, K., Ushida, E. and Chenoweth, A. (2008) “Learners’ and teachers’ perspectives on language online”.
Computer Assisted Language Learning, Vol 21, No. 2, pp 125-142.
Nicol, D.J., and Mcfarlane-Dic, D. (2006). “Formative assessment and self-regulated learning: A model and
seven principles of good feedback practice”. Studies in Higher Education, Vol 31, No. 2, pp 199-218.
Price, M., Handley, K., Millar, J. and O’Donovan, B. (2010). “Feedback: all that effort, but what is the effect?”
Assessment & Evaluation in Higher Education, Vol 35, No. 3, pp 277-289.
Race, P. (2006) “The Lecturer’s Toolkit – A Practical Guide to Assessment”, Learning and Teaching, 3rd Edition,
Routledge, London.
Sagarra, N. and Zapata, G.C. (2008) “Blending classroom instruction with online homework: A study of students
perceptions of computer-assisted L2 learning”. ReCALL, Vol 20, No. 2, pp :208-224.
Stracke, E. (2007) “A road to understanding: A qualitative study into why learners drop out of a blended language
learning (BLL) environment”. ReCALL, Vol 19, No. 1, pp 57-78.
Walker, M. (2009) “An investigation into written comments on assignments: do students find them usable?”
Assessment & Evaluation in Higher Education, Vol 34, No. 1, pp 67-78.
Weaver, M. (2006) “Do students value feedback? Student perceptions of tutors’ written responses”. Assessment
& Evaluation in Higher Education, Vol 31, No. 3, pp 379-394.
354

Model of eLearning Project Evaluation
Jana Kapounova, Jana Sarmanova and Marketa Dvorackova
University of Ostrava, Czech Republic
Jana.Kapounova@osu.cz
Jana.Sarmanova@osu.cz
D10576@student.osu.cz
Abstract: The paper deals with the model of eLearning evaluation process, specifically with a proposal of
methods and means for eLearning project evaluation. There are many approaches to eLearning projects
evaluation. Let us consider eLearning as an educational project. An educational project, just like other projects, is
comprised of design phases, proceeding one after another and interrelating. An eLearning project passes
through several phases of its development: from planning through the development of study supports and the
pilot stage, to usage of project results in instruction, namely in distance learning. eLearning project can be
evaluated during each of its phase and as a whole project as well. It means for each phase of the project we
have to collect reliable information to determine criteria of the phase quality. After evaluation of the phase we
define feedback as a response to the results of performed analysis. In the present paper we want to demonstrate
our way of dividing the eLearning project into phases like phases of the system development life cycle. We
evaluate each individual phase. We have determined activities of evaluation, input information, outputs and
staffing of the activities. We chose information that employs as measures needed for quality evaluation of each
phase and of the project as a whole.
Keywords: evaluation of eLearning, eLearning project, project life cycle, eLearning project phases, project
evaluation
1. Introduction
The expansion of eLearning since the last century was very fast especially in universities and
corporate training sector. Information and Communication Technologies (ICT) were first used for
communication between students and teachers (via email), in the area of supporting study materials
(study texts, PowerPoint presentations, video recordings of lectures), and finally for the controlling
and management of studies (Lojda, 1999).
eLearning plays a crucial role in distance education, but it is also important in supporting face-to-face
studies. The most common form of study using eLearning is a combined form, sometimes called
blended learning.
With the rise of eLearning, many, especially managers of educational institutions, thought that the
eLearning form of instruction will be cheaper. This belief of eLearning being a not expensive solution
is still widespread among the public, even though some experts have warned about higher costs
associated with the implementation of eLearning from the very beginning. What affects the eLearning
projects then, what should we not neglect during their development and implementation? (Poulová,
2006)
Within the ‛eLearning Project Evaluation – the system approach’ project we intend to establish a
procedure (procedures) and determine factors that can affect quality, effectiveness and economy of
an eLearning project.
There are many approaches to eLearning projects evaluation. Most of theoretical studies and
examples of good practice deal with evaluation of study supports and ways of communication
between learners and instructors (Eger, 2004). The effectiveness of eLearning is with no doubts
affected by other factors as well which we will try to determine in our work (Freibergová, 2000). We
presume that the best way of how to prepare the methodology for eLearning evaluation should be the
system approach.
2. Some approaches to the problematic
The discussion of eLearning, or of its evaluation, requires us to define several terms. From a wide
range of definitions that are to be found throughout the relevant bibliography, we have accepted the
following.
355

Jana Kapounova et al.
eLearning is an educational environment which uses information and communication technologies to
achieve the educational goal: it includes creation of educational objects, distribution of study content,
implementation, communication between participants of the educational process and management of
studies.
The greatest emphasis is placed on the term ‛learning’ while the ‛e-’ is the electronic-related prefix,
notifying of using ICT.
The assessment and evaluation we have based on pedagogical principles. In pedagogical science
there are two most often forms of evaluation/assessment (Zlámalová, 2002).
Assessment of the learner’s achievements within the process of learning, in other words the
comparison of his/her entry and output competencies.
Evaluation of effectiveness, in terms of the effectiveness of the system including economic
evaluation of the investment into the educational product.
Theories of eLearning evaluation approaches are very commonly discussed. We have reviewed many
theoretical studies and examples of good practice. Such, which can be used to determine factors
suitable for our evaluation model, have been chosen. For the purposes of our work, we prefer the
following theoretical approaches:
Kirkpatrick’s Four Levels of Evaluation (Winfrey, 1999);
A Framework for the Evaluation of eLearning (Huges, Attwell, 2002);
The Model of assessing quality in eLearning elaborated by The Swedish National Agency of
Higher Education (Åström, 2009);
Managing eLearning Strategies (Khan, 2005).
First we remind in brief, what is the main point of these methods, particularly in relation to our
objectives – to prepare a methodology for eLearning project evaluation
2.1 Kirkpatrick’s four levels of evaluation
Donald Kirkpatrick’s four-level model of evaluation (quoted from Winfrey, 1999) can be applied to
technology-based training as well as to more traditional forms of learning. In Kirkpatrick’s four-level
model, each successive evaluation level is built on information provided by the lower level. According
to this model, evaluation should always begin with level one, and then, as time and budget allows,
should move sequentially through levels two, three, and four. Information from each prior level serves
as a base for the next level’s evaluation.
Level 1 Evaluation – Reactions
Evaluation at this level measures how participants in a training program react to it. It attempts to
answer questions regarding the participants’ perceptions. Did they like it? Was the material relevant to
their work? This type of evaluation is often called a ‛smile sheet’. According to Kirkpatrick, every
program should at least be evaluated at this level to provide for the improvement of a training
program.
Level 2 Evaluation – Learning
Assessment at this level measures the amount of learning that has occurred due to a training
program, evaluations often use tests conducted before training (pre-test) and after training (post-test).
Methods range from formal to informal testing to team assessment and self-assessment.
Level 3 Evaluation – Transfer
Evaluation at this level measures the transfer that has occurred in the learners’ behaviour due to the
training program. Evaluating at this level attempts to answer the question: Are the newly acquired
skills, knowledge, or attitude being used in the everyday environment of the learner? For many
trainers this level represents the truest assessment of a program’s effectiveness.
Level 4 Evaluation – Results
356

Jana Kapounova et al.
Evaluation at this level assesses training in terms of business results, measures the success of the
programme in terms that managers and executives seek: increased production, improved quality,
decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return
on investment. Return On Investment (ROI) is sometimes mentioned as a possible 5 th level or can be
included in original 4 th level Results.
2.2 A framework for the evaluation of eLearning
J. Hughes and G. Attwell presented a new framework for the evaluation of eLearning (Huges, Attwell,
2002).
They tried to define a number of variables, which potentially impact on the effectiveness of the project
and to decide what constitutes dependent, independent and irrelevant variables in a given situation.
Five major clusters of variables have emerged:
Individual learner’s variables:physical characteristics, learning history, learner’s attitude, learner’s
motivation, familiarity with the technology;
Environmental variables:immediate (physical) learning environment, institutional environment,
subject environment;
Contextual variables:socio-economic factors, political context, cultural background, geographic
location;
Technology variables:hardware, software, connectivity, media, mode of delivery;
Pedagogic variables:level and nature of learner support systems, accessibility issues,
methodologies, flexibility, learner autonomy, selection and recruitment, assessment and
examination, accreditation and certification.
2.3 The model of assessing quality in eLearning
An approach to the problematic of aspects and quality evaluation criteria definitions, described by the
team of the Swedish National Agency for Higher Education, is very similar to our attitude. They have
determined two basic quality criteria (Åström, 2009).
Functional and system approach to eLearning implementation including the following partial
aspects:
Material/content;
Structure/virtual environment;
Communication, cooperation and interactivity;
Student assessment;
Flexibility and adaptability;
Support (student and staff);
Staff qualification and experience;
Vision and institutional leadership;
Resource allocation;
The holistic and process aspect.
Internal evaluation, updating and improving of using holistic approach to the resolution of the
problematic.
2.4 Managing eLearning strategies
The author, Badrul Khan (Khan, 2005), presents critical eLearning factors as questions that can be
asked when planning, designing, evaluating and implementing eLearning modules, courses and
programmes. eLearning represents a new paradigm both for learners and for other stakeholders
(instructors, trainers, administrators, technical and support service staff) and the institution. The eightdimensional
eLearning framework was created to comprehend all of the following issues: institutional,
management, technological, pedagogical, ethical, interface design, resource support and evaluation.
357

Jana Kapounova et al.
Each dimension includes several sub-dimensions and each sub-dimension consists of items or issues
focused on a specific aspect of an eLearning environment.
Another idea very important for our approach is the following. In eLearning projects we need to
understand people, process and products concerned in eLearning (called P3 continuum). People are
involved in the process of creating eLearning materials and making them available.
Khan divides eLearning process into two phases:
Content development – planning, design, production, evaluation of eLearning content and
resources;
Delivery and maintenance – implementation of online course offerings, ongoing updating and
monitoring of eLearning environment.
From the above mentioned theoretical methods we take some items that suit our purposes – to
prepare a methodology for evaluation of eLearning. In our model we intend to employ the system
approach, to specify structure and to find its characteristics.
3. eLearning project and its phases
Let us consider eLearning as an educational project. An educational project, just like other projects, is
comprised of design phases, proceeding one after another and interrelating.
Now let us define the term eLearning project:
eLearning project is such a project within of which an eLearning subject, or subjects forming one
complex (e.g. a study programme), or just a part of a subject is planned, developed, implemented and
introduced in instruction.
Note: By the term “subject” we mean a school subject; we prefer the term “subject” to the word “course”.
An eLearning project represents:
Creation of a set of eLearning subjects or eLearning components (most often);
Addition or construction of Learnimg Management System (LMS) functions, e.g. adaptive
instruction option.
An eLearning subject includes:
Relevant content (study support, multimedia components, sources and links, tests, etc.);
Study system (student registration, task completion, test results, etc.);
LMS (study flow monitoring).
3.1 Some models of project phases
An eLearning project passes through stages of development – from planning through study supports
development, testing to usage of project results in instruction. There are many ways how to split down
into stages of developed works. To determine the phases of eLearning project we used some theories
and models presented in bibliography, e.g.
System development life cycle (SDCL)
Instructional System Design (ISD)
Statement of work (SOW)
The individual phases of an eLearning project are determined similarly as the system development life
cycle.
System development life cycle is a process that is used in connection with information systems
development (Cummings, 2006, SDLC in Wkipedia, 2011). Essential phases of the cycle are:
System analysis – analyse the situation, project goals, end-user needs and requirements definition.
358

Jana Kapounova et al.
Design – design functions and operations in detail. The new system is described as a collection of
modules or subsystems.
Testing – unit, system and user acceptance testing are performed; the whole system is tested as well.
Operations and maintenance – include changes and enhancements before the termination of the
system; the system maintenance is an important aspect; new changes require system updates.
Instructional System Design (Clark, 2002) is another possible approach. For our modelling we accept
this very scheme. It is also described as an educational parallel to the software development process.
It breaks down into five phases, an acronym of which (ADDIE) is used as an equivalent to the term
ISD. The design of an educational process must contain all of the five phases, so that the result would
be a product that teaches.
Analysis – identify needs and constraints.
Design – define learning activities, assessment and media.
Development – produce, perform formative evaluation, and revise.
Implementation – deliver the instruction.
Evaluation – evaluate results.
It is necessary to point out that a pre-design phase precedes every project. It includes a statement of
work providing a description of what must be done and in what scope. The stated objectives must
thus fulfil the conditions of measurability, specificity, and attainability. This concerns the preliminary
project plan (Schwalbe, 2007).
4. Model of eLearning project evaluation
When considering the eLearning project evaluation we established
phases of an eLearning project which enabled us to determine
phases of evaluation, within of which we have specified:
activities necessary to be carried out when evaluating and
people responsible for their fulfilment, and further
activities inputs and
activities outputs.
All of the above mentioned we have organized in a following table.
Table 1: eLearning project table structure
eLearning project evaluation activity who performs it (is
phase
phase
responsible)
resource analysis: HR analysis HR analysis manager (of the
human (HR)
...
evaluation check
project)
activity inputs activity
outputs
checklist completed
checklist
4.1 Phases of eLearning project
The individual phases are preceded by a pre-design phase containing a definition or a requirement to
establish an eLearning project.
0. Requirement or an idea to realize, update or extend an eLearning project; it can also be a project
application. Includes objectives, project justification, suitability, effectiveness, target group
specification, expected contribution, added value, risks and options to eliminate them, continuing the
359

Jana Kapounova et al.
project, connections (to other projects, to the institution, to educational intentions of the institution,
etc.), management and organization concept, and other.
After this pre-design phase being assessed and approved, the phases of the system development life
cycle follow.
1. Analysis of the resources, subjects contents, readiness of the institution.
2. Design of author requirements, teaching and learning objectives, concept of the content,
specification of the study matter, teaching structure, teaching strategies and teaching forms
determination, learning management plan layout, specification of feedback mechanisms, learning
aids, instructional units, personal coverage and time schedule of the project, maintenance and
operation plan, operation and instruction termination.
3. Development involves assigning roles or tasks to developers, task definition and specification,
milestones and deadlines layout, development of the educational environment graphic design,
creation of instructional objects, subjects behaviour setup, individual components check up, study
supports completion, subject(s) complex check up, functionality verification before the pilot operation.
4. Implementation starts with experimental or pilot operation, followed by feedback adjustments and
preparation for routine operation, i.e. ensuring operation conditions, subject administration, student
activities evaluation. We must not leave out the project and its parts updates.
5. Evaluation of the project When evaluating an eLearning project it is necessary to distinguish the
evaluation of the project as a whole from the evaluation of the individual subjects. It is also essential
to distinguish evaluation from the executors’ point of view and those who participate in the project
implementation: learners, teachers, personnel concerned, etc.
For the evaluation of the project as a whole we analyse data acquired from evaluation materials, e.g.
study support evaluation reviews, but also the quality of the evaluation ‛secondary products’, for
example the content and composition of the study support evaluation form. In the final evaluation the
whole eLearning project construction view is also considerable, what will for example happen, if we
take out one of the subjects?
4.2 Evaluation phases
Individual phases of an eLearning project life cycle can be assigned with evaluation phases. In these
we determine:
Objectives of evaluation phase;
Activities necessary to be carried out and by whom, or who is responsible for their execution;
Input information are to be required for each of the activities;
Output information;
Measures of a feedback.
Table 2: Example of the eLearning project evaluation activities
eLearning
project phase
readiness of the
institution
analysis
curricular accord with the
educational
orientation of the
institution
technical functional
hardware,
software, internet
evaluation phase activity who performs it
(is responsible)
assess the
institution
comparison of
the content of
instruction with
the orientation
ICT requirements
with available
technologies
360
activity
inputs
manager comparison
of the
educational
technician,
manager
content
comparison
of ICT
equipment
activity outputs
evaluation
statement, e.g.
completed
checklist
evaluation
statement, e.g.
completed
checklist

human
resources
tutors, lectors,
administrators,
etc.
Jana Kapounova et al.
HR requirements
with professional
and time
potentials
manager,
personnel officer
eLearning evaluation phase activity who performs it
project phase
(is responsible)
financial financial project budget accountant,
sufficiency with the
institution
financial plan
manager
organizational ability/readiness interview with manager,
of the institution management of management of
to accept the
project
the institution the institution
client approach e.g. awareness information Public Relation
of the people materials review (PR) specialist,
concerned (PR)
manager
4.3 Activity outputs
comparative
tables
activity
inputs
comparative
balance
sheet
e.g. in a
form of a
questionnair
e
set of
information
materials
evaluation
statement, e.g.
completed
checklist
activity outputs
evaluation
statement, e.g.
completed
checklist
evaluation
statement, e.g.
completed
checklist
evaluation
statement, e.g.
checklist
As evaluation activity outputs we have chosen the following: questionnaire, review, checklist,
evaluation report, survey record, evaluation statement, data analysis conclusions.
Results of the evaluation should among others show, if the people preparing the project have
considered all of the issues, how thoroughly and in what depth. To make the process of evaluation
less demanding and consuming, we need to optimize the set of evaluation materials. It is possible to
cumulate some of the outputs, e.g. in the above mentioned table we can merge all of the evaluation
statements of the readiness of the institution to a single checklist.
5. Conclusion
Every eLearning project is unique. Every developer chooses his own priorities within the project.
Our approach to eLearning project evaluation we based on dividing a project into phases (like phases
of the system development life cycle) and evaluating these individual phases. For each of the phases
we have determined activities of evaluation which are necessary to be realized, input information,
outputs and personnel coverage of the activities.
The determined items have been reviewed by experts in the field of eLearning at universities in Czech
Republic. According to their suggestions, adjustments have been made to the set of activities and
their attributes.
Subsequently some tools for individual phases and sub-phases evaluation are being designed.
First a study support evaluation form has been designed, an evaluation tool from the eLearning
education methodology usage point of view (Mechlová, 2008), (Šarmanová, 2009). The evaluation
form is a structured questionnaire consisting of twenty-six questions split into four areas:
Basic characteristics of text.
Encouragement of students.
Planning and organizing of study activities.
Feedback and evaluation.
The results of this evaluation on 57 study supports have been analysed with statistical methods
(Kapounová, Šarmanová, 2010).
At present two other evaluation tools are being verified:
An implementing of multimedia objects evaluation form;
Analysis and assessment of the suitability of the project in form of a checklist for reviewing the
readiness of the developer institution for implementation and execution of an eLearning project.
In this paper we present our approach to systematic evaluation of eLearning projects that suits our
learning environment, particularly at universities and high schools in the Czech Republic.
361

Acknowledgments
Jana Kapounova et al.
This work was supported by the Czech Science Foundation under the contract No. 406090242.
References
Åström, E. (2009) E-learning quality. Swedish National Agency for Higher Education, Stockholm : Sweden.
Clark, R. C., Mayer, R. E. (2002) e-Learning and the Science of Instruction: Proven Guidelines for Consumers
and Designers of Multimedia Learning. San Francisco : Pfeiffer.
Cummings, H. (2006) Management Information Systems for the Information Age. Toronto : McGraw-Hill Ryerson.
Eger, L. (2004) E-learning, evaluace e-learningu + případová studie z projektu Comenius. Plzeň : Západočeská
univerzita,
Freibergová, Z. (2000) Hodnocení kvality studijních programů. Praha : CSVŠ.
Huges, G, Attwell, J. (2003) A framework for the evaluation of e-learning [online], London : KnowNet,
.
Kapounová, J. (2007) Approaches to the evaluation of eLearning. Proceedings of the Computer Based Learning
in Science conference, pp. 208–214.
Kapounová, J., Šarmanová, J. (2010) The Evaluation of eLearning Study Supports. Proceedings of the Computer
Based Learning in Science conference, pp. 103–109.
Khan, B. (2005) Managing e-learning: design, delivery, implementation and evaluation. Information Science Pub.
Lacko, B. (2009) Zásady moderního projektového řízení. [online], http://lacko.otw.cz/eseje/Co_je_projektoverizeni.doc.pdf.
Lojda, J. (1999) Význam kvality v distančním vzdělávání. In Distanční vzdělávání v České republice – současnost
a budoucnost. Praha: CSVŠ – NCDiV,
Mechlová, E. (2008) Tvorba e-learningových kurzů pro technické obory. Ostrava : Technical University of
Ostrava,
Poulová, P. et al. (2006) Efektivita e-learningových kurzů. In Distanční vzdělávání v České republice –
současnost a budoucnost. Praha: CSVŠ – NCDiV,
Šarmanová, J. (2009) E-learningové prvky pro podporu výuky odborných a technických předmětů (eLearning
Elements Used for Support in Teaching of Specialities and Technical Subjects). Project report. Ostrava :
Technical University of Ostrava.
Schwalbe, K. (2007) Information Technology Project Management. UK : Amazon.
Systems Development Life Cycle (2011) [online], http://en.wikipedia.org/wiki/Systems_Development_Life_Cycle .
Winfrey, E. C. (1999) Kirkpatrick's Four Levels of Evaluation. In Hoffman, B. (Ed.), Encyclopedia of Educational
Technology. [online], http://coe.sdsu.edu/eet/Articles/k4levels/start.htm.
Zlámalová, H. (2002) Principy distanční vzdělávací technologie a možnosti jejího využití v pedagogické praxi na
technických vysokých školách. [online], http://virtual.cvut.cz/telel/zlamalova.html.
362

Bridging the Gap – From Teacher to eTeacher
Andrea Kelz
University of Applied Sciences Burgenland, Campus Pinkafeld, Austria
andrea.kelz@fh-burgenland.at
Abstract: Virtual learning environments have become part of nearly every educational programme in higher and
further education. Hence, competent teachers, tutors and trainers are a major key to success in higher education
and need a thorough grounding in how to design, produce, use and evaluate e-learning and blended learning
resources. But how do teachers actually gain the skills and insights you need to practice e-learning and blended
learning effectively and efficiently and how do they develop a critical understanding of the pedagogical issues
involved in the design, development and implementation of e-learning in their professional context? This paper
focuses on the support offered to acquire and extend the skills needed to “teach online” from the institutional
point of view of an Austrian university of applied sciences. It is shown in what way an institution tries to support
teachers to gain the know-how to “transform” their teaching methods, to support them to make effective use of
learning management systems and the web as an educational resource. Finally, the paper refers to what kind of
nationwide training is offered to university teachers to develop the competencies needed to teach online
effectively.
Keywords: e-teaching, e-qualifications, train the trainer, professional development
1. Starting point
In line with the principles of the treaty of Bologna to converge higher educational systems towards a
more transparent system, a transformation of all diploma programmes at Campus Pinkafeld into
bachelor’s and master’s degree programmes started in 2004. Since then Campus Pinkafeld has been
offering part-time degree programmes using a blended learning approach, allowing for a combination
of on-site and off-campus study modules. Our part-time students can pursue degrees in two core
competences, either in energy and environmental management or in health management and health
promotion. In addition, students without an ‘A’ level equivalent may qualify to study at our university of
applied sciences if they pass the accreditation examination in the required subjects. Similar to our
bachelor’s and master’s part-time courses the preparatory courses for the required subjects are also
based on a blended learning approach (cf. Walitsch 2008). Generally, all students at Campus
Pinkafeld enjoy the same level of academic assistance, regardless of whether they opt to study full- or
part-time. The 250 full-time students use the infrastructure of the campus and its adjoining technology
centre Monday through Friday while the 350 part-time students complete their programmes through
blended learning, which mixes a schedule of on-campus studies on weekends with additional studies
using an internet-based platform. This resulted in the overall reduction of the amount of on-site
teaching hours by at least one third.
The overall design of the part-time programmes at Campus Pinkafeld implies two major organisational
differences. First, courses only take place at weekends and second, the amount of face-to-face
teaching was reduced by about one third. Offering part-time programmes only at weekends, i. e.
Friday afternoons and Saturdays, contributed decisively to a wider acceptance of the technical degree
programmes in the field of energy and environmental management as they do not only enable more
local people to continue to work alongside their studies but also allow people living in remote areas to
participate in the degree programmes. Second, a new constructivist-based didactic framework and its
corresponding teaching methods and learning strategies were established to help part-time students
acquire the relevant skills and competences for their future field of work. Both objectives could be
achieved by technology-aided blended learning settings as they allow for a considerable reduction of
face-to-face sessions and help implement new learning scenarios which are based on open,
computer-aided, learner-centred, self-directed and collaborative learning arrangements.
2. Blended learning strategy in general and didactic framework in particular
Within the part-time degree programmes at Campus Pinkafeld blended learning is the term used to
describe learning activities based on a combination of traditional face-to-face with computer and webbased
online learning settings, multiple theories of learning and pedagogic approaches and a variety
of media and tools employed in face-to-face and virtual learning environments, including traditional
print materials as well as all kinds of technology-based materials and activities. The constructivistbased
didactic approaches and their corresponding teaching methods and learning strategies
pursued at Campus Pinkafeld are supposed to successfully help the students acquire the relevant
363

Andrea Kelz
skills and competences for their future field of work. As most of the learning and teaching at Campus
Pinkafeld was based on behaviourist and cognitive assumptions when the first degree programme in
building engineering started in 1994, the pedagogy and curriculum of technical education in part-time
degree programmes implied a rethinking of how we determine, structure and deliver the content of
technical education and try to find ways to restructure the overall learning environment in which the
learning process itself has gained in importance. As far as curriculum design is concerned, each
programme is comprised of individual study modules, each consisting of a number of courses which
are split into face-to-face and blended learning modules. The modules and courses are defined in
terms of learning outcomes, i. e. what a student is expected to achieve in terms of knowledge,
understanding, skills and competencies. The (virtual) classroom should be a place for knowledge
construction and exchange of knowledge between members of the group, the teacher should be a
motivator, facilitator and guider of the individual learner progress rather than a transmitter of
knowledge and the student should interact with his group and learn in an autonomous way rather than
perceive contents passively (Kelz 2008). A lot of tantalising challenges which needed and partly still
need to be overcome.
From the very beginning of the implementation of a blended learning approach at Campus Pinkafeld a
strategic paper was written by an appointed blended learning coordinator and made available to all
part-time teachers. The paper is divided into two main sections, a theoretical part on the overall
organisational and pedagogic design of the new part-time programmes and a practical part which
outlines how to actually design and implement blended learning activities. The latter focuses on
student requirements, modularisation, learning and teaching strategies, practical approaches to
learning and teaching, ICT and learning management systems, student assessment, course
evaluation and teacher development. As we didn’t expect teachers to read the full 47-page strategic
paper a much shorter teacher manual was created and provided for download. The main contents
were also published as a learning module on our LMS, providing further information and links to
relevant pages. The strategic paper and teacher manual were presented and discussed in teacher
workshops alongside practical activities to plan and implement blended learning activities.
Furthermore, teacher workshops on pedagogic and didactic issues in general and blended learning
scenarios in particular have been offered on a regular basis for the last seven years. These
workshops are held by experienced blended learning trainers and are freely accessible to all
members of the teaching staff (full-time teachers, private tutors, freelancers etc.).
3. Learning management system
Currently there are two teachers responsible for the implementation of the blended learning strategy
pursued at Campus Pinkafeld. They spend up to half of their working hours to help teachers design
and implement blended learning activities, train students and teachers on how to use the information,
communication, and collaboration tools of the platform provided and give teacher workshops on how
to use its authoring tools to design online learning materials and activities.
The primary vehicle for the delivery of our blended learning activities is the open-source platform
ILIAS which has been in use at Campus Pinkafeld since 2005. It is an open learning management
system which serves as a knowledge and collaboration platform and supports a variety of uses and
methods and allows to efficiently create learning and teaching materials by both teachers and
students. As ILIAS is versatile, some more Austrian universities deploy it as a campus-wide blended
learning system in which online modules and courses complement classes (e. g. Pedagogical
University Vorarlberg and University of Applied Sciences Upper Austria). For further information
please got to http://www.ilias.de.
In order to fully utilize the advantages of ILIAS workshops to help lecturers use the online platform in
a meaningful and efficient way are offered at the beginning of each term. Besides, all lecturers of the
technical part-time degree programmes are encouraged to share resources and materials in an open
manner. For this purpose online workspaces particularly designed for the lecturers of technical parttime
programmes were designed which allow communication, exchange of information, and
collaboration between all members of the teaching staff.
4. Teacher and student training
Technology itself, however, doesn’t mean more efficient teaching and improved learning. Effective
blended learning settings proved to require a lot of commitment of the teaching staff and a lot more
364

Andrea Kelz
self-discipline and motivation of students than traditional face-to-face teaching. Moreover, not all
lecturers and students are convinced of a constructivist-based blended learning framework from the
start. The major cause lies in our partly still traditional educational system which emphasizes content
memorization and individual student performance. Particularly older lecturers and part-time students
in their thirties have hardly been exposed to web-based constructivist and collaborative teaching and
learning techniques which are essential components of our blended learning approach. They haven’t
been trained in constructivist and collaborative methods and so the pencil- and paper-based lecture
method is still the method of their choice. From the teachers’ point of view the main reasons for
concern have been that they fear loss of control in the lecture if they give more responsibility to the
students for their learning progress. It takes a great deal of self-confidence in oneself and one’s
students to transfer the responsibility of learning to the student. Some also fear the loss of content
coverage because allowing group interaction often takes longer than giving mere lectures. There have
also been concerns when it comes to assessing students’ efforts. Some teachers are unfamiliar with
continuous assessment techniques or how to assess group efforts. Another major cause for concern
lies in some students’ resistance to collaborative learning techniques. They also still need to become
comfortable with new methods of teaching and learning and express concerns and doubts in the
beginning. Particularly collaborative assignments are initially viewed critically. Students often express
concerns about lacks of contribution of fellow students of the same group or complain about
interpersonal conflicts they have to deal with. As they are not likely to change their attitudes to
learning from one course to another they also gradually need to become acquainted with
constructivist and collaborative learning techniques.
As indicated above, both students and teachers feel that they are in a new learning situation. With
regard to student needs the successful implementation of blended learning designs include knowing
about their own learning styles and needs and helping them improve on how to learn successfully,
i. e. helping them develop a repertoire of learning skills and strategies to support their development as
learners in virtual learning environments. In order to enable technical part-time students to engage
actively in the management and design of their own learning a blended learning course was designed
to make them reflect on their learning styles and processes, help them learn how to learn, enhance
their creative and innovative thinking and eventually improve on their overall performance. In this
course the LMS provided as well as other open web-resources are used for individual and
collaborative e-learning experiences to make them acquainted with various learning and teaching
styles they will be exposed to during their studies.
As part of the teachers are also not prepared for blended learning settings, particularly those who
haven’t had any e-teaching or e-learning experiences themselves, they also need support. Due to the
lack of first-hand experiences some of them try to teach in the same old way simply utilising some
technologies and calling it blended learning or e-learning. They now have to face the challenge of
having to update their knowledge and skills to be able to make appropriate use of ICT in their courses
and learn how to utilise virtual learning environments in ways which best meet the needs of their
students as well as the current challenges of higher education such as the move towards lifelong
learning with its demand for continuous professional development.
The question arises of what it actually is that the teacher needs to do to make a success of blended
learning settings. What competencies and skills are required to teach in a virtual learning environment
effectively and how will the teachers eventually make the change? Smith identifies and describes 51
instructor competencies needed by online instructors and outlines the “key components of a training
program to enable a traditional brick-and-mortar college to transition to a university that has a
significant online component” (Smith 2005). He outlines a variety of competencies needed prior to the
start of a course, competencies needed during the course and competencies needed after the course.
To sum up roughly, the strategy for training our teachers at Campus Pinkafeld also implies that in a
successful blended learning setting the teacher must have sufficient ICT, pedagogical and social
skills. The particular skills required are being able to use the technology (LMS) provided, skills to
design and implement online activities, organization and moderation skills, being able to integrate
different teaching and learning styles, interact actively and give students constant feedback.
But how are teachers supposed to modify their practice to work in this new environment? First of all,
all teachers are advised to start simple and move towards the more complex. These gradual
enhancements range from supporting students via e-mail, moderating online discussion forums,
uploading files onto the platform, providing further materials and resources on the web, to integrating
365

Andrea Kelz
web-based activities into the course, developing and implementing online collaborative activities and
authoring online course materials. In order to do this they need theoretical and practical training,
which is provided by the campus. Teacher training workshops are conducted on a regular basis by
our blended learning administrator and are supposed to help teachers update their ICT skills and
improve their blended learning teaching designs, explore didactic strategies with their peers and
generate new ideas and ways to innovate their teaching.
Finally, for all courses taught at Campus Pinkafeld the teachers are required to provide a course
outline which was designed to help them with the preparation and delivery of the courses. These
course outlines do not only document the curriculum at the course level, they also support the
teaching and learning process by identifying specific course objectives, contents, activities, resources,
and evaluation methods. The outline also ensures that the purpose of the course is clear not only to
the student but also to the teacher as it involves asking questions about student requirements, course
objectives and its context within the overall curriculum. Particularly working with part-time students
who mostly work alongside their studies and thus often have extensive work experience requires
asking questions such as what knowledge and skills do the students already bring to the course and
what do learners actually still need to learn or be able to do in order to succeed in their future work
environments. In addition, it is important to think about the teaching resources already available and
blended learning material and activities which still need to be developed. Brainstorming about such
questions has proved to help set the priorities for course planning and delivery.
Hence, prior to each course, the teachers are provided a course outline form which he/she is
supposed to fill in and upload on the platform for everyone to see. The outline already provides
general information on the course such as its number, title, name of the instructor and contact
information, main course contents according to the curriculum, instructional method such as lecture,
seminar, laboratory etc., and student workload. The instructor is required to add the following
information:
Short description of overall course objective/learning outcomes: statement of the knowledge, skills
and competencies the students are expected to have acquired by the end of the course.
List of required and recommended literature.
Information on each face-to-face session as well as blended learning activities including short
description of learning objectives, contents covered, learning material provided, description of
required student tasks with due date and submission mode.
Detailed description of assessment mode: statement of what percentage of the final grade each
assignment and exam will represent.
5. Student evaluation of blended learning activities
All course evaluations are carried out by means of EvaSys at the end of each course. EvaSys allows
for student feedback of the different types of part-time courses (lectures, seminars, laboratory
exercises etc.) and their blended learning activities. For further information on EvaSys please go to
http://www.electricpaper.biz/products/evasys-education.html. For each course the students fill in a
course questionnaire evaluating student satisfaction in the following areas: course administration,
course design, course contents, teacher, blended learning activities and overall impression of the
course.
With regard to blended learning modules there are three questions to be answered:
There has been sufficient time to work on the blended learning assignments.
The blended learning activities have been supported sufficiently by the teacher.
The learning management system has been used efficiently by the teacher.
The possible ratings range from ‘1’ (fully agree) to ‘5’ (completely disagree). The institutional results
provided by the head the bachelor’s degree programme in energy and environmental management
and the master’s degree programme in building technology and management are as follows:
The tables show that there have been overall continuous improvements of the ratings over the years.
The ratings of the bachelor’s degree students, however, are a bit weaker than those of the master’s
degree students. One reason might be that the master’s students, most of whom completed the
366

Andrea Kelz
bachelor’s programme in energy and environmental management at Campus Pinkafeld, are already
more familiar with our blended learning approach. The same holds true for the teachers as some of
them also teach in the bachelor’s degree programme. The ratings are quite good but there still
remains room for improvement, particularly when it comes to designing collaborative, web-based
activities and making more effective use the learning management system in terms of creating
interactive learning materials. Moreover, there will have to be some sort of qualitative assessment of
blended learning materials and activities on an institutional level.
Table 1: Ratings of bachelor’s degree programme in energy and environmental management
Average student ratings of blended learning modules (all semesters/135 students)
Winter Term Summer Term Winter Term Summer Term
2008/2009 2009
2009/2010
2010
Sufficient time to work on the
blended learning assignments
1.65 1.42 1.43 1.40
Sufficient monitoring and
support by the teacher
1.68 1.51 1.47 1.40
Efficient use of the learning
management system (ILIAS)
1.76 1.60 1.54 1.50
Table 2: Ratings of master’s degree programme in building technology and management
Average student ratings of blended learning modules (all semesters/60 students)
Sufficient time to work on the
blended learning assignments
Sufficient monitoring and
support by the teacher
Efficient use of the learning
management system (ILIAS)
Winter Term Summer Term Winter Term Summer Term
2008/2009 2009
2009/2010
2010
1.53 1.25 1.29 1.20
1.43 1.32 1.21 1.10
1.54 1.35 1.35 1.30
6. Nationwide training and initiatives
About ten years ago a nationwide initiative was started by the Austrian government to expand the use
of e-learning and blended learning at Austrian universities. The programme “New Media in Teaching
at Universities and Universities of Applied Sciences” was started and provided the basis for a cooperation
of all Austrian universities in an association called “Forum Neue Medien Austria” (Forum
New Media Austria). This forum sees itself as a lobbyist for innovation in teaching in higher education.
Their interactive information and communication platform, conferences and newsletters encourage
scientific discourse and are open to everybody who is interested in e-learning. At present the network
of the association encompasses 21 Austrian universities and 16 universities of applied sciences. This
association has been working on general ICT policies and e-learning issues, especially issues of
quality management, legal questions in e-learning, e-learning and career development and e-content
development. For further information please go to http://www.fnm-austria.at. Furthermore, in 2005 the
Austrian ministry of education developed a framework paper for an e-learning strategy and funded
projects of universities which aimed at developing their individual institutional e-learning and eteaching
strategies. Within this initiative nine Austrian universities of applied sciences developed
components for implementing e-learning-strategies in a cooperative way.
Meanwhile all Austrian higher education institutions provide some sort of virtual learning environments
to support and facilitate learning. There are staff members assisting and advising lecturers in
didactical matters as well as in the technical implementation of e-learning and blended learning
concepts and scenarios. They provide support and development in areas such as the didactical
design and production of online learning materials, diagrams and animations, the programming of
interactive learning software, the production of training videos and automated self-control questions
with feedback, as well as the continual adjustment of the e-learning and blended learning courses and
the learning management system in use. As far as learning management systems are concerned,
only recently a web-survey has been conducted, trying to find out what platforms are used at different
universities of applied sciences in Austria. The survey obtained 16 replies. The results so far provided
to the responding universities/campuses show that 10 out of 16 universities of applied sciences in
Austria use Moodle to support online learning. One of the reasons of course is that the Austrian
ministry of education supported the installation and service of this open source learning management
367

Andrea Kelz
system. Most other universities taking part in the survey also use open source solutions such as ILIAS
or some kind of combination of in-house and open source learning management systems.
While competence building in e-learning and blended learning at universities still mostly takes place
autonomously, the following universities are jointly offering a nationwide “E-Learning Certificate”: the
Universities of Innsbruck, Salzburg, Linz, Graz and Klagenfurt; the University of Business
Administration Vienna, the Medical University Graz, the University of Arts in Linz, the Universities of
Applied Sciences of Upper Austria, Salzburg, Vorarlberg and Kufstein, the University College Tyrol,
the Education Service Tyrol and the Management Centre Innsbruck. The course programme consists
of three modules which are offered at beginners’ and advanced levels. Module 1 comprises media
didactics in teaching at universities (use of media in teaching, installation and use of e-learning
courses in LMS systems and collaborative e-learning), module 2 focuses on media production (digital
photography, e-learning and law, e-learning and web 2.0) and module 3 covers questions such as
media organisation and e-moderation. If participants have completed all three modules at a minimum
of two course providers within 18 months they can apply for an e-learning certificate. The certificate
course is understood as a professional development instrument and is offered free of charge. For
further information please go to http://www.zertifikat-elearning.at/?home=1.
Finally, the Austrian Federal Ministry of Education, Arts and Culture defined the pedagogical and
didactical use of IT (especially EPICT) as one of several innovative and integrative principles. Taking
account of the principle is demanded in all areas of further education and training and supported by
specific initiatives. EPICT – the European Pedagogical ICT Licence – is a comprehensive, flexible and
efficient in-service training course introducing a European quality standard for the continued
professional development of teachers in the pedagogical integration of information, media and
communication technologies (ICT) in education. The European Pedagogical ICT Licence can be
acquired after successful completion of a minimum of eight modules of which four are compulsory and
four can be chosen out of ten optional modules. The obligatory modules focus on understanding the
pedagogical role of ICT in teaching, acquiring basic ICT skills, searching and communicating with
internet tools, creating and sharing content, and understanding the requirements of bringing
innovation to schools and universities. For further information please go to http://www.epict.org/.
7. Conclusion
We have been witnessing a paradigm change in higher education for quite some time now and like in
most other universities ICT has become integrated into all courses of our degree programmes.
However, as ICT needs to be more than the technology itself and the content it carries, it is not
sufficient to provide technical support and technical training to both students and teachers. As
students and teachers act in both face-to-face and virtual classroom situations they have to build new
learning and teaching concepts. Students need to become more autonomous learners and teachers
need to develop the skills to teach blended learning courses effectively.
In view of the importance of a successful long-term, campus-wide, blended learning strategy we
constantly have to review and analyse the development of our own blended learning practices. The
factors critical to the success of blended learning approaches are definitely sufficient training and
continual support of the concerned students, teachers and administrators at all levels. Students and
teachers do not necessarily mind change but being made to change and become blended learning
students and teachers in a new learning and teaching environment turned out to be quite a challenge.
So far there has been no direct reward or motivation for members of the teaching staff to make the
effort to incorporate ICT and blended learning activities into their courses. The primary incentives
have been intrinsic rewards such as personal professional development in terms of teaching
techniques, positive student feedback and peer recognition. The main problems which still need to be
solved don’t concern the implementation of a blended learning approach in terms of ICT, changing
learning environments and roles of the students and teachers, but the impact of blended learning on
teacher workload. Compared to traditional teaching blended learning designs involve more time to
develop activities and proper course materials. So the next question to be tackled is how the teachers’
workload can be measured effectively when it comes to the preparation of online materials and
blended learning activities. With many teachers there’s still a sort of unwillingness to experiment with
innovation and the possibilities the use of ICT offers as they do not only lack time but also some sort
of extrinsic motivation. We have to find a way to encourage more faculty members and freelancers to
develop web-based activities. Financial incentives are being discussed at the moment, but so far we
haven’t found any real solution to the problem.
368

Andrea Kelz
Within the last seven years it has proved that the use of ICT requires a continuous process of
development of competencies online teachers should have and demands professional preparation
and proper pedagogical and didactical training. Face-to-face teaching has remained central in our still
campus-based institution and blended learning is mainly received as distance learning, enriched by
ICT, supplementing on-campus degree programmes. In future even more emphasis will have to be
put on the pedagogy and didactics of blended learning as some teachers unfortunately still think that it
is possible to simply transfer their teaching material and activities used in the classroom to the web.
Many teachers still have to learn how to cope with the role of becoming facilitators in the learning
process and have to realize that a learning management system is more than a digital library to store
and provide learning materials. Only then they will stop to wonder why no one uses the discussion
forums they provided in their courses.
References
Apel, Heino et al (2003) Online Lehren. Planung und Gestaltung netzbasierter Weiterbildung, Bertelsmann,
Bielefeld.
EvaSys – Education Survey Automation Suite (2011) [online], Electric Paper Official Website,
http://www.electricpaper.biz/products/evasys-education.html.
European Pedagogical ICT Licence (2011) [online], EPICT Official Website, http://www.epict.org/concept.shtml.
Forum Neue Medien Austria (2011) [online], FNM Official Website, http://www.fnm-austria.at/Home/Home/.
Garrison, D. Randy and Vaughan, Norman D. (2008) Blended Learning in Higher Education: Framework,
Principles, and Guidelines, John Wiley & Sons, San Francisco.
Gierke, Christiane et al (2003) Vom Trainer zum E-Trainer. Neue Chancen für den Trainer von morgen, GABAL,
Offenbach.
ILIAS Open Source e-Learning (2011) [online], ILIAS Official Website,
http://www.ilias.de/docu/goto_docu_cat_580.html.
Kelz, A. (2008) “E-Learning Strategies in Technical Part-Time Studies – Constructivist and Collaborative
Approaches to Learning and Teaching”, 7 th European Conference on e-Learning, Volume 1, APL, Reading,
pp 557-563.
Reinmann-Rothmaier, Gabi (2003) Didaktische Innovation durch Blended Learning. Leitlinien anhand eines
Beispiels aus der Hochschule. Huber Psychologie Praxis. Lernen mit neuen Medien, Verlag Hans Huber,
Bern.
Smith, T. C. (2005) “Fifty-One Competencies for Online Instruction”, [online], The Journal of Educators Online,
Volume 2, Number 2, July 2005, http://www.thejeo.com/Ted%20Smith%20Final.pdf.
Walitsch, M. (2008) “Successful Blended Learning Concept for Non-Traditional University Access in Austria”,
Proceedings of the IADIS International Conference on e-Learning 2009, Algarve, Portugal 17-20 June 2009,
pp 253-256.
Zertifikat eLearning (2011) [online], Official Website, http://www.zertifikat-elearning.at/?home=1.
369

Open Courses: The Next big Thing in eLearning?
Kaido Kikkas 1, 2 , Mart Laanpere 2 and Hans Põldoja 2
1 Estonian Information Technology College, Tallinn, Estonia
2 Institute of Informatics, Tallinn University, Tallinn, Estonia
kaido.kikkas@itcollege.ee
mart.laanpere@tlu.ee
hans.poldoja@tlu.ee
Abstract: During the last 15 years, eLearning has undergone a number of changes regarding openness of the
learning environment, learning resources as well as the process of teaching and learning. After the initial period,
when eLearning used mostly the tools of 'ordinary' Web, the first-generation of eLearning emerged – large,
mostly proprietary environments which firmly separated the chosen (students and tutors) from the 'barbarians at
the gates' (the rest of the Internet) by using accounts and passwords. The tools themselves were shaped by
creators, not users – due to closed source and restrictive licenses. WebCT and early Blackboard were prime
examples of this generation. The second generation of eLearning rode the wave of free and open-source
software, bringing along a much greater variety of tools as the environments became open for modifications (e.g.
Moodle, Ilias). This generation also emerged along with the rise of open content (powered by free licenses like
the Creative Commons family) which in turn established the Open Educational Resources (OER) movement,
examples of which are Connexions, LeMill, MIT OpenCourseWare and others. We propose that the emergence
of Personal Learning Environments combined with collaborative wikis signify the next stage of eLearning. Open
in both the environment and the process, it facilitates a more flexible and also more challenging model of
learning. Massive Open Online Courses (MOOC) represent the radical conception of openness in eLearning, as
all MOOC courses are open for enrolment for any Internet user. We started to use Wikiversity and personal blogbased
learning environments in 2008. This paper provides a summary of our experience with teaching 18 open
courses in 2008-2011, with more than 560 enrolled students. We analyse the design, learning process and
learning outcomes of these open courses using the knowledge building theory by Bereiter as our main frame of
reference and the framework analysis of the courses.
Keywords: open courses, personal learning environment, wiki, blog
1. eLearning: a glance on history
During the last 15 years, the mainstream practices of eLearning have undergone a number of
changes with regard to openness of the learning environment, learning resources as well as the
process of teaching and learning.
Leinonen (2005) sees the main phases of using computers in learning. Various other authors have
suggested slightly different versions of the timeline, but it seems to be rather close to the current
consensus. However, one of the main roots of eLearning – an orderly, top-down approach rather than
a distributed, community-based activity – is clearly visible here. This is likely the reason why the first-
generation eLearning systems shared the same mindset.
Thus after the initial starting period, when eLearning used mostly the tools of 'ordinary' Web, the firstgeneration
eLearning emerged – large, mostly commercial and proprietary environments which firmly
separated the chosen (students and tutors) from the 'barbarians at the gates' (the rest of the Internet)
by using accounts and passwords. The tools themselves were shaped by creators, not users – due to
closed source and restrictive licenses. The popular learning management systems (LMS) like WebCT
and early Blackboard were prime examples of this generation.
As Leinonen (2005) points out, these early systems inherited the mindset of the previous, computer-
based training phase – the same top-down, teacher-to-student, low-interaction approach. Even
though the later LMS's became more aware that learning requires social activities among the learners
themselves and the learner and the teachers, these were often added as an afterthought.
As first pointed out by David Wiley in 1998 (OECD 2007), the second generation of eLearning rode
the wave of free and open-source software (FOSS). Wiley concluded that a lot of principles that
applied to software were also applicable to other online content, resulting in the birth of the first free
content license in OPL (Open Publication License) and the subsequent Open Content movement
which in turn gave rise to the wider Free Culture movement as described by Lessig (2004).
370

Kaido Kikkas et al.
The term 'open educational resources' (OER) was born at the beginning of the 21st century, along
with the establishment of Creative Commons (the organisation and their novel system of content
licensing) in 2001 as well as the decision of the Massachusetts Institute of Technology (backed by the
Hewlett Foundation and the Mellon Foundation; OECD 2007) to make their course materials
available online. The term 'OER' was first used the following year (Johnstone 2005).
On the one hand, the OER movement had notable influence on LMS's, promoting the next generation
of eLearning tools where the environment was also open for modifications (e.g. Moodle, IVA, Ilias).
On the other hand, it gave rise to dedicated online repositories like Connexions, LeMill, MIT
OpenCourseWare and others. Still, while the technical, environmental side became more distributed
and open, the methodological side widely remained top-down and restricted (e.g. even if a FOSS
environment like Moodle was used, the course in it was still locked to outsiders).
During the recent years, the emergence of Personal Learning Environments combined with
collaborative wikis signify the next stage of eLearning. Open in both the environment and the process,
it facilitates more flexible and also more challenging model of learning. As described by Fini (2009),
Massive Open Online Courses (MOOC) represent the radical conception of openness in eLearning,
as all MOOC courses are open for enrolment for any Internet user.
2. Open courses
While the most visible outcome of the Open Education movement are numerous OER repositories
and OpenCourseWare initiatives we must point out that open education has a much wider meaning
than just openly available educational content. According to the Cape Town Open Education
Declaration (2007), “open education is not limited to just open educational resources. It also draws
upon open technologies that facilitate collaborative, flexible learning and the open sharing of teaching
practices that empower educators to benefit from the best ideas of their colleagues. It may also grow
to include new approaches to assessment, accreditation and collaborative learning.”
In recent years there has been a rising discussion about open educational practices (Ehlers, 2011).
Ehlers defines open educational practices as “practices which support the (re)use and production of
OER through institutional policies, promote innovative pedagogical models, and respect and empower
learners as co-producers on their lifelong learning path.” Online courses with open enrollment could
be seen as an example of open educational practices.
The first widely known open course was Introduction to Open Education course that was organized in
autumn term 2007 by David Wiley (Wiley, 2007). This course inspired several other educators to open
up their courses with free enrollment. In spring term 2008 Teemu Leinonen and Hans Põldoja
organized an open course about composing free and open online educational resources. This course
was special because Wikiversity was used for running the course (Leinonen, Vaden and Suoranta,
2009). Both of these courses were official university courses that were made open for external
participants. The requirements for institutionally enrolled students and informally participating students
were the same, but only institutionally enrolled students received credits for the participation.
Most well-known open courses have been organized by George Siemens and Stephen Downes. Their
courses include Connectivism and Connective Knowledge (CCK08, CCK09, CCK11), Personal
Learning Environments Networks and Knowledge (PLENK2010), and Learning and Knowledge
Analytics (LAK11). These courses had less formal requirements for participation. The participants
were free to choose to what extent they will participate in the course activities. This resulted a large
number of enrolled participants from which only a small part were active. About 2200 participants
enrolled to CCK08 course, about 700 to CCK09 and about 1700 to PLENK2010 (Downes, 2011).
Because of these large numbers the participants started to call these courses massive open online
courses (MOOC) (Fini, 2009).
There are other examples of open courses where the number of participants is limited. P2PU is an
initiative that runs short 6 week online courses that have typically about 20 seats (P2PU, 2011). Small
number of motivated participants guarantees that they will all receive feedback from the facilitator and
their peers.
371

Kaido Kikkas et al.
However, in the Estonian context we can not speak about “massive open online courses” because the
number of non-credit participants that enroll to the courses is typically rather small. Therefore we will
use the term “open course” in this paper.
3. The Wikiversity experience
In our case we have used Wikiversity to provide open courses. Wikiversity is one of the projects under
Wikimedia Foundation that also governs Wikipedia. It is somewhat less structured than Wikipedia,
supporting learning communities, their learning resources and learning activities. In our courses
Wikiversity is typically used for course syllabus, participant enrolment and learning resources. All the
participants have their personal blogs to submit their course assignments. This kind of use of wikis
and blogs is inspired by open courses organized by David Wiley (2007) and Teemu Leinonen
(Leinonen, Vaden and Suoranta, 2009).
We claim that in our daily teaching practice, open courses have evolved beyond the initial
experimentation phase and reached the high level of scalability. From 2009 to 2011, we have run
(separately from each other) more than 20 full-scale open courses in three different higher education
institutions in Estonia:
Social Software and Network Communities (SSNC), 4 ECTS M.Sc. course with 226 participants
(total for all years).
Ethics and Law in New Media (ELNM), 5 ECTS international M.Sc. course with 44 participants.
Ethical, Social and Professional Aspects of Information Technology (ESPAIT), a 4 ECTS
introductory-level B.Sc. course with 161 participants
Standards and Specifications for eLearning Tools (SSET), 4 ECTS M.A. course with 20
participants
Learning Environments and Learning Networks (LELN), 3 ECTS M.A. course with 28 participants.
Creating Digital Learning Resources (CDLR), 3 ECTS M.A. course with 29 participants
Working with Sources, Referencing and Reference Management in ELearning (WSRRME), a 10 -
week, 100% online professional development course for teachers and educational technologists,
38 participants.
Creating and Using Open Educational Resources (CUOER), a 10-week, 100% online course for
teachers and educational technologists, 31 participants.
Didactics of Informatics, a 4 ECTS M.A. course with 42 participants.
Basics of Instructional Design, a 3 ECTS M.A. course with 45 participants.
Active learning in Informatics Lessons, a 3 ECTS M.A. course with 26 participants.
Professional Development with ePortfolio, a 12-week. 100% online professional development
course for teachers, 12 participants
M-Learning, 12 week long 100% online professional development course for teachers, 10
participants
As Wikiversity-based courses can be run in a variety of ways, the brief descriptions of two possible
approaches used by us wll follow below.
3.1 Approach 1: Wiki-centred course designs
Each student was expected to have a blog (RSS feed for posts was required – both free services like
Blogger or WordPress and standalone installations were allowed), a Skype account and an e-mail
address. Also at the SSNC and ELNM courses, they also needed a wiki space (either using a free
service like Wikidot.com or setting up their own) for teams of 4-5.
372

Kaido Kikkas et al.
A typical task set included weekly 'lecture' texts (1 or 2 per week; available from the course page at
Wikiversity), a weekly blogging task at the end of the lecture text (to be completed by the next
Monday noon) and a weekly Skype text chat meeting (at SSNC and ELNM; lasting typically 1-1.5
hours). In addition, everyone was to post to the course forum following the weekly topic, but this task
did not have fixed weekly regimen. Finally, the SSNC and ELNM had to write a wiki-based group
paper and review another group's work (as an individual blog post).
The SSNC and ELNM courses lacked formal examination, the ESPAIT featured an optional written
exam (a 'backup' option for less active people or also an extra way to raise the grade – but it was
possible to earn the highest grade without it). Thus, the SSNC and ELNM did not have the 'end rush'
at all, while at the ESPAIT it was much milder than is typically seen at traditional courses.
The exact graduation systems varied somewhat by course, but a typical one consisted of:
40% - the wiki-based group paper (or in the case of ESPAIT, a written exam essay)
35% - blogging tasks (typically compact pieces of opinion, ~1 page each)
25% - forum posts (of substance – the 'me too!'-type and offtopic were typically not counted in)
10% - weekly Skype chats (for consulting as well as community building).
Note: the course point system featured a ~10% surplus, designed to give students some choice.
3.2 Approach 2: Blog-centred course designs
These courses were organized as blogging seminars with weekly topics and assignments. New
assignments were given every Monday, the students had to write a blog post by the end of the week.
In addition to Wikiversity, a course blog was used in all courses. Wikiversity was used for course
syllabus and enrolments while weekly assignments and summaries were published in the course
blog.
The 10-week online courses WSRRME and CUOER included a video conference during the first and
the last week of the course. Weeks 5 and 10 were left without assignments so that people could
complete the assignments they had not managed to do in time. In case of Master level courses there
were also 4...6 classroom meetings during the course. Besides individual blogging, the Master level
courses required substantial group work. Students typically used a shared blog, a wiki or Google
Docs to complete the group assignment.
To emphasize openness, the learning environment consisted mostly of tools that were open source or
provided with free hosting. The course blog was hosted at WordPress.com, the learning resources
were published at LeMill and SlideShare and links were shared through Delicious. In the CUOER
course also Skype was used for text chat. In 2010 it was decided to use Twitter instead of Skype for
rapid communication. However, in Estonian context Twitter is not still actively used by the educational
community. Another service that was added in 2010 was Mendeley, used to share references to
research papers. The only closed tool used during the courses was FlashMeeting that was used for
video conferences in the WSRRME and CUOER courses.
A complicated issue in open courses is to manage the list of participants and to aggregate all their
blog posts. In the first courses, Wikiversity was used for course enrolments and blog posts were
aggregated using Pageflakes. This required a lot of manual work from the facilitator. That led us to
design and develop the EduFeedr tool that handles course enrolments, manages assignments and
aggregates all the blog posts (Põldoja, 2010). Since 2010 EduFeedr has been used for managing the
courses and Wikiversity is used only for course syllabus that can be collaboratively edited.
4. Discussion
To start with, we have to agree with Seely Brown and Adler (2008) in that the world is becoming both
flat
(in that the virtual communication has eliminated many obstacles by reaching almost any place in the
world) and spiky (in that the local level of productivity and innovation will determine the success of a
location, and the differences will increase). Thus, it is necessary to not only supply the next
373

Kaido Kikkas et al.
generation of workforce but also do it dynamically and provide flexible environment to facilitate it
(various authors like Benkler, Barnes, Himanen, Theobald and others also point to that direction).
We used the Ritchie and Spencer (1994) framework analysis method to study the feedback gathered
from the participants of open courses. The data originates from two sources - transcripts of the weekly
chat sessions and the reflections on learning experiences collected from ELNM and SSNC students
after the end of the course in 2009 and 2010. After familiarizing ourselves with the collected data, we
identified five emerging themes. Eventually, we indexed and charted the sections of the data using
EverNote. These five themes are listed below with some samples of data belonging to each theme.
1. Enlightening experience: open course as a new way to learn together
The first aspect clearly visible at the courses was their novel form for students. While a number of
them were familiar with eLearning, almost nobody had any previous experience with open courses.
This resulted in technical questions (“"Are the blogs bundled up somewhere?"), some doubts in the
overall process ("Do we have to be constructive? :P") as well as problems with time management
probably familiar from all e-courses (“What was the deadline? Sorry, I was in Finland last week").
However, the overall feedback reflects the courses being successful. The initial awkwardness ("This
course is my nightmare :)") was replaced by joy ("Loved it!” - the same person after graduation). The
feedback evaluated the courses very well (e.g. the SSNC 2010 received 92 points out of 100) and
also suggested good overall quality of the course ( "And as educational technologist, I now know why
some courses get the quality certificate and some don't :)”, "I enjoyed this course a lot, probably one
of the best this semester. Thanks, Kaido, and hope to take more of your courses next semesters.").
Another point was the ability of the facilitators to provide continuous feedback ("Thanks for the
immense work you did in weekly grading of tasks. It was an excellent course.").
2. The culture of sharing: daring to write in public
Writing in public was not very easy to adopt either - reasons were various like authorship concerns (“I
firmly believe that one's intellectual property is an asset and should be addressed as one”), lack of
time (“Blogging proved to be very difficult with such a hectic schedule"), organisational issues ("Are
references and such necessary?", “Do you REALLY read ALL the blog posts?") and difficulties in
formulating one’s opinion in writing (“The topics themselves are interesting enough and the main
obstacle may be how to stop thinking and just to write something in short words.").
However, several people pointed out that the publicity in fact works as a good method for quality
assurance, stimulating them to produce quality output ("My friend asked me about that mysterious
course which makes me write such interesting things in my blog.").
3. Real-time chat as a community tool: what it gives and what it takes
The weekly real-time chat sessions (we mostly used Skype in text mode) was received well by the
participants ("Thanks folks! It was fun. I'll look forward to the next chat") - among the outlined features
were immediateness and interactivity compared to other componentes of the course (“What are we
discussing? Or isn't it such a structured conversation at all?") as well as the wide range of topics
discussed (“"One of the best courses. and that’s because I expected those Wednesdays.a bit scary
and thrilling at same time and so interesting!"), although there were questions about privacy ("Will we
really have discussions that we won't dare to share? The course topic would imply openness") or
simply the need to be active in discussion (“Will we get credit for speaking or just being present?”).
4. Community gravity: nurturing the social ties within the group
The community gravity which measures how strongly a user might be attracted to a community was
first researched by Matsuo and Yamamoto (2009). We find the concept to be applicable also in the
context of eLearning (see also Väljataga and Põldoja, 2011). We suggest that open courses do
possess a good potential for strong community gravity.. Of the factors influencing the gravity, relaxed
atmosphere and a modest level of formality (“"Sorry, it is easier and more interesting than I thought"),
informativeness (not strictly limited to the course programme ("Thanks - I'd have never heard about
Pekka Himanen without this course", but also "So, let's start with Angelina Jolie and her part in "The
374

Kaido Kikkas et al.
Hackers""), changes in requirements when dictated by circumstances ("if we would drop one task,
which one should it be?") and a healthy dose of humour ("... Everything has fallen so silent... doo-doodoo"
- "Nii vaikseks kõik on jäänud", a popular Estonian song; a remark on chat inactivity). Plus,
another notion shared with the next category is the low level of hierarchy (see below).
5. Facilitation: changing the role of teacher
Open courses tend to have facilitators enabling the learning process rather than professors teaching.
This was also witnessed at our courses where most tasks were analytical in nature and without a
predetermined ‘correct’ answer. Rather than demanding the ‘party line’ in thinking, sometimes a direct
(albeit well-argumented) opposition to the facilitator was graded above the like-mindedness (“I totally
disagree, but that's OK :)"). This resulted in sharp remarks (“The happiest people are those who
reside in infirmaries...", "Internet is the reflection of the real world - perhaps just a bit more cruel"),
some really down-to-earth approaches to sensitive topics ("I just tested out Net Nanny (a web
censorship software). Was still able to surf some porn and book a couple of hookers.") and a range of
thought-provoking questions presented to the peers ("What can a parent do if his/her child is
cyberbullied when he/she knows nothing about it?", "About today's topic, people what do you think is
a hacker a modern anarchist?" etc). However, the overall reaction was again very positive ("I'm
grateful for the course, it taught another angle to look at the world, introduced problems which are not
met in everyday life and forced us every week to think and write about the world.").
Our findings from the Wikiversity courses were the following:
The open content model functions as an effective quality assurance system – students are
motivated to produce quality work because of the possibility of immediate (both in-course and
from outside) peer review. Other fields of Free Culture (e.g. free and open-source software, as
suggested by Mockus et al 2002) tend to share the phenomenon.
Extra students joining from outside should be treated as a possibility, not a liability. The main
point here is to design the course in such a way that substantial increase in student numbers
would increase the supervisor's workload the least possible amount. One option is to make
distinction between for-credit and non-credit students. For-credit students have to complete all the
assignments and will get university credits for the course. Non-credit students participate the
course in various less formal ways and do not have to complete all the assignments. They will
receive feedback from other participants but the facilitator will not take responsibility to give them
the same amount of feedback that is given to for-credit students. This model is used by Alec
Couros (2010).
The model is especially suitable for the fields where rapid changes in the course materials are
needed in order to follow the state of the art (new media, information technology). As evident from
the history of Wikipedia, wiki as a media form is very flexible and can follow changes well. The
experience suggests that adequately supervised Wikiversity courses can have attrition rates far
less than conventional courses – especially if such a method is new for students. This has one
condition – the structure of the course and the tasks must be clear from the beginning, making a
very well laid-out and written course guide a must.
Community management and also conflict solving skills are very valuable – especially in
multicultural courses or when a large share of participants have other first languages. In some of
our courses, the supervisor was very thankful for having previous experience as a chatroom
administrator.
While it is possible to facilitate large courses in a single chat (SSNC 2010 made likely a record
with weekly Skype text chats involving ~140 people), it is advised to split large groups for chat. If it
is impossible, stricter chat etiquette must be maintained to minimise noise, a simple helpful tip is
to make the supervisor type with Caps Lock turned on (even if it is usually interpreted as shouting
and therefore discouraged).
For best results, the homework should contain tasks with varying response cycle (e.g. wiki-based
group paper vs weekly Skype chat session – ample thinking room vs 'on the toes' situation). Also,
some people enjoy writing large papers, some like short opinions or direct discussion – this is
where the surplus in total points counts.
375

Kaido Kikkas et al.
Occasional guests participating in the weekly chat (1-2 per course) are very stimulating for the
sense of community (e.g. ELNM 2009 had a lawyer and a Microsoft specialist) Creating a
dedicated learning community rather than stressing a teacher-student dichotomy is crucial to
success. Our experience shows that the weekly chat sessions are crucial here – these should not
be neither just formal exchanges or just chit-chats, but a mixture of both with a strong accent in
the 'we'-aspect, or community building. Also, these sessions, when directed skillfully, can have a
large impact on students' self-study motivation and skills. A good enough proof was the end of the
SSNC 2009 course, with students grumbling during the last chat session: 'What the heck will we
do on Wednesday nights from now on?'.
5. Conclusions
In this study, we applied action research approach and framework analysis method to make sense of
our experiences with open courses in 2008-2011 (more than 20 courses at three different higher
education facilities in Estonia). Our analysis shows that open courses have evolved beyond the phase
of experimentation by few enthusiasts and are a likely candidate to be the next major step in the
evolution of eLearning.
Acknowledgements
This research was funded by Estonian Ministry of Education and Research targeted research grant
No. 0130159s08.
References
Brown, S.J. and Adler, R.P. (2008) ‘Minds on Fire: Open Education, the Long Tail, and Learning 2.0.’,
EDUCAUSE Review, vol 43, no 1, January/February, pp. 16-32.
Couros, A., (2010) ‘Developing Personal Learning Networks for Open and Social Learning’, in Veletsianos, G.
(ed.) Emerging Technologies in Distance Education. Athabasca: Athabasca University Press, pp. 109-128.
Downes, S. (2011) Connectivism and Connective Knowledge, [Online], Available:
http://www.huffingtonpost.com/stephen-downes/connectivism-and-connecti_b_804653.html [9 Jun 2011].
Ehlers, U., (2011) ‘From Open Educational Resources to Open Educational Practices’, eLearning Papers, no 23,
March, pp.1–8.
Fini, A. (2009) ‘The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course
Tools’, International Review of Research in Open and Distance Learning, vol 10, no 5.
Johnstone, S.M. (2005) ‘Open Educational Resources Serve the World’, EDUCAUSE Quarterly, vol 28, no 3, pp.
16-32.
Lessig, L. (2004) Free Culture: How Big Media Uses Technology and the Law to Lock Down Culture and Control
Creativity. New York: The Penguin Press.
Leinonen, T. (2005). ‘(Critical) history of ICT in education – and where we are heading?’, [Online], Available:
http://flosse.blogging.fi/2005/06/23/critical-history-of-ict-in-education-and-where-we-are- heading/ [4 Jun
2011].
Leinonen, T., Vadén, T. and Suoranta, J., (2009) ‘Learning in and with an open wiki project: Wikiversity’s
potential in global capacity building’, First Monday, vol 14, no 2.
Matsuo, Y. and Yamamoto, H. (2009) ‘Community gravity: measuring bidirectional effects by trust and rating on
online social networks’, WWW '09 Proceedings of the 18th international conference on World Wide Web,
pp. 751-760.
Mockus, A., Fielding, R.T. and Herbsleb, J.D. (2002) ‘Two case studies of open source software development:
Apache and Mozilla’, ACM Transactions on Software Engineering and Methodology, vol 11, no 3, pp. 309-
346.
OECD (2007) Giving Knowledge for Free: The Emergence of Open Educational Resources, OECD Publishing.
P2PU (2011), [Online], Available: http://p2pu.org/ [9 Jun 2011].
Põldoja, H., (2010) ‘EduFeedr: following and supporting learners in open blog-based courses’, Proceedings
Barcelona Open ED 2010, The Seventh Annual Open Education Conference, Barcelona, pp. 399-407.
The Cape Town Open Education Declaration (2007),[Online], Available:
http://www.capetowndeclaration.org/read-the-declaration [9 Jun 2011].
Väljataga, T. and Põldoja, H. (2011), ‘Enhancing community gravity in open online courses’, ECER 2011 Urban
Education (accepted for presentation/publication).
Wiley, D. (2007) Intro Open Ed Syllabus, [Online], Available:
http://www.opencontent.org/wiki/index.php?title=Intro_Open_Ed_Syllabus [9 Jun 2011].
376

Using a Social Networking Environment to Facilitate
Transition Into Higher Education
John Knight and Rebecca Rochon
Bucks New University, High Wycombe, UK
John.Knight@bucks.ac.uk
Rebecca.Rochon@bucks.ac.uk
Abstract: Transition into higher education (HE) can be challenging for incoming students. Literature identifies
three main areas where students may benefit from support: social, practical and academic. This paper discusses
a case study that explores the potential of a social networking environment to provide support in these areas. The
Learning Development Unit (LDU) at Bucks New University launched Startonline in 2010. This online presessional
environment used the social networking platform Ning to provide new students with access to nonsubject-specific
academic activities (e.g., critical thinking), social networking tools and practical information. As a
pilot, the aim was to observe where students focussed their attention and explore the affordances of a social
networking environment for facilitating transition. Startonline ran for the month leading up to the beginning of the
academic year. Around 300 students participated. Quantitative analysis of platform user data was conducted and
student and staff participants were interviewed, providing useful qualitative data. Postings and replies were
counted and organised into the following themes: social, practical and academic. Findings highlighted that
students’ engaged intensely in social and informational aspects of the environment, but remained resolutely
uninterested in generic academic activities. There was, however, considerable self-directed interest in finding
subject-specific information and learning activities. Personal involvement of subject-teaching staff seemed the
determining factor in this. This project underlines the usefulness of SNSs in providing powerful opportunities for
students to establish social networks as they transition into higher education. Methods for engaging students
academically in social networking environments are also suggested.
Keywords: transition, social networking, social capital, affordances, engagement
1. Introduction
A review of theories related to student retention by Braxton and Hirschy (2005) suggests that while
the issue of transition is complex, two models remain important in understanding: Tinto’s (1975, 1993)
Academic and social integration and Astin's (1977, 1985) Theory of Involvement . Both these models
suggest that the more engaged a student is with the institution, the higher likelihood of student
retention. By focussing on the “known” elements, such as academic, social and practical information,
it is possible for universities to contribute actively and positively during the transition phase to facilitate
this engagement. This case study considers the use of a social networking site (SNS) to provide
support to students in their transition into higher education.
The Learning Development Unit (LDU) at Bucks New University (Bucks) has been addressing
transitional issues for a number of years through face-to-face pre-sessional campus-based
programmes. These programmes have been highly successful in terms of providing students with
opportunities to develop academic skills and engage socially. To provide opportunities for a wider
range of students, the LDU launched Startonline in 2010, a bespoke SNS based on the Ning platform
which went live one month prior to induction. The environment included video, podcasts, discussion
forums and web pages, including:
Non-subject-specific academic activities: critical thinking; logical reasoning, writing skills
Social networking tools: profiles, ‘friending’, messaging and chat
Practical information ranging from student services to local entertainment
In total, just over 300 people became members. Of these, 25 were staff. Student-initiated discussion
activity was counted and organised by theme. A total of 332 discussion threads and replies were
recorded, of which staff activity accounted for 91 (28%). This initial analysis was complemented with
interviews with students as well as one staff member.
2. Emphasising the social in social networking
Establishing friendships and social networks has been described as key to transition (Lowe and
Cooke, 2003; Maunder et al., 2010). A study of undergraduate students at Bucks noted that first-year
students depend on their network of friends for everything from moral support to more general
377

John Knight and Rebecca Rochon
guidance, concluding that universities should do more to ensure that students have the opportunity to
create these networks as soon as they start their studies (Wickens et al., 2006).
SNSs are well-placed to provide students with opportunities for establishing contact with each other;
they are well established as part of students’ existing communication systems (Phipps, 2007) and
play an acknowledged role in the ways in which they manage and maintain their ‘social capital’
(Ellison et al., 2007). Indeed, it was clear from Startonline activity and from interviewees that students
were keen to use the technology to lay the groundwork for future university-based relationships.
I like how it was set up so that students that were just starting could like get to know
people that could be on their course or who they might be living with things like that... so
that you don’t feel too scared or anything when you come in to the Uni thinking “Oh no I
don’t know anyone.” (Excerpt from student interview)
Statistical data demonstrated that the most popular areas were member profiles and discussion
forums. Just over 60% of discussions initiated by students related to social issues, in particular finding
others who were either studying on the same course or living in or near the same accommodation
(see Fig. 1). This highlights that building social capital was the primary concern of users, and that
there was interest in transferring this beyond the Startonline environment.
Theme Threads Replies Total (threads and replies)
% of total
discussion
activity
Social 37 173 210 63
Practical 7 24 31 9
Academic 0 0 0 0
Figure 1: Summary of student discussion forum activity (threads and replies)
Using a bespoke SNS solution rather than building on students existing online profiles (e.g., in
Facebook), allowed Startonline to circumvent those privacy issues identified in other studies
(Ribchester, 2009; Sturgeon and Walker, 2009) and located control firmly with the students in terms of
the extent to which they could choose to make links between their personal online ‘spaces’ and their
more public university ‘faces’. Interestingly, there is clear evidence of students establishing initial
contact in Startonline and then moving out into the private domain by exchanging profile information
and ‘friending’ each other on Facebook. Similarly, there is also evidence of interest and intention of
meeting within the physical University environment at an event just prior to induction
3. Managing practicalities
Managing the practicalities of living away from home in an unfamiliar environment and obtaining
information related to the university was another focus for concern, accounting for slightly less than
10% of discussion activity (see Fig. 1). Students were also pro-active in starting their own discussion
threads if they needed information. Indeed, almost 20% of all student-initiated threads were requests
for information. Equally, students seemed content to answer one another, or direct others to
information. In cases where specialised information was needed, the Students Union or appropriate
staff member provided information. Students were both interested in practical information and proactive
in obtaining it:
I’m disabled. So I was a bit unsure about how I was, you know, how I’d go about finding
the Disabled Unit and what I could do, and what sort of help they’d give[...] I did ask a
few questions I wasn’t sure about. So, I asked those myself. (Excerpt from student
interview)
4. Learning
Student problems during first year are often related to a lack of study skills (Winterson and Russ,
2008; Cook and Leckey, 1999). However, none of the three learning generic activities were engaged
with by students (see Fig.1). Reasons for students’ lack of engagement in this area remain unclear,
but a possible cause is suggested in the way that SNSs can be said to afford interaction and provide
opportunities for learning. Where the technology was used in a way that ‘played to its strengths’ (i.e.,
to connect with others and convey information) learning opportunities seemed to occur
spontaneously. Where subject-related teaching staff members were available, students self-directedly
sought information and engaged with materials in a way that could be usefully built on within the
378

John Knight and Rebecca Rochon
physical teaching environment. Students also showed interest in finding subject-specific information
and learning activities. It is telling that this learning-related activity took place solely via direct
messages to staff profiles, rather than in the public discussion forum, perhaps reflecting Greenhow’s
(2009) assessment of the usefulness of SNSs to afford useful academic behaviour. Direct, personal
involvement of subject-teaching staff seemed the determining factor in encouraging this. Staff who
engaged with students directly, providing subject-related content, found the experience rewarding and
were able to draw on shared Startonline experiences during initial face-to-face contact:
...[it] was very useful in terms of that engagement: breaking it down rather than being just
a cold lecture [...] it became something that was a little bit more vibrant, really.(Excerpt
from staff interview)
5. Conclusion
Using SNSs has the potential to widen access to useful pre-entry information and allow students to
engage easily and at a distance with other students and members of staff. Issues of privacy are easily
circumvented by using bespoke solutions, allowing students to manage the degree of overlap
between academic and private social networks. It is apparent that SNSs are at their most beneficial
when employed for their intended purpose. Using SNSs empowers students to engage in the useful
activity of developing friendships, something that they are demonstrably interested in doing.
Attempting to manipulate the affordances of the platform to provide generic pre-entry learning
activities via online discussions was not successful. However, the social engagement of staff
members encouraged students’ pro-active engagement with subject specific resources. The
development of students as self-directed learners, who take responsibility for their own engagement,
is a key benefit of using SNSs that could be usefully be explored further.
References
Alcock, J. (2009) “Using Facebook Pages to reach users: the experiences of University of Wolverhampton”,
Association of Libraries and Information Professionals in the Social Sciences (ALISS) Quarterly, Vol. 4,
pp.2-6.
Astin, A.W. (1993) What matters most in college: Four critical years revisited, San Francisco, Jossey-Bass.
Braxton, J. M. and Hirschy, A. S. (2005) “Theoretical Developments in the study of college student departure”. In
Seidman, A. (Ed.). College student retention: Formula for student success, Westport, CT, Praeger Press,
pp.61-87.
Cook, A. and Leckey, J. (1999) “Do Expectations Meet Reality? A survey of changes in first year student
opinion”, Journal of Further and Higher Education, Vol.23, No. 2, pp.157-171.
Ellison, N. B., Steinfield, C., and Lampe, C. (2007) The benefits of Facebook friends: Social capital and college
students use of online social network sites. Journal of Computer-Mediated Communication. Vol.12, No.4,
pp.1143-1168.
Greenhow, C. (2009). Social network sites and education: emerging research within CSCL, in A. Dimitrakopoulou
(Ed.), CSCL 2009: Proceedings of the International Society of the Learning Sciences Computer-supported
Collaborative Learning Conference, Rhodes, Greece, June 10-12.
Lowe, H. and Cook, A. (2003) “Mind the Gap: Are students prepared for higher education?”, Journal of Further
and Higher Education, Vol. 27, No.1, pp.53 -76.
Maunder, R., Mjali, S. and Cunliffe, M. (2010) “Exploring transition in HE for first and second year undergraduate
psychology students: the role of expectations, personal growth and social identity in shaping experiences”.
Paper read at 2nd Annual Aim Higher West Yorkshire Symposium: What is Transition?, University of Leeds,
England, 16 July 2010.
Phipps, L. (2007) “Web 2.0 and social software: an introduction”, [online], JISC.
http://www.jisc.ac.uk/publications/publications/web2socialsoftwarev1.aspx.
Ribchester, C. (2009) “Enhancing the student experience through the use of online social networks, with
particular reference to induction and employability”. Proceedings of the ‘Hitting the Ground Running’
Conference at Glasgow Caledonian University, Glasgow.
Sturgeon, C.M. and Walker, C. (2009) “Faculty on Facebook: Confirm or Deny?” Paper read at the Annual
Instructional Technology Conference (14th, Murfreesboro, TN, Mar 29-31, 2009)
Tinto, V. (1975). “Dropouts from higher education: a theoretical synthesis of recent research”, Review of
Educational Research, Vol. 45, pp.89-125.
Tinto, V. (1993) Leaving college: rethinking the causes and cures of student attrition, 2 nd ed, Chicago, University
of Chicago Press.
Wickens, E. Forbes, A. and Tribe, J. (2006) Listening, understanding and responding to leisure and tourism
undergraduates. Journal of Hospitality, Leisure, Sport and Tourism Education, Vol. 5, No. 2, pp.4-13.
Winterson, J. and Russ, M. (2008) “Understanding the transition from school to university in music and related
subjects”. Project Report. University of Huddersfield,
http://eprints.hud.ac.uk/3692/2/Understanding_the_Transition_from_School_to_University.pdf.
379

Evaluation of Quality of Learning Scenarios and Their
Suitability to Particular Learners’ Profiles
Eugenijus Kurilovas 1, 2, 3 , Inga Zilinskiene 2 and Natalija Ignatova 4
1
Centre of Information Technologies in Education, Vilnius, Lithuania
2
Vilnius University Institute of Mathematics and Informatics, Lithuania
3
Vilnius Gediminas Technical University, Lithuania
4
Vilnius Pedagogical University, Lithuania
eugenijus.kurilovas@itc.smm.lt
inga.zilinskiene@gmail.com
natalija.ignatova@itc.smm.lt
Abstract: The aim of the paper is to investigate and present a comprehensive scientific model and several
methods suitable for the expert evaluation of quality of learning scenarios. A special attention is paid to their
suitability to particular learner groups (i.e., profiles). The solution of learning scenarios quality evaluation and
optimisation problems could help educational institutions to select suitable learning scenarios for the particular
learner profiles. The research results will be implemented in iTEC – a four-year, pan-European research and
development project focused on the design of the future classroom funded by EC 7th Framework Programme.
The main objectives of iTEC are to develop and refine a range of teaching and learning scenarios for the future
classroom, to develop decision support criteria that facilitate the selection of scenarios that can be mainstreamed
and taken to scale; and to carry out large-scale pilots in up to 1,000 classrooms in at least 12 European countries
exploring both the integration of technologies and how these impact on teaching and learning practices.
Suitability of several iTEC scenarios to particular learner groups is analysed in the paper. A number of multiple
criteria decision analysis principles are applied to create a comprehensive quality model (criteria tree) for
evaluating learning scenarios. Several optimisation methods are explored to optimise learning scenarios in
conformity with particular learner’s profile. Several practical examples of iTEC learning scenarios have been
evaluated against the proposed model and methods. The research results have shown that both Analytic
Hierarchy Process (AHP) method to establish the weights of quality criteria and several fuzzy optimisation
methods are suitable to solve learning scenarios’ multiple criteria evaluation and optimisation tasks for particular
learner profiles.
Keywords: learning scenarios, evaluation of quality, multiple criteria decision analysis, learner profile,
optimisation methods, Analytic Hierarchy Process (AHP)
1. Introduction: Evaluation of learning scenarios in iTEC project
The aim of the paper is to investigate, propose, and demonstrate examples of practical application of
a model and methods suitable for the expert evaluation of quality of learning scenarios (LS). A special
attention is paid to LS suitability to particular learner groups (i.e., profiles).
1.1 What is learning scenario?
Unit of Learning (also known as ‘Learning Scenario’ (LS) – this term is commonly used in European e-
Learning community, e.g., in iTEC project) is referred here as an aggregation of learning activities that
take place in particular virtual learning environments (VLEs) using particular learning objects (LOs).
This notion is based on Koper and Tattersall (2005) work and IMS Learning Design (IMS LD, 2003)
specification. Quoting Koper and Tattersall (2005), “a 'Unit of Learning' refers to a complete, selfcontained
unit of education or training, such as a course, a module, a lesson, etc”. IMS LD (2003)
specification’s conceptual vocabulary clarifies that a 'unit of learning' is an abstract term used to refer
to any delimited piece of education or training. It is noted that a 'unit of learning' represents more than
just a collection of ordered resources to learn, it includes a variety of prescribed activities (problem
solving activities, search activities, discussion activities, peer assessment activities, etc.),
assessments, services and support facilities provided by teachers, trainers and other staff members.
A learning design as an integral part of any unit of learning is a description of a method enabling
learners to attain certain learning objectives by performing certain learning activities (LA) in a certain
order in the context of a certain learning environment. According to IMS LD (2003), activities are one
of the core structural elements of the 'learning workflow' model for learning design. They form the link
between the roles and the learning objects and services in the learning environment. The activities
describe a role they have to undertake within a specified environment composed of learning objects
380

Eugenijus Kurilovas et al.
and services. Activities take place in a so-called 'environment', which is a structured collection of
learning objects, services, and sub-environments.
Therefore, we can conclude that in general case LS could consist of learning activity (or method),
learning objects and learning environment referred here as services package. This kind of services
package in e-Learning theory is commonly known as Virtual Learning Environment (VLE) (also known
as Learning Management System, etc.).
Therefore, we can divide LS into three components, namely LA, LOs and VLE.
1.2 Learning scenarios in iTEC project
Research results will be implemented in iTEC project. iTEC (Innovative Technologies for an Engaging
Classroom) is a four-year, pan-European project focused on the design of the future classroom. The
main iTEC objectives are (iTEC, 2011):
To develop and refine a range of teaching and learning scenarios for the future classroom, and to
develop decision support criteria (technological, pedagogical and policy-related) that facilitates the
selection of scenarios that can be mainstreamed and taken to scale; and
To develop specific teaching and learning activities, based on the scenarios, and test these in a
pre-pilot phase, and to carry out large-scale pilots in up to 1,000 classrooms in at least 12
European countries exploring both the integration of technologies and how these impact on
teaching and learning practices and the engagement of a wider group of stakeholders outside the
school;
1.3 Learning scenarios personalization strategies
The paper is aimed to evaluate quality of LS paying especial attention to their suitability to particular
learners’ groups (i.e., profiles).
Learning groups (styles) were developed by Honey and Mumford (1992), based upon the work of
Kolb (1984), and they identified four distinct learning styles or preferences: Activist, Theorist;
Pragmatist and Reflector.
Activist: Activists are those people who learn by doing. Have an open-minded approach to
learning, involving themselves fully and without bias in new experiences. Their preferred activities
are: brainstorming, problem solving, group discussion, puzzles, competitions, and role-play.
Reflector: These people learn by observing and thinking about what happened.
Pragmatist: These people need to be able to see how to put the learning into practice in the real
world.
Theorist: These learners like to understand the theory behind the actions.
As far as the format of the paper does not allow presentation of research results for all
aforementioned learner profiles in the same paper, the authors aim to present quality and suitability of
LS to the only learner profile, namely, activist learners.
The literature review (Mulwa et. al. 2010, O' Keeffe et. al. 2006, Essalmi et. al. 2010) shows that there
are a lot of personalisation parameters and when several personalisation parameters are used, the
question is: what personalisation parameters are to be used for personalising each course (LS)?
Essalmi et. al. (2010), discussed 4 personalisation strategies (i.e., alternatives A1, A2, A3, and A4):
A1: Using all the personalisation parameters for each course
A2: Using the subset of personalisation parameters which includes divergent characteristics of
learner
A3: Using the subset of personalisation parameters to include only the most significant ones for
each course
A4: Using the subset of personalisation parameters recommended by the professor responsible
for the course
381

Eugenijus Kurilovas et al.
Essalmi et. al. (2010) approach is based on the fourth alternative, that the teacher is the expert, who
decides which personalisation parameters (criteria) are most significant by selecting learning scenario
for appropriate course (LS).
Figure 1: Proposed framework for evaluation of suitability of learning scenario to learner
In this paper, the authors play the role of experts-evaluators (lecturers) who decide which LS quality
criteria are more significant for the activist learners, and which are less significant, as well as evaluate
quality of each LS alterative against the proposed quality criteria model.
The rest of the paper is organised as follows: methodology of the research is presented in chapter 2,
literature analysis and research results are presented in chapter 3, and conclusion and
recommendations are presented in chapter 4. The main chapter containing research results is divided
into two separate parts – creation of LS quality evaluation model (criteria tree) and practical
application of the method for evaluating LS which in its turn is divided into (a) the use of the novel
method of consecutive triple application of AHP to establish the weights of quality criteria, (b)
application of triangular and trapezoidal fuzzy numbers to establish the values of criteria, and (c)
practical example of evaluating several real-life LS alternatives used in iTEC project.
2. Methodology of the research
The main problem analysed in the paper is how to establish
A ‘proper’ set of LSs quality evaluation criteria that should reflect the objective scientific principles
of constructing a quality model (criteria tree) for LS suitable to particular learners’ groups (i.e.,
activist learners in our case), and
‘Proper’ methods for evaluation of LS quality and suitability to particular learners’ profiles paying
especial attention to establishment of proper weights of quality criteria by the novel method of
consecutive triple application of AHP.
Expert evaluation is referred here as the multiple criteria evaluation of LS alternatives aimed at
selection of the best ones based on score-ranking results (Kurilovas and Dagiene 2009a,b).
According to (Zavadskas and Turskis 2010), there is a wide range of multiple criteria decision making
problem solution techniques, varying in complexity and possible solutions. Each method has its own
strength, weaknesses and possibilities to be applied. Usually the experts have to deal with the
problem of optimal decision in the multiple criteria situation where the objectives are often conflicting.
In this case, an optimal decision is the one that maximises the expert’s utility.
382

Eugenijus Kurilovas et al.
Evaluation of quality of LS alternatives is a typical case where the criteria are conflicting, i.e., LS could
be very qualitative against several criteria, and not qualitative against the other ones, and vice versa.
Therefore, the authors propose to use multiple criteria decision analysis (MCDA) approach for
creation of LS quality evaluation model. In order to construct a proper comprehensive scientific quality
criteria system (model), the authors use the well known principles of identification of quality criteria
that have been proposed by Belton and Stewart (2002) in multiple criteria decision analysis (MCDA)
theory related research work. Practical application of these principles will be described below while
analysing LS quality criteria system (tree).
LS multiple criteria evaluation method used by the authors is referred here as the experts’ additive
utility function represented by formula (1) below including LS evaluation criteria, their ratings (values)
and weights. According to (Kurilovas and Serikoviene 2010b), this method is well-known in the theory
of optimisation methods and is named “scalarisation method”. A possible decision here could be to
transform multi-criteria task into one-criterion task obtained by adding all criteria together with their
weights. Therefore, here we have the experts’ additive utility function:
m
∑
i=
1
f ( X ) = a f ( X ) , a = 1,
a > 0 .
i
i
m
∑
i=
1
i
i
where f i (X j) is the rating (i.e., non-fuzzy value) of the criterion i for the each of the examined LS
alternatives X j. The weights here should be ‘normalised’ according to the ‘normalisation’ requirement
m
∑
i=
1
a = 1,
a > 0 .
i
i
According to (Zavadskas and Turskis 2010), the normalisation aims at obtaining comparable scales of
criteria values. The major is the meaning of the utility function (1) the better LS meets the quality
requirements in comparison with the ideal (i.e., 100%) quality (Kurilovas and Dagiene 2009ab).
The complexity of the analysed problem influences the application of more complex methods for
evaluating the quality of LS from the point of view of different learner groups. In this paper, a novel
method of consecutive triple application of Analytic Hierarchy Process (AHP) is used to establish
proper weights of LS quality criteria in the case when there are several experts evaluators. Triangular
and trapezoidal fuzzy numbers methods are used to establish proper values of LS quality criteria.
After that, formula (1) is used to calculate the values of additive utility functions for each of the
explored LS alternatives.
3. Literature analysis and research results
This section is aimed to apply the aforementioned scientific approaches in order:
To propose a suitable scientific model for evaluation of quality of LS paying especial attention to
their suitability to particular learners’ profiles (i.e., activist learners in our case);
To propose suitable scientific methods for evaluation of quality of LOs paying especial attention to
the use of novel method of consecutive triple application of AHP for establishing quality criteria
weights, and different fuzzy numbers methods to establish the values (ratings) of LS quality
criteria; and
To present the experimental evaluation results using proposed evaluation model and methods.
3.1 Learning scenarios quality model
Belton and Stewart (2002) have identified the following principles of identification of quality criteria
that are relevant to all MCDA approaches: (1) Value relevance; (2) Understandability; (3)
Measurability; (4) Non-redundancy; (5) Judgmental independence; (6) Balancing completeness and
conciseness; (7) Operationality; and (8) Simplicity versus complexity.
LS quality model based on these MCDA criteria identification principles is presented in the Fig. 2. The
model consists of three groups of quality criteria (i.e., components of LS), namely learning objects
383
(1)
(2)

Eugenijus Kurilovas et al.
(LOs), learning activities (methods) (LAs), and virtual learning environments (VLEs). The model
consists of already created LOs and VLEs quality criteria systems accordingly presented in (Kurilovas
et. al. 2011) and (Kurilovas and Dagiene 2009b), and LA quality criteria system created specially for
the presented research. LA quality criteria were proposed by iTEC working groups, and a number of
scientific papers were additionally analysed (Shee and Wang 2008, Sun et. al. 2008, iCOPER 2011)
to propose LA quality criteria. Belton and Stewart (2002) principles were applied to form the system of
criteria. The authors have paid special attention to Non-redundancy, Judgemental independence,
Balancing completeness and conciseness, and Simplicity Vs complexity principles to create the
comprehensive criteria tree presented in Fig. 2.
Figure 2: Learning scenario quality evaluation model (criteria tree)
There is a clear division of all the criteria into ‘internal quality’ and ‘quality in use’ criteria here. In the
model, the authors apply criteria classification principle claiming that one should evaluate learning
384

Eugenijus Kurilovas et al.
software alternatives using two different groups / types of evaluation criteria – ‘internal quality’ and
‘quality in use’ criteria. According to (Kurilovas and Serikoviene 2010b), ‘internal quality’ is a
descriptive characteristic that describes the quality of learning software independently from any
particular context of its use, and ‘quality in use’ is evaluative characteristic of software obtained by
making a judgment based on criteria that determine the worthiness of software for a particular project
or user / group (e.g., activist learners).
Therefore, if we want to analyse general quality of LS, we should consider all quality criteria a priori
equal, but if we want to analyse suitability of LS to particular learner profile, we should pay special
attention to ‘quality in use’ criteria by establishing higher weights.
3.2 Learning scenarios quality evaluation methods
As it was mentioned above, in order to evaluate quality of alternatives or their suitability to particular
learner profiles, one should establish the weights of quality criteria, their values, and after that to apply
experts’ additive utility formula (1) to get final evaluation results.
3.2.1 Use of Analytic Hierarchy Process (AHP) to establish the weights of quality criteria
According to (Kurilovas et. al., 2011), the weight of the evaluation criterion reflects the experts’
opinion on the criterion’s importance level in comparison with the other criteria for the particular
needs.
In this paper, the authors propose an original method of consecutive triple application of AHP to
establish proper weights of LS quality evaluation criteria in the case when there are several experts
evaluators.
According to Saaty (1990), AHP is a useful method for solving complex decision-making problems
involving subjective judgment. In AHP, the multi-attribute weight measurement is calculated via pairwise
comparison of the relative importance of two factors (Lin 2010). The design of the questionnaire
incorporates pair-wise comparisons of decision elements within the hierarchical framework. Each
evaluator (three in our case) is asked to express relative importance of two criteria in the same level
by a nine-point rating scale. After that, we have to collect the scores of pair-wise comparison, and
form pair-wise comparison matrices for each of the K evaluators.
According to Saaty (2008), the fundamental scale of absolute numbers is as follows:
Table 1: Pair-wise comparison scale for AHP preferences
Numerical rating Verbal judgements of preferences
9 Extremely preferred
8 Very strongly to extremely
7 Very strongly preferred
6 Strongly to very strongly
5 Strongly preferred
4 Moderately to strongly
3 Moderately preferred
2 Equally to moderately
1 Equally preferred
After that, we have to construct a set of pair-wise comparison matrices (size n x n) for each of the
lower levels with one matrix for each element in the level immediately above by using the relative
scale measurement shown in Table 1. The pair-wise comparisons are done in terms of which element
dominates the other:
There are n(n-1)/2 judgments required to develop the set of matrices in this step. Reciprocals are
automatically assigned in each pair-wise comparison.
385

Eugenijus Kurilovas et al.
Then hierarchical synthesis is used to weight the eigenvectors by the weights of the criteria and the
sum is taken over all weighted eigenvector entries corresponding to those in the next lower level of
the hierarchy:
In order to check the correctness of calculation, having made all the pair-wise comparisons the
eigenvalue λ max is calculated:
The method of consecutive triple application of AHP consists of application of aforementioned AHP in
three consistent stages:
Establishment of comparative weights of three different groups of LS quality criteria and weights a
i of all quality criteria;
Establishment of comparative weights of ‘internal quality’ and ‘quality in use’ criteria groups from
activist learner point of view; and
Establishment of final weights of quality criteria from activist learners point of view by application
of AHP once again only for ‘quality in use’ criteria.
3.2.2 Use of triangular and trapezoidal fuzzy numbers to establish values of quality criteria
The widely used measurement criteria of the decision attributes’ quality are mainly qualitative and
subjective. Decisions in this context are often expressed in natural language, and evaluators are
unable to assign exact numerical values to the different criteria. Assessment can be often performed
by linguistic variables: ‘bad’, ‘poor’, ‘fair’, ‘good’ and ‘excellent’. These values are imprecise and
uncertain: they are commonly called ‘fuzzy values’. Integrating these different judgments to obtain a
final evaluation is not evident (Kurilovas and Serikoviene 2010ab).
Therefore, the authors have proposed to use fuzzy group decision making theory (Ounaies et. al.
2009) to obtain final assessment measures. The fuzzy numbers are: (1) triangular fuzzy numbers, (2)
trapezoidal fuzzy numbers, and (3) bell-shaped fuzzy numbers (Zhang Li Li and Cheng De Yong
1992, Kurilovas et. al., 2011). In the presented paper, the authors use triangular (Fig. 3) and
trapezoidal (Fig. 4) fuzzy numbers for evaluating quality of LS and their suitability to activist learner
profile:
Figure 3: Triangular fuzzy numbers Figure 4: Trapezoidal fuzzy numbers
386

Eugenijus Kurilovas et al.
According to Kurilovas et. al. (2011), in the case of using average triangular fuzzy numbers, linguistic
variables conversion into non-fuzzy values of the evaluation criteria should be as follows:
‘excellent’=0.850; ‘good‘=0.675; ‘fair’=0.500; ‘poor’=0.325; ‘bad’=0.150, and in the case of using
average trapezoidal fuzzy numbers – ‘excellent’=1.000; ‘good‘=0.800; ‘fair’=0.500; ‘poor’=0.200;
‘bad’=0.000.
There are three experts-evaluators (i.e., the authors of the paper) in our case, and therefore it was
necessary to calculate average values for each LS quality criterion.
3.3 Practical example of evaluation of learning scenarios in iTEC project
Two LS alternatives proposed by iTEC experts were chosen by the authors to demonstrate
application of the aforementioned methods for evaluating LS quality and their suitability to chosen
activist learner profile:
LS1: “A Breath of Fresh Air” (Cycle 1 detailed scenario available at
http://itec.eun.org/web/guest/scenario-library )
LS2: “Online Repositories Rock” (Cycle 1 detailed scenario online at
http://itec.eun.org/web/guest/scenario-library )
The scenarios do not contain any LOs and there is no explanation of what kind of VLEs should be
used to implement the scenarios. Use of particular LOs and VLEs is up to the decision of every
country participating in iTEC, and the project experts propose a number of widgets for each scenario
to enrich VLEs to be used while implementing scenarios at schools.
Therefore, the authors decide to consider ‘good’ LOs only for the chosen LS. This means that all LOs
quality criteria in the model presented in Fig. 2 should be evaluated 0.800 according to trapezoidal
fuzzy numbers method, and 0.675 – according to triangle fuzzy numbers method.
VLE Moodle was chosen by the authors as a proper environment to implement both LS. Since VLE
Moodle was already evaluated by the experts in (Kurilovas and Dagiene 2010, 2009a), the authors
use the same values to evaluate the VLE component of LS quality criteria.
Application of the aforementioned method of consecutive triple application of AHP has shown the
following results:
Stage 1 (establishment of comparative weights of three different groups of LS quality criteria and
weights a i of quality criteria) results have shown that the evaluators prefer LOs and LA
components more in comparison with VLE component (LOs – 39.7%, LA – 39.7%, and VLE –
20.6%). Weights a i for all 24 LS quality criteria were also calculated in conformity with AHP.
Stage 2 (establishment of comparative weights of ‘internal quality’ and ‘quality in use’ criteria
groups from activist learner point of view) results have shown that the evaluators prefer ‘quality in
use’ criteria in comparison with ‘internal quality’ criteria (respectively 69.4% Vs 30.6% for LOs,
72.2% Vs 27.8% for LA, and 61.1% Vs 39.8% for VLE) to analyse suitability of chosen LS to
activist learners.
Stage 3 (establishment of final weights of LS quality criteria from activist learners point of view by
application of AHP once again only for ‘quality in use’ criteria) has re-established the weights of
‘quality in use’ criteria.
Final evaluation results after using utility function (1) are as follows:
In general (G) case when experts do not take into account particular learner profile: 72.7%
according to trapezoidal method, or 63.8% according to triangle method for LS1 Vs 68.5%
according to trapezoidal method, or 61.0% according to triangle method for LS2:
a ⋅ f ( X ) = ( 0,
727 0,
685)
a ⋅ f ( X ) = ( 0,
638 0,
610)
a
iG j
iG j
In particular case when experts take into account particular (activist) learner (A) profile: 76.4%
according to trapezoidal method, or 65.6% according to triangle method for LS1 Vs 69.7%
according to trapezoidal method, or 61.8% according to triangle method for LS2:
iA
⋅ f ( X j ) =
( 0,
764 0,
697)
a
iA
⋅ f ( X j ) =
( 0,
656 0,
618)
387

Eugenijus Kurilovas et al.
If we look separately at learning activity (LA) component of the scenarios, we’ll get the following
results for LA comparative weights in LS:
In general case when experts do not take into account particular learner profile: 28.3% according
to trapezoidal method, or 25.2% according to triangle method for LA1 Vs 24.1% according to
trapezoidal method, or 22.3% according to triangle method for LA2:
⋅ f ( X ) = 0,
283 0,
241 ⋅ f ( X ) = 0,
252 0,
223
aiG j
a
iA
⋅ f ( X j ) =
( )
( 0,
298 0,
232)
aiG j
( )
In particular case when experts take into account particular (activist) learner profile: 29.8%
according to trapezoidal method, or 25.7% according to triangle method for LA1 Vs 23.2%
according to trapezoidal method, or 21.8% according to triangle method for LA2:
a
iA
⋅ f ( X j ) =
( 0,
257 0,
218)
In general case these results mean that, according trapezoidal fuzzy numbers method, LS1 meets
72.7% quality in comparison with the ideal, LS2 – 68.5%, and using triangle fuzzy numbers method –
LS1 – 63.8%, and LS2 – 61.0% respectively.
In particular case (for activist learner) these results mean that according trapezoidal fuzzy numbers
method, LS1 meets 76.4% quality in comparison with the ideal, LS2 – 69.7%, and using triangle fuzzy
numbers method – LS1 – 65.6%, and LS2 – 61.8% respectively.
The same tendency in ranking the alternatives is noticeable in evaluation of LA. It is understandable,
because the experts have decided that LOs for each alternative should be evaluated equally, and in
this case LA remains the most significant component of LS.
The obtained results mean that LS1 is a better alternative in comparison with LS2 both in general case
and in particular case of suitability to activist learner, and both for learning activity component and the
whole learning scenario.
4. Conclusion and recommendations
The research results presented in the paper show that (1) complex application of principles of multiple
criteria decision analysis for identification of quality evaluation criteria, (2) quality criteria classification
principle, (3) fuzzy group decision making theory to obtain final evaluation measures, and (4) the
original method of consecutive triple application of AHP to establish criteria weights:
Are applicable in real life situations when schools have to decide on use of particular learning
scenarios for their education needs, and
Could significantly improve the quality of expert evaluation of learning scenarios by noticeably
reduce of the expert evaluation subjectivity level.
Use of the method of consecutive triple application of AHP leads to establishing different weights of
criteria for particular leaner groups, and respectively – to different LS alternatives evaluation results.
Application of both triangle and trapezoidal fuzzy numbers shows similar alternatives’ quality
evaluation results, i.e., the ranking of analysed alternatives have not changed while applying different
fuzzy numbers methods. The experimental evaluation results show that proposed scientific
approaches are quite objective, exact and simply to use for selecting qualitative LS alternatives for
particular learner groups. On the other hand, proposed LS personalised quality evaluation approach is
applicable for the aims of iTEC project in order to select LS suitable for activist learners. Therefore,
these approaches have been recommended by the authors to be widely used by European policy
makers, publishers, practitioners (teachers), and experts-evaluators both inside and outside iTEC
project to evaluate quality and personalisation level of learning scenarios. Method of consecutive
triple application of AHP presented in the paper is absolutely novel, and these new elements make
the given work distinct from all the other earlier works in the area.
5. Appendix 1
The work presented in this paper is partially supported by the European Commission under the 7th
Framework Programme – as part of the iTEC project (Grant Agreement Number 257566). The
authors are solely responsible for the content of this paper. It does not represent the opinion of the
388

Eugenijus Kurilovas et al.
European Commission, and the European Commission is not responsible for any use that might be
made of data appearing therein.
References
Belton, V., Stewart, T.J., (2002) “Multiple criteria decision analysis: an integrated approach”. Kluwer Academic
Publishers, 2002
Essalmi, F., Jemni Ben Ayed, L., Jemni, M., Kinshuk, Graf, S. (2010) “A fully personalization strategy of Elearning
scenarios”. Computers in Human Behavior 26(4): 581-591 (2010)
iCOPER (2011). EU eContentplus Programme’s iCOPER (Interoperable Content for Performance in a
Competency-Driven Society) Best Practice Network web site. D. 3.1. Annex A. Available online at
http://www.icoper.org/
IMS LD (2003) “IMS Learning Design Information Model. Version 1.0 Final Specification”. Available online at
http://www.imsglobal.org/learningdesign/ldv1p0/imsld_infov1p0.html#1495449
Honey, P. & Mumford, A. (1982) “Manual of Learning Styles”. London: P Honey, 1982
iTEC (Innovative Technologies for an Engaging Classroom) project website (2011). Available online at
http://itec.eun.org/web/guest/
Kolb, D.A. (1984) “Experiential Learning: experience as the source of learning and development”. Englewood
Cliffs, Prentice Hall, 1984
Koper, E.J.R., and Tattersall, C. (Eds.) (2005) “Learning Design: A handbook on Modelling and delivering
Networked Education and Training”. Heidelberg: Springer
Kurilovas, E., Bireniene, V., Serikoviene, S. (2011) “Methodology for Evaluating Quality and Reusability of
Learning Objects”. The Electronic Journal of e-Learning, Vol. 9, Issue 1, 2011, pp. 39–51. Available online
at www.ejel.org
Kurilovas, E., Dagiene, V. (2010) “Multiple Criteria Evaluation of Quality and Optimisation of e-Learning System
Components”. The Electronic Journal of e-Learning, Vol. 8, Issue 2, 2010, pp. 141–150. Available online at
www.ejel.org
Kurilovas, E., Dagiene, V., (2009) “Multiple Criteria Comparative Evaluation of e-Learning Systems and
Components”. Informatica. 2009, Vol. 20, No. 4, pp. 499–518
Kurilovas, E., Dagiene, V. (2009) “Learning Objects and Virtual Learning Environments Technical Evaluation
Criteria”. The Electronic Journal of e-Learning, Vol. 7, Issue 2, 2009, pp. 127–136. Available online at
www.ejel.org
Kurilovas, E., Serikoviene, S. (2010) “Application of Scientific Approaches for Evaluation of Quality of Learning
Objects in eQNet Project”. In: M.D. Lytras et. al. (Eds.): WSKS 2010, Part I, CCIS 111, pp. 437–443.
Springer, Heidelberg 2010
Kurilovas, E., Serikoviene, S., (2010) “Learning Content and Software Evaluation and Personalisation Problems”.
Informatics in Education, 2010, Vol. 9 (1), pp. 91–114
Lin, Hsiu-Fen (2010) ”An application of fuzzy AHP for evaluating course website quality”. Computers & Education
54 (2010), p. 877–888
Mulwa, C., Lawless, S., Sharp, M., Wade, V. & Arnedillo-Sanchez, I. (2010) “Adaptive Educational Hypermedia
Systems in Technology Enhanced Learning: A Literature Review”. ACM Special Interest Group for
Information Technology Education Conference (SIGITE), Central Michigan University, Midland, MI,
USA, October 7th-9th, 2010, pp. 1-12
O' Keeffe, I., Brady, A., Conlan, O. & Wade, V. (2006) "Just-in-time Generation of Pedagogically Sound, Context
Sensitive Personalized Learning Experiences", International Journal on E-Learning (IJeL), Special Issue:
Learning Objects in Context," International Journal on E-Learning (IJeL), vol. 5, pp. 113-127, 2006
Ounaies, H.Z., Jamoussi, Y., Ben Ghezala, H.H., (2009) “Evaluation framework based on fuzzy measured
method in adaptive learning system”. Themes in Science and Technology Education, 2009, Vol. 1, Nr. 1,
pp. 49–58
Saaty, T. L. (1990) “How to make a decision: The analytic hierarchy process”. European Journal of Operational
Research, 48(1), 9–26
Saaty, T. L. (2008) “Relative Measurement and its Generalization in Decision Making: Why Pairwise
Comparisons are Central in Mathematics for the Measurement of Intangible Factors - The Analytic
Hierarchy/Network Process”. RACSAM (Review of the Royal Spanish Academy of Sciences, Series A,
Mathematics) 102 (2), 2008, p. 251–318.
Shee, Daniel Y., Wang, Yi-Shun (2008) “Multicriteria evaluation of the web-based e-learning system: a
methodology based on learner statisfaction and its application”. Computers and Education 50(48), 894-905.
Sun, P.-C., Tsai, R. J., Finger, G., Chen, Y.-Y., & Yeh, D. (2008) “What drives a successful e-learning? An
empirical investigation of the critical factors influencing learner satisfaction”. Computers and Education, 50,
pp. 1183–1202
Zavadskas, E. K., Turskis, Z., (2010) “A new additive ratio assessment (ARAS) method in multicriteria decisionmaking”.
Technological and Economic Development of Economy 16(2): 159–172
Zhang Li Li, Cheng De Yong, (1992) “Extent analysis and synthetic decision”. In: Support Systems for Decision
and Negotiation Processes, Preprints of the /FAC/IFORS/IIASA/TIMS Workshop, Vol. 2. System Research
Institute, Warsaw, pp. 633-640
389

Models of eLearning: The Development of a Learner-
Directed Adaptive eLearning System
Stella Lee 1 , Trevor Barker 1 and Vive Kumar 2
1
Computer Science Department, University of Hertfordshire, Hatfield, UK
2
School of Computing & Information Systems, Athabasca University,
Edmonton, Canada
stellaylee@gmail.com
t.1.barker@herts.ac.uk
vive@athabascau.ca
Abstract: In many eLearning contexts, materials are designed to be self-paced, with the content being available
anytime, anywhere for learners to study independently. Commonly, without the presence and immediate
feedback of an instructor, distance learners are left to their own devices to negotiate their learning path and to
monitor their own progress. Furthermore, learning a complex topic structured in terms of various media and
learning materials requires learners to make certain instructional decisions concerning what to learn and how to
go about their learning. In other words, self-paced learning requires learners to self-regulate their own
learning(Hadwin & Winne, 2001). Very often, learners have difficulty regulating learning in higher education when
topics are complex and unfamiliar and it is not always clear to the learners if their instructional decisions are
optimal.(Azevedo, Cromley, Seibert, & Tron, 2003) Research into adaptive eLearning systems has attempted to
facilitate this process by providing recommendations, classifying learners into different preferred learning styles,
or highlighting suggested learning paths(Brusilovsky, 1998).The aim of this research is to explore how learners
can self-directed and self-regulate their online learning both in terms of domain knowledge and meta knowledge
in the subject of computer science with a flexible and adaptive eLearning system. Two educational theories:
experiential learning theory (ELT) and self-regulated learning (SRL) theory are used to aid learners’ in their
learning paths. As a result, changes in domain-knowledge, meta-knowledge, learner experience, learner
satisfaction, perceived controllability, and system usability are being measured. All in all, this paper sums up the
research work being done on the initial development of the system, instructional design framework based on the
two theories, experimental design plan and course material examples as well as related issues.
Keywords: adaptive systems, eLearning, instructional design, learning design
1. Introduction
eLearning has become so prevalent in higher education and corporate training that we have seem
every effort being made to reshape and digitized contents online in the past decade or so. ELearning
uses the internet to deliver instructions to the learner and it may be argued that one of the most
popular forms of eLearning is web-based distance learning (Koohang & Du Plessis, 2004) In many
cases, these web-based learning materials are designed to be self-paced, with the content being
available anytime, anywhere to allow learners to study independently. Commonly, without the
presence and immediate feedback of an instructor, distance learners are left to their own devices to
negotiate their learning path and to monitor their own progress. Furthermore, learning a complex topic
structured in terms of various media and learning materials requires learners to make certain
instructional decisions concerning what to learn and how to go about learning. In other words,
eLearning requires learners to self-regulate their own learning(Hadwin & Winne, 2001). Very often,
learners have difficulty regulating learning when topics are complex and unfamiliar and it is not always
clear to the learners if their learning decisions are optimal(Azevedo et al., 2003). Research into
adaptive eLearning systems has attempted to facilitate the learning process by providing
recommendations with respect to classifying learners into preferred learning styles and by associating
recommended learning paths with these learning styles(Brusilovsky, 1998). Indeed, research has
shown the importance of adapting online course material to support learners with different
background knowledge and skills(Brusilovsky, 1998; Weber, 1999). Another work has described a
user modeling approach that is beneficial to learners who are interacting with complex learning
applications in an online environment(Adisen & Barker, 2007).
Broadly speaking, user modeling is a technique employed to provide users with options with respect
to performing tasks and interacting with systems differentially. The common variables to model
include the user’s personalities, abilities, prior knowledge, preferences, performances and
intentions(Barker & Adisen, 2005). User modeling can also be defined as a model of users residing
inside a system or computational environment(Fischer, 2001). However, many of these models
attempt to “match” students with a certain learning styles or learner characteristics that it falls short on
390

Stella Lee et al.
placing the initiative on the learners themselves and help them make aware of their own learning and
in control of their learning paths. In fact, there has been no evidence in matching teaching to learning
styles as the best way to improve learning. Instead, there is an optimal way to teach chunk of content
to all students(Pashler, McDaniel, Rohrer, & Bjork, 2008). In this study, our modeling approach is
centred around a constructive cognitive model of learning (Liu & Matthews, 2005). This approach
involves a model that helps make learning constructive while providing adaptive feedback to learners.
As a result, an adaptive eLearning system has been designed that is task-based and support learners
to self-direct their own learning paths by choosing the learning preferences (or learning modes as we
called them) as they see fit. Additionally, contextually appropriate meta level skills (i.e. study skills)
information and tools are provided along with the domain-level material. Adaptive feedback is
provided at the end in the form of a self quiz to evaluate whether the learners have progress through
their learning with adequate domain and meta knowledge.
The domain we have chosen for this study is in computer science. Specifically, we are conducting our
research study within an introductory Java programming course at a distance learning university in
Canada. Two relevant educational theories: experiential learning theory (ELT) and self-regulated
learning (SRL) have been selected as the underlining instructional design principles for this eLearning
system. We will discuss more about these theories in Section 2. Section 3 will cover the planned
research and the experimental design in details while Section 4 provides examples of the material we
have redesigned to suit this learning approach. Finally we conclude with discussion and future work in
Section 5.
2. Background on the two educational theories used
Two educational theories have emerged from authors’ experiences in designing eLearning models
and reviewing relevant literature in learning preferences and online learning - the Experiential
Learning Theory (ELT) and Self-Regulated Learning (SRL) theory. The fundamental ideas underlying
the two theories are explained below.
2.1 Experiential Learning Theory (ELT)
Developed in the early 1970's by David Kolb, Experiential Learning Theory (ELT) emphasizes
experience as the central focus in learning. Learning-styles theories(Coffield, Moseley, Hall, &
Ecclestone, 2004) raised questions about what learning strategies we operate with and how we use
learning strategies to enhance student learning. By studying these theories, one can also gain
insights into what motivates learners and how to help them to understand more about their own
strengths and weaknesses as learners. Kolb's model of learning styles could be represented as
"flexible stable learning preferences"(Coffield et al., 2004) as ELT is not about fixed learner traits, but
rather a "differential preference for learning, which changes slightly from situation to situation”. For
instance, a student working in a group setting during a field trip for geo-caching might prefer to take a
turn at hands-on interaction with the device even though he would normally prefer reading textual
instruction at home alone. In his research, Kolb observed that some students have a definite
preference for certain learning activities(Kolb, 1984). For example, one student might prefer reading
lecture notes whilst another student might prefer working on an interactive simulation. “From this
emerged the idea of an inventory that would identify these preferences by capturing individual
learning differences.” (Kolb & Kolb, 2005)
Figure 1 shows how a learner can progress through the experiential learning cycle: concrete
experience (and new information acquired) is translated through reflection into concepts, which are in
turn used as guides for active experimentation and the choice of new experiences. Kolb stated that a
learner can begin the learning cycle at any one of the four modes, but that learning should be carried
on as a continuous spiral. As a result, knowledge is constructed through the creative tension among
the four modes and learners will be exposed to all aspects of learning: experiencing, reflecting,
thinking and acting.
Furthermore, there are four dominant learning styles that are associated with these modes:
converging, diverging, assimilating and accommodating(Kolb, 1984).
According to Kolb, the four learning styles are:
391

Stella Lee et al.
Converging (AC and AE focus) - interested in practical applications of concepts and theories. A
learner who has this preference is good at problem solving, decision making and practical
application of ideas.
Diverging (CE and RO focus) - interested in observation and collection of a wide range of
information. A learner who is strong in this preference is imaginative and aware of meanings and
values. They are interested in people and are socially inclined.
Assimilating (AC and RO focus) - interested in presentation and creation of theoretical models. A
learner leaning toward this preference is more concerned with ideas and abstract concepts than
with people.
Accommodating (CE and AE focus) - interested in hands-on experiences. A learner with this
preference likes doing things; carry out plans and trial-and-error method.
Figure 1: Kolb's Experiential Learning Cycle
Our research is grounded in individualized instruction and the intention is to make explicit to the
learners why we are employing this instructional design approach. To further Kolb's take on distinctive
learning preferences and to be able to provide options on learning activities accordingly, ELT has
been used to design task-level content and skills in Java programming along the two dimensions of
Concrete Experience-Abstract Conceptualization (CE-AC) and Reflective Observation-Active
Experimentation (RO-AE). Content for the Java programming course has been redeveloped to
present the course online content along the four modes of learning. The four modes of learning will be
discussed in details in Section 3.
2.2 Self-Regulated Learning (SRL) theory
Self-Regulated Learning (SRL) fits in well with a learner-initiated eLearning model because in an
individualized learning environment, learners are often left to their own devices, and a flexible
eLearning system can help make them aware of their own thinking, monitoring, planning and
evaluating personal progress against a standard, and motivation to learn(Boekaerts & Corno, 2005;
Butler & Winne, 1995; Perry, 2006; P.H. Winne & Perry, 2000; Zimmerman, 1990). When learners
adapt their approaches to learning, learning is said to be self-regulated(Winne, 1997). Self-regulated
learning (SRL) states that learners not only need to regulate their performance, but also how they
learn. Literature shows that self-regulation can positively affect learners’ achievement(Azevedo,
2005). Self-regulated learning (SRL) is “an active, constructive process whereby learners set goals for
their learning and then attempt to monitor, regulate, and control their cognition, motivation, and
behavior” (Pintrich, 2000). Furthermore, Hadwin, Wozney, & Pontin (2005) revealed that ‘ordinary’
collaboration (as found in traditional online forums and chats) is insufficient to transform learners’
everyday “reflection” into productive “regulation”.
392

Stella Lee et al.
What sets self-regulated learners apart is their awareness of when they know a skill or fact and when
they do not, at a meta knowledge level - i.e. they plan, set goals, organize, self-monitor, and selfevaluate
thorough out their studies(Zimmerman, 1990). In addition, self-regulation works best when
learners are provided with continual feedback concerning the effectiveness of their learning approach.
This is something that an adaptive system can provide as it can support and reinforce self-monitoring
techniques as well as self-regulated learning strategies. We mainly focus on this aspect of selfregulation
in relation to the design of an eLearning system wherein students can receive feedback if
they choose to do so.
Our model’s SRL interface includes components that allow students to self-reflect along with
components that engage them in self-regulation. On one hand of the spectrum, self-reflection could
be as simple as a thinking process made self aware, the intangible 'ah ha' moment of a conceptual
breakthrough; on the other hand, self-regulation could be more tangible as a system or a tutor can
observe what students do after they self-reflect. For example, in debugging, a student can self-reflect
on errors identified by the compiler at the end of each compile of the program being developed. The
system can track/record the number of errors and warnings faced by the student at the end of each
compile. The system can also classify the types of errors encountered by a single student over
multiple sessions of program development. Looking at this list of errors is an act of self-reflection.
However, self-regulation takes it one step further. The student may try to identify most common errors
and warnings he/she faced, take notes on how he/she resolved these common errors and warnings,
and refer to these notes when he/she encounters these common errors and warnings when writing
another program. This "proactive self-reflection" is what we identified as self-regulation even thought
at this stage, we are not planning on tracking the end results of using the study skill tools, it is the
provision of these “how to” study skill guides and tools embedded as options for students to assist
them in becoming better self-regulated, self-reflected learners that we are interested in.
3. Program of research
3.1 Participants
In order to undertake this program of research, 60-80 voluntary participants who enroll in COMP 268 -
Introduction to Java Programming course at a distance learning university in Canada will participate in
the study. The participants are adult learners in various age groups with various programming skills.
This is a 100 level (all introductory computer science courses start with the number “2” at this
institution) course and there is no prerequisite. We expect participants to have at least basic
computing and distanceLearning skills.
3.2 Material and procedure
One unit from COMP 268 - Introduction to Java Programming course consists of five learning
concepts has been redesigned and redeveloped to presents a series of learning activities designed
according to the ELT theory - i.e. the material has been designed to be presented in four different
learning modes: watching, discussing, diagramming, and trying out. These four modes correspond to
ELT’s four learning dimensions as discussed in Section 2. For example, for “watching” mode, the
material might be presented in a YouTube video tutorial with narration and textual explanation. A
learner is free to choose any of the learning activities to begin his/her study. The system doesn’t
initiate or suggest any particular learning paths. The idea is that we are giving the learner complete
control with respect to which learning activity suits his/her learning preference at the time of the study
while providing learning options. According to ELT, a learner can begin at any stage of the learning
cycle, but it is most beneficial for acquiring new knowledge if he/she would go through them all
eventually to fully understand and apply this new knowledge. He/she might or might not choose to go
through them all; the decision is his/her to make. If at the end of any chosen learning activity, the
learner feels that he/she has a good grasp of the material, he/she can opt to skip the rest of the cycle
and go to the next topic. A post-test is available at the end of the unit for learner to self-assess his/her
knowledge both at the domain and meta level. In another word, learners can test how well they have
learn a certain programming concepts in Java as well as whether the associated key study skills are
helpful to them in regulating their own learning and deploying them as learning strategies. The posttest
is a formative test and it will not affect the participants’ grade in their overall course performance.
In addition, as supplementary learning material, a SRL theory based interface titled “Key Study Skills”
is available on the upper right-hand corner of the webpage to assist with learning about meta
393

Stella Lee et al.
knowledge. Meta knowledge in terms of key study skills for studying programming have been selected
as: note taking skill, reflection and discussion skill, conceptualization skill, and problem solving skill.
For example, within the “Watching” activity, the key study skill we emphases is “note taking”. Learners
can choose to access this supplementary material at any given time during his/her study. The design
consideration is that the key study skills appear as an unobtrusive interface that sits on the upperright-hand-corner
of the main content. Learners can choose to engage and learn how the “note
taking” skill helps with the “watching” activity (i.e. how to take better notes while watching a tutorial
video online and what are the appropriate tool to use for note taking), or he/she can just ignore it and
carry on with watching the YouTube tutorial video on its own.
One pre-test has been developed with 25 questions for domain-knowledge and 20 questions for
meta-knowledge (study skills). In this test we are looking at two types of data:
Test score for the domain-knowledge - to gauge how much a student has already know about
control structures in java programming prior to this unit. This test score will be compared to the
post-test domain-knowledge test score.
Test score for meta-knowledge (study skills) - to gauge a student’s study skill competency. This is
more of a self-report rather than an objective measurement of specific study skills. This test score
will be compared to the post-test meta-knowledge test score to identify whether the study skills
provided help improve students’ overall study strategies and make them aware of certain metalevel
skills.
Similar to the pre-test, one post test is available with 25 questions for domain knowledge and 20
questions for meta-knowledge (study skills). It is designed to measure the efficacy of each learning
mode. Questions are generated based on the content from select mode of each learning concept. For
each answer, the system will then give feedback and suggestions according to the test results, and
direct the student to focus more in the learning mode in which the student is weak. The aim is to help
learners to become more efficient and aware in self-monitoring their learning.
It is intended that there will be an end-of-unit survey and a follow-up structured interviews with 6-10
participants two weeks upon the completion of the study. The survey aims to measure learner
experience, usability, user satisfaction and perceived controllability of the system. At the end of the
survey, we will invite participants to take part in a 40-min structured interview using Adobe Connect.
Participation in the interview is entirely voluntary and has no effect on their grades. The purpose of
the interview is to discover more details about the learners’ attitudes and impression about the
experiment, and give learners a chance to elaborate on the survey results. Questions will also be
geared towards whether they find the system and the options provided helpful, in both the domain
specific content as well as the meta-cognitive activities, i.e., study skills.
3.3 Research questions
This study was designed to answer the following research questions:
How would the two named learning theories, ELT and SRL, be useful in the production of
constructive adaptive system?
How might such a system be tested and evaluated in a real context?
What are the effects of such a system on:
Domain-knowledge competency
Meta-knowledge competency
What are the effects of such a system on:
Learners’ experience
System usability
User satisfaction
Perceived controllability
394

3.4 Modeling approach
Stella Lee et al.
The form of modeling we used is a learner-directed model. A learner initiated system as a system that
places full learning control in the hands of the learner. The system does not automatically adapt or
assign any differentiated content to the learner(Marzano, 1992). Simply put, the system serves a
suggested guide to the learner where he/she can accept or reject the suggestions. This approach
enables personal autonomy, active involvement and engagement with the content and serves as a
motivational factor for online learning. For learning styles, learners will self-select from the multiple
learning content representations, so there is no need for an automated system intervention. For prior
knowledge and skills, the comparisons of the pre-test and post-test results will inform the system as to
what to recommend and adapt at the end. If the post-test results indicate a deficiency in certain
domain knowledge area, then the system will provide appropriate feedback to the students. It is not so
much a specific prescription; rather, it is used as reference guide.
4. Example of the course
Prototype of the eLearning system has been developed in Moodle (an open source Learning
Management System) providing alternative content for Unit 3 of the course for our study. Traditionally,
Unit 3, similar to the rest of the course, is written in a linear, textual format with heavy reference to the
textbook.
Currently, after the redesigned of the unit, unit 3 composes of five learning concepts and each
concept starts with the same home page interface that present the four modes of learning. The five
learning concepts are: If-Else Statement; Loops; Break, Continue, and Return; Switch, Try-Catch-
Finally, and Labels; and Recursion. Figure 2 below illustrates the Unit 3 layout design for the course
while Figure 3 shows the home page of each learning concepts.
Figure 2: Unit 3 home page as viewed by students entering into the experiment
Once a student access the learning concept home page, a circular wheel interface with four
quadrants will be presented to them. Each quadrant is linked to a different but comparable learning
activity based on Kolb’s experiential learning cycle: experiencing, reflecting, thinking and acting. For
instance, the experiencing, the learning activity is presented as a series of tutorial videos on YouTube
on how to program If-Statements (see figure 4).
395

Stella Lee et al.
Figure 3: Home page for learning concept 1 - If-Else Statement
Figure 4: YouTube tutorial videos on how to program If-Statements under the “Watching” quadrant
learning activity
Twenty study skills --five successive skills for each of the four learning modes-- have been developed.
The four key study skill areas we focus on are: note taking, communications, conceptualizing, and
problem solving skills. Each study skill is paired with a tool for learners to try out and practice these
skills. Figure 5 shows that the Note Taking skill is being presented with Evernote, a note-taking
software. The rest of the study skills are paired with the following: communications – Moodle
discussion forum; conceptualizing – FreeMap (mind mapping tool); and problem solving – BlueJ (an
integrated development environment tool).
396

Stella Lee et al.
Figure 5: One of the key study skill – Note taking skill along with suggested tool to use
5. Discussion
The research reported here relates to the design of a quite significant implementation of an eLearning
system in a pragmatic educational context based on constructive models of learning. The approach
differs in several ways from other approaches we have used in the past. In the first place the
modeling approach is intended to foster a constructive approach to learning. To this end two learning
theories have been employed in the design of the system. Modeling approaches we have used in the
past have been to some extent instructive in that they have been used (Adisen & Barker, 2007;
Adisen, Barker, & Britton, 2004; Barker, Jones, Britton, & Messer, 2002; Brusilovsky, 1998). In the
authors’ opinion, eLearning 2.0 implies self-directed learning and a student taking personal
responsibility for his or her learning. The question remains: how can a system based on a modeling
approach foster constructive rather than instructive learning? The ways that student models have
been used in the past to recommend or prescribe presentation can be seen as fostering instruction
rather than construction of knowledge. Secondly in our approach the model we use relates directly to
the tasks that learners are undertaking and their approaches to them, rather than to models of an
individual learner. Our modeling approach does not attempt to model the characteristics or
knowledge of a learner, but rather models the relationship between the task the learners are
undertaking and the level of confidence, skill and general strategy they are using. The support and
presentation strategy provided for learners is therefore not intended to deliver instruction differentially
based on settings of a student model, but rather is intended to help the learner to construct his or her
own learning path through the material in order to solve a problem or to learn new material related to
a problem. The use of learning theory or cognitive style was previously used in our research to
classify or categorize learners(Barker et al., 2002). In the current approach we have avoided this
method and have instead modeled the tasks and the approaches to solving them that best fits the
needs of an individual learner. In the next stage of the research a full-scale evaluation of the initiative
is currently underway with learners at an online university in Canada. It is hoped to gain an
understanding of how this approach may be of benefit to learners.
References
Adisen, A., & Barker, T. (2007). Supporting the diversity of the ELearning 2.0 learners: The development of a
psychological student model. Paper presented at the E-Learn 2007 Conference, Quebec City, Canada.
Adisen, A., Barker, T., & Britton, C. (2004). Investigating the potential of mental models in adaptive user
modelling. Paper presented at the Proceedings of HCI 2004: Design for Life, Leeds Metropolitan University,
Leeds, UK.
Azevedo, R. (2005). Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of
Self-Regulated Learning. Educational Psychologist, 40(4), 199-209.
397

Stella Lee et al.
Azevedo, R., Cromley, J. G., Seibert, D., & Tron, M. (2003). The role of co-regulated learning during students’
understanding of complex systems with hypermedia. Paper presented at the Annual meeting of the
American Educational Research Association.
Barker, T., & Adisen, A. (2005). Experiments on visual and verbal skills: The development and testing of a skills
profile. Paper presented at the Proceedings of the European Learning Styles Information Network
Conference, Univerity of Surrey, Surrey, England.
Barker, T., Jones, S., Britton, C., & Messer, D. (2002). The use of a co-operative student model of learner
characteristics to configure a multimedia application. User Modelling and User Adapted Interaction, 12, 207-
241.
Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and
intervention. Applied Psychology: An International Review, 54(2), 199-231.
Brusilovsky, P. (1998). Adaptive educational systems on the World-Wide-Web. Paper presented at the Proc. of
Workshop "Current Trends and Applications of Artificial Intelligence in Education" at 4th World Congress on
Expert Systems.
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of
Educational Research, 65, 245-281.
Coffield, F., Moseley, D., Hall, E., & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning: A
systematic and critical review. London: Learning and Skills Research Centre.
Fischer, G. (2001). User modeling in human-computer interaction. User modeling and user adapted interaction,
11, 65-86.
Hadwin, A. F., Wozney, L., & Pontin, O. (2005). Scaffolding the appropriation of self-regulatory activity: A sociocultural
analysis of changes in teacher-student discourse about a graduate research portfolio. Instructional
Science, 33, 413-450.
Hadwin, F. H., & Winne, P. H. (2001). CoNoteS2:A Software Tool for Promoting Self-Regulation. Educational
Research and Evaluation, 7(2-3), 313-334.
Kolb, A., & Kolb, D. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher
education. Academy of Management Learning & Education, 4(2).
Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs,
NJ: Prentice-Hall.
Koohang, A., & Du Plessis, J. (2004). Architecting usability properties in the eLearning instructional design
process. International Journal on ELearning, 3(3).
Liu, Y. & Matthews, K.S., (2005), Vygotsky’s philosophy: Constructivism and its criticisms examined, International
Education Journal, 2005, 6(3), 386-399.
Marzano, R. J. (1992). A different kind of classroom: Teaching with dimensions of learning. Alexandria, VA:
Association for Supervision and Curriculum Development.
Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning Styles: Concepts and Evidence.
Psychological Science in the Public Interest, 9(3), 105-119.
Perry, N. E., Phillips, L., & Hutchinson, L.R. . (2006). Preparing student teachers to support for self-regulated
learning. . Elementary School Journal, 106, 237-254.
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In P. P. M. Boekaerts, & M. Ziedner
(Ed.), Handbook of self-regulation (pp. 451-502). San Diego, CA: Academic Press.
Weber, G. (1999). Adaptive learning systems in the World Wide Web. Paper presented at the 7th International
Conference on User Modeling, UM99, Banff, Canada.
Winne, P. H. (1997). Experimenting to bootstrap self-regulated learning. Journal of Educational Psychology,
89(3), 397-410.
Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. B. P. Pintrich, & M. Seidner (Ed.),
Handbook of self-regulation (pp. 531-566). Orlando, FL: Academic Press.
Zimmerman, B., J,. (1990). Self-regulated learning and academic achievement: An overview. Educational
Psychologist, 25(1), 3-17.
398

Can eLearning Enhance Practice-Based Design Courses?
Jake Leith, Joanna Zara and Malcolm McInnes
University of Brighton, Brighton and Hove, UK
J.Q.Leith@brighton.ac.uk
J.M.Zara@brighton.ac.uk
M.R.McInnes@brighton.ac.uk
Abstract: In recent years the focus for UK Higher Education has been on improving student experiences;
technology has played a crucial role in transforming the curriculum, evidenced in a series of initiatives funded by
the Joint Information Systems Committee (JISC) (HEFCE, 2009). The adoption of online learning and teaching
techniques on practice-based courses, within the field of art and design, have often been seen as controversial,
due to the vocational nature of the discipline. This paper seeks to understand, through the application of action
research, what affect the incorporation of blended learning has had on the professional practice element of a
practice-based course and compare students’ responses to results published by one of JISC’s completed
programmes, ‘Learner Experiences of eLearning.’ (JISC, 2009) Objectives To identify how a blended programme
of eLearning and face-to-face teaching of business and professional practice can support and enhance learning
in design-led practice-based courses To understand how the eLearning method of teaching delivery can
contribute to flexibility of learning in design-led practice based courses. To identify areas where such an
approach contributes to the best use of resources in response to the impact of socio-economic conditions on the
education sector. Work to be carried out To design and live test the delivery of a business and professional
practice programme using a blend of eLearning and face-to-face teaching. To identify the perceived/real benefits
and limitations of such an approach by the analysis of interviews/focus groups/questionnaires with students and
staff from the School of Architecture and Design at the University of Brighton. To identify the elements of course
delivery that best suit an online approach and those that are more suited to a face-to-face model. To identify the
key resources required for this eLearning method of delivery and how they differ from those required for a more
traditional face-to-face approach. Compare cohorts who have adapted to the blended learning model of teaching
from the more traditional face-to-face learning, with those who have experienced blended learning from the
beginning of their course.
Keywords: online, blended, practice-based, eLearning, design
1. Methodology
Art and design education within the Faculty of Arts at the University of Brighton, follows a recognised
constructivist tradition within the disciplines, encouraging a continuous process of inquiry,
experimentation and innovation that developments a student’s working methods and identity. This
personal construction of knowledge is augmented by a collective learning experience that enables
students to place their work within the professional context of the creative industries.
The professional practice components of the course at level 4 and 5, also adopt a constructivist
approach, with the production of a student-centred group project that is facilitated by the tutor, which
encourages students to work together and take responsibility for their learning against a set of defined
criteria. The pedagogical design of the online materials and activities were underpinned by this
existing Constructivist approach. The principles outlined by Meyes & De Freitas (2004) were
employed to provide a constructivist learning environment that gave students ownership of the task,
helped model thinking skills, provided a scaffold for the project, guided discovery and afforded an
opportunity for reflection.
Activities and learning materials re-purposed or produced for online delivery were aligned with the
learning outcomes of the project and were specifically designed to help students complete
components of the assignment alongside face-to-face delivery. (Biggs, 1999).
An action research model outlined by the Open University (2005) was adopted as a method to assess
the effectiveness of the changes implemented and recognized the commitment of the tutors to the
incorporation of technology as a continuous cycle of development. It enabled them to be active
participants in the process of exploring student perceptions of the learning experience, while gaining
feedback and reflecting how to affirm good practice and make further adaptations to the blended
learning mix. This paper represents the findings from the first cycle of this action research.
Blended learning within the context of this paper is defined as the delivery of a combination of face-toface
and computer-mediated learning (Graham, 2005). But also recognizes the complexities and
399

Jake Leith et al.
opportunities blended learning can provide through a variety of new media resources to enrich the
student experience, described by Littlejohn and Pegler (2007).
2. Introduction to the project
The two practice-based design courses examined in this project are the Fashion & Textiles and 3D
Design & Materials Practice undergraduate courses run within the School of Architecture and Design
at the University of Brighton. Both courses have contained a “Business and Professional Practice” unit
- in varying forms - for over fifteen years.
Students studying on these 2 courses are predominantly taught in studios and workshops for the
practical aspects of the course and these facilities do not have access to computers. Professional
Practice is one of the academic components of the course that requires students to produce written
work for assessment that requires IT.
The make-up of the first and second year cohorts (who are the focus of this research to date) is
similar to previous years, with a high percentage of females (86% across both year groups) and a
high percentage of students with dyslexia (23.5% across both year groups, compared to what is
already considered a high level of around 6.5% within the School of Architecture and Design as a
whole).
Delivery and assessment of the first year unit is based on a “Business Case Study”: students work in
groups (of 6 or 7) developing a business case study by observing a local creative business.
Delivery and assessment of the second year unit is based on a “Group Marketing and Finance
Assignment”: students again work in groups to investigate either a high street retail brand or a design
practitioner, and use the results of their investigations to propose a priced and costed range/body of
work which could be produced and sold by the brand/practitioner they have examined.
In the academic year 2010/11 new online content delivery was introduced into these Business and
Professional Practice Units. The aim was to provide an online enriched environment for both students
and staff that would also have the potential to significantly streamline the 2009/10 programme, in
terms of delivery, co-ordination, the volume of assessment-related administration and in handling
increasing sizes of cohorts.
This paper is intended to illustrate the impact of such an approach in the context of a practice-based
design course; clearly the methods and technologies used are not new, and have been used in other
non practice-based design courses both within the University and further afield, for some considerable
time now.
Whilst there has been some individual engagement with online learning tools for some final year
students on the courses referred to (e.g. e-portfolios and the online submission of assignments), the
programmes studied here represent the first strategic shift towards a fully blended learning
programme within the School of Architecture and Design, with specific online learning activities for
students and online support tools for teaching staff.
3. Reasons for adopting a blended learning approach
The change to a blended learning approach had three main drivers:
A recognition that this was an opportunity to provide an enriched learning environment for both
students and staff.
A decision to break the assessment process down into more “manageable chunks” across the
academic year, thus avoiding the problems associated with a heavy assessment burden on staff
at one time of year, but without adding to the overall time spent by staff on assessment related
administration.
Socio-economic conditions: in particular the need to streamline resources and reduce HPL (hourly
paid lecturer) staffing whilst handling increasing student numbers. We have yet to see the full
impact of funding changes upon the University, but this change in approach should enable us to
respond as flexibly as possible, whilst maintaining the quality of the courses we deliver. We need
400

Jake Leith et al.
to bear in mind students’ expectations – to graduate with a level of employability skills that reflect
the investment they have made in the course.
The sector has placed increasing emphasis on the importance of effective eLearning in higher
education. The HEFCE report ‘Effective Practice in a Digital Age’ highlights the current position. “Yet
technology-enhanced learning remains a source of concern for institutions. This finding may reflect
the extent to which supporting such practice makes demands on institutional resources, but also
suggests a need to understand better how to design and support learning involving technology.
Access, especially to the internet and social software, may have increased, but this does not mean
that technology is always used to its best advantage, either by teachers or learners.” (HEFCE, 2009)
4. Course changes and development
During this first year of implementation the focus has been upon the identification of and the
appropriate adaptation of existing content as a practical approach to undertaking learning re-design,
given the limitation of available resources. The project demonstrates how existing materials can be
repurposed to take advantage of some of the recognised benefits of eLearning (e.g. flexibility,
learning at one’s own pace - perhaps particularly important in the case of the cohorts here, given the
high proportion of students with dyslexia). It also acknowledges the findings of the LXP2 final
synthesis report that technology should and can discretely facilitate learning for those with disabilities,
e.g. podcasts for dyslexic students. (Sharpe, 2009)
There was also a deliberate decision taken on the types of technology chosen. For instance, in order
to retain the “personalisation” that exists in face-to-face delivery, regular audio podcasts and screen
cast presentations were developed. The use of Turnitin software for online submission also enabled
the use of formative peer assessment.
Course content was structured around a “week by week” approach. Students accessing the unit were
able to click on a link which would take them to the specific online content for that week, together with
a note of any “live” delivery scheduled. This was supplemented by a traditional full printable academic
year timetable available for download from the VLE (Virtual Learning Environment).
Online content and tools provided included:
Weekly podcasts from the course leader
Staff introducing themselves in short video clips
Videos of previous years’ students talking about the group projects they had undertaken
Online access to previous years’ work submitted
Online lectures in various formats (PowerPoints captured with Camtasia Relay; PowerPoints with
recorded audio, recordings of live lectures)
Online exercises;
Links to online resources (for instance, i-player, Box of Broadcasts, market intelligence
databases, reading lists);
Online group tools within the VLE (email, blog, file exchange);
Discussion boards (for use by students when undertaking group projects);
Online submission and peer assessment (Using Turnitin software).
5. Research methods
Following the delivery of the units, initial questionnaires were sent to each year group studied; non
course-specific questions were based as closely as possible upon those used in the LEaD (Learners’
Experiences Across the Disciplines) project, undertaken by the University of Edinburgh as part of
phase 2 of the JISC Learner Experiences of eLearning Programme conducted between 2005 and
2009. This project had a similar focus, looking at the experiences of students upon entering HE, but
was across a range of disciplines. The intention was to make use wherever possible of baseline data
in order to understand whether there were any identifiable differences that could be explained in the
context of the particular practice-based course focused on in our own project.
401

Jake Leith et al.
Data was collected anonymously after the group projects had been handed in; access to the survey
questions was kept open for both year groups for one month. Some students may have received
marks for the units in question by this stage, but due to the nature of the questions asked this should
not have had a significant impact upon the replies given.
20 responses were received from the first year cohort of 113 students, and 13 responses from the
second year cohort of 96 students. We recognise that this is a small sample, but the responses seem
to be representative of the cohort as a whole taking into account the qualitative data in the form of
observations captured by staff over the course of the unit.
One feature of the data captured so far is that there appears to be little discernable difference
between the responses of the different year groups surveyed – for the next stage of this research we
will be conducting focus groups from within each year group and will be looking for further evidence
on this point.
We have also documented the experiences of the professional practice staff team involved in the
implementation of the programme.
6. Findings
6.1 Ownership and use of technology
Computer ownership is almost ubiquitous: only one respondent did not have a computer of his/her
own for exclusive use for University studies. This is comparable to the data collected from the LEaD
project (94% PC ownership). Ownership of other electronic devices was similar across both cohorts
as shown in Figure 1:
Figure 1: Ownership of electronic devices, with comparison between cohorts
Over 80% of students from both years owned an MP3 player, digital camera and some form of
external memory (e.g. usb sticks or external hard drive). This finding matches that of the LXP2 report
which states "There is agreement that the majority of learners arrive at university confident, positive
and enthusiastic about their use of technology (LeAD, 2009); (STROLL, 2009); (LexDis, 2009). The
projects that conducted surveys show that most learners arrive with their own personal technology,
notably laptops and mobile phones." (Sharpe, 2009)
402

Jake Leith et al.
Figure 2 below shows the interesting diversity in the popularity of internet-based services: for example
social networking sites (e.g. Facebook) were used regularly by the vast majority of students, whilst
Twitter was used regularly by only 3 respondents. This demonstrates that we should not take for
granted students’ use of social media across the board.
Figure 2: Levels of usage of internet based services across both cohorts (Level 4 + Level 5)
The popularity of social networking sites might explain the responses to a later question about
organising group assignment activities, where communication by Facebook was more popular than
the group discussion areas set up in the VLE, particularly amongst the second year students where
no respondents used these.
6.2 Levels of confidence in the use of technology
Students in the sample expressed varying levels of confidence in their use of technology, but fewer
than 10% across both year groups considered themselves unconfident as shown in Figure 3:
Figure 3: Levels of confidence in the use of technology
Most students saw themselves as technologically literate. However, some criticism of the way the VLE
worked (see comments below) is likely to come from their own experience with more “user friendly”
social networking sites. We therefore need to recognise the need to educate students on the use of
403

Jake Leith et al.
their informal skills in the formal environment of a VLE, whilst trying, within the limitations of the
learning platform provided, to emulate some of the experiences students prefer. Again these findings
are backed up by the LXP2 Final Synthesis Report that found "most learners, despite access to
technology and self-reported confidence, aren’t appropriating the technology they know into their
study.” (Sharpe, 2009)
6.3 Response to the blended learning approach
Email and announcements via the VLE were considered the most helpful form of communication
between staff and students (83% considered email useful or very useful, 75% considered
announcements useful or very useful); towards the end of the year the department introduced a text
messaging service which was not included in the questionnaire. Anecdotally this appears to be very
popular and follow up interviews will be designed to cover this. Figure 4 shows the responses of both
cohorts combined:
Figure 4: Forms of online communication considered the most useful across both cohorts
There was no conclusive evidence on the perceived benefits of online lectures, with both groups
divided approximately equally between finding them useful/not useful. A significant number (28%) of
respondents answered “don’t know/unsure” when asked if videos had been useful or helpful – in this
regard a comment by one respondent may be significant here: “I think some people had trouble
accessing podcasts and videos. My internet access at home sometimes cannot support these
functions”.
Comments on the online materials in general indicated that it was the accessibility of materials that
students found most helpful: “they were always there and accessible”; “could access it at a computer
at any time”; “could individually work at own pace/when convenient yet keep group updated without
sending large files via email”.
Comments on the content of materials (both online and face-to-face) indicate that, as might be
expected, there is a wide range of abilities and previous experience in the cohorts examined with
regards to business and professional practice, and consequently students’ expectations varied widely
(“the topic was approached without first covering the basics so lots of people felt quite lost and
bewildered”; “I felt some of the “introduction” lectures were incredibly basic..”). Online learning
activities will be particularly useful in providing more differentiation in future years.
Difficulties highlighted included a lack of consistency in terms of material location within the VLE:
“sometimes – and this is across all courses – where things are located can be confusing…”; student
central [the VLE] is difficult and illogical”.
404

Jake Leith et al.
One comment in particular highlighted the significance of problems associated with lack of internet
access at crucial points in the course: “for the entire first term we had problems installing the internet
at home, this made connecting with information that was only available on the internet an obstacle in
accessing information. It shouldn’t be assumed that everyone has full internet access”.
Students still valued the face-to-face elements of the teaching – in particular talks from visiting
lecturers (“I enjoyed the Tom Savigar lecture the most, it was very interesting”; “BJ Cunningham
lecture was great EVERYONE was talking about it”), and face-to-face tutorials. We will be looking at
the possibility next year of including live web conferencing that should simulate many of the benefits
of face to face talks from external practitioners whilst reducing the number of hours required of their
time.
Opinion was divided on the usefulness of online peer assessment (40% found it useful or very useful).
7. Staff team experiences on the impact of the design and implementation of
blended learning
7.1 Designing the new course content
The short timeframe in which the new course content had to be implemented meant that in many
cases the material was being produced “as we went along”.
The weekly audio podcasts provided by the Course Leader during this academic year were intended
to aid students with general feedback in an informal, ‘human’ manner, more akin to the aural qualities
associated with face to face delivery. These included ‘tip of the week’, which became a weekly feature
of the podcast. The Course Leader, new to the production of podcasts, found it helpful to make bullet
points and prepare written elements of text before making the audio recording. An added advantage
on one or two occasions was the ability to record podcasts in advance and then use the VLE’s ‘timed
release’ function to publish it for students at a predetermined future date.
There was a steep learning curve for staff and HPL’s involved in the production of captured lectures.
Again short notice and lack of time available resulted in perhaps not enough “redesign” of content. We
also recognise the need to address intellectual property issues surrounding the capture of lectures by
HPL’s.
7.2 Delivery
As noted above, email was a prime method of communication for the majority of students.
Unfortunately due to circumstances beyond our control, some first year students were not able to
access their university email for several weeks at the start of the first term. This caused considerable
problems, but validated the need for a blended approach to learning, as the face-to-face introductory
and project briefing sessions at the start of the term became essential in allowing us to recognise and
deal with the problem.
The online format of the “week by week” programme noted above was supplemented by a hard copy
timetable and project briefing document. These were also stored on the VLE in downloadable form.
We realised early on that the documents (in particular the timetable) needed to be more visually
consistent with the online format, as this could have assisted the students in navigating the VLE.
7.3 Assessment
Online formative peer assessment (using Turnitin software) was conducted to help focus the students’
thinking and provide an opportunity to critically reflect on each other’s projects. This aspect of the
assessment process was complemented with a face to face tutorial session. Students appreciated
being able to see what other groups were doing and assess their progress against this.
Several additional points of assessment were introduced by breaking the projects down into “chunks”.
This was helpful not only to give students structure to their project but also in providing staff with
clearly defined stages to pick up on progress with students. It was also recognised during the delivery
of the project that these stages needed to be more clearly integrated with the delivery of online
materials to facilitate the students’ understanding of these.
405

Jake Leith et al.
For the summative assessment staff were obliged to use the School’s prescribed paper based
summative assessment feedback forms, it was therefore not possible to provide feedback
electronically, via the Turnitin software, and so staff did not benefit from the anticipated efficiencies in
administering this process. This will be addressed in the next academic year so that online marking
can be incorporated.
7.4 Technical support
Course design and implementation was conducted on an ongoing basis throughout the year of live
testing. Neither of the main staff involved have a technical IT background, so it was vital to have a
flexible and responsive point of contact for IT support within the University. One of the University’s
Learning Technologies Advisors supported the unit’s development and was available throughout the
course’s delivery. This proved invaluable in enabling staff to undertake the work involved.
8. Review of work carried out
The design and live testing carried out so far has highlighted a number of recommendations which are
set out below, and will be supplemented with further information gathered from student focus groups
in September 2011:
Elements of the course already identified as best suiting an online approach are those lectures
with more “technical” content – online delivery allows learners to follow these at their own pace,
and also allows differentiation of delivery to take place via online exercises;
Elements of the course that best suit a face-to-face approach fall in to two categories: the initial
introductory and briefing sessions (these proved invaluable in identifying online problems early
and also “humanise” the blended learning environment from the start) and secondly the lectures
from visiting practitioners.
Key resources: The infrastructure required for online delivery was already in place. Hardware
requirements for staff were minimal (e.g. large flat screen monitors for online review and
assessment, MP3 recorders for podcasts and P.C.’s with up-to-date web browsers).There has
been a reduction in the need for physical space for lectures – this is significant as the cohort size
requires the use of a large capacity lecture theatre. Staff have also been able to use their time
more flexibly – for instance podcasts and lectures being recorded in advance, and assessment
conducted without the need to access and share hard copy documents.
Comparison of cohorts: to date as already noted there has been little discernable difference in the
responses obtained from the two year groups. We will follow this up in our focus group work.
9. Conclusions
From a student perspective the online element has not been detrimental to the subject area.
The issues and concerns raised by students, are that they feel that they are computer literate but, do
not have the skills to translate this into formal learning.
Whilst they feel confident with their usage of computers and internet services in specific areas
(Facebook is used externally by many), students cannot translate the skills they have into the formal
learning arena. However their experience of these services make them a critical audience when
considering the design and quality of the online learning delivered through the University’s learning
environment.
It is important that students are provided with clear technical standards and understand the
technologies that will be used to deliver the online content and that time is built into the curriculum
allow time to try these out, to provide them the opportunity to resolve any technical glitches.
Whilst students’ response to the delivery online is broadly neutral, they are quick to recognise the
benefits associated with this form of delivery and the flexibility that this affords them.
Staff need to consider carefully the layout and design of the online space within the virtual learning
environment to ensure that this provides a logical and consistent experience to the students.
406

Jake Leith et al.
The online developments undertaken by Professional Practice has generated interest from the studio
based provision who have now begun to think how aspects of online learning might be used to deliver
instructional elements in the course, thereby freeing staff to devote greater time to working with
students on the creative process.
One example of this is in Fashion Design, through the delivery of digital pattern cutting. The traditional
face-to-face teaching for this unit will be supported by the delivery of online instructional tutorials,
demonstrating how to use the pattern cutting software. Another project recently launched within the
Faculty of Arts allows students to build e-portfolios through their own usage of an e-portfolio and
social networking web application (Mahara). One of the aims of this project is that the portfolios will
also feed into a central database from which material can be collected for alumni profiling.
10. Recommendations
The layout of the online content within the VLE will be kept as streamlined and user friendly in
order to emulate where possible the social networking spaces students are familiar with.
Supporting hard copy documents will be designed to be visually consistent with the online
presentation of the programme delivered.
However, we also need to manage expectations from the start, and we are examining the
possible adoption of the checklist for new students developed from the LEaD project.
Face-to-face contact at the start of the academic year is considered vital in orientating new
students and picking up problems early on.
Contracted visiting lecturers for the next academic year should be asked to provide an
introductory podcast to their lecture / workshop. This should aid students in their preparation for
sessions and provide a good ‘taster’ to the forthcoming lecture, ‘flagging’ its relevance to the
programme of study.
Online exercises should be developed to improve differentiation of teaching in the more technical
topics taught.
Summative assessment should be carried out online within the Turnitin software in order to
maximize the time saved by staff on assessment administration.
The introduction of live webinars with HPL’s and other visiting speakers should be investigated.
Acknowledgements
We would like to thank Adam Bailey University of Brighton Learning Technologies Advisor for his
support during the first year of this new course delivery and his contributions to this research paper.
References
Biggs, J. B. (1999) Teaching for Quality Learning in University, Society for Research in Higher Education and
Open University Press, Buckingham.
The Open University, (2005) Action Research, A Guide for Associate Lecturers. [online]
http://www.open.ac.uk/cobe/docs/AR-Guide-final.pdf
Mayes, T and De Freitas, S. (2004) JISC Design for learning key outcomes, Work package 1b, eLearning Models
desk study, Review of eLearning theories, frameworks and models. [online]
http://www.jisc.ac.uk/uploaded_documents/Stage%202%20Learning%20Models%20(Version%201).pdf
Littlejohn, A. and Pegler, C. (2007) Preparing for Blended eLearning, Routledge, London.
Graham, C. (2005) The Handbook of Blended Learning. [online], Wiley
http://media.wiley.com/product_data/excerpt/86/07879775/0787977586.pdf
407

Sophisticated Usability Evaluation of Digital Libraries
Stephanie Linek and Klaus Tochtermann
ZBW – Leibniz Information Centre for Economics, Kiel, Germany
s.linek@zbw.eu
K.Tochtermann@zbw.eu
Abstract: Digital libraries are an important headstone for distant education. Thereby, modern digital libraries
encounter not only the classical provision of documents, but also other related services like publishing portals or
lists of hot topics. Another upcoming challenge is the connection with Web 2.0. Thus, the functionalities of online
libraries will change and becoming more and more complex. Accordingly, the usability evaluation of these
complex functionalities has to be assured and adapted to the new challenges. The proposed methodology of
sophisticated usability evaluation follows a spiral model for prospective recommendations. The usability
evaluation is done by an iterative process. This is in line with the state of the art and most modern models in
usability engineering. However, in contrast to the existing models, we propose an explicit combination between
systematic quantitative investigations and focused usability studies. The proposed multi-method approach
addresses different important elements of a sophisticated scientific usability evaluation. The core elements of the
cyclic process are repeated usability benchmarking, focused usability studies, derivation of recommendations
and decision on the planned improvements in the face of the overall strategy. The repeated benchmarking allows
a quantitative measurement of advantages and drawbacks as well as improvements and impairments. The
quantitative benchmarking data can also help assigning the gravidity of usability-problems or the importance of a
specific innovation. Depending on the requirements of the users and the developments of the digital library, the
specific usability-studies could be qualitative or quantitative or a mixture of both. The overall aim of these studies
is the formulation of concrete practical recommendations. The recommendations have to be aligned with the
technical possibilities and strategic decisions. After the improvements and innovations were implemented, a new
benchmarking cycle can take place. The four elements can be flexible combined. This procedure ensures not
only a holistic and sophisticated usability evaluation, but also the openness for new challenges and opportunities
of the 21st century. The described approach will be exemplarily explained by the ZBW – Leibniz Information
Centre for Economics.
Keywords: iterative evaluation, usability, digital libraries, multi-method approach
1. Introduction: Digital libraries and the importance of usability evaluation
In modern times, it’s quite usual to regard to electronic literature instead of written documents.
Information (including specialist literature) is more and more searched among the internet and social
platforms are often used for knowledge sharing.
Even though a lot of libraries are online, many of the digital libraries appear a little bit old fashioned. In
most cases, there is only a lose connection with social platforms like Facebook or Xing. Also new
forms of learning and knowledge sharing (e.g., by means of serious games) are not considered. Even
worse, the usability of the web-based digital libraries and its services is often problematic and usability
evaluation is still an exception.
There are many slightly different definitions of usability. Besides accessibility, most of the definitions
are based on four elements: Effectiveness, efficiency, usefulness (appropriateness for subjective aims
of the user) and joy of use (see for example Rubin and Chisnell, 2008). The usability of Web sites in
general and digital libraries in specific is of rising importance. Nowadays the end-users expect webservices,
software and online platforms not only to be useful for the original purpose, but also to be
self-explaining, and quick and easy to handle. In the best case, the use of software is not only helpful
but also includes “joy of use”.
Thus, usability evaluation is of rising importance in the modern media world. For digital libraries an
adequate usability evaluation is the fundament for being capable of competing with simple search
instruments (like Google) that deliver quick and easy results. While classical digital libraries offer a
complete and high class literature list, many end-users prefer the “quick and dirty” search by Google,
Alta Vista or similar services – just because of the better usability.
If digital libraries take their role as educational service seriously, they have to assure a good (or at
least sufficient) level of usability as prerequisite to reach their target end-users.
408

Stephanie Linek and Klaus Tochtermann
2. Aims of usability evaluation of digital libraries
Usability evaluation should not only aim at the usability improvement of an existing online platform,
but also has to account for upcoming innovations and improvements. Additionally, usability studies
can also accompany the establishment of a completely new online platform or digital library (i.e., if a
normal library wants to go online.)
Overall, the usability evaluation of web-based digital libraries and their services can address multiple
goals:
Identification of specific and general usability problems of existing digital libraries
Addressing specific and isolated usability questions
Enabling a comparison with general usability standards and comparison with similar Web sites or
online-services
Quality check of innovations and improvements: Usability validation and measurement of
improvements and enhancements
Possibility to formulate concrete recommendations for usability improvements
Enabling and fostering the development of an online-platform of a library, that accounts for the
basic requirements of usability
Enabling and fostering the development of new online services of an existing digital library in the
face of basic requirements of good usability
Flexible methodological procedures of usability evaluations that could be applied in the case of
new forms of online services, e.g., Web 2.0 applications and game-based learning
The listed goals are not unique for digital libraries. Especially, the assurance of good usability and the
openness for innovations are main aims of every modern Web site and online services that have to
consider new technological possibilities and the growing requirements of end-users (Dumas and
Redish 1999). In this sense, the proposed methodology could also by applied for other online-services
and Web sites.
It is important to note, that not all of the listed goals are of equal importance. Depending of the actual
status of the Web site and online-services as well as the planned developments, different goals can
be in the centre of usability evaluation. For existing Web sites, that want to incorporate innovative new
applications the most important goal might be to have a flexible methodology that is apt for the
usability evaluation of the new applications. In contrast, for Web sites that are still in their early
development phases, a basic fundament of general usability standards could be the most important
goal.
Depending of the main goal of usability evaluation one has to choose the appropriate method. For a
comparison with general usability standards a quantitative standardized measurement instrument is
necessary. Contrariwise, for addressing prospective usability questions of planned innovations,
qualitative interview data might be more fruitful.
Nowadays there are many various methods available for usability evaluation, ranging from qualitative
methods with single expert-ratings to quantitative methods with a large group of end-users. Each of
these methods has its advantages and drawbacks. It has to be carefully selected which method is the
best to address a specific usability problem in a specific context. Furthermore, from a practical point of
view, it is also a question of resources. Even though, usability testing with end-users is often seen as
the “silver bullet” for usability evaluation, sometimes it is too costly and time intensive. In literature,
there is an ongoing debate which of the available methods is most efficient and effective for
identifying usability problems (overview is given by Rubin and Chisnell 2008; Dumas and Reddish
1999). However, this paper is not about the comparison of different methods of usability evaluation
but rather propagates a multi-method approach that is oriented on valid identification and elimination
of usability problems in the face of practical possibilities.
In most cases, usability evaluation has to address more than one goal and thus, more than one single
(assessment) method is necessary. Furthermore, the development of a Web site as well as the
accompanying usability evaluation is a repetitive process. On the one hand, usability
409

Stephanie Linek and Klaus Tochtermann
recommendations are needed for further developments of the Web site. On the other hand, the
ongoing development of a Web site results in new usability questions.
3. Overview of the proposed multi-method approach
The proposed methodological approach accounts for the interdependence between technical
developments and usability evaluation. Thereby, the need for multiple methods is explicitly
addressed. Overall, usability evaluation is composed of iterative cycles with the following elements:
Repeated benchmarking of the overall usability by means of standardized questionnaires: These
data deliver quantitative indicators of improvements based on a larger panel of users
Specific usability-studies on selected concrete usability questions based on a small sample of
users: These studies work with qualitative data, but in case might also require quantitative
indicators
Recommendations based on the results of usability-studies and the quantitative benchmarking
Improvements in the face of the usability-recommendations and strategic decisions
The iterative procedure is an important characteristic of the described approach. Usability evaluation
is seen as an ongoing process that has to account for new technical possibilities as well as for
changing requirements and expectations of the end users. This this is in line with the state of the art
and most modern models in usability engineering (Rubin and Chisnell 2008; Dumas and Reddish
1999). However, most existing models propagate iterative testing without a specification of the
measurements or how different methods could be combined in the iterative cycles.
Contrariwise, our proposed multi-method approach combines general benchmarking and specific
usability studies in the face of the needed requirements and strategic decisions. In this context, it is
important to note, that the whole process of usability engineering is conceptualized as an iterative
cycle, including repeated benchmarking and specific usability test as well as the formulation of
usability recommendations and strategic decisions.
The listed elements above could be arranged and combined in a flexible way. Thereby, quantitative
and qualitative methods will be combined to enable concrete usability recommendations that can be
adapted to rapid changing technology and growing user requirements.
4. Description of the single elements
4.1 Quantitative benchmarking with standardized questionnaires
Quantitative benchmarking with standardized questionnaires is necessary for a systematic ongoing
usability evaluation and development of a digital library (as well as for any other long lasting
professional Web service). Even though usability studies with pure qualitative data might be helpful to
identify the most urgent usability problems, quantitative data with standardized measurement
instruments are necessary to receive the required input for ongoing developments and a quality check
of a digital library.
Quantitative usability data have the following main advantages (see also Langdridge and Haggerjohnson
2009):
Possibility of a priority ranking of usability problems
Quantification of improvements and enhancements (as quality check)
Possibility of a comparison with general standards of usability
Possibility of a comparison with competitors and alternative digital libraries (by the use of
standardized existing questionnaires)
Especially the priority ranking of usability problems is of high practical relevance, because normally
there are not enough resources for addressing all usability issues from the developer’s side.
Additionally, there might be a conflict between different aspects of usability. A classical example is the
balance between simplicity of handling and the appealing design of a Web page. While qualitative
data identify both problems, the quantitative measurement enables a decision for the developers: to
make it either more simple (even though design will be boring) or to make the design more
410

Stephanie Linek and Klaus Tochtermann
sophisticated (even though it will be more complicated). This illustrates also the necessity to select
quantitative instruments that are apt for the specific case.
In principle, many different quantitative measurements are thinkable for usability benchmarking.
Objective behavioral video data or logfile protocols as well as subjective interviews or questionnaire
data can be analyzed in a quantitative way. However, the analysis of video data and logfile protocols
is very resource-intensive and thus, often not feasible for a big sample or a broad variety of variables.
Contrariwise, a questionnaire is timesaving and a less resource-intensive instrument. It can be easily
distributed among a big sample of users; in case also online-surveys are possible.
The use of a standardized existing questionnaire allows a valid and reliable comparison with other
Web sites or services and thus, enables a comparison with competitors or alternative digital libraries
as well as a comparison with general standards of usability. For the assessment of general usability
several standardized questionnaires are available: short ones (e.g., System Usability Scale - SUS by
Bangor, Kortum and Miller 2009) as well as long ones (e.g., ISONORM by Prümper 1999 or
IsoMetrics by Gediga, Hamborg and Düntsch 1999).
However, sometimes the use of standardized questionnaires might not be sufficient since they are
very general and not apt for a benchmarking of special services, e.g., the quality of the specialized
content. In such cases, an additional scale has to be developed for the evaluation of specific
important features, for example for the content quality of the results of a literature search.
4.2 Focused usability studies
Focused usability studies target at specific usability questions. These questions could regard to the
usability of new features, to the improvement of identified usability problems, to planned innovations,
to the redesign of the Web site and to many more. In this sense usability studies are aligned with the
requirements of end-users, the specific questions of developers and planned innovations of the
management or the general policy of the library. Accordingly, the selection of the concrete method
has to be oriented on the status quo, practicability and urgency of usability issues. Thereby, many
different methods are possible including quantitative as well as qualitative data. For an existing page,
usability test with end users can be seen as the “silver bullet”. Contrariwise, also heuristic evaluations
can be an alternative. For the establishment of a new Web site, rapid prototyping with paper-based
mock-ups could be a good way (Snyder 2003). With respect to planned innovations one might prefer
a focus group or semi-structured interviews. (Like mentioned above, it would go beyond the scope of
this article to explain and compare the different methods.)
Each method has its advantages and drawbacks. The crucial thing is to select the most appropriate
method with respect to the concrete situation, usability issue, and (last but not least) the available
resources and strategic decisions. Thereby it is essential to conduct the usability study in a way that
enables concrete recommendations. The “questions shapes the answers” (title of the review by
Schwarz 1999). In the case of usability recommendations this implies to ask in a way that produces
improvements and alternatives – and not only critique and the identification of problems.
4.3 Derived recommendations based on usability studies
Recommendations address not only the elimination of problems or the improvement of the usability in
general, but also target at the accentuation of the favorite applications and the best aspects of the
digital library.
Recommendations for usability improvements have to be concrete and practicable. In principle, the
recommendations have to be based on the results of the focused usability studies, but also the data
of the benchmarking can deliver important input (depending on the chosen questionnaires and
additional questions.) Benchmarking data can be used for screening, i.e. identification of overseen or
hidden usability issues. Additionally, the comparison with competitors could be helpful for finding
gaps, and obstacles as well as to identify the main incentive or appeal of the own Web site.
Furthermore, also a literature research on existing usability guidelines could be a starting point,
especially for the establishment of a complete new digital library or a redesign of a Web site. For more
specific, concrete recommendations one has to carefully interpret the results of the own usability
studies. The same is true for recommendations for planned innovations.
411

Stephanie Linek and Klaus Tochtermann
4.4 Making improvements: Balancing usability-recommendations and strategic
decisions
In a perfect world, usability recommendations could be fully implemented and we will have happy endusers
and a proud development-team. However, in reality improvements can only be made in the face
of available resources and strategic decisions regarding the overall policy of the (digital) library. If
resources are spare, one has to address the most urgent usability issues first. However, if there is the
strategic vision to become the most innovative digital library, this might lead to the decision to address
first required innovations and second the elimination of usability problems. Additionally, researchbased
recommendations could also contradict the policy of a digital library. For example, end-users
might have the desire for a broader assortment of and easier access to the genre fantasy or horror
while the management of the library wants to establish the image of an expert for sociocritical
literature. It is a difficult decision, how to handle such contradictories and how to find a compromise.
But think of Alice’s Adentures in Wonderland (Charles Dodgson alias Lewiss Caroll, 1865): It is
sociocritical as well as fantasy literature.
5. Cyclic interplay of the four elements: Adaptive iterative process
The interplay of the four elements described above can be characterized as follows:
Multi-method approach: Combination of quantitative and qualitative methods
Iterative cycles of usability evaluation and improvements
Flexibility: Flexible combination of the single elements and flexibility of the single elements
High interdependency between the elements
5.1 Multi-method approach
The combination of quantitative and qualitative methods addresses the divers demands at different
stages of usability evaluation (see also Langdridge and Hagger-johnson 2009). Qualitative methods
are often a timesaving and cost-effective way to identify a broad range of the existing usability
problems and user requirements. However, qualitative methods do not allow a comparison with other
digital libraries or search instruments. Analogously, it is not possible to quantify the improvements
made in comparison with prior version of the digital library and its services. Thereby, quantitative
measurements provide the necessary information. Quantitative measurements allow prioritizing
usability problems as well as user requirements. Additionally, the use of quantitative standardized
questionnaires enables the comparison with other software and online platforms (including other
digital libraries). An important advantage of the quantitative benchmarking is the possibility to make
improvements and enhancements measureable. Just the fact that a problematic feature or application
was modified doesn’t mean that this modification was actually an improvement. Indeed, end-users
don’t like changes and don’t want to deviate from their ordinary handling of a Web site. Comments
like “The old version was much easier” or “The old version was not perfect, but I really don’t like the
new one” are not an exception. Thus, changes of an existing Web site and its services have not only
to be communicated, but it has also to be evaluated if the changes are actual an improvement for the
end-users.
The combined interpretation of quantitative and qualitative data enables the provision of detailed
concrete usability recommendations. Thereby, it is also possible to make a priority ranking between
the recommendations. Based on the different methods it might be possible to make alternative
suggestions how to handle a specific usability problem.
5.2 Iterative cycles of usability evaluations and improvements
The quantification of usability improvements is also a good example for the iterative character of
usability evaluation and usability improvements. It is very hard to create good usability of a web-page
or online service, especially for the more complex ones. Moreover, it is impossible to create an
everlasting high usability. In line with the technological progress and the developments of the Web,
e.g., the ongoing development of Web 2.0 technologies, also the requirements of the end-users will
change. This in turn implies that existing Web sites and online-services have to change, too.
Otherwise they will be old fashioned – or in other words they will lack usability since they are no
longer efficient, effective, useful, and provide no joy of use. Thus, iterative usability measurements are
necessary. On the one hand, specific usability tests have to be repeated in the face of innovations
412

Stephanie Linek and Klaus Tochtermann
and new technical possibilities. Thereby qualitative data can be used to identify new possibilities or to
identify old-fashioned applications. On the other hand, quantitative measurements and repeated
benchmarking can be used as quality check if the Web site keeps up to date.
5.3 Flexibility
The iterative process is a flexible combination and arrangement of single elements. The iterative cycle
can start at different points and the arrangement of the elements has to be adapted to the concrete
context.
Also the single elements are flexible. Quantitative benchmarking can be made by different
questionnaires and the qualitative studies can follow different approaches that should be selected with
respect to the concrete practical aims and the available resources.
For an existing Web site it makes sense to start with quantitative benchmarking, to assess the status
quo as a baseline and thus enable a comparison of the different versions after substantial changes
have been made. Such a comparison ensures that the changes are actually an improvement for the
end-users and makes improvements quantifiable.
However, for the construction of a completely new Web site one should start with small focused
usability studies. Methods like rapid prototyping are advantageously to avoid wasting resources of the
developer-team. As a prerequisite, the group of end-users has to be known.
In some cases also usability-recommendations could be a starting point. Until now many usability
guidelines are available and could deliver fruitful input for the construction of every Web site. Even
though most of the guidelines are too general to allow specific design recommendations some of
them could be very helpful when open design questions come upon, e.g. how to arrange the
navigation menu. A very good and valuable source of such general guidelines (based on research
findings) is available at http://www.usability.gov/guidelines/guidelines_book.pdf .
Besides these rather practical, user-oriented starting points in some cases also a strategic decision
could be the first step. For example, a public institution wants to invest in Web 2.0 tools or refuses the
usual conventions of navigation menu to make their innovative and vanguard character visible. Such
strategic decisions are a challenge for the usability researchers and require a very flexible use of
usability methods.
5.4 High interdependency between elements
The elements of the approach are highly interconnected. Independent of the concrete starting point or
the arrangement of elements, none of the elements can be managed isolated from the others.
Quantitative benchmarking as well as focused usability studies enable concrete usability
recommendations. Thereby, the measurement methods have to be carefully selected with respect to
the concrete practical demands of end-users and the capacities of development. It doesn’t make
sense to ask the end-users for innovations that could never be implemented. Analogously,
recommendations have not only to be concrete but also practicable and in line with strategic
decisions.
6. Usability evaluation of the digital library ZBW – Leibniz Information Centre
for Economics
In this section the described methodological approach will be explained by the concrete example of a
digital library, namely the ZBW – Leibniz Information Centre for Economics (http://www.zbw.eu/indexe.html).
6.1 The digital library ZBW and its online services
The ZBW is the world’s largest specialist library for economics, with locations in Kiel and Hamburg.
The ZBW provides numerous services like EconBiz for literature search, EconStor as a publishing
portal, EconDesk as a reference service and many more. The services are frequently used and very
welcomed by the end-users. For example, for EconBiz there are about 15 000 queries per month with
20 000 to 28 000 visitors per months. Also the GooglePageRank of 7 demonstrates the high visibility
and degree of popularity of this service.
413

Stephanie Linek and Klaus Tochtermann
Until now, the usability of the online services and the Web site itself has not been evaluated in a
systematic way. Furthermore, the services will be enhanced and improved in the future. Therefore, a
new internal task force for usability evaluation was established at the ZBW. The aim of this task force
was not only to enhance the usability of the existing Web site and its online services, but also to
regard for the future development and the planned innovations with respect to Web 2.0.
6.2 Strategic decision as starting point
The starting point for usability evaluation was a strategic decision: To become a modern digital library,
that is capable to face the challenges of Web 2.0. Thereby, it was recognized, that good usability is
the key to attract end-users that would be otherwise only use Google-search or similar search tools.
6.3 Benchmarking questionnaire as foundation of the usability evaluation
In a first step a benchmarking questionnaire was created, to assess the status quo of the already
existing Web site and the services of the ZBW.
The benchmarking questionnaire comprises standardized existing scales, an additional scale for
assessing the quality of literature search results and several prospective questions. These three main
parts of the benchmarking questionnaire regard to different purposes:
Standardized questionnaires (ISONORM & SUS) serve two main aims. First, the comparability with
other software products and Web sites, and second, the establishment of an internal standard, that
makes improvement measureable with respect to general standards of usability.
Additional scale for assessing the quality of literature search results of the online service EconBiz tries
to establish a measurement instrument for an internal quality standard of the very heart of a digital
libraries online service: The literature search. The scale was constructed in close collaboration with
the product manager of EconBiz and librarians. The wording and format of the scale was analogous
to the SUS. Even though the validity and reliability was not proven, the scale assesses quality criteria
that were judged as important cornerstones of the service EconBiz. Thereby, the scale included
general criteria (e.g., completeness of the literature list) as well as specific aspects that regard to
enhancements and improvements (e.g., sophisticated ranking and filter options).
The prospective additional questions were included to assess end-users’ requirements and to receive
a more holistic view. Additionally, the questions tackle also planned innovations. Besides, the endusers
are asked about their typical work with the ZBW services, especially literature search with
EconBiz, to receive the required information for the creation of test scenario for subsequent usability
test scenarios. (More details on the benchmarking questionnaire can be found in Linek and
Tochtermann 2011.)
6.4 Focused usability studies for the future development, innovations and practical
needs
The usability studies of the ZBW will focus on two main areas: First, on the planned innovations based
on the strategic decision (modern digital library with connection to Web 2.0 applications), and second,
on the main usability issues revealed by the baseline data. These two main areas have several
intersections, especially with respect to the core service of the ZBW, i.e., EconBiz, the online service
for literature search. Additionally, it is planned to redesign the ZBW homepage.
The usability studies at the ZBW will mainly be done by means of videotaped usability tests. The
created test scenario will be based on the typical tasks of different end-user groups. Besides usability
test, the focused usability studies are designed in the face of the concrete requirements, e.g. rapid
prototyping with paper-based mock-ups as information source for the redesign of the homepage and
interviews with end-users as method for requirement engineering for the planned innovations.
6.5 Research-based recommendations in the light of the strategic decision
The recommendations for the improvements and further developments/innovations will be based on
the benchmarking-questionnaire as well as the focused usability studies. For example, rapid
prototyping and interviews with end-users will enable concrete recommendations for the structure of
414

Stephanie Linek and Klaus Tochtermann
the sites and the labels for navigation. The quantitative benchmarking data deliver the necessary
base for a priority ranking of required improvements. Additionally, the open questions of the
benchmarking questionnaires provide additional information on user requirements.
After the redesign of the Web site including essential changes and innovations, the benchmarking
questionnaire will be presented again to a big sample of end-users. This procedure enables a quality
check, if the changes made are in fact an improvement. Thereby, the standardized usability-scales
(ISONORM & SUS) as well as the scale on content quality remain constant. However, the open
questions will be modified in the light of the future development and planning of the ZBW.
7. Resume and outlook
The proposed methodological approach offers a flexible way for a sophisticated usability evaluation
that is on the one hand research-based and on the other hand oriented on practical needs. The
combination of quantitative and qualitative data provides a more holistic view of the situation and
allows the derivation of concrete usability recommendations. Additionally, quantitative benchmarking
enables the comparison with older versions of the Web site as well as the comparison with general
standards of usability and competitors. The flexible arrangement of elements allows a wide spread
application of the methodological approach. Thereby it is important to consider the high
interdependency between the elements and to adapt on the concrete situation.
The described example of the digital library ZBW demonstrates the different possibilities of the
described approach. Even though the preliminary evidence (of pilot tests) is promising, it is far too
early to draw conclusions yet. The described example of the ZBW is only one application case and
thus, the methodology has to prove its value for other instances. Thereby, the proposed multi-method
approach and the described benchmarking questionnaire are open for adaptive variations,
modifications and enrichments.
References
Bangor, A., Kortum, P. and Miller, J. (2009) “Determining what individual SUS score mean: Adding an adjective
rating scale“, Journal of Usability Studies, 4(3), 114-123.
Dodgson, C. alias Lewis Caroll, (1865) Alice’s Adventures in Wonderland, Macmillan: United Kingdom. Full text
online available at http://www.cs.cmu.edu/~rgs/alice-table.html or at:
http://ebooks.adelaide.edu.au/c/carroll/lewis/alice/
Dumas, J. S. and Redish, J. C. (1999) A practical guide to usability testing, Intellect Books, Oregon, USA.
Gediga, G., Hamborg, K. C. and Düntsch, I (1999) “The IsoMetrics usability inventory. An operationalisation of
ISO 9241-10 supporting summative and formative evaluation of software systems”, Behaviour and
Information Technology, 18, 151-164.
Langdridge, D. and Hagger-johnson, G. (2009). Introduction to research methods and data analysis in
psychology (2 nd ed.), Prentice Hall.
Linek, S. B. and Tochtermann, K. (2011) “Assessment of usability benchmarks: combining standardized scales
with specific questions”, Proceedings of the 14 th International Conference of Interactive Collaborative
Learning (ICL 2011), Piešt’any/Slovakia, September 2011.
Prümper, J. (1999) “Test IT: The ISONORM 9241/10”, in H. J. Bullinger and J. Ziegler (Eds.), Human-Computer
Interaction – Communication, Cooperation, and Application Design (1028-1032), Erlbaum: Mahwah, New
Jersey.
Rubin, J. and Chisnell, D. (2008) Handbook of usability testing, Wiley Publishing Inc., Indianapolis.
Schwarz, N. (1999) “How the questions shape the answers”, American Psychologist, 54 (2), 93-105.
Snyder, C. (2003) Paper prototyping: The fast and easy way to define and refine user Interfaces, Morgan
Kaufmann, San Francisco.
415

Social Networks, eLearning and Internet Safety: Analysing
the Stories of Students
Birgy Lorenz, Kaido Kikkas and Mart Laanpere
Institute of Informatics, Tallinn University, Estonia
birgy.lorenz@gmail.com
kaido.kikkas@tlu.ee
mart.laanpere@tlu.ee
Abstract: eLearning has recently become more social and open, involving open educational resources, open
courses, open learning using personal learning environments or social networks. We believe the schools are not
ready for this yet, as strategies and regulations supporting open learning are not up to date. It may seem easier
to restrict the use of Twitter, Facebook etc rather than integrate them into the learning process. Our study which
was based on the qualitative analysis of 201 stories written by secondary school students shows that a typical
safety incident is not solved adequately when existing regulations are used by the schools. We carried out the
qualitative analysis of 201 e-safety related short stories presented by students (aged 12 to 16), parents, teachers,
school IT managers and police, collected through the Safer Internet in Estonia EE SIC campaign. 2/3 of the
stories are fictional – they may be based on urban legends which however appear to refer to real stories. 1/3 of
the stories reflect real incidents. We mapped typical behaviour patterns and templates, beliefs regarding privacy
as well as regulations and limitations concerning the use of social networks at schools. We found that most of the
solutions used by schools to ensure e-safety are either technical or purely regulation-based, only some schools
appeared to have studied or elaborated on pedagogical or behavioural aspects. Problems are defined by
limitations and regulations while actual safety incidents (whether in- or outside school) remain largely unsolved
(or even undetected). Schools tend to excuse their unawareness with „we'll react when it happens“ (secretly
believing that it will not happen). Most schools also try to delegate such problems to parents - who in turn look
upon schools for help, as their only reaction to safety incidents is often just applying time limits on Internet use.
Thus there is an urgent need for information and working guidance mechanisms for managers, teachers, parents
and students. These matters must be solved well before schools reach the critical mass in using eLearning,
social networks and modern gadgetry as parts of curriculum. Schools are expected to apply new technologies in
teaching and learning, but safety of student and teachers is paramount in this context.
Keywords: online safety, schools, policy, new technologies, social media
1. Introduction
The internet safety issues in the context of eLearning are quite often addressed in a simplified, blackand-white
manner, using rare cases of criminal privacy violation to scare teachers and parents. We
believe that fear is not the best teacher in this domain.
Our goal was to:
Evaluate what kind of e-safety stories are students and experts (teachers, it-managers, and
police) telling and how they are related;
Find typical e-safety stories to promote development of regulations in these areas;
Previous studies in this matter can be divided into 6 bigger groups: cyber-bullying (Berson 2002);
moral issues – pedophilia, inappropriate content, behavioural errors (Akdeniz 1997; Carr 2004;
Mitchell 2004; Peters 2009); programs and materials for schools (Wishart 2004); threat analysis for
eLearning (Alwy 2010); normal teenage internet usage (Bullen 2000; Enochsson 2005; Dworschak
2010); Internet usage analysis (Livigstone, 2011; The Safer Internet for Children Qualitative Study in
29 European Countries 2007; Towards a Safer Use of the Internet for Children in the EU – a Parents’
Perspective 2008). While according to the EUKids Online II survey (Livingstone 2010) Estonian
students are among the top users of Internet and have had substantial exposure to online security
risks, there has been next to no attempt of national-level regulation in this field.
We analysed the typology and sources of safety incidents, the real solutions offered, the students'
thoughts and feelings stemming from the situation, the solutions suggested by students and whether
these typical solutions actually apply in real-life situations. The practical experience of students,
teachers, IT managers and police indicates a gap between the measures used in education and real
life.
416

2. Background
Birgy Lorenz et al.
The new National Curriculum for Primary and Secondary Schools prescribe the use of multitude of
digital tools in the learning process: virtual learning environments (VLE), personal learning
environments (PLE) and other web 2.0 resources like social networking to be used in a classroom
(National Curriculum 2010). A recent study shows that even with the good existing training
opportunities and guided by the requirements of the new curriculum, most of the Estonian teachers
are not ready to facilitate students in the matters of e-safety (Maadvere 2010).
In the teachers' communities there are some discussions about the differences between the real life
needs of a 21st century learner and how the schools should meet these needs. There are rising new
challenges like digital divide, globalization as being a part of global internet village, digital sociality,
familiarity in communicating; the changes in the roles of student and teacher and in understanding the
policies and responsibilities (Veldre 2010, Murumaa 2010).
There are a lot of materials (usually in English) that urge teachers to discuss these matters with
students. But they are usually used as a reaction rather than a preventative act. At first nobody
believes it could happen to them; yet if something happens, it's not announced and discussed as
people feel ashamed (Hoiser 2009).
Teachers are usually reluctant about e-safety - likely most schools have some video posted in the
internet featuring a teacher who is unaware that he/she is a 'movie star' or some students secretly film
other students (Vasli 2011). The discussions of incidents on e-safety tend to lead to internet usage
restrictions at school. Schools and parents are interested in monitoring children’s behaviour in the
name of preventing cyber-bullying or meeting with strangers (Hunter 2005).
Studying school policies and practices, we could only find technical rules regarding computer
wellbeing, time or operational policies (TDL arvutiklasside kasutamise reeglid 2004). Most schools
regulate nothing at all, being afraid to create new precedents, or rely on the 'ostrich effect' claiming
that having no incidents yet gives them no reason to act. In comparison, British schools opt for much
more regulation (Children & Young People's Services 2011). In the United States, similar documents
regulating teacher-student relations are given to sign by both parties (Ohio Policy reference manual
2011).
3. Methods
We carried out two studies of e-safety stories and bottom-up policy development.
The first study (Stage I) used the Grounded Theory (Chamaz 2006).
The aim of Stage I was to decide who will be the focus group and what the main topics in the stories
are. We used the 'go-around' exercise (Fundamental Team and Meeting Skills 2003) with empty cards
and clustered the results with 50 participants (10 teachers, 6 experts, others were students aged 12-
16). People were divided into 8 groups (6-7 people in the group); 7 groups with students and 1
teacher, 1 group containing teachers and experts) and the goal was to:
Decide who is involved in e-safety incidents and whom children look upon to get the information
or help from;
Gather and cluster data on what kinds of incidents can usually happen (the overall topics) and
what would be important to the community.
The results of Stage I were 13 different categories to code the e-safety stories into.
Stage II used storytelling as a method. The story is a useful tool for learning - when we analyse
stories, we can understand better how the world works for children (Vilke 2000).
The stories were gathered from the people selected at Stage I: experts (police, teachers,
psychologists, ICT experts) and students. The experts were selected from the Estonia EE SIC project
using directed focusing (Teddlie 2009). Later we also used the snowball method (Gray 2007) as esafety
educational experts are rare in Estonia. The stories from students were collected using a
storytelling competition at InSafe project workshops in Estonia. We analysed the collected stories in
line with qualitative case study method (Gagon 2010).
417

Birgy Lorenz et al.
135 students participated in the workshop titled “tell me your e-safety story”. We cannot prove
them as authentic e-safety stories but we assume that they are not all myths. These are the
stories that parents tell their children and children tell each other.
19 stories that children claimed to be real.
20 experts' stories were gathered from the interviews or surveys.
27 stories provided by the police are a representative collection of what they think mostly happens
to children that end up reported to the police.
The experts presented their stories as situations with solutions.
To code stories we used 13 themes (found in the results). They are more human behaviour related
than typically found in the studies about technical e-safety risks in eLearning (Alwy 2010) or e-safety
studies like EUKids Online II survey (Livingstone 2010), where the main goal is to address sexual
themes. We were more interested in average students' typical internet/computer interaction
experience. For that we used 180 stories out of 201, some stories were excluded due to being too
short.
4. Results
We used the open coding approach to categorise the stories into 13 topics: spam, gaming, computer
overuse, virus, fraud, passwords, fake accounts, cyber-bullying, harassment, slandering, privacy,
pornography and meeting strangers. These were the topics or labels that emerged after reading the
stories several times. Every story was then labelled with one topic (primary code), which appeared to
be the main focus of this story. The Figure 1 illustrates the differences of stories collected from police,
experts and children.
Figure 1: The distribution of primary codes after the first round of coding
As most of the stories actually contained references to multiple aspects and not just to the primary
topic, we decided to do the second round of coding so that each story could be assigned multiple
secondary codes. The Figure 2 illustrates the differences of occurrences of secondary codes in
stories collected from police, experts and children.
The analysis of the interviews suggests that neither teachers nor students do not really understand
what e-safety is about (or there are too many different understandings what is or what is not e-safety)
and who is responsible in this matter to help the child.
Claims of the teachers:
We have not faced any real e-safety incidents in past 3 years;
Our school is e-safe because our students are not allowed to use WiFi or phones during the
school time or they use Web filtering;
418

The e-safety of children is a parents' concern;
Birgy Lorenz et al.
When something happens we will create new procedures.
Figure 2: Frequency of secondary codes
Claims of the students:
We can access the Internet whenever we want, the filtering of the Web does not work;
Cyber-bullying is an everyday act, it can start anywhere and it will not stop when teachers try to
stop real-life bullying, sometimes they also punish wrong people.
We also found evidence that the story usually starts with connection to one environment and later
changes to another; for example, the first connection is established via web or e-mail, but afterwards
it moves to a social network or direct messaging system. Most of the connections are made in social
networks (Facebook, Rate, Orkut, gaming sites); 90% of the stories took place at home, but
schoolmates are usually involved in some extent as well. So we can conclude that the stories are also
discussed at school. Compared to the police stories, the children's stories do not involve mobile
phones yet.
The children's stories are mostly about gaming, fraud and passwords. There is a relationship between
gaming and computer overuse and viruses, some stories can state relationships between gaming and
harassment and pornography. Usually the games are single-player or standalone role-playing games
but there was growing evidence about online gaming as well. The fraud category has direct
relationships to passwords, spam, harassment, slandering and privacy. Fraud is usually seen in web
or social networks, less in direct messaging. Passwords (scamming, phishing or hacking) are related
to the web, less so to direct messaging or social networking. When we clustered harassment and
cyber-bullying then it can relate to spam, fake accounts and slandering, while harassment is related to
gaming, fraud, fake accounts and pornography.
The real cases from children tell us typical stories about fraud, stolen passwords, harassment and
slandering. Fraud was mentioned in relation to privacy and pornography. The stolen passwords were
mostly related to cyber-bullying and slandering. Slandering and harassment were related to privacy
infringement, and mentioning of pornography co-occurred with ones of gaming. There were no stories
about spam, computer addiction and meeting strangers. There were lots of indications that direct
messaging is taking place in social networks or chat rooms and strangers do not get added to one's
MSN or Skype account – but this is what parents and teachers typically address in e-safety trainings.
419

Birgy Lorenz et al.
The experts' stories address mainly cyber-bullying, slandering, privacy infringement, somewhat less
fraud and stolen passwords. There are direct relationships between privacy and slandering, bullying
and passwords. Most stories are related to networking or using the web.
The police stories show us what kind of help parents need from the legal system. They are usually
related to fraud and pornography, less related to passwords, harassment and meeting strangers.
There are some relationships between gaming and passwords; harassment and pornography,
meeting strangers and pornography. Fraud is usually related to mobile phone or is carried out via
mobile phones. As a rule, parents turn to police when they have lost money. The police cannot
usually interfere when there are only moral problems. There are some relationships between direct
messaging, pornography and meeting strangers because in direct messaging one can use video
transfer.
The stories from police indicated which paragraphs in the Estonian Penal Code are applicable for
Internet crimes: PenC § 179 for showing pornography to minors, PenC § 217 for password hacking,
PenC § 157-2 for identity theft, PenC § 213 for other computer frauds (Penal Code 2002).
Our analysis of e-safety policies found on the official Web sites of ten Estonian secondary schools
revealed that many schools address e-safety issues not with policy measures, but with technical
access restrictions, e.g. filtering some web sites or blocking some services. Only a small number of
schools had e-safety policies, which addressed topics like cyber-bullying or sexting (sending sexuallythemed
texts or images via mobile phones). Some schools had regulations on taking pictures in
school premises or sharing the visual material originating from the school (see Table 1). The
government does not impose any relevant regulations on schools, besides generic legislation (e.g.
penal code). While there are regulations used to develop schools' internal rules, they contain no
mention of e-safety. Moreover, while schools are subjected to Lesser ISKE (Estonian three-level IT
baseline protection system) framework, no practical support is provided.
Table 1: Typical school rules regarding to use of ICT (based on policies of 10 schools)
Type Rules
Technical Every class or user has a different account
The right to install or run software or print files is restricted or limited
Technical filtering of the web
People There are goals and priorities of the tasks what you can do with the school computers
There are rules regarding the use of social network sites and direct messaging
Computer lab working hours, health related rules (how long can you use a computer)
Physical well-being of the computer workstation
Monitoring the students
Gaming is usually not allowed
Other Penalties for breaking the rules
Information where to turn for help
One can make suggestions
From the stories we also found the indications about what kind of regulations are needed. The typical
recommendation was that the school must be present in the network because it influences students to
behave better; the school must develop rules regarding to mobile and other devices usage in the
schoolhouse; the school must filter programs and the web in the computer class; an "x" school
student is not to make a fake account or post under someone else’s name; an "x" school student
should not share his/her passwords with others; an "x" school student cannot slander others in the
web; an "x" school student is to ask permission from other people before taking pictures in the school
premises.
5. Discussion
The analysis of the students’ e-safety stories revealed several issues. It is clear that many students
don’t apparently understand what e-safety means. Students do not usually think that they are in any
way involved in an e-safety incident, even if they have been harassed or bullied on the Internet e.g. in
a YouTube video about the teacher. The collection of stories from the “storytelling’” exercise and from
the web-based competition provided similar results. Also, in a test with a control group giving students
an e-safety topic (like privacy or virus) to write about, they first wrote something 'to please us' and
then changed the story to something that they wanted actually to write about. It was quite interesting
that they were not thinking about the given topic but rather writing about their real problem. It is
something we should study further on in a deeper level.
420

Birgy Lorenz et al.
The students’ stories are highlighting the privacy issues in social networks, for it is easy to make fake
accounts, gather personal information from search engines or even take pictures with mobile phones
and post them online. Instead, people tend to believe that privacy is someone else's problem. They
often also state that it is difficult or they do not have time to change privacy settings - but after an
incident, they suddenly come to believe that they could learn to do that.
The children's stories do not include stories about pedophilia and meeting strangers what is usually
considered the biggest threat regarding e-safety. Children do not understand the differences between
harassment and cyber-bullying – these terms are foggy to them. It can point to weak sexual education
in Estonian schools because sexual topics are identified only 4 times out of 17 cases about
harassment. But it can also be a topic that is not openly discussed among students. Also, the line
between privacy and personal data protection is not really understandable to students. They tend to
prefer black-and-white solutions: it's all private or not at all.
The stories collected from police and IT experts differ because police deals with stories mostly only
when there is direct monetary loss. On the other hand, schools often prefer to deal with incidents
secretly, striving to find a solution between the parties without releasing the case to public.
Looking at the results from the e-safety related policies of schools, we will see an interesting
phenomenon - even when we did present typical sex-related stories to be discussed they were not
considered a priority issue for schools. As seen by principals, harsh E-safety regulations at schools
could be one of the future reasons for teachers not to use VLE, PLE and m-learning with students and
only stay using the teacher's computer and projector for presenting their own materials, which is
considered safe.
Although our study focused on analysing e-safety incidents, it also informs the eLearning community
about the need to raise awareness of teachers and students about potential threats to their privacy.
This need becomes even more evident in the light of new trends related to the use of social media
(blogs, wikis and social networking sites) as a new type of online learning environment (Becta 2008).
And the most interesting is that in e-safety trainings usually are addressing MSN conversations and
strangers issue where the prescribed solution is to block unpleasant persons, but today's interaction is
moved to social networks where there is also opportunity to chat. When a child feels pressure to have
more friends than other people then blocking an unpleasant person is not really an option. This also
creates a privacy problem – the child opens his/her life to strangers even without direct
communication.
6. Conclusions
Our overall conclusion is that typical e-safety policies must stress topics that all stakeholder groups
agree being important: gaming, fraud, password, harassment, pornography and meeting strangers.
There are direct relations between gaming and passwords; fraud and privacy in social networks;
passwords and slandering in social networks; harassment and pornography. Students also point out
problems regarding viruses, fake accounts, cyber-bullying, slandering – these are topics at which
police is usually powerless to help. Our analysis shows that only a few schools have explicit policies
which target e-safety issues. Yet, even these few existing school-level policy documents fail to
address the topics which were most frequently mentioned in the stories written by students.
Next, we should turn our attention towards evaluating the e-safety risks by themselves and influence
of dangerous behaviour patterns to the online learning activities. As Internet usage has changed
children’s values and patterns of online behaviour, the question is how we (as parents and educators)
can adjust children's behaviour on the net if we still live in a different e-world. We should acknowledge
that although we mostly use the same digital tools with the new generation, we still identify and
handle the threats related with our online activities differently.
References
Becta (2008) Analysis of emerging trends affecting the use of technology in education, Research to support the
delivery and development of Harnessing Technology: Next Generation Learning 2008–14
Berson, I., Berson, M. and Ferron, J. (2002) Emerging risks of violence in the digital age: lessons for educators
from an online study of adolescent girls in the United States, Meridian: A Middle School Computer
Technologies Journal Vol. 5 No. 2, NC State University, Raleigh, NC
421

Birgy Lorenz et al.
Akdeniz, Y. (1997) The Regulation of Pornography and Child Pornography on the Internet, The Journal of
Information, Law and Technology (JILT), UK
Alwy, N. and Fan, I. (2010) Threat analysis for eLearning, Int. J. Technology Enhanced Learning Vol. 2 No 4, UK
Bullen, P. (2000) The Internet: its effects on safety and behaviour implications for adolescents, Department of
Psychology University of Auckland, Netsafe, New Zealand
Carr, J. (2004) Child abuse, child pornography and the internet, ISBN 0900984805, NCH, London
Chamaz, K. (2006) Constructing grounded theory: a practical guide through qualitative analysis, SAGE Publ.
London, GB
Children & Young People's Services (2011) Internet safety and model policies on the acceptable use of the
internet for schools [online] http://www.bristol-cyps.org.uk/services/ict/acceptable.html
Dworschak, M. (2010) The Internet Generation Prefers the Real World, Spiegel Online Internetional
Enochsson, A. (2005) A gender perspective on Internet use – Consequences for information seeking on the net,
Information Research, 10(4) p. 237
Fundamental Team and Meeting Skills: Tools and Techniques (2003) New York State Governor's Office of
Employee Relations [online]
http://www.goer.state.ny.us/Training_Development/Online_Learning/FTMS/300s4.html
Gagon, Y.(2010) The Case Study Research Method: practical Handbook, Presses de Universitete du Quebee,
Canada
Gray, P., Williamson , J., Karp, D. and Dalphin, J. (2007) The Research Imagination: an introduction to
quantitative and qualitative methods, Cambridge University Press, GB
Hoiser, F. (2009) Countering “it won’t happen to me”, Safety News Alert [online]
http://www.safetynewsalert.com/countering-it-wont-happen-to-me/
Hunter, N. (2005) “Jenny´s Story” – An internet safety resource developed to combat child abuse on the internet,
Lancashire Constabulary, Pretton, UK [online] http://www.popcenter.org/library/awards/goldstein/2005/05-
06.pdf
Linstone, H. A., Turoff, M. and Helmer, O. (2002) The Delphi Method - Techniques and Applications, Murray
Turoff and Harold A. Linstone.
Livingstone, S., Haddon, L., Görzing, A., & Olaffson, K. (2010) Risks ans Safety on the Internet: The perspective
of European children. London: LSE: EU Kids Online.
Maadvere, I. (2010) IKT uues põhikooli riiklikus õppekavas [online]
http://tiigrihypeharidustehnoloog.blogspot.com/2010/06/uusoppekava2.html
Mitchell, K., Finkelhor, D. and Wolak, J. (2004) Victimization of Youths on the Internet, Victimization of Children:
Emerging Issues (pp 1-39) New York: The Haworth Maltreatment and Trauma Press, NY
Murumaa, M. (2011) Digisotsiaalsus õpetaja ja õpilase suhetes, Õpetajate leht Vol. 16 [online]
http://www.opleht.ee/?archive_mode=article&articleid=5367
National Curriculum - Põhikooli Riiklik õppekava (2010), Riigi Teataja [online]
https://www.riigiteataja.ee/akt/13273133
Ohio policy reference manual (2011) Sample Regulation computer/Online services [online]
http://www.ohioattorneygeneral.gov/getattachment/48c7a1bc-6ccc-4486-8c79-9190f9a3f97a/Teacherstudent-relations-sample-policy-for-accept.aspx
Okoli, C. and Pawlowski, S. D. (2004) The Delphi method as a research tool: an example, design considerations
and applications, Information & Management Vol. 42 pp.15–29
Penal Code - Karistusseadustik (2002) Riigi Teataja [online] https://www.riigiteataja.ee/akt/184411
Peters, R. (2009) How Adult Pornography Contributes To Sexual Exploitation of Children, [online]
http://www.obscenitycrimes.org/news/HowAdultPornographyHarmsChildren.pdf
Rubtsova, P. (2011) Privacy in Social Media, Tallinn University.
Safer internet for children qualitative study in 29 european countries, summary report (2007) Eurobarometer,
[online]
http://www.internetsafety.ie/website/ois/oisweb.nsf/0/F7650C73182B83B6802574D5004A932B/$File/Safer
%20Internet%20for%20Children-%20Summary%20Report-march-may07.pdf
TDL arvutiklasside kasutamise reeglid (2004) Tartu Descartes'i Lütseum [online]
http://portal.tdl.ee/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=45&MMN_positi
on=65:59
Teddlie, C. and Tshakkori, A. (2009) Foundations of Mixed Methods Research: integrating Quatitative and
Qualitative approaches in the social behavioural sciences, SAGE Publ. London, GB
Towards a safer use of the Internet for children in the EU – a parents’ perspective Analytical report (2008)
Eurobarometer, [online] http://www.internetsafety.ie/Website/OIS/OISWeb.nsf/page/DPCY-
7MELY61694917-en/$File/Eurobarometer%20Survey%202008.pdf
Vasli, K. (2011) Sotsiaalmeedia imed: 11 aastat vana raadionalja "Jah narkootikumidele" taassünd, Õhtuleht
Online [online] http://www.ohtuleht.ee/427422
Veldre, A. (2011) Kohutavate muutuste anatoomia, Õpetajate leht Vol. 16 [online]
http://www.opleht.ee/?archive_mode=article&articleid=5368
Vilke, M. (2000) Research study on children’s story-telling, Psycholinguistics on the Threshold of the Year 2000,
pp. 453-461. Porto: Faculdade de Letras da Universidade do Porto.
Wishart, J. (2004) Internet safety in emerging educational contexts, Computers & Education archive Vol. 43 No.
1-2, August 2004 Elsevier Science Ltd. Oxford, UK, UK
422

Learning Management Versus Classroom Management in
Technology-Supported Blended Learning
Arno Louw
Centre for Technology Assisted Learning (CenTAL), University of
Johannesburg, South Africa
alouw@uj.ac.za
Abstract: The comprehensive University of Johannesburg (UJ), South Africa, had to adapt to a fast growing
student and academic community. Since 1998 a learning management system (LMS) has been introduced as to
broaden the physical boundaries of classrooms over four campuses, student residences and residential areas
(worldwide). The Centre for Technology Assisted Learning (CenTAL) was established to accommodate students,
teaching and research staff and empower them to keep up with current and future trends in technology-assisted
learning. Currently a blended learning approach, with direct face-to-face (F2F) contact with students, is followed
and driven by the “learning to be” philosophy of teaching and learning. F2F contact is supplemented with a
remote e-learning environment hosted in an LMS. Consequently, the implementation of technology in teaching
and learning became an adopted phenomenon and promised increased success in higher education. However,
changing curricula, staff turnover and increasing student numbers resulted in the efficacy of the LMS used, to be
questioned. Moreover, the actual learning experiences of students are often dismissed as a new paradigm and
statistical evidence leads to a contradiction about blended learning efficacy during annual course evaluations.
This occurrence often causes confusion when effective use of an LMS is discussed. Subsequently, this inception
forces the perception of key role players in two directions: managing virtual learning environments and managing
learning within an electronic learning environment. Ultimately, learning management as viewed by the
constructivist, and classroom management as viewed by the behaviourist, are aligned by presenting various
examples from an LMS for operational e-learning. Therefore, this paper points out the different perceptions of
lecturers towards the effective use of an LMS as remote component for blended learning at UJ. Emerging from
the above rationale, this paper points out traditional perceptions of lecturers towards e-learning and further poses
the question: Is learning management or classroom management the platform for effective blended learning
design? Findings and conclusions further addresses the consolidated data substantiated over a period of three
years in nine faculties during research in professional development of faculty members related to effective,
blended learning. The data and examples are presented as qualitative and, where more clarity is required,
substantiated by quantitative data. The actual examples presented, point out teaching practices where classroom
management was used as point of departure and then contrasted with examples where learning management
was used as a starting point. This paper further reveals e-learning environments where this dichotomy becomes
distinguishable but inseparable. An in-depth discussion extrapolates instructivist and constructivist approaches to
be extremes of a continuum whereby ideal balance becomes a superlative teaching application for both F2F and
online learning environments. In this case study this pedagogical paradigm escalates to an imperative for
implementing effective technology-supported blended learning at the University of Johannesburg.
Keywords: technology-assisted learning; e-learning environment; learning management system (LMS); learning
management; classroom management; technology-supported blended learning
1. Introduction
Blended learning at UJ mainly uses ICTs and specifically an LMS whereby on and off line activities
and participation encompasses the style of teaching. This approach is conducted from a policy on
teaching and learning strategies. The background to this is presented followed by a brief description
on the methodology followed. Thereafter, the perception of students, as the Y generation is described.
The discussion about the impact and considerations escalates into the perceptions of students and
lecturers. After the perceptions have been clarified, the context of classroom management and
learning management within blended learning is discussed. This is then followed by examples as part
of the findings and conclusions followed by a summary.
2. Background and the adopted philosophy
Recognising the change which enhances teaching and learning strategy, the University of
Johannesburg (UJ) has adapted a philosophy for teaching and learning. This philosophy has become
university policy and is circumscribed under Section 1 of the Principles of a Teaching and Learning
Strategy as follows: UJ recognizes the complexity and rapidly changing nature of the social, economic
and intellectual environment for which its students are being prepared. It is imperative, therefore, that
teaching and learning at UJ should transform from a primary concern with the transmission of
knowledge (learning about) to a primary concern with the practices of a knowledge domain (learning
to be). Therefore, in its teaching and learning activities and in the design of its modules and
423

Arno Louw
programmes, and within the parameters of reasonably practicable implementation. (UJ Senate
Teaching and Learning Committee 2009).
A proof to the growth in student numbers and the accompanying lecturer growth is shown in reports
from CenTAL for 2008 to 2010. The reports reveal an increase in students and lecturers using the
LMS. One can arguably agree that these numbers will inevitably become unlimited as usage of the
Internet, ICTs, Web 2.0 learning tools, and an LMS as learning tool increases over the next few years.
Consequently, the same approach for using these numbers was change when the 2010 report was
published. Therefore, the approach to what is seen as effective use of the Internet and World Wide
Web (WWW) as learning tools, has become biased toward the actual teaching needs from lecturers
and the learning needs of students. This discrepancy highlights an immediate need for lecturers to be
related to the rate of adoption to incorporating the Internet and the WWW into blended learning.
3. Methodology and data gathering
Over this decade, ICTs, Web 2.0 facilities and a new generation of students have also become part of
daily teaching practices and many faculties had to adapt to new and innovative ways of teaching and
learning. Since the University of Johannesburg has adopted a blended learning approach, the LMS
has been upgraded in 2002, 2006 and in 2010 the pilot launch of the new Blackboard system had
been launched in 2011. I have been involved in these processes and implementations and have
worked closely with over one hundred lecturers in nine faculties over four campuses. The process
followed for presenting the research in this paper, comes from notes, conversations, being involved in
implementation procedures and training many lecturers and students since 1998. The subsequent
involvement in many research project in this time, has delivered trends which I share within the scope
and limitation of this paper. It is intriguing to observe how lecturers adapt and come up with new
initiatives in spite of growing student numbers and limited physical classroom space on campuses.
Therefore, the discussions, conclusions, and given facts in this report can be substantiated with
literature, data from and data generated by the LMS currently in use. Apart from using data from
many cases, I will mostly focus on projects involving large classes in modules for Commercial Law, B
Tech bridging courses, Language Skills for Science, and Business Management. The findings,
conclusions and arguments are presented together with the discussions as to promote the integrated
nature of this research. However, a separate section for findings and conclusions will also be given as
an accumulation of what is assumed to be effective blended learning.
4. Impact and considerations
The diffusion theory has its origins in work done in 1943 in rural sociology. According to Jones,
Jamieson and Clark (2005), the Diffusion, per se, is defined as “a process that occurs over time and
can be seen as having five distinct stages – knowledge, persuasion, decision, implementation, and
conformation.” The diffusion theory framework involves four main elements: innovation,
communication channels, time and the social system. Furthermore, these elements “influence the
adoption or rejection of an innovation in a complicated, interdependent way” (op. cit.). The same
authors refer to Jacobsen (2000) who used the five stages in adopting technology for teaching and
learning and various other authors (Bronack & Riedl 1998; Schefstrom et al. 1998; Surry & Farquahar
1997) who used several diffusion theories used and relates specifically to an instructional technology
model. The common elements of most diffusion theories are quoted in table 1 as proffered by Jones,
Jamieson and Clark (2005) and given in relation to the UJ context.
Table 1: Common elements of diffusion theory
Influential Elements Description
Diffusion is a process that occurs over time and can be
Diffusion process
seen as having five distinct stages:
Adopter categories
Perceived attributes
Rate of adoption
Lecturers vary greatly in their willingness to adopt a
particular innovation. Individual characteristics can be
used to divide the population into categories:
How innovation is perceived, influences the adoption
decision. Five perceived attributes of an innovation
have been shown to have strong influence:
The relative speed with which lecturers in a social
system adopt innovation. Innovations are diffused over
time in a pattern that resembles an s-shaped curve.
424

Arno Louw
Devedšić poses (op. cit.) that organising and linking materials on the Web, as well as extracting and
interpreting the content for the sake of effective learning, places the burden on the human user. I
argue that the perceptions of lecturers and students have an influence on the rate of adopting blended
learning because hard and soft skills, accompanying values, and expected attitude(s) must be part of
a curriculum. I maintain further that a power play for the ideal learning process in blended learning
causes misperceptions. Therefore, it is not only stakeholders of the learning process who should
adopt, but also the content which needs to be aligned with demands in industry upon graduating a
student (suggested by the teaching and learning philosophy of UJ).
4.1 Students perception and expectations from blended learning
To know the background of students might be considered cliché. However, within the South African
context, various background and cultures, 11 official languages, accommodating people with
disabilities, growing student numbers, and comprehensive 4 campuses will always influence
perception. However, common trends of the current student generation contribute to enlighten our
understanding of the student of today.
According to Smith and Grant-Marshall (2004), the South African generational divide differs from the
international division. They define the generations in South Africa as follows: Baby Boomers were
born between 1950 – 1969; Generation X people were born between 1970 – 1989 and Generation Y
were born between 1990 – 2005. This division, as with all divisions, is however not clear cut. People
who were born, either early or late in a generation, is known as cuspers. Cuspers show traits of both
generations. To understand today’s students, it is necessary to briefly look at some of the traits of
each of the generations, with particular emphasis on technology as there is definitely a digital divide
between the various generations. Baby Boomers grew up following World War II. They grew up with
one-way technology, being the radio. Their introduction to technology was in the 80s with the advent
of the personal computer (PC).
Generation X (also known as the lost generation) loves to communicate by e-mail and want instant
answers to questions and know how to get speedy access to information. They are far more adept
with technology than their parents (Baby Boomers). However, they often lack the technology savvy of
Generation Y (or the Millenials). Generation Y grew up in the full swing of technology – 24/7
television, the Internet, social networking including Facebook, Blackberry applications, iPhone
facilities etc. Generation Y wants lots of knowledge and they want it instantly. They thus rely on
technology to assist them in obtaining the knowledge and to share their knowledge with others. In
addition, expensive cell phones, bigger head phones, and iPods are displayed. A recent report from
UJ (Bester, 2011) conveys that computer orientation for first year students delivered results shown in
Figure 1 and Figure 2.
Figure 1: First year access to different technologies
425

Arno Louw
It should be clearly stated that this is personal access to these technologies and the interpretation to
the question is often confused with “owning” the technology rather than have actual access. In
addition, many students have access to computers and the Internet via hotspots in residences, per
immediate family households or communes, or at home where a computer is shared by many people.
Informal discussions with students brought forth that the computer is used for social networking (e.g.
Facebook) and gaming, yet applications such as the word processor, and spread sheet is never used
for formal writing although presentation software is not all that unfamiliar to them. Hence figure 4
reveals the computer proficiency of current second and first year students.
Bearing all said in mind, Baby Boomers were taught by the Traditionalists (born during 1930 – 1949),
in accordance with the behaviourist theory, i.e. rote memorisation. Baby Boomers rebelled against
this format and taught Generation X in line with the cognitivist approach to teaching and learning.
Rote memorisation still forms the cornerstone of the cognitivist approach, but it goes one step further
in that the information that was memorised then is used by applying it to a set of facts as one would
experience it in real life. Generation Y’s education follows the social constructivist approach, which
strongly relies on collaborative learning, which taps neatly into Generation Y’s preference for working
in groups. Even more so, the learning theory of Connectivism emerges from the learning styles of
these students (Siemens 2005). This trend, in teaching and learning in society, enlightens our
paradigm for understanding the learning needs of our students.
Figure 2: Computer proficiency of first years 2010 – 2011 (comparison)
These students adapt easy to e-learning when analysed against the diffusion theory and are classified
in five categories, namely: innovators, early adopters, early majority, late majority, and laggards
(Keller & Cernerud 2002). In addition, familiarity with e-learning, an LMS, and ICTs as components of
e-learning, are largely accepted by modern students. In a later study, Buzzetto-More (2008) finds that:
“When prior educational exposure [to e-learning components] was examined, the majority of students
indicated that they had used a computer to solve a problem as part of a class assignment,
participated in group work that involved using computer software, and delivered a presentation using
computer software.” She also found that “students’ perceptions and experiences with online learning
were similar to findings reported from studies conducted at majority institutions where students have
reported that they want to see traditional learning supported by e-learning strategies; however, faceto-face
instruction is preferred over fully online learning.” (op. cit.).
4.2 Pedagogical design for blended learning interventions
The affordance of the Web bears the following constraints that must be addressed as it influences the
theory behind applying learning in the online environment. Anderson (2004) describes the following
426

Arno Louw
limitations that I will list without an in depth discussion as it is recognisable and contentious within the
South African educational realm.
Anderson (2004) proffers the following model that shows the types of online interaction that occurs in
a virtual educational environment.
By using the model in Figure 3, online instructors and designers are faced with crucial decisions
based on different learning outcomes that will be best learned through specific types of learning
activities in an online environment. The decision on the design of the different learning activities, as
part of the instructional design process, arguably is not based on how learners learn but rather on
How do they learn what?
Figure 3: Online learning and types of interaction (Anderson, 2004:49)
4.2.1 The dimension of user activity
Two stereotypes of learning environments are distinguished, namely, mathemagenic and generative.
These types of learning environments also become the distinguishable ranges on the continuum in
the pedagogical dimension of user activity. Reeves (1997) describes a mathemagenic environment,
according to Hannifin (1992), to “access various representations of content.” Contrarily, other
generative learning environments are described to “engage learners in the process of creating,
elaborating or representing knowledge.” (op. cit). I draw on a conclusion that the online learning
environment should attempt to become more generative in its nature. The environment should by
nature attempts to engage both learners and facilitators to interactively mentor, facilitate and promote
peer discussions and participation, and is socially constructivistic. Furthermore, the activities should
be designed to introduce virtual and mental collaboration tools to the participants. Reeves (1997)
substantiates by stating that “Generative learning environments are aligned more closely with
contructivist pedagogy whereas mathemagenic environments are often based on instructivist
pedagogy, but is not necessarily always obvious.” Thus, the activity design becomes cardinal in the
promotion of constructivist pedagogy in blended learning.
4.2.2 The dimension of learner control
This pedagogical dimension has been the most researched as a topic. This dimension ranges from
allowing learners to have control by allowing them to “make decisions about what sections to study
and/or what paths to follow through interactive material.” (Reeves 1997). The range in this
pedagogical dimension varies the continuum from unrestricted to non-existent. Furthermore, an
accurate explanation of this dimension is given by Powell (2000) stating that the issue of control “is
427

Arno Louw
somewhat an unwritten contract between the site user and the developer. There is a give and take in
the relationship.” The author adds that “... users are imperfect like everyone else; if we give them
complete control, they make serious errors. If you control things too much and the user notices that
they can’t resize their window or press certain buttons, they may become angry or frustrated.”
Moreover, the same author argues that “Users can do everything they need to do and nothing more.
People need to feel like they are in control but the control should have limits. Good interfaces exhibit
this control.”
4.2.3 The dimension of flexibility
The continuum of a learning program can be unchangeable or more learner accommodative.
Therefore, this dimension ranges between a teacher-proof programme to an easily modifiable
programme. A programme must make provision for changing content, adaptation to the immediate
need(s) of the learners as well as the design of activities that can be adapted according to changes in
the dynamic curriculum that might be influential to good classroom practice. Earle (2002) explains that
flexibility within an online environment refers to the customisability. She refers to Schatz who believes
that “we are on the verge of being able to provide learning customized for each specific learner at a
specific time, taking into account their learning style, experience, knowledge and learning goals.”
Moreover, the author focuses the discussion on flexible learning as to be the successor of
programmes designed to be flexible. She poses the following ten stipulations as to describe flexible
learning. In this regard, Earle (2002) further draws on Reigeluth (1999) and lists the inclusion
embedded in these values, as: authenticity, ownership, and relevance of the learning experience for
students; rich social contexts and multiple perspectives for learning... the critically of “what to teach”
and the considerable variability of “how to teach it”; learning that is driven by an ill-structured problem
(or question, case, project). The overseer of a workable pedagogy is the lecturer and therefore the
perception of the lecturer during the diffusion process is important.
4.3 Lecturers perception and expectations from blended learning
Bombarding lecturers with all that is said in this section is certainly overwhelming. Even more so,
considering the different generations and the rate of adoption, poses a challenge toward the process
of professional development. Systematically introducing lecturers to the blended learning approach,
integrated between F2F and online teaching in an LMS, caused just as much acceptance, resilience
and resistance. The acceptance of blended learning usually calls for immediate implementation. After
considering whether students should really be encouraged to bring cell phones to class, Orlando
(2011) says: “For one, students will eventually be entering the working world where these devices are
ubiquitous. Look at any business meeting and you will find everyone with smart phones or laptops.
The restriction on their use amounts to telling students that they need to go back to using pen and
paper for notetaking, like the 60’s. We are preparing students for a world that no longer exists.” He
pleads further that instead of “fighting the digital movement, try treating it as a collaborator in the
learning process and a way to get all of your students involved in class.” This positive perception
motivates older lecturers until the realisation, with the amount of planning for successful blended
learning, becomes certainty.
The LMS becomes more of an organisational tool. Lecturers all start to use the LMS for managing
groups of students. The following uses often occur when lecturers adopt using an LMS to extent their
F2F meetings:
Large classes receive class notes (mainly Power Point slides) after the lecture via the LMS to
ensure that students pay attention in class.
Announcements for venue changes, due dates, and quick arrangements are ensured to reach all
students.
The burden of checking for plagiarism in paper submissions becomes the responsibility of the
student.
Assignments can be submitted electronically via the LMS and hard copies do not get lost. A
proper submission record is also viable.
Assessments (usually tests) can be done online and record keeping is assured.
Statistical analysis of “activities” can deliver data for quick reports.
There is better control over class lists and group submissions.
428

Arno Louw
Occasionally other resourceful web sites can be visited as to allow for access to more resources.
Consultation times are setup so that all students are aware.
Teaching assistants and tutors become maintainers of the online component of certain courses.
(The young ones know how the system works…).
The LMS can be remotely accessed when necessary.
The evaluation of the online component of courses, proofs delivers ‘dump’ sites and the
functionality is largely underutilised.
This occurrence is observable with many lecturers although there are the few champion users who
use the integrated approach differently. However, I question the students’ involvement in the learning
process as commended by Orlando (2011). Thus far, I conclude that the stipulated occurrences tend
more towards classical classroom management and not the actual learning management expected
from an LMS in use. Herein lies the topic of this paper and I shall now refer to these concepts.
5. Classroom management versus learning management
Many authors describe the term Classroom management. Objective to being partial, classroom
management emphasises a manager in a classroom (in this paper referred to as the ‘lecturer’), who
mediates the process of implementing a curriculum as efficiently as possible. (Allyn & Bacon, 1986;
Moskowitz & Hayman, 1976; Berliner, 1988; Brophy & Good, 1986; Allen,1986; Gootman, 2008;
Pintrich & De Groot,1990).
On the other hand, it becomes a challenge to describe Learning management. However, bearing this
caveat in mind, the research followed a direction that greatly resembles the discussions of many
authors. Bart (2011) denotes Jacoby who recommends four steps towards designing and planning
critical reflection which adds to breadth and death to student learning. Furthermore, also bearing in
mind that learning should be designed as a blended approach interrelated between a F2F and online
ecology, I refer to chain the work of Bart (2011) and Shibley (2011). In this instance, Shibley (2011)
accurately explains that: “Blended learning combines the best elements of face-to-face classes and
online classes. Get proven strategies on how to create a blended course that improves the teaching
and learning experience.”
5.1 The confusion between classroom management and learning management
Lecturers and students are both exposed to the opportunities available for implementing successful
blended learning. To make a decision on which aspects are classroom management related and
which are learning management related is difficult at this stage although examples can be given as to
which is the most appropriate for each aspect. At UJ, the following list of contributing factors causes
confusion for distinguishing between classroom management and learning management when using
the LMS:
There is a general confusion about administration and housekeeping of certain courses;
Unexpected increases in student numbers;
Changing curricula;
Late student registration;
First generation students;
Freedom of choice from students;
Late organisational adjustments;
Large classes (600+ students);
Increased responsibility for both lecturer and students;
Getting class lists in time after registration;
More frequent assessments which results in shorter tests;
Student tracking capabilities;
Saving paper, and
Extended degree courses to give all students an equal opportunity.
429

Arno Louw
Lecturers assume that learning only occurs in the LMS environment and use the classroom to
present content only.
Consequently, these factors also become reasons for lecturers finding an online component (LMS)
more lucrative to use. This is as far as the LMS is used and cause true blended learning never to
escalate to its full capacity in certain courses.
5.2 Considerations for proper classroom and learning management
Within an online environment, the following should always be considered when assigning students to
work independently from classroom facilitation:
What is there is no teacher present?
The use of netiquette.
Is navigation an obstacle for learning? (Number of clicks.)
How is learning managed in a classroom as opposed to how learning is managed in a classroom
with shifting borders?
I argue and maintain that an LMS is a tool for teaching and learning and not only for teaching or
learning. Perceptions of the lecturers over the past few years have been that the LMS becomes more
an organisational tool to consolidate large classes and ensure the integrity of disseminating
information. However, information is often distinguished from actual subject content and thus does not
form part of actual blended learning pedagogy. This often becomes the assumption upon which the
use of the LMS is based. It should also be mentioned that, in a few exemplary cases, the exact
opposite is also true which presumes that a course could have been presented online in its totality.
Does this really equip students with hard and soft skills upon graduation? Moreover, does this fulfil the
goal of the university’s learning to be philosophy of teaching and learning as opposed to learning to
do?
6. Findings and conclusions
The contextual use of F2F and online component of blended learning gives of view of classroom
management and learning management. The context will be given, briefly described from the
literature followed by an example. The contexts can be: learner-centred, assessment-centred,
collaborative, or community-centred.
A lecture is presented by DVD in many
venues or by streaming from a remote
classroom. The lecturer is shown on a big
screen in the laboratory.
Figure 4: Example 1: Language and skills for science (DVD)
Students are watching the lecture from a computer
laboratory on campus and are interactively involved
with activities on the computer. An assignment has
to be completed by the end of the lecture. Tutors
assist in the classroom management.
430

Lecturers and tutors are online after
the F2F meeting.
Figure 5: Example 2: Law discussions
Students download a video clip
with new vocabulary verbally
given to them. This is done via
the LMS.
Figure 6: Example 3: Cell phone vocabulary
A community is available in the
LMS.
Arno Louw
Students participate in a formal discussion group under
the headings:
“What you want us to know” and
“What we want you to know”
LIBRARY
Thereafter, a
dictionary should be
consulted in the
library to find the
correct spelling and
definition of the given
vocabulary.
Figure 7: Example 4: Writing Centre Community
A paragraph must be submitted online
using the vocabulary.
Participation begins online with individual students and
grows to workshops with groups in a F2F classroom.
431

A treasure hunt has to be done to find
locations for pedagogical design. Students
have to describe the route in order to reach
their outcome. This is done independent of the
LMS.
Figure 8: Example 5: Google maps example
Arno Louw
This activity culminates in a workshop session
that happens F2F.
Students have to make a journal entry after every lecture. They are rewarded marks if they can
explain what knowledge and skills they have gained by making examples of real life situations.
Figure 9: Example 6: Reflection journal (ONB)
7. Summary
This paper presented an argument that there is a meaningful difference between classroom
management and learning management. The perception causing this difference is situated in the
adoption of lecturers toward using blended learning. Blended learning at UJ mainly uses ICTs and
specifically an LMS whereby on and off line activities and participation encompasses the style of
teaching. This approach is conducted from a policy on teaching and learning strategies. The
background to this was presented followed by a brief description on the methodology used.
Thereafter, the perception of students, as the Y generation was described. The discussion about the
impact and considerations escalated into the perceptions of students and lecturers was presented.
After the perceptions had been clarified, the context of classroom management and learning
management within blended learning was discussed. This is then was followed by examples as part
of the findings and conclusions. Thereafter, a summary of the paper was given.
432

References
Arno Louw
Allen, J.D. 1986: Classroom management: students' perspectives, goals, and strategies. American Educational
Research Journal, 23, 437-459.
Anderson, T 2004: Online Learning Theory. Towards a theory of online learning. Chapter 2. Anthabasca
University. Online book. Available:http://www.cde.athabascau.ca/online_book/ch2.html Accessed: 22
November 2006.
Bart, M. 2011 Critical Reflection Adds Depth and Breadth to Student Learning. FACULTY FOCUS, May 11 2011.
FacultyFocusSupport@magnapubs.com also available: http://www.facultyfocus.com
Berliner, D. C. 1988: Effective classroom management and instruction: A knowledge base for consultation. In J.
L. Graden, J. E. Zins, & M. J. Curtis (Eds.), Alternative educational delivery systems: Enhancing
instructional options for all students (pp. 309-325). Washington, DC: National Association of School
Psychologists.
Brophy, J. E., & Good, T. L. 1986:. Teacher behavior and student achievement. In M. C. Wittrock (Ed.),
Handbook of research on teaching (3rd ed., pp. 328-375). New York: Macmillan.
Buzzetto-More, N.A. 2008: Student perceptions of various e-learning components. Interdiciplinary Journal of E-
Learning and Learning Objects. 4(2008).
Cambridge, D.; Kaplan, S. & Suiter, V. 2005 Communities of Practice Design Guide.
http://www.educause.edu/ir/library/pdf/NLIO531.pdf.
Clark, W. 2009 Beyond Web 2.0: mapping the technology landscapes of young learners Journal of Computer
Assisted Learning. Article first published online: 16 JAN 2009 DOI: 10.1111/j.1365-2729.2008.00305.x
Devedzic, V.B. 2003: Key issues in next-generation web-based education Available:
http://scholar.google.com/url?sa=U&q=http://ieeexplore.ieee.org/iel5/5326/27780/01238675.pdf%3Farnumb
er%3D1238675 Accessed: 20 January 2006.
Drucker, P. 1999 Management Challenges for the 21st Century ,Harper Business
Garrison, D.R. & Akyol, Z. 2009 Role of instructional technology in the transformation of higher education.
Journal of Computing in Higher Education. Volume 21, Number 1, 19-30, DOI: 10.1007/s12528-009-9014-7
Gootman, M.E. 2008: The caring teacher's guide to discipline : helping students learn self-control, responsibility,
and respect, K-6. 2008, p.36
Keller, K. & Cernerud, L 2002: Students’ perceptions of e-learning in university education. Journal of Educational
Media. 27(2002) Nos. 1-2.
Kim, B. 2001 Social constructivism. In M. Orey (Ed.), Emerging perspectives on learning, teaching, and
technology. http://www.coe.uga.edu/epltt/SocialConstructivism.htm
Kim, B. 2010 Web 2.0 and Emerging Learning Technologies.
http://en.wikibooks.org/wiki/Web_2.0_and_Emerging_Learning_Technologies /Innovative_Pedagogies
Kohonen,V. 2009 Learning to learn through reflection – an experiential learning perspective.
archive.ecml.at/mtp2/Elp_tt/.../Supplementary%20text%20E.pdf
Moskowitz, G., & Hayman, J. L., Jr. 1976: Success strategies of inner-city teachers: A year-long study. Journal of
Educational Research, 69, 283-289.
Orlando, J. 2011: Bring your computer to class day. FACULTY FOCUS, June 6 2011.
FacultyFocusSupport@magnapubs.com also available: http://www.facultyfocus.com
Pintrich, P.R., & De Groot E. V. 1990:. Motivational and self-regulated learning components of classroom
academic performance. Journal of Educational Psychology, 82, 33-40.
Reeves, T.C. & Hedberg, J.G. 2003: Evaluating interactive learning systems. Athens, GA: University of Georgia,
College of Education.
Siemens, G. (2005). Connectivism: A Learning Theory for the Digital Age. Internationa Journal of Instructional
Technology & Distance Learning , 2 (1).
Shibley, I 2011 Putting the Learning in Blended Learning. FACULTY FOCUS, February 21 2011.
FacultyFocusSupport@magnapubs.com also available: http://www.facultyfocus.com
Smith, G. & Grant-Marshall, S. 2004: Mind the Gap. Johannesburg: Penguin Books.
University of Johannesburg Senate Teaching and Learning Committee: Teaching and Learning Strategy,
3/2009(1))
Wenger, E 2011: Communities of Practice (on line: http://www.ewenger.com/theory/.accessed 16 May 2011.
Wolfgang, C.H. & Glickman, C.D. 1986: Solving Discipline Problems. Allyn and Bacon.
433

How to Represent a Frog That can be Dissected in a Virtual
World
Robert Lucas
Department of Physical Sciences, The Open University, Milton Keynes UK
r.j.lucas@open.ac.uk
Abstract: One of the problems in creating a realistic virtual dissection is how to represent the solid nature of a
substance and any subcomponents it may have using surface modellers such as SketchUp (Sketchup 2011) and
Maya (Maya 2011). Although solid modellers are available, models produced by them are not generally usable by
modern game engines. This paper describes a practical method of representing and manipulating objects so that
they can be cut at arbitrary points by a planar knife and behave as though they were solid. Although this paper is
mostly concerned with cutting through 3D models in order to give the illusion of them being made of a
homogenous material, some discussion of how other aspects of the dissection process is also included.
Background on how models are represented by meshes made from lists of triangles is given. The representation
and the algorithms to support the cutting operations are given at a fairly high level. Complete implementation
details and discussion of all possible pathological cases are not included. Some examples of the cutting process
being used on a simple and more complex object are given. In particular we show how a model of a Malteaser
can be cut in half where its internal texture is a photograph of the actual insides of a real Malteaser. This gives a
very realistic cut that entirely creates the illusion of the 3D model being solid. We also show how the skin of a frog
can be represented so that it appears to be solid when cut. This paper will be of interest to implementers of any
3D virtual application requiring the realistic representation of arbitrary planar object cutting.
Keywords 3D, virtual, dissection, cutting, surface modelling
1. Introduction
For many years 3D graphics has been used to provide games on PCs and consoles and the huge
profits obtainable from successful games have been a huge driving force in the parallel development
of hardware such as graphics cards and consoles, and software that implements 3D modelling and
animation environments. Recently there has been an explosion of 3D simulation in science including
many examples by the author, which include a 3D Celestial Sphere (Lucas 2010), A Telescope
Simulator used to train students embarking on a course in observational astronomy (Lucas, Kolb
2009), some science experiments (Lucas 2009) and an archery simulation for teaching the motion of
projectiles (Lucas 2010).
Many simulations of scientific processes can be achieved fairly straightforwardly as the above
applications demonstrate. However, in order to create a realistic simulation of a dissection of an
animal such as a frog many components need to be in place. Organic materials range from being soft
to hard. They often contain liquids and gases. They will sometimes be elastic and sometimes plastic
in how they deform under pressure. There will often be some surface tension that will pull the sides of
an incision apart for example. A common operation in dissection is pulling a surface tissue back and
perhaps pinning it. These are all issues that need to be addressed when constructing a 3D simulation.
There are many issues surrounding dissection and whether we should be doing it with actual animals
and in particular with frogs (Save the frogs 2011). These issues, although interesting are not
discussed here. However, if we are to replace real dissection with something virtual then we want this
something to be as realistic as we can possibly make it. Existing dissection applications consist of
models with discrete components that simply allow the user to take them apart, such as in the case of
a virtual frog, where it is possible to remove its organs as entire entities. More realistic applications
(Digital Frog 2011) allow cuts to be followed with a mouse and then play a movie of an actual
dissection of that part. This is a far cry from an actual dissection where a scalpel is used to cut
through tissue, organs and whatever the blade comes up against. In all 3D graphic systems that are
used in games and virtual worlds such as second life; objects are created using a mesh to represent
the geometry of the surface and a texture used alongside a shader (i.e. a material) to render
something that looks solid. Cutting into such a model would instantly reveal it to be hollow (or not
visible at all). Although it is possible to put one model inside another, and then another model inside
this one and so on, this is a poor approach as this method is incapable of representing the
homogeneity of a substance such an amorphous tissue which might fill the space between the
surface of one object and the surface of another. What is required is a method of cutting the meshes
that constitute the geometry of an organ, and as the mesh is cut (or as soon as the cut is completed),
434

Robert Lucas
rebuilding the mesh on-the-fly with the appropriate texture. Cutting is a common technique in 2D
graphics, sometimes used in mapping applications to enable arbitrary regions to be cut from a larger
map. Often this will be done so that where a public utility gets cut, for example, the properties that
belonged to the original are re-attached to the cut parts. This is the approach taken here and is
extended to 3D. We use a set of materials consisting of textures and shaders that are associated with
the various components that construct the object to enable us to recreate the cut objects and its
components. Components are recursively built from subcomponents to the level of detail required,
everything else is assumed to be homogeneous and hence can be represented by its associated
texture when cut. The Unity 3D development environment, which uses the PhysX physics engine from
NVIDIA, is used (Unity3D 2011). This gives complete programming control over mesh construction
and texturing.
2. Some 3D graphics basics
Polygonal 3D models are generally built out of meshes that are a collection of triangles. Triangles
consist of three vertices, three edges and a single surface called a face. The single surface means
that a triangle can only be seen from one side. In general it is standard that if a triangle has had its
vertices defined clockwise from the viewing angle, then it is visible, and from the opposite side it is
invisible. Thus if you cut a sphere (built from triangles) in half and remove the half nearest you, you
won’t see anything at all of the remaining hemisphere as all the faces will be orientated away from
you and can only be seen from the far side. Triangles are the chosen primitive for 3D polygonal
models because they have the property that all three vertices always lie in a single plane and
graphics cards can be optimised to take advantage of this. Quadrilaterals or quads do not share this
property. However, a quad can always be split into two triangles by adding a diagonal. Although it is
the case that some modelling packages will let you develop in quads, there will always be a
‘triangulation’ facility. This paper is only concerned with polygonal models built from triangles. Figure 1
shows a polygonal model of a frog’s poison gland as shown within a perspective view in the Maya
surface modelling application (Maya 2011).
Figure 1: A frog’s poison gland as a smooth shaded polygonal model
To understand the descriptions and algorithm a reasonable familiarity with vectors is needed. A good
reference is [Spiegal 1974]. The first chapter introduces most if not all of what is needed. Note that in
this paper vectors will nearly always be 3 vectors, that is 3D vectors with x, y and z components.
435

Robert Lucas
Occasionally we will also refer to 2 vectors that are used to map textures onto surfaces. In both cases
vectors are represented by a bold letter, such as V1 in any formula or sentence.
3. Basic data structures
In the Unity (Unity 2011) world a mesh consists of an array of triangles. This is a list of indices into
another array of vertices where each vertex is a 3-vector. This defines the position of the vertex in 3D
space. Each consecutive three entries of the triangles array define a triangle. To make this tangible
figure 2 shows how a cube is represented by twelve triangles where each triangle consists of three
vertices. The ‘triangles’ array that corresponds to this contains: 3, 1, 2, 2, 1, 0, 5, 3, 4, 4, 3, 2, 7, 5, 6,
6, 5, 4, 1, 7, 0, 0, 7, 6, 5, 7, 3, 3, 7, 1, 2, 0, 4, 4, 0, 6. In fact, it is not quite as simple as this as there is
a good deal of redundancy in the vertices array with the same point being represented as many times
as it appears in the definition of a triangle. The very first operation applied in our cutting process is to
remove this redundancy to make the triangles array as defined above. This makes it very much easier
to compare vertices as the vertex number in the triangles array uniquely defines a point in space.
Thus if two entries in the triangles array are the same, then they must refer to the same point in 3D
space.
Figure 2: The vertices of a cube drawn as a mesh of triangles
4. Developing a cutting algorithm
Developing software for complex operations can sometimes be rather akin to the old joke: when you
ask an Irishman the way, he replies ‘Well I wouldn’t want to start from here’. This is because the
space the problem spans can just be too large to get to grips with. However much we like to pretend
in the software world that all things can be achieved by a top-down design approach (Lucas 2000), in
practice we commonly use a combination of top-down and bottom up design approaches, if for no
other reason than it can prevent us going insane from the complexity of what we are trying to deal
with. The bottom-up approach does allow us to start from ‘somewhere else’ which will indeed make it
easier to get to our destination.
4.1 Representing a planar cut as a data structure
In this case it seems sensible to start with a definition of a cut as a data structure that will give us a
handle on actually performing the cut. Then we can address the issue of how we might create such a
cut data structure at a later time when we are entirely happy that our representation has made the
problem tractable. This concentrates the mind wonderfully, what do we mean by a cut? What do we
want to be the result of the cut? These questions come to mind as soon as we address the problem of
representing the cut. For simplicity we will consider planar cuts. Elementary geometry tells us that a
plane can be represented by a normal to the plane and a single point on the plane, so there is
certainly no problem in representing that; two 3-vectors will suffice. If we look at Figure 3 and imagine
a planar cut caused by a blade occupying a plane containing the x and y axes passing through the
436

Robert Lucas
origin at the centre of the cube, we can imagine the points of intersection of this plane with each
affected triangle.
Figure 3: Visualizing the cut points of a plane with a cube
This gives us the necessary insight to define a cut-point as consisting of two vertices and a number
representing how far from the first vertex toward the second is this cut-point. Thus a single cut point is
represented by the triple:
(VertexA, VertexB, lambda)
Where the cut occurs on this edge between VertexA and VertexB at the point
(VectorB-VectorA)* lambda
Where VectorA is the vector from the origin to VertexA and VectorB is the vector from the origin to
VertexB.
Thus the entire planar cut can then be represented by the ordered list:
{CutA, CutB,…,CutH}
It is clearly the case that this representation can be used on any arbitrary complicated mesh, just as
long as the mesh is composed of triangles. More importantly, we can now perceive how a cutting
algorithm might proceed by taking each cut point and creating a vertex at the point of the cut and
rebuilding each triangle that has a vertex on the edge containing the cut point. Finally we will need to
fill in the missing faces that the cut has created. In pseudo code (Lucas 2000) the algorithm is:
Begin
CutPoint1 = Cut[0]
VertexA = CreateVertex(CutPoint1)
For each remaining cut point do
CutPoint2 = Cuts[next];
VertexB = CreateVertex(CutPoint[next])
If both cut points share same first vertex then
CreateTriangle(VertexA, VertexB, SharedVertex)
Else
CreateTriangle(VertexA, VertexB, CutPoint1.StartVertex)
CreateTriangle(VertexA, CutPoint2.StartVertex, CutPoint1.StartVertex)
Endif
VertexA = VertexB
Repeat
End
437

Robert Lucas
Note the need to create two triangles where there isn’t a shared vertex between the pair of cuts. The
algorithm needs to be extended to remember all the created vertices as these will be needed to fill the
hole created by the cut.
4.2 Filling the hole created by the cut
This is a well-established problem with several solutions. It is commonly referred to as triangulating a
polygon. The added vertices form a 2-D polygon. In the example of the cube, this polygon is simply
defined by the set of 3-vectors given by the points A through H. For a solution see (Chazelle 1991) or
(de Berg 2000) or perform a Google search on ‘triangulating a polygon’ for algorithms, comments and
related problems and diversions, such as the Art Gallery Problem proposed by Victor Klee in 1973
when he was a young mathematician at Montreal University. One thing to note is that, unless we want
to create a lot of unwanted and useless triangles (i.e. they are not really triangles at all) we need to be
careful about vertices being collinear, that is we don’t want to create a triangle from three vertices that
are in a straight line. We can easily test for two vectors being collinear using the vector dot product,
but just how we implement this in terms of how accurate we want this test to be can be a source of
problems. Where the test is too stringent we won’t catch all the collinear vertices, where it is too lax,
we will be leaving visible gaps in our filled face.
4.3 Creating the cut data structure
In order to create all the cut points we need to take every edge of the mesh and calculate the
intersection of this edge with the plane that represents the blade performing the cut. Given two points
represented by the vectors V1 and V2 and a plane represented by the normal N and a point on the
plane given by the vector P. The point of the intersection is given by the vector (Bourke 1991):
I = V1 + lambda (V2-V1)
Where lambda is the scalar given by:
lambda = N dot (P – V1) / N dot (V2 – V1)
where dot is the vector dot product.
Where lambda is between zero and one, then the intersection is actually on the line between the two
points. This is exactly what we want to test for. So the algorithm for determining the cut points needs
to take each edge of the original mesh and calculate the value of lambda and where this is between
zero and one we have a cut point formed from the two vertices and the value for lambda. The cut
points then need to be topologically sorted so that we deal with each adjacent edge containing a cut
in succession to ensure that the triangles are created correctly.
4.4 Texture mapping
Textures are applied to a surface using what are commonly known as uv coordinates. These are
defined for every vertex of every triangle. The uv coordinates are two dimensional and in Unity are
represented by a 2-vector. These uv vectors dictate how the texture is to be mapped onto the triangle.
They range from zero to one, where one would dictate the rightmost position on the texture (for u) or
highest position on the texture (for v). Thus a particular patch of a texture can be mapped onto each
triangle by manipulating the uv coordinates for each vertex.
Being able to map a texture onto a surface is very important. This is what gives our model realism.
We can also use photographs for our textures and as shown in [Lucas 2010] this can be of huge
advantage in scientific simulations by giving extra authenticity to the model. With a cut surface the
situation is complicated by the fact that this surface has been programmatically created to fill the hole
left by the cut using some number of different shaped triangles. The texture must be mapped such
that there is continuity of the texture over the edges of one triangle to the next. This is not as
complicated as it may first appear. Consider the vertex marked A Figure 4 that is common to many
triangles. We want there to be continuity in the textures appearance as it is mapped over all the
triangles meeting at Vertex A.
438

Robert Lucas
Figure 4: A set of triangles filling a planar hole
If we calculate the uv values for A based on the coordinates in space of A then clearly this will map
identically for all the triangles that contain A (A has the same coordinates regardless of which triangle
you pick!). This is true of any other vertex and as the mapping from one vertex to another is linear, will
quite automatically get continuity at the edges as long as we calculate the uvs from the coordinates.
To map the texture uniformly onto the surface we need to find the horizontal and vertical coordinate
axes on the cut face. You can consider this to be like placing a sheet of graph paper onto the face.
We already have the normal to the cut face as this is the same as the normal to the cutting plane. If
we take the cross product of the normal and the ‘up’ vector we get the horizontal coordinate, and the
cross product of the normal and the ‘right’ vector we get the vertical coordinate. The up and right
vectors are defined for you in Unity. The algorithm is then (in pseudo code):
Begin
N = Cut plane normal
H = N cross Right
V = N cross Up
For every triangle T do
uv[1st Vertex] = Vector2(Vertex1 dot H, Vertex1 dot V)
uv[2nd Vertex] = Vector2(Vertex2 dot H, Vetex2 dot V)
uv[3 rd Vertex] = Vector2(Vertex3 dot H, Vertex3 dot V)
Next
End
We can scale the texture in both directions by putting a multiplying factor in front of the Vector2
constructor.
We can choose to treat textures differently for different objects by separating the texture for the
original surface from that intended for the cut surface. For example we can have one texture for the
surface of an egg, the shell, and another for the interior white. This effectively makes the shell
infinitely thin, but saves on the number of meshes and triangles needed and can be a reasonable
compromise where memory is tight. The materials associated with the textures are being stored within
the code as this makes it unnecessary for external references to be made at run-time. Figure 5 shows
a cross section of a Malteaser. This is constructed from two polygon models of a sphere with twelve
vertical and horizontal divisions. The outer sphere has a brown chocolate texture and the middle
sphere is given a cut-face texture made from a photograph of the centre of an actual Malteaser.
439

Robert Lucas
Figure 5: A model of a Malteaser which has been cut to make it appear solid with a generated face
and a cut face texture made from a photograph of the real thing
This was good enough to fool my four and six year old into thinking it was a photograph of a real
Malteaser and there is no doubting their expertise in this area! When it was explained that it was a 3D
model and were shown it being rotated, one of them asked if it was possible to print it in such a way
as they could eat it!
Figure 6 shows the cross section of some frog skin which has been created as a model. This has
been constructed from polygons representing the main skin volume, the poisonous glands (bulbous
bottle shaped as given in figure 1) and the mucous glands (white).
Figure 6: Cross section through a frog’s skin
In this case all the textures were drawn from a diagram obtained from the internet (R Hamley 2011),
but it should be clear that the textures could have been obtained from photographs. Clearly this view
is at some higher level of magnification than normal. However, there is no reason why we cannot
include magnified views in our 3D world. The virtual spectroscope described in (Lucas 2010) includes
a telescope that is implemented using a software camera capable of yielding whatever magnification
is required of whatever it is pointed at in the virtual world.
5. Discussion and future development
There is much that can be done with what has been implemented. It certainly felt at first quite strange
to be able to cut through models that we know to be hollow and have them display internal structure
as though they are solid. Effectively it is an illusion very much like a magic trick. However, any 3D
graphics application is actually an illusion in any case! There is still some exploration to be done with
this tool as it stands without further development as the results are quite dramatic, being both visually
and intuitively appealing. We can consider dissecting objects that might be too difficult for standard
tools, such as small insects and microscopic organisms. Considering performance, the array
operations that are required to manipulate a mesh in real time have adequate performance for these
type of cutting operations on complex meshes. The Unity Scripting Reference (Unity 2011) states that
two million vertices can easily be handled within a second. As even complex models do not usually
have more than a few hundred thousand vertices, we can see that we are well within the performance
needed for real-time operation.
440

Robert Lucas
However, in a real dissection, cuts are not usually made in a single action but are the result of a
number of incisions. Having successfully implemented the planar cutting algorithm with the associated
texture mapping to the cut surface, it is now possible to see how incisions can be implemented. A
knife blade can be represented as a 3D model and the collision between it and the object being cut as
well as the geometry of the blade can be used to calculate the appearance of the incision. The
incision can be implemented in much the same way as the cuts in the approach described above, but
now we will want to add in some behaviour of the cut material, such as how much it pulls back from
the blade to create a groove or valley in the cut surface. The sides of this groove can be constructed
as per our cut surface above. Some experimentation with how a ‘peeling back’ type operation can be
performed. We can identify a contact point and force from a dissection tool indicated by its movement.
We can warp a mesh differentially by moving its vertex positions along the direction of the force from
the tool whilst maintaining distances between adjacent vertices to maintain the surface integrity of a
substance like skin. We can even introduce some stretching if we need to. We can propagate the
force from the tool through the tissue being ‘peeled back’ so that points nearer the tool move the most
whilst those distant from the tool behave as though anchored. With a suitable controlling device, there
is no reason why we couldn’t offer some feedback to the user on the force required to perform an
incision or operation to move or peel-back an object. At this point we will be well on the way to
providing a realistic, if not total, dissection experience. This will be a very different experience from
that offered by current dissection packages as there will be no constraints on where you can put the
knife in, nor what you can put it into.
6. Conclusions
We cannot realistically expect to perform authentic dissections using 3D models in a games
environment without putting in place some algorithmic and representational solutions to some of the
many facets of the various processes that make up a dissection activity. This paper has shown how
we can achieve planar cuts on surface models that make them appear to be solid by cutting the faces
of the original object and replacing them with faces along the cut and filling the hole created by the
cut. It has also been shown that textures can be added to the cut face that can span many triangular
faces and present a coherent continuous texture that can be derived from a photograph to add extra
realism. It is proposed that these techniques can be extended to provide partial cuts or incisions. It is
asserted that these techniques take us a significant step towards performing virtual dissections using
standard tools available to games programmers.
References
Bourke P (1991) Intersection of a point and a plane http://paulbourke.net/geometry/planeline/ (accessed on
07/06/11).
Chazelle, B (1991). "Triangulating a Simple Polygon in Linear Time." Disc. Comput. Geom. 6, 485-524, 1991.
de Berg, M.; van Kreveld, M.; Overmans, M.; and Schwarzkopf, O. "Polygon Triangulation: Guarding an Art
Gallery." Ch.3 in Computation Geometry: Algorithms and Applications, 2 nd rev. ed Berlin: Springer-Verlag,
pp.45-61, 2000.
Digital frog 2011 http://www.digitalfrog.com/ (accessed on 07/06/11).
R Hamley 2011 “Comparing Frog and Human Skin”.
http://slohs.slcusd.org/pages/teachers/rhamley/Biology/Frog%20Dissection/integumentary%20system.html
(accessed on 07/06/11).
Lucas R. (2000) “Programming Methodology”
http://www.keylinkcomputers.co.uk/Tom/Programming/programming%20methodology.pdf (accessed on
07/06/11).
Lucas R J. (2009) “3-D Immersive Screen Experiments”. Proceedings GIREP 2009. , (Groupe International de
Recherche sur l'Enseignement de la Physique) 17-21 August 2009 Leicester University. (ISBN 978-1-4461-
6219-4).
Lucas R J, Kolb U. (2009) “Use of 3-D Virtual Environments in Teaching Physics and Astronomy”. Proceedings of
Frontiers In Science Education Research. Cyprus March 2009 Eastern Mediterranean University press
2009.
Lucas, R J. (2010). “Using a virtual world for teaching the motion of projectiles”. In Norton, Andrew ed. Electronic
Resources for Teaching and Learning (In press). The Open University, pp. 58–64.
Lucas, R J (2010). . “The Celestial E-Sphere”. In Norton, Andrew ed. Electronic Resources for Teaching and
Learning (In press). The Open University, pp. 58–64.
Maya (2011) http://usa.autodesk.com/maya/ (accessed on 07/06/11).
Save the frogs (2011) http://www.savethefrogs.com/gifts/digital-frog.html (accessed on 07/06/11).
Sketchup (2011) http://sketchup.google.com/ (accessed on 07/06/11).
Spiegel M. R. 1974 “Theory and Problems of Vector Analysis” McGraw-Hill Book Company, New York.
Unity 3D (2011) http://unity3d.com/ (accessed on 07/06/11).
441

Learning by Wandering: Towards a Framework for
Transformative eLearning
Marie Martin and Michaela Noakes
Duquesne University, Pittsburgh USA
Martin684@duq.edu
Noakes495@duq.edu
Abstract: As technology continues to flatten the world and as Web 2.0 changes the way knowledge is created
and shared, tertiary education institutions are turning increasingly to eLearning to extend access to students
globally as well as to improve the quality of their learning experience. Learning Management Systems (LMS)
currently dominate the delivery of eLearning at this level. Though these systems have extended functionality by
including some Web 2.0 tools, they are generally perceived as a ‘walled garden’, essentially embodying the
traditional transmission paradigm of teaching and learning rather than the philosophy of Web 2.0. This is leading,
particularly in the blogosphere, to calls to break down the walls of the LMS and to explore more open online
courses. There is, however, an emerging view that Web 2.0 ideals can be realised within an LMS environment,
provided the environment is aligned with these ideals. This paper supports that view. It presents a case study of
an eight-week eLearning course based on this premise, offered in spring 2011 as part of a doctoral programme in
Instructional Technology by Duquesne University, Pittsburgh, USA, and designed and delivered within an LMS by
an instructor living in Northern Ireland. The course is underpinned by the concept of learning by wandering. The
pedagogy is aligned with the fundamental Web 2.0 philosophy. Within broad parameters, it is flexible, studentcentred
and, from an early stage, student-led. Students are encouraged to use a variety of Web 2.0 tools,
according to their preferences, to collaborate in preparation for their leadership role and as a language to express
their ideas and to share their learning. The teacher’s role is identified as sage at the side. This case study is
intended to contribute to the provision of a framework for transformative eLearning through fostering a Web 2.0
ethos within a traditional learning environment.
Keywords: learning management systems; Web 2.0 ethos; case study; learning by wandering; sage at the side;
transformative eLearning
1. Introduction
As technology continues to flatten the world (Friedman, 2007), and as Web 2.0 changes the way
knowledge is created and shared (Guth and Helm, 2010), tertiary education institutions are turning
increasingly to eLearning to extend access to students globally as well as to improve the quality of the
learning experience (Beetham and Sharpe, 2007). Learning Management Systems (LMS) currently
dominate the delivery of eLearning at this level (Kyong-Lee and Bonk, 2006). Though these systems
have extended functionality by including some Web 2.0 tools, they are still generally perceived as a
“walled garden”, essentially embodying the traditional transmission paradigm of teaching and learning
rather than the philosophy of Web 2.0 (Katsifli, 2010; Lee and McLoughlin, 2011). This is leading,
particularly in the blogosphere, to calls to explore ‘massive open online courses’ (Stein, 2008;
Siemens, 2011). Huijser and Sanker (in Lee and McLoughlin, 2011: 267-283) argue, however, that
Web 2.0 ideals can be achieved within an LMS, provided the environment is aligned with these ideals.
This paper offers a descriptive case study of an eight-week eLearning course that endeavoured to
create this environment and to bring the new Web 2.0 mindset to bear on the delivery of eLearning
within a traditional LMS. The study is based on qualitative data derived from the following sources:
notes made by the lead author as participant observer (Quinn Patton, 2002), archives of the
synchronous weekly class, discussion board activity, student assignments, and written feedback from
students. It aims to highlight the human dimension and provide an inside view of the eLearning
experience. The focus was on two questions: How can this course, delivered within a traditional LMS,
embody the Web 2.0 ethos? Does this process lead to a transformative eLearning experience for the
students?
The eight-week online course on Education in a Global Society was offered as part of a doctoral
programme in Instructional Technology by Duquesne University, Pittsburgh, USA and designed and
delivered by an instructor living in Northern Ireland (the lead author). The paper will outline the
rationale, aims and objectives of the course, the demography of the doctoral cohort, the course
design in the context of high expectations of delivery by eLearning, the pedagogy, the technologies
used, and the lessons learned. It is hoped that this case study can contribute to a framework for
transformative eLearning for both students and teachers and support the argument that the Web 2.0
ethos can be realised within a traditional LMS environment.
442

2. Rationale, aims and objectives
Marie Martin and Michaela Noakes
The rationale for the course was to address the need to raise the level of global awareness of
students with regard to education in order to prepare them to function effectively as educators in a
multicultural society and in a world without borders. The aim was to enable students to understand the
socio-cultural context of education globally, beginning with their own as a basis for the comparative
study of other systems. The objectives were to understand the history of educational ideas in the
western world and evaluate these in the context of a global society; to compare and contrast
American education with education systems in other selected countries; to assess the students’
personal educational experience and philosophies, and to critique how their philosophies affect the
roles of educators, students, and organizations.
3. The doctoral cohort
The doctoral cohort comprised eight graduate students, all with responsible positions in education or
training, all with limited exposure to other cultures, and all with previous experience of eLearning. The
level of digital literacy ranged from good to very high. By contrast, the level of digital literacy of the
teacher was quite basic, a situation that led to unexpected and invaluable outcomes which will be
discussed later in the paper.
4. The course design
The design was grounded in the firmly held view of the teacher/designer that eLearning could actually
break down the barriers to learning encountered in the ‘limited and closed world of the traditional
classroom’ (Martin, 2010: 75) and, with appropriate e-pedagogy, could ‘sustain a form of learning that
is equivalent, if not superior, to that provided by traditional classroom settings’ (Kuriloff, 2005). It was
also grounded in the belief that this form of learning could provide students with a challenging,
enjoyable and transformative eLearning experience. The design was underpinned by the concept of
learning by wandering – using technology, in accordance with one’s own way of learning, to embark in
a spirit of ‘serious playfulness’ and with an ever-open mind on a largely uncharted voyage over the
seas of cyberspace in an endless quest of other ways of knowing, thinking and being in the world
(Martin, 2010: 85). This way of learning necessarily involves being willing to ‘travel tangentially … and
to share with and learn from others’ (Martin, 2010: 24), and the course design sought to facilitate that
approach. The sharing with and learning from others was to take place asynchronously in Blackboard
and in real time in Wimba Classroom where weekly class sessions were to be held. Virtual visits were
to be undertaken first as a class group to a small number of identified educationally high performing
countries. Additionally, each student was given complete freedom to select a country - other than one
already visited - for individual wandering and exploration. All such learning and reflections were to be
shared. For this purpose, students were given the option of using Web 2.0 technologies.
5. An appropriate e-pedagogy
Mindful that many educators tend to regard ‘on ground’ classroom-based teaching as the optimal
learning experience and therefore tend to see online learning simply as an ‘alternative delivery system
for traditional pedagogy’ (Kuriloff, 2005), the teacher sought to create an appropriate e-pedagogy that
would help release the transformative potential of eLearning. Foremost in the pedagogy was the
establishment of a high quality relationship with the students. Hargreaves (2003) stresses that good
teachers understand the importance of caring relationships and emotional engagement with learning.
This applies particularly in eLearning because of the potential in this environment for personal and
social isolation and disaffection (Croft, Allison and Duff, 2010). Additionally, within broad parameters,
in accordance with the philosophy of learning by wandering, the e-pedagogy was flexible – to allow for
some productive off-course wandering and reversal of roles. It was also aligned with the open,
collaborative and relational mindset of Web 2.0 (Guth and Helm, 2010: 22). Blackboard and Wimba,
and - in accordance with students’ preferences – Web 2.0 technologies were used as shared spaces
for ‘collective intelligence’, and there was a strong focus on ‘participative and collaborative user
experiences’ and on ‘dialogical conversations’ (Guth and Helm, 2010: 41). Specifically, the epedagogy
was student-centred and, from an early stage, student-led, as students worked in pairs to
lead a part of the group learning journey, as well as undertaking independent individual explorations
to a country of their choice. A shared learning approach within a community of learners was fostered
throughout the course to allow them to experience learning as a collaborative, social and enjoyable
activity, inclusive of both students and teacher. The underpinning metaphor for this approach is
Thornburg’s (2004) Campfires in Cyberspace, with the campfire as the more formal learning place
where the elders (the teacher or the student leaders) tell the story and initiate discussion, and the
443

Marie Martin and Michaela Noakes
watering hole as the informal space where wanderers take turns to be storytellers and listeners and
where peer learning takes place.
6. Implementation
In the context of maintaining a caring relationship in which each student felt that his/her progress
mattered to the teacher, the latter made two major commitments. One was to offer students who were
unsure about the direction of their assignments the opportunity to submit them as work-in-progress for
monitoring by the teacher without prejudice to the final grade. This option was welcomed by the
students and availed of responsibly - usually in the form of Google docs to facilitate pre-submission
sharing and editing as the student deemed appropriate. The second commitment was to email
individual formative feedback after each assignment. This was enormously appreciated by the
students and increased their motivation to give of their best. One student’s reaction was typical: I
appreciate that you personally email each of us after a project or assessment. ... I find it validates the
hard work we put into our assignments. ... I understand it takes time, but it does mean so much to me
that you send a personal email with strengths and weaknesses.
The first form of learning by wandering to which the students were introduced was ‘time travel’ to
enable them to follow in outline the story of educational ideas in the western world from ancient
Greece to the Digital Age. This served as context for the ongoing exploration into the direction
education should be taking in our global society. Another form of tangential wandering introduced at
an early stage was ‘The Journey into Self’. This encouraged students to keep a reflective journal to
monitor their own inner wandering and to consider whether this journey was transformative of them as
learners.
Travelling tangentially was encouraged throughout the course. Initially, this took the form of looking
briefly at the theme of learning by wandering in myth and legend (Martin, 2010) as well as in ancient
and more modern history and in literature. The students saw tangential travelling as a rich
contributory source to their learning and felt comfortable with going off course to share readings and
learning experiences from their ‘real life’. An example was the sharing at one point by a student
discussion leader of an issue, which, though introduced under the subject title of ‘entirely unrelated’,
evolved into a fecund sharing of experiences on the importance of the teacher in the learning process.
7. Scaffolding students as leaders of learning
The strategy of having students lead the learning by putting two in charge of a particular section of the
group wandering proved to be highly effective in terms of learning as well as being an enjoyable and
potentially transformative experience for the leaders. Over the period of eight weeks, the class as a
group virtually visited four countries selected from the course textbook with emphasis being placed on
additional shared research. This meant that all students could take turns at being both teacher and
learner – or storyteller and listener according to Thornburg’s campfire and watering hole metaphor.
The teacher provided scaffolding for the leaders in a number of ways prior to their undertaking their
roles. Simple guidelines for leading asynchronous discussion were made available. The teacher also
drew up a content analysis model for online discussion, based on the model devised by
Gunawardena, Lowe and Anderson (1997). The model illustrates how knowledge construction and
negotiation of meaning online proceed through five phases, identifying at each stage the Process
Variables (how learners negotiate and interact) and the Product Variables (content and outcomes of
learner interactions). For convenience, the teacher’s version of this model is reproduced in table form
in Table 1.
This provided all students with a tool to enable them to analyse the content of the particular group
discussion they were leading. It also helped them to self-evaluate their own contributions to
discussions.
Finally, the teacher held a short tutorial in the Wimba Classroom with each pair of leaders just prior to
their undertaking the role. This was totally non-directive. Its purpose was simply to give the leaders a
voice, to ensure they understood their mission and assure them of teacher intervention only when or if
requested. They appreciated this freedom and empowerment, and without exception, rose to the
challenge, using mainly wikis and Google docs to collaborate in preparation for their task. The weekly
synchronous class was the forum where each pair of leaders prepared their peers for the next stage
of the learning journey. They also took great care with the identification and formulation of the main
discussion question to be followed up asynchronously in Blackboard, where they succeeded in
444

Marie Martin and Michaela Noakes
initiating and monitoring well-reasoned and stimulating posts, often enhanced by reference to
websites researched by students and illustrated by videos from YouTube and other video-sharing
websites.
Table 1: Model for online discussion content analysis (based on Gunawardena, Lowe and Anderson,
1997)
Phase Process variables Product variables
1
Sharing and comparing
information
Statements, observations
2 Discovery and exploration Questions, clarifications, elaborations
3
4
Negotiation of meaning
Co-construction of knowledge
Testing ideas
Revising ideas
5 Awareness of new knowledge
Joint meaning making
Shared understanding
Testing & revising against personal knowledge
Metacognitive statements
Reflections
Summarising (to reflect consensus or diversity of perspectives)
While the class travelled as a group to four selected countries, the teacher was engaged in individual
exploration of other parts of the world. The weekly synchronous class was the ‘campfire’ around
which the students gathered with the teacher as listeners to the tales of their peer leaders, before
turning their attention to the teacher in the role of storyteller. In practical terms, the teacher’s input
enriched the learning journey by increasing the number of countries the class could ‘visit’ in the short
period of eight weeks. In terms of class dynamics, it subtly blurred the distinction between teacher
and learner. This was to become a positive feature of the course and to lead to ‘a model in which the
old teacher/student relationship [was] replaced by learning together’ (Papert, 1999).
8. Teacher-as-learner
The teacher-as-learner role became more pronounced when the class returned from their individual
wandering. As indicated earlier, the teacher’s digital literacy was quite basic, but, aware of the
students’ superior prowess in this domain, she encouraged them to work in whatever medium they felt
they could best articulate their stories. She asked only that those who chose the digital option would
provide her with a ‘dummy’s guide’ to their selected medium to help her create an assessment rubric.
Half the class – four students – were in this category. The learning curve for the teacher was steep
and exhilarating, as her mind and senses were drawn into stories told in the form of movies, a Glog,
and a Prezi. In different ways, these digital formats vividly captured and imparted to teacher and
students alike not just the factual, but the emotional reality of the social fabric and the education
systems of the countries visited, illustrating that ‘technology is a language in which powerful ideas can
be expressed’ (Papert, 1999) and multiple perspectives dramatically conveyed. Explaining his choice
of medium for sharing his story of the ‘level of chaos for students and educators’ in a particularly
troubled part of the world, one student wrote: No amount of data on politics, economics, or
governmental oversight can ever adequately paint a picture of turmoil in areas currently afflicted by
poverty, injustice, and violence, [so] I assembled a Glog which can be found below. The videos and
songs paint a picture that, at least to my eyes, [is truer] of the daily world experienced by those under
immense political and social pressures. From the teacher’s point of view, the discovery of the Glog
with its apparently chaotic collage of videos was enlightening and invaluable. It seemed a near-perfect
match of content and expression in that it captured visually the chaos which the student wanted to
convey and which was authenticated by the scenes in each video and by the ‘insider view’ of the
people caught up in the chaos. The fact that the videos could be viewed in any order without
diminishing the impact also made the Glog a most appropriate form of expression for learning by
wandering.
445

Marie Martin and Michaela Noakes
The student who used the Prezi explained her choice during a synchronous Wimba class by saying
that, having bought into the freedom and the curvilinear approach of learning by wandering, she
wanted some way of expressing herself in a non-linear, more impactful way than by the linear format
of text or PowerPoint. A classmate with whom she discussed this introduced her - as part of our
shared learning ethos - to the Prezi which is curvilinear in design, allowing the user to zoom as
appropriate either on the big central picture in view on the screen or on the smaller pictures or texts
surrounding it. This layout proved in effect to be a metaphor for her theme of the ‘mosaic’ of the
country she was describing. The ‘dummy’s guide’ provided by one of the ‘movie makers’ gives an idea
of the complex and time-consuming process required by her choice of medium: I used Google docs to
create slides of relevant … information that I researched and read about both in books and online. I
took screenshots as jpgs and inserted them into iMovie. Also looked for images that would enrich the
movie. Then I worked on editing, cutting, cropping and animating everything. The last step was to
research free Creative Commons podcast music and create background sound while editing for
volume and ducking in and out. The same student remarked ruefully that it would have been so much
simpler to write a paper, but she felt the movie was a more powerful form of expression. It should be
stressed at this point that those students who did choose to write a paper also exploited their medium
and enriched the learning of peers and of the teacher – albeit more within the latter’s comfort zone!
Peer response to the digital stories was uniformly enthusiastic. The following is a typical example: I
am so honoured to have such a talented cohort! You guys have given me so many great ideas for
future projects. What a great way to teach kids about other cultures - instead of PowerPoints or
lecture! Wow! These and other similar responses to the experience of sharing and learning with and
from one another might perhaps give some indication of the strong element of what Guth and Helm
(2010: 16) call the Web 2.0 philosophy, a ‘relationship revolution’ driven by ‘ideals such as sharing,
openness, collective intelligence, flexibility and collaboration’.
9. Teacher 2.0
Dooley (in Guth and Helm, 2010: 277-303) speaks of ‘Teacher 2.0’ whose teaching approach is
learner-centred, not technology-centred, who focuses on being able to use available technology as a
means of collaboration and development of shared knowledge and of equipping students with the
skills needed for professional life in today’s globalised world. She also depicts some of the traits of
Teacher 2.0. Among these are a willingness to experiment and take risks, to integrate technologies
that are more familiar to the learners than to the teacher, and effect a ‘symbiosis of truly collaborative
learning ... by actively involving teachers in the learning process and students in the teaching
process’. Teacher 2.0 also uses technology to provide opportunities for students to take ownership of
the learning process. Above all, Teacher 2.0 makes the required shift from the transmission education
paradigm to the ‘mutual sharing of knowledge-building between teacher-student, student-student and
student-teacher’. Dooley asks bluntly: ‘Does Teacher 2.0 exist?’ The experiences of the lead author
as teacher in the eLearning course which is the subject of this paper would suggest that it is only if
this type of teacher does exist that eLearning can begin to reach its transformative potential.
10. Teacher as sage at the side
In moving away from transmission education in eLearning, the teacher has also to move away from
the traditional role of being sole repository of learning and to seek a role that best empowers the elearner.
This is frequently seen as moving from teacher to facilitator, from sage on the stage to guide
at the side. The role adopted by the teacher in the case of the eLearning course described in these
pages was rather that of sage at the side, setting the students free to learn by wandering and take
ownership of their learning, scaffolding them when necessary, but always endeavouring to ‘bring
wisdom, perspective and [experience] to the learning’ (Papert and Caperton, 1999). In this way, it
becomes possible to move toward the goal that Hargreaves (2003) suggests is the ultimate
achievement of the teacher - not to deliver learning, but to develop learners. This is exactly how the
authors of this paper understand transformative learning – developing learners who love learning and
for whom it becomes a lifelong commitment.
11. Students as critical friends
Students also responded as critical friends to the teacher’s request for suggestions for improving the
course. In general, these took two forms. One, surprisingly, was a suggestion to do more learning by
wandering along the ‘road less travelled by’– from Robert Frost’s poem which we had also made our
own. One student felt that all would have benefitted from having ‘more room to wander’ as this might
446

Marie Martin and Michaela Noakes
have led to some interesting tangential discussion. The second suggestion, unsurprisingly, was to
introduce other technologies, including Skype or videoconferencing. Ironically, the teacher of this
course is a champion of the use of videoconferencing in education, has used this medium in previous
eLearning courses, and had hoped to try Skype group conferencing with this cohort, but was
frustrated by time and other logistics. She occasionally used the video feature in Wimba, but it
functioned only on a one-to-one basis and tended to slow down the pace of the class. This will be
more thoroughly explored for the next iteration of the course. The ideal would be, as the same student
suggested, to seek videoconferencing opportunities with people from the target countries. Another
very useful suggestion that will hopefully be acted upon was to ‘encourage more PBL (project-based
learning) during the Wimba sessions, such as break-out activities, online whiteboard activities, etc’.
As the course continues to evolve, it will obviously be important to incorporate the use of desktop
sharing and to place more emphasis on integrating Web 2.0 technologies into the Blackboard
management learning system. Ideas suggested by the co-author include screen capture video
software for digital storytelling, desktop application sharing tools to enable future students to begin to
build a timeless and virtual ‘Wandering Library’ to share their experiences as a teaching tool for those
that follow them, and possibly Twitter to support community building among students and, if
appropriate, to create unique user names to allow them reach out to the web in search of ‘virtual tour
guides’ for them as they wander.
12. Conclusion
How did this course, delivered within a traditional LMS, endeavour to embody the Web 2.0 ethos? It
sought to do so, firstly, by highlighting the importance of the teacher’s attitude and the impact of this
on students (Katsifli, 2010). In particular, emphasis was placed on the course teacher’s high
expectations of quality eLearning and awareness of importance of the human dimension. Secondly,
the focus was on the need for appropriate pedagogy, leading to a paradigm shift from transmission
teaching to student-centred and student-led learning. Additionally, the open mind aspect of the
underpinning concept of learning by wandering was stressed in order to facilitate the necessary
change of mindset. Finally, Blackboard and Wimba were used as shared learning spaces for
collective intelligence, and creating space for students to use Web 2.0 tools for creative expression
and sharing of learning.
Did this process lead to a transformative eLearning experience for the students? Written feedback
from them would suggest that something of this nature was experienced by at least a few. In an email
to the teacher one student wrote: I thank you for being the kind of teacher that allows me to grow [not
just] as a student but as an educator! As part of a Blackboard discussion about ‘teaching to the test’
and the consequent lack of ownership of the learning process, a student’s post read: I think that is
why [the] idea of wandering ... and student generated learning is so much more meaningful. And from
the student who chose to present her final assignment as a “Jog”, a tool that enables multiple
websites to be presented as a package: For the final reflective project I would like to put together a
brief synopsis of learning from this class: A Jog on the Web (wandering). I would like to use this
technology because it will help me in the future [to] refer back to some of the learning and contain the
pertinent links to information we learned about. In the introduction to the ‘Jog’, she states: Like
Ulysses, we as educators must not remain stationary, but wander and learn from the world around us.
She also includes (with attribution) in her ‘Jog’ the wandering tales of some of her peers and she
posted a note to them on the discussion board: Thanks to the class for sharing their learning this
semester! ... It has been a blast! I am still adding to the learning from this course but here is the start
of my Jog. Students also welcomed being given the ‘leeway to learn [and to share their learning] in
the manner that [suits them]’ (personal email from a student). Another student wrote to express
appreciation of the teacher’s ‘willingness to push the envelope with Web 2.0 technologies’. Above all,
students valued the teacher as sage at the side. One student expressed this in her final email: I
appreciate all of the wisdom and insight you were able to offer as you guided us through our journey.
… It truly was a spectacular learning experience!
The literature reflects the tension between two opposing views of the LMS. The first is the widely held
view that, even with the addition of Web 2.0 tools, an LMS remains essentially an electronic
replication of existing practice. The more recent view stresses the transformative potential of LMS
mediated learning when it is informed by the Web 2.0 ethos. The eLearning course described in this
paper supports the latter view. While very much a work-in-progress, it still serves to illustrate that the
quality of eLearning is determined not by simply adding Web 2.0 tools to the traditional environment,
but rather by embodying the Web 2.0 mindset within that environment. In this context, the case study
447

Marie Martin and Michaela Noakes
is offered tentatively as a contribution to the development of a framework for transformative
eLearning.
References
Beetham, H. and Sharpe, R. (2007) Rethinking pedagogy for a digital age: designing and delivering eLearning,
London and New York: Routledge.
Croft, N., Allison, D. and Duff, M. (2010) Overcoming isolation in distance learning: Building a learning community
through time and space, [Online], Available: http://ctiweb.cf.ac.uk/jebe/pdf/NicholasCroft5(1).pdf
Friedman, T. L. (2007) The world is flat, New York: Picador/Farrar, Straus and Giroux.
Gunawardena, C., Lowe, C. and Anderson, T. (1997) ‘Analysis of a global online debate and the development of
an interaction analysis model for examining social construction of knowledge in computer conferencing’,
Journal of Educational Computing Research, vol. 17, no. 4, pp. 395 – 429.
Guth, S.and Helm, F. (eds.) (2010) Telecollaboration 2.0: Languages, literacies and intercultural learning in the
21 st century, Bern: Peter Lang.
Hargreaves, A. (2003) Teaching in the knowledge society: Education in the age of insecurity, New York:
Teachers College Press.
Katsifli, D. (2010). The impact of Blackboard software on education globally over the past 10 years. [Online],
Available:
http://www.lms.unimelb.edu.au/elo/resources/The_impact_of_Blackboard_software_on_education_globally_
(20100204W).pdf
Kuriloff, P. (2005) ‘ Breaking the barriers of time and space: More effective teaching using e-pedagogy’, Innovate
[Electronic], vol.2, no. 1, Available: http://www.innovateonline.info/
Kyong-Jee, K. and Bonk, C. J. ( 2006) [Online]. ‘The future of online teaching and learning in higher education:
The survey says ...’ Education Quarterly [Electronic], vol. 29, no. 4, Available:
http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/TheFutureofOnline
TeachingandLe/157426
Lee, J. W. and McLouglin, C. (eds.) (2011) Web 2.0-based eLearning: applying social informatics for tertiary
teaching, Hershey, New York: Information Science Reference.
Martin, M. (2010) Learning by wandering: An ancient Irish perspective for a digital world, Bern: Peter Lang.
Papert, S. and Caperton, G. (1999) Vision for education: The Caperton-Papert platform [Online], Available:
http://www.papert.org/articles/Vision_for_education.html
Papert, S. (1999) Technology in schools: To support the system or render it obsolete, Milken Family Foundation,
[Online], Available: http://www.mff.org/edtech/article.taf?_function=detail&Content_uid1.
Quinn Patton, M. (2002.) Qualitative research and evaluation methods, 3 rd edition, Thousand Oaks, London, New
Delhi: Sage Publications.
Siemens, G. ( 2011) Re: Massive open online courses, blog ( http://www.elearnspace.org/blog/), 4 Aug.
Stein, J. (2008) Re: Massive open online courses, blog ( http://jaredstein.org/2008/02/29/lmss-ples-walledgardens-and-yearnings-for-debate/),
29 Feb.
Thornburg, D. D. (2004) ‘Campfires in cyberspace: Primordial metaphors for learning in the 21 st century’,
International Journal of Instructional Technology and Distance Learning, vol. 1, no. 10, October, [Electronic],
Available: http://www.itdl.org/Journal/Oct_04/invited01.htm.
448

Online Student Engagement: Unfulfilled Promises or
Promises Unfulfilled?
Linda Martin, Gary Spolander, Imran Ali and Beulah Maas
Coventry University, UK
hsx185@coventry.ac.uk
aa2912@coventry.ac.uk
aa2446@coventry.ac.uk
aa4895@coventry.ac.uk
Abstract: Engagement is often seen as a reliable proxy for learning (Coates, 2005) but also in general abilities
and critical thinking (Gellin, 2003), student satisfaction (Kuh et al, 2007), cognitive development (Pascarella &
Terenzini, 2005) and improved grades (Tross et al, 2000). Whilst Coventry University have made significant
efforts to support early engagement and promote good technology support, experience indicates that some
students of on line programmes continue to experience "engagement challenges". These include choosing to
study in isolation; students maximizing offered flexibility of eLearning to the detriment of synchronous
collaboration, accessing learning materials only shortly before the submission of assessed work is due and as a
result failing to move from surface level to in-depth approaches of cognitive processing (Henri, 1992). As a result
what was intended as a dynamic, interactive medium for shared learning can become a pragmatic approach to
individualistic achievement. In response to these concerns on a distance based online Health and Social Care
degree, a project was launched to explore the impact of collaborative peer assessment and whether this would
result in greater motivation to develop and share approaches to learning. Would developing this increased
"critical space" (Jankowska & Atlay, 2008) result in an improved level of academic work being presented for
assessment?
Keywords: on line; peer assessment; collaborative; student engagement; shared learning
1. Introduction
ELearning presents a dynamic approach to study which has opened a multitude of flexible
opportunities for students not previously experienced. Innovative, creative resources can contribute to
the development of the knowledge base within the student group through more responsive
participative methods, especially when compared to the large and small group discussions so
frequently used in traditional 'classroom' teaching in higher education. ELearning presents a panacea
to the challenges of flexible learning, and increased opportunity for student engagement experienced
through offering both synchronous and asynchronous opportunities to communicate verbally, visually
and in writing within learning communities. The flexibility of eLearning enables lecturers to respond to
the individual learning needs of students.
2. Developing communities of learning
At Coventry University, the Foundation, BA and Master’s degrees in Social and Health Care
Management were launched three years ago as wholly online programmes to offer increased
flexibility and innovative course delivery. In this time, significant differences and concerns emerged in
the responses from students on these three programmes. Notably the Foundation degree students
demonstrated lower levels of engagement and academic learning. Further analysis by the course
team suggested that this was influenced by a complex array of factors linked to developments in the
social care and health sector, HEI sector responses to government initiatives and challenges and the
individual characteristics of students. In response the course team developed a pilot peer assessment
process as a means of promoting engagement and with a view to developing a better understanding
of academic expectations for the Foundation degree students. This paper outlines this process, by
exploring the rationale, approach taken and the initial results of the pilot. This is a work in progress
and further development is ongoing.
There have been considerable changes in recent years in the social care sector, with the increased
use of independent sector organizations to deliver care services. Although the need for leadership
and management training is undisputed in the sector, it pays low wages (Low Pay Commission,
2011), staffing levels are often stretched (Mansell, 2011) and there is little incentive for senior
managers to support staff in acquiring professional training. In the health sector, the picture is slightly
different with many staff in the position of needing to achieve degree status in order to be eligible for
promotion in the National Health Service (NHS). The level of self motivation is therefore variable
449

Linda Martin et al.
across the sector. Higher education has responded to this situation and government incentives by
promoting Foundation degrees which enable workers with few prior qualifications to access a degree
programme and complete it within two years of full time study. Sector requirements called for flexible
education that took into consideration location, speed of study and mode of delivery (Skills for Care,
2008).
eLearning was seen as providing a range of benefits to learners including; contributing to self
regulation (Salovaara cited in Kelly et al 2010), a key management skill and 'collaborative interaction
between learners' (Jonassen et al cited in Kelly et al, 2010), while Laird and Kuh (2005) found a
positive link between online provision, active and collaborative learning and deep learning. Indeed,
Harper and Quaye (2009) highlight that educational engagement requires students to both participate
and be involved which includes being active, feeling and making sense of materials. This behavioral,
emotional and cognitive engagement (Fredericks et al, 2004) also requires supportive learning
communities to develop active and collaborative learning, academic activity collaboration, formative
learning and a feeling of being supported by those learning communities (Coates, 2007). The
opportunities afforded by the Foundation degree appeared to offer students a range of benefits as
highlighted by sector workforce plans.
3. The learning challenge
The three years of online course delivery and evaluation support the benefits of eLearning, but a
chasm developed between the approach to learning of the undergraduate and post graduate
students. The teams experience was that once post graduate students had overcome initial anxieties
regarding accessing the virtual learning environment, their confidence increased quickly to enable
them to develop communication and support mechanisms within their learning group, but this was not
the case at undergraduate level. . Rather students reported and appeared to be working individually,
at very different paces, engaging only with the lecturer in spite of encouragement to participate in
learning community discussion forums and exercises in which individual responses were shared with
other group members. This appeared to result from differences within the two student groups with the
undergraduates being less experienced managers, having fewer previous educational achievements
and receiving less support from their employers to study.
This pattern of study was not envisaged by the course team during development and whilst it may
have been accepted as an inevitability of the students' work commitments, had they been achieving
educational success in their studies, the academic attainment was a concern, which unveiled the
possibility of students compounding previous negative learning experiences through non
achievement. The course had been designed to offer academic support through the provision of a
study skills module, digital provision of all essential reading materials and individual tutorial support.
Despite this a significant numbers of assignments were written anecdotally, showed little evidence of
academic reading and failed to comply with academic requirements. Students were working to
deadlines, accessing course materials only shortly before the submission of assignments was due, so
failing to move from surface to deep learning (Henri,1992) and conforming to Salmon's (2000) 5 stage
model of learning whereby student failure to access the technology results in failure to learn.
4. Encouraging engagement through peer assessment
The importance of engagement in student learning has gained momentum as universities have found
themselves in an increasingly marketised and competitive environment. The increased use of quality
assurance by universities and the emphasis on demonstrating added value have been instrumental in
supporting efforts to encourage student engagement and academic success. The literature frequently
highlights that engagement is considered a proxy for learning (Coates, 2005) but also in general
abilities and critical thinking (Gellin, 2003), student satisfaction (Kuh et al, 2007), cognitive
development (Pascarella and Terenzini, 2005) and improved grades (Tross et al, 2000, Kuh &
Vesper, 1997). This highlighted the importance of needing to address the lack of student engagement
and poor academic achievement consistently.
Whilst the Foundation degree could theoretically be completed in two years full-time, all students are
part-time due to the requirement that they are also in appropriate employment. The course supports
the development of knowledge essential to professional practice and its application within the
workplace and as a result a key underpinning principle is the commitment to learning collaboration,
the sharing of learning from diverse work settings and reflection on practice. Both synchronous and
asynchronous learning objects were developed to provide opportunities for the group to develop
450

Linda Martin et al.
“critical space” (Jankowska and Atlay, 2008), to work collaboratively with less tutor input and
ultimately to develop independent learning skills, which would enable reflection, critical thinking, and
experimentation as students embraced deep learning (Gellin, 2003; Henri, 1992, Shulman, 2002). All
of this can improve grades (Tross et al, 2000), but only if students engage (Fredericks et al, 2004).
Numerous models of engagement were reviewed including the assessment to online contributions,
but peer assessment was identified as offering potential for students to engage more critically and in
more detail with the process of learning. Defined by Lindblom-Ylanne (2006:52), as learners
assessing 'the achievement, learning outcomes or performance of their fellow students' it offered a
range of potential benefits for the learner. Participation encourages the development of skills in giving
and receiving feedback which are an essential aspect of professional responsibility, judgment and
autonomy (ibid). Additionally the process of formative feedback supports disciplinary understanding,
encourages increased ownership of academic work, encourages understanding of learning outcomes
and facilitates student autonomy (Sadler, 2010). Brew (1999) outlines that these opportunities
therefore encourage deep learning. Falchikov (2005) (cited in Nicol, 2010: 509) argues that peer
feedback was often considered more helpful than that of a tutor alone, whilst others (Race, 2001,
Coleman, 2006) suggest that peer assessment enhances assessment outcomes. In particular, the
team felt it would help the students engage more actively with the process of writing so that students
are required to engage and share their understanding of concepts, scholarly identity and the ways
that they express their ideas (Bass, 2010, Meyer et al, 2009). Peer assessment, it was felt, offered
fresh perspectives to learning communities through sharing how others approach the same topic,
promoting reflexivity by broadening and deepening students’ understanding of the subject and
academic rigour (Yang, 2010) as well as engaging learners in an activity which resembles
professional practice (van der Pol, 2008). From their research, Ballantyne, Hughes and Mylones (in
Vickerman, 2009) report that students found the experience of assessing other students' work
motivating as well as aiding knowledge development of subject content and increasing their
understanding of the assessment process.
Peer assessment is not without critics, Par and Joordens (2008: 528) raise the issue of the quality of
peer reviewers comments and the related justification for 'peer-derived' grades while Brown, Rust and
Gibbs (1994) and Hartley (1998, in Vickerman, 2009) highlight that judging boundaries can be
challenging for staff, let alone learners who are not usually focused on assessing each others' work.
Further Par and Joordens (2008) argue that the choice of assessment tool is important, a point
supported by Fowler and Mayer (in Kelly et al, 2010), who recognise that while a range of
pedagogical approaches can be adopted to support on line learning, appropriate selection of 'internet
tools' based on contextual factors is important. Factors such as the anticipated learning outcomes and
the role of the teacher and learner, i.e. locus of control, all contribute to student learning, performance
and satisfaction. There is considerable concern about the use of peer assessment in summative
assessment; however, research has demonstrated its reliability and validity (DeNisi & Stevens, 1981;
Reilly & Chao, 1982; Bamberger et al, 2005) although this debate continues (Cho and Schunn, 2003).
The decision to undertake summative peer assessment was based on concerns that students may
not engage with formative peer feedback and the importance of the process encouraging improved
course engagement. Two assessments were developed for each module, the first one being peer
assessed and representing 40% of the module mark and the second one being assessed by the
module tutor, representing 60% of the module mark. Students needed to provide feedback on the
assessment and discuss this within a small group before allocating a mark. The mark was then
moderated by the module leader. Due to small cohort numbers, it is too early to tell whether the
change has impacted on student achievement. The tool selected for the assessment process is
PeerMark, a facility of Turnitin, a plagiarism detection programme through which students regularly
submit their work. Familiarity was the key determining factor. To date the process has been used in
three modules involving students at different stages of their degree.
5. Student perceptions
Qualitative data was collected by questionnaire from the students who have been involved in the pilot.
Six out of a possible thirteen students responded to the opportunity to give feedback. They were
asked to comment on their memories of peer assessment, the challenges, the benefits and any
contribution the exercise has made to their learning. It is interesting that although a range of positive
comments were made about the project; the responses to the first question, about their memory of it
were negative. The overriding feelings that have stayed with them are unease, discomfort and lack of
451

Linda Martin et al.
confidence. Although these feelings became more positive as the pilot progressed it is their anxieties
which they recall immediately. Key issues identified by the students that responded can be
summarized as follows:
The challenges for them indicate that the pilot has encouraged them to reflect more on what
contributes to a good piece of work. It was noted that it was easier to be critical of others than it
was to identify the good points which they had to consciously think about.
A significant point was also made regarding their own experience at the time of their involvement
in the pilot. For some this was the first piece of work they had written. They had no indication of
whether their own work was of a satisfactory standard or whether they had understood the
question. This added to the feelings of anxiety.
The benefits noted related to increased engagement. One student commented that prior to the
pilot comments put on the discussion forums had received no response from others, so it
provided an opportunity to communicate. The pilot also provided the opportunity for some group
learning as the final mark had to be agreed by several students working together.
Additionally, the exercise contributed to the students' understanding of the difference between a
good and bad essay by being able to read the work of others. However, they did not feel that the
feedback received from other students about their own work helped their learning as the
comments tended to be short.
For some, it has helped them to be more critical of their own work and has increased their
confidence. However, part of this increase is due to feeling that their own work was better than
the assignments they marked.
6. Staff perceptions
The three staff involved in the pilot were asked to comment on the challenges, the benefits and the
lessons learned. The feedback can be summarized as follows:
For staff implementing peer assessment there have been a series of process issues which have
required an immediate response before being able to reflect on whether the exercise was
achieving its aim. The technology was challenging. Although all the team were familiar with
Turnitin and can use this with ease, the Peer Mark option proved to be much more challenging. In
addition to the added time spent there were ongoing concerns about whether it would work as
intended, providing a smooth efficient submission system which protected the identity of students.
The student response during the pilot was disappointing. Some students had to be constantly
reminded and guided through the process whilst others, who were waiting for them to complete
their marking, had to be appeased. One refused to take part and another went on holiday without
completing the process. This led to an unequal distribution of work and adjustments were made to
the marking criteria on the next occasion to reflect this and encourage participation.
Attempts were made to involve students at a very early stage of the process, determining the
marking criteria, but only one student contributed to this while the others waited to be told what to
do.
Although the process issues created some negative responses, the results of the exercise were more
encouraging. The marks suggested by students were usually closely related to those the lecturer
would have given, although there was greater coherence in relation to the final agreed group mark
rather than the individually suggested student marks. The agreed process was that the lecturer would
only change marks given by the students where there was significant concern about the mark given,
but this was not defined with exactitude. For level 1 student’s precise marks are of less significance
as they do not contribute to the degree classification but for levels 2 and 3 the significance increases
and students question their given mark more and expect the mark to be clearly justified.
7. Discussion
The pilot is currently being reviewed to determine how it will proceed into its second year but a
number of issues have been raised by these preliminary results. The initial purpose of the pilot was to
increase student engagement and as a result increase academic achievement. To date the
experience has called to question whether engagement and achievement are linked, whether there
are other more effective ways of improving student achievement, and whether engagement is a
shared utopia between students and lecturers or only a vision of lecturers.
452

Linda Martin et al.
While it is commonly argued that negative experiences have a greater impact than positive ones
(Normann,2000) it is important to address the overarching anxiety, unease, discomfort and lack of
confidence remembered by the students. These feelings threaten to outweigh and override the more
positive experiences stated by students as the pilot progressed. New challenges by their nature
threaten existing comfort zones but confidence in one's ability to meet the challenge is vital to
success. Failure will only compound one's lack of confidence and feelings of inadequacy. A pilot of
this nature has the propensity to discourage students from further participation on the programme and
a as result limit the potential to deepen their learning. For inexperienced students and for those
already failing, further consideration needs to be given as to whether the benefits promoted from
previous research of increased self confidence and motivation (Race, 1998) and enhanced
assessment outcomes (Race 2001, Coleman 2006) are likely to be realised through this process.
Critically, there are or can be significant differences between contrasting types of course presentation
i.e. on line, distance learning or on campus programmes. This programme seeks to market itself on
the strength of opportunities for interaction and shared learning that derive from online course
delivery. Students initially value this but a common reason given by students who withdraw from the
programme is isolation. As a result interaction and engagement may be processes they expect other
students to provide, but of which they are passive recipients. Peer assessment is underpinned by the
principle that you have to give on order to receive. However, this may be a principle that today's
students are struggling to give priority to. Krause (2005) found that part-time students, who are also in
paid employment, were less engaged than full-time students with more than half of part-time students
stating that their paid work interfered with their studies. Rather than seeing a need to negotiate with
their employers or make adjustments to their commitments, they expected study to fit around their
lives. McInnes (2003) suggests that engagement can no longer be taken for granted as a shared goal,
it needs to be negotiated with students and Yorke (2006) highlights the need to differentiate between
performance and learning goals. For some students this has been a confidence-boosting exercise,
but this seems to have resulted from a sense of their own work being of a better quality than that
which they have marked, rather than from learning from each others' work and feeling able to write a
better assignment as a result. It appears that whilst tutors continue to be dedicated to students'
learning, students may be more focused on their individual achievement. While research suggests a
close link between engagement and achievement, this is based on an understanding of achievement
resulting from a process of learning rather than completion of a task. Without a commitment to
learning, students are unlikely to move from surface to deep learning, focusing narrowly on successful
task completion, and may therefore only gain more surface level understanding of the subject. They
may be extremely committed to complete each task but, reflecting the world in which they work, are
outcome rather than process orientated and fail to see the benefit of tasks which do not appear to
have a direct correlation with the achievement of their immediate goal. Peer assessment is an attempt
to re-engage them with the process of learning, but there may be more fundamental issues which
need to be addressed with students before they value the potential benefits rather than being taskfocused
alone. Zepke and Leach suggest that higher education needs to adapt to the changing
expectations of students by negotiating "a strategic approach to learning in which students choose
which approach to adopt in any given situation" (2010: 173). The implication of this is that while peer
assessment may be a tool that can be used to exercise engagement, it may only be effective when
students have first embarked upon a commitment to learning and see peer assessment as a positive
part of that process. The issue in question then is not their understanding of learning requirements but
their motivation to learn. So the challenge for tutors is how to incentivise approaches to learning. This
needs to be addressed at point of entry onto a programme, prior to students embarking upon
assessed work, for without this commitment students are unlikely to reflect on the peer assessment
process as a learning experience and will judge it only as a task. The dilemma is whether
inexperienced students who are operating in a very different environment in the workplace can
continuously shift between two such contrasting sets of expectations.
For part-time students in employment it can no longer be assumed that they are engaged with a
concept of learning in its own right. It may have become inextricably linked with achievement and
career progression to the extent that they measure their learning through their employment status. It
is with this in mind that the course team will progress to meet the challenge of extricating the two with
a view to introducing students to the joys of learning and the fulfillment that is achieved through the
acquisition of new knowledge and the development of wisdom.
453

8. Conclusion
Linda Martin et al.
The pilot project aimed to developing greater engagement and as a result enhance student academic
achievement. Early findings suggest that where students have engaged with the process they have
enjoyed and benefitted from the interaction. There are also indications that their confidence has
increased and they have appreciated being able to see the work of other students as a means of
locating the quality of their own work.
The project will continue into a second phase but will only be used with level 1 students in order to
address concerns that unreliability of marking may impact on a student’s overall degree classification.
It will also only be used in two of the four possible modules at level 1 so the students can focus on the
quality of their assessments rather than it becoming an administrative burden. Other peer assessment
tools will be considered for use and formative self assessment will be introduced at level two to
observe whether students have developed skills which are beneficial to their own learning and
development.
The pilot has provided a valuable insight into how students approach learning and assessment which
has wider implications for programme development and as the pilot continues, further consideration
will be given to the relationship between work and study which underpins Foundation Degrees.
References
Bamberger, P.A., Erev, I., Kimmel, M. & Oref-Chen, T. (2005) Peer assessment, individual performance and
contribution to group processes: the impact of rater anonymity, Group and Organization Management, 30,
(4): 344-377
Bass, R. (2010) The rise of social learning as a bridge to expertise presented at The Pedagogical Research in
Higher Education Conference, Liverpool Hope University, 25 th October.
Brew, A. (1999) Towards autonomous assessment: using self-assessment and peer assessment, in S. Brown
and A. Glasner (eds) Assessment Matters in Higher Education: choosing and using diverse approaches,
Buckingham: Society for Research into Higher Education & Open University Press.
Cho, K. & C. D. Schunn (2003) Validity and Reliability of Peer Assessments with a Missing Data Estimation
Technique, Proceedings of ED-Media, Honolulu, Hawaii, USA.
Coates, H (2005) The value of student engagement for higher education quality assurance, Quality in Higher
Education, 11 (1): 25-36.
Coates, H. (2007) A model of online and general campus based student engagement, Assessment and
Evaluation in Higher Education, 32 (2): 121-141.
Coleman, I (2006) A Study using Formative Assessment Feedback in Physiology and Pharmacology to
Encourage Engagement (see Case Studies section of www.open.ac.uk/fast)
DeNisi, A.S. & Stevens, G. (1991) Profiles of performance, performance evaluation and personnel decisions,
Academy of Management Review, 24: 592-602.
Fredericks, J. A. Blumenfeld, P. C. and Paris, A. H. (2004) School engagement: potential of the concept, state of
the evidence, Review of Educational Research, 74 (1): 59-109.
Gellin, A. (2003) The effect of undergraduate student involvement on critical thinking: A Meta- Analysis of the
Literature, 1991 – 2000, Journal of College Student Development, 44 (6): 746-762.
Harper, S. R. & Quaye, S. J. (2009) Student Engagement in Higher Education, London, Routledge.
Henri, F. (1992) Computer conferencing and content analysis, in A. R. Kaye, Collaborative Learning Through
Computer Conferencing, Berlin, Springer-Verlag.
Jankowska, M. & Atlay, M. (2008) Use of creative space in enhancing students’ engagement, Innovations in
Education and Teaching International, 45 (3): 271-279.
Kelly, D., Baxter,J. & Anderson, A. (2010), Engaging first-year students through online collaborative
assessments, Journal of Computer Assisted Learning, 26: 535–548.
Krause, K-L. (2005) Engaged, inert or otherwise occupied? Deconstructing the 21 st century undergraduate
student, Keynote paper presented at the Sharing Scholarship in Learning and Teaching: Engaging Students
Symposium, James Cook University, Townsville, 21-22 September.
Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K. and Hayek, J. C. (2007) Piecing Together the Student
Success Puzzle: Research, Propositions and Recommendations, ASHE Higher Education Report, 32, 5,
San Francisco, Jossey- Bass.
Laird, T. & Kuh, G. (2005) Student experiences with information technology and their relationship to other
aspects of student engagement, Research in Higher Education, 46, (2): 211-33.
Lindblom-ylänne, S., Pihlajamäki, H. & Kotkas, T. (2006) Self-, peer- and teacher-assessment of student essays,
Active Learning in Higher Education, 7, 1: 51-62.
Low Pay Commission (2011) National Minimum Wage Low Pay Commission Report 2011, accessed at:
http://www.lowpay.gov.uk/lowpay/report/pdf/Revised_Report_PDF_with_April_date.PDF, [last accessed 20
June 2011]
454

Linda Martin et al.
Mansell, J, (2011) Bristol care home: a failure on every level, The Guardian, accessed at:
http://www.guardian.co.uk/commentisfree/2011/jun/01/bristol-care-home-failure-every-level, [last accessed
20 June 2011]
Meyer, J. H. F. and Ward, S. C. and Latreille, P. (2009) Threshold concepts and metalearning capacity,
International Review of Economics Education., 8 (1), 132-154
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education,
Assessment and Evaluation in Higher Education, 35 (5): 501-517.
Normann, R. (2000) Service Management: strategy and leadership in service business, 3 rd edition, Chichester:
Wiley.
McInnes, C. (2003) New realities of the student experience: How should universities respond? Paper presented
at the European Association for Institutional Research 25 th Annual Conference, Limerick.
Par, D. & Joordens, S. (2008) Peering into large lectures: examining peer and expert mark agreement using
peerScholar, an online peer assessment tool, Journal of Computer Assisted Learning, 24, 6, 526-540.
Pascarda, E. T. & Terenzini, P. T. (2005) How College Affects Students: a third decade of research, vol 2, San
Francisco: Jossey-Bass.
Race, P. (1998) Practical Pointers in Peer Assessment, in S. Brown (ed) Peer Assessment in Practice, (SEDA
paper 102) Birmingham: SEDA.
Race, P (2001) A Briefing Document on Self, Peer and group Assessment, LTSN Generic Centre.
Reilly, R.R. & Chao, G. (1982) Validity and fairness of some alternative employee selection procedures,
Personnel Psychology, 78: 113-123
Sadler, D. R. (2010) Beyond feedback: developing student capability in complex appraisal, Assessment and
Evaluation in Higher Education, 35 (5): 535-550.
Salmon, G. (2000) E-moderating, London: Kogan Page.
Skills for Care (2008) The Sector Qualification Strategy - Implications for employers, accessed at
www.skillsforcare.org.uk%2Fnmsruntime%2Fsaveasdialog.aspx%3FlID%3D208%26sID%3D145&rct=j&q=
The%20Sector%20Qualification%20Strategy%20-
%20Implications%20for%20employers&ei=_mz_TZqaFo2yhAfMltSjCw&usg=AFQjCNFio4Er59rDUbiJ75uN
1TudQVoz8g&sig2=Ql1r5FMnGqIZjgmAfov1LQ&cad=rja, [last accessed 20 June 2011]
Tross, S. A., Harper, J. P., Osherr, L. W. and Kneidinger, L. M. (2000) Not just the usual cast of characteristics:
using personality to predict college performance and retention, Journal of College Student Development, 41
(3): 325-336.
van der Pol, J, van den Berg, B, Admiraal, W, & Simons, P (2008) The nature, reception, and use of online peer
feedback in higher education, Computers & Education, 51, 4: 1804-1817.
Vickerman, P (2009) Student perspectives on formative peer assessment: an attempt to deepen learning?,
Assessment & Evaluation in Higher Education, 34 (5): 221- 230.
Yang, Y. (2010) Students’ reflection on online self-correction and peer review to improve writing', Computers &
Education, 55, (3): 1202-1210.
Yorke, M. (2006) Student engagement: deep, surface or strategic? Keynote address delivered at the 9 th Pacific
Rim First Year in Higher Education Conference: Engaging Students. Griffith University, Gold Coast,
Australia 12-14 July
Zepke. N. & Leach, L. (2010) Improving student engagement: ten proposals for action, Active Learning in Higher
Education, 11, (3): 167-177.
455

Personalized e-Feedback and ICT
Maria-Jesus Martinez-Argüelles 1 , Josep-Maria Batalla-Busquets 1 , Patricia
Noguera-Guerra 1 and Ernest Pons-Fanals 2
1 Economia i Empresa, Universitat Oberta de Catalunya, Barcelona, Spain
2 Departament Econometria, Estadística i Economia Espanyola, School of
Economics, Universitat de Barcelona, Barcelona, Spain
mmartinezarg@uoc.edu
jbatalla@uoc.edu
pnoguera@uoc.edu
epons@ub.edu
Abstract: In the context of the adaptation of the new undergraduate, graduate and postgraduate degrees to the
European Higher Education Area (EHEA), the student centered learning (SCL) is an approach which aims to
overcome some of the problems inherent to more traditional forms of education by focusing on the learner and
their needs, rather than being centred around the teacher’s input. Thus the student becomes the protagonist of
his learning process by means of doing learning exercises or activities. So, these learning exercises or activities
constitute the axis the student’s learning process. As far as they are evaluated and corrected in a systematic and
rigorous form, they turn into the most direct and automatic channel that allows the student to get clear and
concrete information on the foreseen aims’ and competences’ attainment. The aim of this article is to analyse the
effectiveness and the efficiency of giving a more personalized feedback by means of ICT to the learning
exercises or activities done by the students. The way to achieve it is to offer the tutors a diversity of multimedia
tools (text, oral, video, etc...) that facilitates this task. The objective is to reduce the time of preparing the
feedback and thus to make it easier as well as to achieve a higher impact. The work has been undertaken in a
virtual classroom where the tutor is provided with a kit of ICT tools that allows him to send to each student a
personalized comment on his exercises or activities. To make this comment, different complementary alternative
forms to written feedback have been explored in order to achieve greater motivation and greater impact. This
alternative forms have been chosen taking into account the personality of the tutors participating in the project as
well as their origins and profession paying special attention to their abilities in transmitting knowledge. That's why
one of the main proposals is trying to get best communicative capabilities of each one. The results show that the
students find personalized feedback very positive, because it makes learning easier and thus more motivating. In
order to implement personalized feedback in an efficient way from the tutor’s point of view, several actions should
be undertaken such as technical training in communication and pedagogy, modifying the way of correcting the
exercises or activities and also adapting the dimension of the classroom by taking into account that the number of
students is a key element for this tutorial model.
Keywords: personal e-feedback, ICT, student centered learning, improvements in students rendability
1. Introduction
In the asynchronous eLearning contexts of Higher Education, based on written language, the
feedback provided by the teacher is one of the basic elements in the students' learning process. The
student centred learning (SCL) approach aims to overcome some of the problems inherent to more
traditional forms of education by focusing on the learner and their needs, rather than being centred
around the teacher’s input. Thus the student becomes the protagonist of his/her learning process by
means of fulfilling learning exercises or activities.
This study is made at the Universitat Oberta de Catalunya (UOC) The UOC is a totally virtual
university. One of the basic characteristics of its educative pattern is to encourage a range of
continuous evaluation activities to every subject that is part of the diverse learning programs. The
underlying idea is that the student attains the competencies inherent to each subject by carrying out
the different proposed activities. These continuous evaluation activities are called Continuous
Evaluation Tests (PAC, for its acronym in Catalan). We call them tests as they also serve to assess
how the students are performing on a continued basis and to check if they attain progressively the
learning aims that are previewed.
These PACs are, therefore, the axis of the student's work. Furthermore, as they are evaluated
systematically and rigorously, they become the most direct and automatic channel for the student to
receive a clear information on whether he/she is attaining the previewed goals or not, as well as the
mistakes he/she is making and what he/she is lacking.
456

Maria-Jesus Martinez-Argüelles et al.
Nevertheless, one of the aspects of the UOC's learning model that the students have missed most in
recent years is the absence of a personalized feedback to the continuous evaluation activities they
have submitted [Martinez et al. 2010].
This work is organized as follows: in the first part we present the theoretical framework that served us
to think and design the pilot test. In the second part we describe the pilot test's implementation and
we announce the first results. We close with some conclusions and the future lines of investigation
opened out of this first experience.
2. Methodology
Using feedback in the pedagogical process has been broadly studied. Existing literature shows the
difficulties arisen both from the teacher's perspective and the student's perspective, giving or receiving
feedback in the learning process in an eLearning environment. Such difficulties are linked basically to
the high ratio students-teacher in the virtual classrooms, that increases the follow-up task of the
learning that the teacher could do (see [Buchanan, 2010] and [Ley, 1999]). Other existent difficulties
are related to the contents of the feedback, that is to say, how and which feedback is provided to the
student so as for him/her to make the best of it in his/her learning process, and to make sure that it
has been the appropriate feedback [Gibbs et al., 2004]. This issue has not been too studied until now
and even less in eLearning environments. The present work intends to explore how and which is the
feedback to be provided, in order to improve its efficiency and efficacy in virtual classrooms with a
high number of students. With this aim, different multimedia technologies are used (video, audio,
screen captures, etc.) as a complement or substitute to written language. This aspect has not been
studied yet and is the main contribution of this article.
Taking as starting point the definition of the concept of feedback contained in works by Narciss and
her collaborators ([Narciss, 2004, 2008], [Narciss et al., 2004a, 2004b, 2006]), the aim of this article is
focused on its semantic and structural dimension.
The semantic dimension of feedback refers to the contents it transmits, and has been studied in nonvirtual
environments by several authors such as [Kluger et al., 1996], [Kramarski et al., 2001],
[Kulhavy et al.,1989], [Mason et al., 2001] and [Narciss, 2004]. In virtual environments it has been
tackled by [Espasa, 2008], [Espasa, 2010], [Espasa et al., 2010], [Alvarez et al.] and [Guasch et al.,
2010]. Because of the objectives and characteristics of our study, we use the taxonomy elaborated by
this author, which includes four semantic sub-dimensions of feedback (from the lesser to the major
degree of complexity) that are not exclusive amongst them.
Such sub-dimensions are the following:
Sub-dimension 1 – Error identification and correction: the feedback informs about the correction
of errors made.
Sub-dimension 2 – Correct answer: the feedback shows the correct answer or gives the right
solution.
Sub-dimension 3 – Task improvement: the feedback recommends contents and strategies in
order to improve the work, or the test that has been submitted.
Sub-dimension 4 – In-depth information: the feedback suggests in-depth information in order to
continue to move forward in learning in the future.
Out of these four sub-dimensions, our analysis will focus on the first three, (the ones inherent to
graduate studies). The last one is not used as it corresponds to postgraduate studies. Together with
[Kulhavy et al., 1996], [Mory, 2004] and [Mason et al., 2001], we think that, in order for the feedback
to succeed, it is necessary for it to contain both sub-dimensions 1 and 2 (in which the teacher
provides a judgement on whether the answer is correct or incorrect) and sub-dimension 3 (that offers
the student guidelines towards the correct answer).
As regards the structural dimension of feedback, it refers to the presentation or the feedback's shape
in a concrete context. Considering contributions made by [Narciss et al., 2006], others such as
[Hyland, 2001, 2003] and [Espasa, 2008] in a eLearning environment of higher education such as the
university we are analysing, the main elements that define the feedback's structural dimension are:
457

Maria-Jesus Martinez-Argüelles et al.
Agents that take part in the feedback processes: One of the elements inherent to feedback are
the members that take part of its processes. This research has been developed by using a
technological platform that integrates among others two applications that make it possible to
provide the student with feedback. The first one is the continuous evaluation registry (RAC, for its
acronym in Catalan), a space in the virtual classroom where the teacher inserts the marks of the
continuous evaluation and facilitates the personalized feedback (which is sent to a private space
only shared by teacher and learner). The second one is the calendar and it can be found in the
planning space of the virtual classroom. There, the student can accede to the solutions of the
continuous evaluation tests (PAC), prepared by the teacher and published one day after the
students have submitted their solutions.
Space of virtual communication where the feedback processes take place: Another component is
the virtual space where the feedback takes place, be it public or private. In this study we analyse
the effects of feedback in a private space (from now on personalized feedback). Authors such as
[Rice et al., 1994] have proved that personalized feedback (private spaces) has advantages over
feedback in public spaces because it fits better to the needs and characteristics of each student.
Moment of feedback: To provide immediate or differed feedback has been the object of
discussion in several researches on this issue. It is remarkable the proposal made by [Dempsey
et al., 1988], that define some categories used in this work. The first one is immediate feedback (it
provides the student with information on the correct answer) and in this context it corresponds to
the PACs' solutions, published just one day after the students submit their answers. The second
one is differed feedback (where the errors made are commented on, some contents are
recommended as well as strategies to improve the process of learning), that gives added value to
the solution of the PAC.
Extension and form of the feedback: Another special feature of feedback in a virtual environment
is its extension. It is tightly linked to the semantic dimension of feedback and to the kind of
knowledge on which the feedback is given (depending on whether it is more abstract or
conceptual or more applied and procedural). In order for the feedback to be effective it must give
enough information to university students so as for them to attain the aims of learning previewed.
The feedback can have different shapes: text and/or technological multimedia (audio and video
recordings, screenshots with audio as well as the possibility to upload compressed videos). The
student receives it in the RAC as an attached file or in the message itself.
Following Narciss and her collaborators [Narciss 2004, 2008], [Narciss et al., 2004a, 2004b, 2006], in
this article we also state that there are other dimensions apart from the semantic and the structural
dimension with influence on the feedback processes, as in the case of the pedagogical design.
Following we explain the pedagogical framework in which this study has been carried out.
2.1 Pedagogical design
One of the defining aspects of this work is the context in which it is carried out. It is an eLearning
environment where the process of teaching and learning is conveyed through a technological
platform. This work studies the case of the Universitat Oberta de Catalunya (UOC). This university
was created in 1995 and from the very beginning has developed its activities in a virtual campus. In
the UOC's model of education, the agents of the educative activity do not share the same space and
time coordinates. This flexibility that involves that both the student and the teacher enjoy the learning
environment from any space and moment (asynchronous communication) needs the educative
activity to be based upon communication exchanges between both of them.
The UOC's pedagogical model puts the student in the centre of the teaching and learning process
(student centred learning, SCL) carried out in a virtual classroom. This virtual classroom has four
different areas: planning, communication, resources and evaluation. The evaluation area gathers all
aspects related to the evaluation process of a given subject. This study is focused in one of the
spaces of the evaluation area: the Continuous Evaluation Registry (RAC for its acronym in Catalan)
that is where the personalized feedbacks take place. This personalized feedback consists of
introducing marks by the teacher, and/or personalized inputs (sub-dimension 1, error identification
and correction, and sub-dimension 3, task improvement). Despite the fact that sub-dimension 2 is not
directly dealt with from the RAC (nor it is a direct part of the feedback), some guiding solutions to
evaluated activities are given once the submission term is over. Furthermore, when necessary, part of
the feedback uses the solutions as a support.
458

Maria-Jesus Martinez-Argüelles et al.
During the teaching and learning process, the student counts with the tutor's and the teacher's direct
support. The latter is the one that teaches in the virtual classroom, accompanies the student in the
development of the teaching and learning process acting both as a facilitator of the educative activity
and as expert on the subject he/she is teaching.
The UOC's evaluation system is based upon the combination of an educational and time-continuous
evaluation, with a final evaluation. The continuous evaluation is based on a series of proposed
evaluation tests -continuous evaluation test (PAC)- that are carried out on a time-continuous basis.
To this point, the theoretical basis of this study and the context in which it has been developed have
been described. It follows the empirical analysis and its results.
3. Implementation
It is in the context of research on processes of feedback with the aim to promote construction of
knowledge and regulation of learning, still rather incipient in eLearning environments, where this study
takes shape. Thus, attaining the objectives proposed in the context of this article will allow us to get
results at two different levels: on the one hand, efficiency for the teachers and, on the other hand,
efficacy for the students (performance). In this context, considering costs and benefits of
implementing personalized feedback, it is necessary to question its usefulness, in order to extend it to
other subjects or studies. With the aim to analyse the impact that has the incorporation of
personalized feedback to the solution of activities and correction of exercises during the continuous
evaluation process, and to make available several technological methodologies as well as their
feasibility, diverse pilot tests have been designed according to the following criteria: groups of
reduced dimensions (maximum 35 students) per subject have been chosen in order to ensure the
teacher's adequate dimensioned task. Subjects have been selected according to the sort of
knowledge (structural dimension of feedback) and diverse methodology, apart from considering
teachers with different aptitudes, assessing the action's transversality and its possible extension to all
of the subjects in the degree. The involved subjects, all of them from the Degree in Business
Administration, were: Introduction to business (01.500), Statistics essentials (01.501), Behaviour of
economic aggregates (01.508), Economic structure (01.510) and Introduction to accountability
(01.520). As pointed out, there was a will to find subjects covering different fields of the degree. From
applied economy (Economic structure and Behaviour of economic aggregates) to more
methodological subjects (Statistics essentials), including accountancy subjects (Introduction to
accountability) and business organization subjects (Introduction to business) (see detail in table 1).
Table 1: Summary charter of forecasts made in each subject when carrying out the pilot test
Subject
Introduction
to
accountability
Statistics
essentials
Economic
structure
Behaviour of
economic
aggregates
Introduction
to business
In all the
activities?
Yes, with
different levels
of intensity
Yes, with
different levels
of intensity
Yes, with
different levels
of intensity
Yes, with
different levels
of intensity
Yes, with
different levels
of intensity
To whom?
PAC1 to everyone. To
the rest, depending on
the results (D, C- and
C+ enough to
everyone) and to the
rest depending on the
errors and evolution
Depending on marks
categories: C+, C- and
D.
PAC1 to everyone. To
the rest, depending on
the results and needs.
To everyone with a
weak mark and
especially to those
who fail
PAC1 to everyone, to
the rest depending of
results and needs
Level of
personalization
Individual
Depending on
the type of error
made
Individual
How? Proposal of feedback
PAC1/PAC2:
audio and/or
textual. The rest
video or screenshots
The most difficult
(PAC2/PAC3):Ca
ptures and video.
To the rest
text/audio.
Video and/or
audio
Depending on
Captures/video/te
the type of error
xt/audio
made
459
Individual Video/text/audio
To correct errors and
guide/improve the
learning process
To correct errors and
guide/improve the
learning process
Firstly, to motivate.
Secondly, to correct
errors and
guide/improve the
learning process
To correct errors and
re-orient/improve the
learning process
To correct errors and
re-orient/improve the
learning process and
to motivate

Source: Own elaboration from surveys to teachers.
Maria-Jesus Martinez-Argüelles et al.
There has also been a will to find subjects with different evaluation models. Whereas some of them
had five activities, four of which are compulsory (Economic structure, Behaviour of economic
aggregates and Business introduction), others have five compulsory activities (Statistics essentials)
and even others (Introduction to accountability) has five continuous evaluation activities with the
obligation to submit the last one and three more out of the other four activities).
The design of the pilot test has a double dimension. A technical part focussed on contributing to the
maximum efficiency when making the teacher's development of feedback possible, and a teaching
part that focusses on how to get a major impact on the student out of this intervention. In order to
assess the action's impact it was rendered necessary to introduce monitoring classrooms (in order to
grant the major degree of equalization possible in the monitoring classroom, it was made over the
same subject, in the same semester, guided by the same teacher and on the same dimension, except
for the subject Statistics essentials).
The design of the technical part was made taking into account different possibilities of feasible
multimedia feedback according to capabilities of the technological platform (audio, video, commented
screen-shots...). In order to make it possible, some technical improvements have been implemented
to the RAC, with the aim to facilitate elaboration of a personalized feedback. In fact, technological
possibilities in a technological platform when applying these techniques, is a limitation to be taken into
account when making it extensive to the rest of the degree.
As regards the teaching part, the discussion on each of the feedback options' suitability and the
possibility for it to be applied to one or several activities took place between the Teacher Responsible
for the Subject (PRA for its acronym in Catalan) and the teacher. Where the multimedia feedback was
not implemented, the option has been the written feedback (individual or individualized group). Having
in mind these components, plus each teacher's abilities and concerns, a pilot test for each subject
was defined. As common elements, individualised feedback was made in all activities of all subjects,
though with different levels of intensity. In all cases, students have been given an answer to the
generic PAC, in line with what is being done in the rest of the subjects in the Economy and business
Studies (sub-dimension 2 in the previous part).
Both the multimedia options adopted to carry out the personalized feedback and the number of targetstudents,
are according to each subject and result from the planning made by the teacher and the
PRA. Whereas in some cases, the option was a generalized personalized feedback in the first
activities, in other cases, feedback was focussed on those persons that have not passed the activity
or passed it with difficulties, in order to encourage them to go ahead as well as for them to try and
correct the errors made so as to better face next activities, and also for them to be able to set up
concepts that will allow them to pass the Final Test without difficulties (working sub-dimensions 1 and
3 of the previous part). In other cases the option was for a more generic one, comprising students
with similar typologies, whether because they made the same error when solving the activity or they
had a similar final result in the activity. Another of the options adopted was the generalized
personalized feedback (to all of the students), independently of their mark in order to correct errors of
those who had a worse result and encourage those who had a good result to maintain their level.
As already pointed out, the pilot test was adapted to each teacher's abilities and to each activity's
specific needs. As said, the first activity in each subject had a generalized personalized written
feedback in line with the evaluation criteria of the Economy and Business Studies degree. From the
second activity, implementation of the multimedia feedback started in a generalized manner in all the
subjects. In Economic Structure, feedback was made with audio oriented to those students that did
not pass the subject, that made important errors or that did not submit it. From the third activity, audio
was complemented by video in search of a generalization of messages so as they can be used with
more than one student. In Statistics Essentials the option was a personalized feedback based on
commented screen-shots this option was chosen because the subject's format allows a very efficient
exploitation of this tool. Resolution of problems in the context of statistics requires certain approaches
and mathematical developments that the commented screen capture dynamics makes it easier to
understand (it is very useful in elements such tables reading, statistics calculations solving from a
statistics program, etc.) In this sense, this feedback has not been generalized to the whole range of
activities of the subject but was centred on those activities where major profits were foreseen. As
460

Maria-Jesus Martinez-Argüelles et al.
regards the personalized feedback's addressee, this was chosen according to errors made while
carrying out the activity or because he/she did not pass the activity. This criteria intends to minimize
the number of answers to be elaborated by the teacher, but involves a lesser degree of
personalization. In order to fulfil this lack the multimedia feedback was complemented by written
feedback (in some occasions, reiteration of errors made by the student can provoke two multimedia
feedback entries). The cost of implementing such option is high but can be reduced in time for
different reasons: improvement of the teacher's abilities, generalized use of answers to concrete
problems and accumulation of examples and exercises solutions for the future (to be used in future
feedback or as complementary tools to be used in the learning process).
The pilot test carried out in the subject Behaviour of economic aggregates shows clearly the increase
of the amount of work that involves the personalized feedback, even if it is written (this excess work
was valued as double the work, though there have been attempts to save time with common answer
strategies). The strategy followed here varied according to results obtained by students and to the
test's features. In PAC 2 audio feedback was implemented when searching a major impact (the PAC
was not passed, the mark could have improved in relation with the previous PAC, …) whereas in the
rest of cases the option was written feedback (good results in PACs, non-submission of two of the
PACs,...) For the rest of PACs this has been the criteria followed when considering which feedback
should be implemented. Furthermore, video was introduced in cases where a rather common error
was made. In this case, technological abilities also become a key factor to be taken into account.
Here, the assessment made by the tutor is very positive due to the degree of proximity between
student and teacher that this technique allows. In this case, instead of facilitating understanding of
certain concepts, as happened in subjects such as Statistics Essentials, the most valued element is
improvement in the student's answer due to the impact that personalized feedback had on him/her.
Nonetheless, again it is remarkable the importance that dimension has when offering a correct
personalized feedback.
In Introduction to Accountability there was a general written answer to everybody in the first PAC,
then audio personalized feedback to those that submitted the second PAC and written to those that
had not done so. Video was used in the rest of activities. In this case, the only discrimination was
between students who has submitted the activity and those who had not. The degree of performance
of students in the classroom where the personalized feedback took place was higher.
In Business Introduction, feedback was also performed according to the mark, in text for the first
activity. From then on, the option was to incorporate audio / video feedback depending on the mark
obtained after the second activity, with a clear intention to improve the result in those cases where
they had not been positive. Together with these messages, written feedback was used when it was
necessary to complement some information. In this case, the teacher assessed that video or audio
feedback messages did not bring significant quality improvement with respect to the personalized
feedback in text. What is really considered as critic in this case is the possibility to perform
personalized feedback with all the students. The option to extend the use of such tools in the learning
process as a complement in cases where there are important doubts about the contents and the
teacher can clarify them, is raised.
Once all pilot tests have been presented, it is time to draw up a balance. The benefits of implementing
multimedia personalized feedback go in two aspects. The first one shows how useful is such
methodology in cases that it leads to improving understanding and learning of the subject, from the
correction of errors and the extension of contents. In this sense, it must be taken into account that the
fact of considering personalized and adapted to each student's needs feedback, is in itself a highly
positive elements that is worth counting on. The second aspect to be considered is the impact that
multimedia personalized feedback has on the student, due precisely to the fact that it has been done
in audio or in video. Some teachers pointed out the positive assessment of students on this option
and, therefore, it must be understood as an element to be taken into account in the future.
The costs involved in the adoption of this methodology are high and are directly related to the
dimension. Teachers must acquire technical and communication skills that need to be developed.
This is the reason why the need of training the teachers before implementing this option is suggested.
Time invested in preparing generalized and personalized feedback, both written and multimedia, is
quite high and, thus, it is necessary to assess how effective is to incorporate it. In fig. 1 we see that,
as an average, some 20% of the total time invested by the teacher, can be directly attributed to
461

Maria-Jesus Martinez-Argüelles et al.
generation of personalized feedback. Moreover, it must be considered that this is directly related to
the size of the group. Over 40 students, timely-costs in the elaboration of feedback go far beyond the
time teachers must dedicate to it. This forces us to opt for several strategies. Reducing the number of
people in the classroom leading to an optimal size could be an example, or also cost-reducing
strategies can be raised, starting with the creation of generic messages (oriented to solve concrete
doubts and errors which are common to several students).
50,0
45,0
40,0
35,0
30,0
25,0
20,0
15,0
10,0
5,0
0,0
Training, Multimedia personalised feedback Correction Multimedia implementation Personalized feedback
01.520 01.500 01.508 01.501 01.510
Figure 1: Hours destined to correction and personalized feedback, by type of work (ource: Own
elaboration from surveys to teachers).
One element that has not been taken into account when implementing the pilot test and could add
some significant distortion to the analysis is the fact that in the subject Statistics Essentials there was,
at the same time, another pilot test incorporating important changes to the structure of the student's
classroom being carried out. This fact reduced our frame of reference to work with when making the
comparison. That is to say, whereas comparison between classrooms can be analysed with no fear,
comparative analysis with previous semesters is difficult because they would not be strictly
comparable. Moreover, in this case, the teacher did not manage two small-sized groups but only one.
4. Conclusions
Implementation of multimedia personalized feedback arises as a need to improve the student's
learning capacity and performance, making the best out of generalization and spreading of multimedia
tools, beyond the feedback in text. From this element and after developing a set of pilot tests in
several subjects of the Degree of Business Administration the following conclusions can be drawn:
Firstly, with the aim to make multimedia feedback more efficient and to make the best of its options,
teachers need specific training both technological and pedagogical, as well as to develop
communication abilities through video. Secondly, it has been proved that personalized feedback
requires an important time investment by the teacher, therefore it is necessary to plan the intensity
and frequency in which inputs will take place both regarding students and PACs. Thirdly, the
effectiveness of feedback and accuracy of using one or another channel, one or another type of
feedback depends a lot on the typology of the subject and the teachers' communication and
technological abilities. Fourthly, it can be stated that personalized feedback brings a closer
relationship between learner and teacher, this having favourable and encouraging effects on learning
on the students. It is also certain that, on the other hand, this more personalized relationship between
learner and teacher is to the detriment of the relationship classroom-group. Fifthly, it is apparent that
personalization of feedback changes the way PACs are corrected and, therefore, lastly we can
conclude that using other ways to give feedback different from the written one, does not reduce the
462

Maria-Jesus Martinez-Argüelles et al.
teachers' dedication and makes it hard to be implemented generally in groups of 70 students with the
current contractual relationship of dedication.
These conclusions must be compatible with the fact that students judge very positively the
personalized feedback and has a facilitating as well as motivating effect on learning. Nevertheless,
according to what the teacher suggests, the most important element is what is said rather than the
tool that is used to say it.
On the other hand, in order for this feedback to be implemented by teachers, a technical training on
communication and pedagogy is needed. This personalization requires additional dedication and new
ways to correct PACs that make it difficult to be extrapolated to a 70-student classroom-group,
particularly taking into account that the longer this personalized feedback takes place, the more
demanding students become and the teacher-student interaction increases.
Finally, from this study several lines of future investigation arise. Data analysis regarding academic
performance between pilot classrooms and monitoring classrooms is a key element to be valued
when considering that the academic performance of the over effort that personalized feedback
means. In this sense, it must also be considered, when assessing its possible spreading to the rest of
subjects within the context, whether or not different types of subjects correspond to different types of
feedback.
Acknowledgements
This article is the result of a collective work resulting from the project “TOWARDS AN
IMPROVEMENT OF THE E-FEEDBACK”, funded by AGAUR (Catalan Goverment), leaded by María
Jesús Martínez Argüelles (2010MQD00145), We are grateful to the work done by the rest of members
of the project: Marc Badia-Miró, Carolina Hintzmann, Dolors Plana-Erta, Muriel Garreta Domingo,
David Trelles Bertran and Antoni Mangas. We also want to thank the collaboration of Anna Espasa
Roca when improving the methodological work around feedback. Lastly, we want to thank the
economic support given by the Economy and Business Department of the Universitat Oberta de
Catalunya.
References
Alvarez, and., Espasa, A. & Guasch, T. (in press). “The value of feedback in improving collaborative writing
assignments in an online learning environment”. Studies in Higher Education.
Buchanan, T. (2000). “The efficacy of a World-Wide Web mediated formative assessment”. Journal of Computer
Assisted Learning, 16, 193-200.
Dempsey, J.V. & Wager, S.U. (1988). “A taxonomy for the timing of feedback in computer-based instruction”.
Educational Technology, 28(10), 20–25.
Espasa, A. & Meneses, J. (2010). “Analysing feedback processes in an online teaching and learning
environment: An exploratory study”. Higher Education, 59-3, 277-292.
Espasa, A. (2008). “El Feedback en el marc de la regulació de l’aprenentatge: caracterització i anàlisi en un
entorn formatiu en línia”. Doctoral Thesis. On line, available
at:http://www.tdx.cat/browse?value=Espasa+Roca%2C+Anna&type=author
Espasa, A. (2010). “Temporal and assessment dimension: characterisation of feedback after assignments”.
eLearn Center Research Paper Series, Issue 1. Time factor in eLearning and assessment.
http://elcrps.uoc.edu/ojs/index.php/elcrps/article/view/issue1-espasa
Gibbs, G. & Simpson, C. (2004). “Conditions under which assessment supports students’ learning”. Learning and
Teaching in Higher Education, 1, 3-31.
Guasch, T.; Espasa, A. & Álvarez, and.(2010). “Formative e-feedback in collaborative writing assignments: the
effect of the process and time”. eLearn Center Research Paper Series, Issue 1. Time factor in eLearning
and assessment. http://elcrps.uoc.edu/ojs/index.php/elcrps/article/view/issue1-guasch-espasaalvarez/issue1-guasch-espasa-alvarez
Hyland, F. (2001). “Providing Effective Support: Investigating feedback to distance language learners”. Open
Learning, 16(3), 233-247.
Hyland, F. (2003). “Focusing on form: Student engagement with teacher feedback”. System, 31, 217-230.
Kluger, A.N. & DeNisi, A. (1996). “The effects of feedback interventions on performance: A historical review, a
meta-analysis, and a preliminary feedback intervention theory”. Psychological Bulletin, 119(2), 254-284.
Kramarski, B. & Zeichner, O. (2001). “Using technology to enchance mathematical reasoning: Effects of feedback
and self-regulation learning”. Educational Media International, 38 (2-3), 77-82.
Kulhavy, R.W. & Stock, W.A. (1989). “Feedback in written instruction: the place of response certitude”.
Educational Pyschology Review, 1(4), 279-308.
Ley, K. (1999). “Providing feedback to distance students”. Campus-Wide Information Systems, 16(2), 63-69.
463

Maria-Jesus Martinez-Argüelles et al.
Martínez, M.J.; Juan, A.A.; Castan, J. (2010). “Using the Critical Incident Technique to Identify Factors of Service
Quality in Online Higher Education”. International Journal of Information Systems in the Service Sector,
2(4), 57-72.
Mason, J. & Brunning, R. (2001). Providing feedback in computer-based instruction: what the research tell us.
University of Nebraska-Lincoln: http://dwb.unl.edu/Edit/MB/MasonBruning.html
Mory, E. H. (2004). “Feedback research revisited”. A D. Jonassen (Ed.), Handbook of research on Educational
Communications and Technology. Mahwah, NJ: Lawrence Erlbaum, 745-785.
Narciss, S. (2004). “The impact of informative tutoring feedback and self-efficacy on motivation and achievement
in concept learning”. Experimental Psychology, 51(3), 214-228.
Narciss, S. (2008). “Feedback strategies for interactive learning tasks”. A J.M. Spector, M.D. Merrill, J. Narciss,
S., Körndle, H., Reimann, G. & Müller, C. (2004a). Feedback-seeking and feedback efficiency in web-based
learning– How do they relate to task and learner characteristics? A. P. Gerjets, P. A. Kirschner, J. Elen, &
R. Joiner (Eds.), Instructional design for effective and enjoyable computer- supported learning. Proceedings
of the first joint meeting of the EARLI SIGs Instructional Design and Learning and Instruction with
Computers (pp. 377-388). Tuebingen: Knowledge Media Research Center.
Narciss, S. & Huth, K. (2004b). How to design informative tutoring feedback for multimedia learning. A H. M.
Niegemann, R. Brünken & D. Leutner (Eds.), Instructional Design for Multimedia Learning (pp. 181-195).
Münster: Waxmann.
Narciss, S. & Huth, K. (2006). Fostering achievement and motivation with bug-related tutoring feedback in a
computer-based training for written subtraction. Learning and Instruction, 16(4), 310-322.
Van Merriënboer & M. P. Driscoll (Eds.). Handbook of Research on Educational Communications and
Technology (Aect). New Jersey (EUA): Lawrence Erlbaum.
Rice, M., Mousley, J & Davis, R. (1994). “Improving student feedack in distance education: a research report”. A
T. Evans & D. Murphy (Eds.), Research in distance education (pp. 52-62). Geelong (Austràlia): Deaking
University Press.
464

Evaluation of Multimedia Tools and e-Feedback in Virtual
Learning Environments
Maria-Jesús Martínez-Argüelles 1 , Marc Badia-Miro 2 , Carolina Hintzmann 1 and
Dolors Plana-Erta 1
1
Economia i Empresa, Universitat Oberta de Catalunya, Barcelona, Spain
2
Departament Història Econòmica, Universitat de Barcelona, Barcelona, Spain
mmartinezarg@uoc.edu
mbadia@ub.edu
chintzmann@uoc.edu
dplana@uoc.edu
Abstract: There is a high consensus that one of the key elements to ensure knowledge attainment in a virtual
university context is the existence of regulatory processes of learning, which allows students to regularly evaluate
their learning process. In asynchronous learning environments one of the usual strategies to facilitate this
regulation is the use of feedback mechanisms between the student and tutor. But in the environment of large
groups of students written personalized feedback may be too laborious for the tutor. In this paper we present a
pilot project focusing on the context of Degree in Business Administration from the Universitat Oberta de
Catalunya. We have explored ICT tools to give feedback (audio, video and screenshots). The aim is to improve
the effectiveness and efficiency of the learning regulatory process. During one semester the project was
implemented by taking a virtual classroom with a reduced number of students in specific subjects using
alternative ways to written feedback. In order to evaluate the effects of using these alternative multimedia tools,
the results obtained have been compared with the other benchmark groups taking into account the following
aspects: academic results, assessment of students, evaluation of tutors, and hours of dedication of tutors. The
implementation of the pilot test and evaluation of results have allowed to draw some conclusions about the
conditions under which the use of multimedia tools in feedback may in fact help the adjustment process in virtual
learning.
Keywords: feedback, virtual learning, ICT, personal feedback in large groups
1. Introduction
In the framework of the implementation of new degrees to the European Higher Education Area
(EHEA), the student becomes the protagonist of his/her learning process, which carries out by
fulfilling learning activities. In order to ensure that the student really attains the competencies
associated to each subject, the function of accompaniment and personalised relationship between
tutor and learner becomes a priority.
The UOC is a totally virtual university. One of the basic characteristics of its educative pattern is to
encourage a range of continuous evaluation activities to every subject that is part of the diverse
learning programs. The underlying idea is that the student attains the competencies inherent to each
subject by carrying out the different proposed activities. These continuous evaluation activities are
called Continuous Evaluation Tests (PAC, for its acronym in Catalan). We call them tests as they also
serve to assess how the students are performing on a continued basis and to check if they attain
progressively the learning aims that are previewed.
These PACs are, therefore, the axis of the student’s work. Furthermore, as they are evaluated
systematically and rigorously, they become the most direct and automatic channel for the student to
receive a clear information on whether he/she is attaining the foreseen goals or not, as well as the
mistakes he/she is making and what he/she is lacking.
Nevertheless, one of the aspects of the UOC’s learning model that the students have missed most in
recent years is the absence of a personalised feedback to the continuous evaluation activities they
have submitted (Martínez, M.J.; et al (2010)). Taking as starting point the definition of the concept of
feedback given by Susanne Narciss in her multiple scientific works (Narciss, S. (2004, 2008) and
Narciss, S. & Huth, K. (2004, 2006)), this study is focussed on the semantic and structural dimension
of feedback.
465

1.1 Semantic dimension
Maria-Jesús Martínez-Argüelles et al.
The semantic dimension of feedback refers to the contents it transmits, and has been studied in nonvirtual
environments by several authors such as Kluger, A.N. & DeNisi, A. (1996), Kramarski, B. &
Zeichner, O. (2001), Kulhavy, R.W. & Stock, W.A. (1989), Mason, J. & Brunning, R. (2001) and
Narciss, S. (2004). In virtual environments it has been tackled by Espasa, A. (2008). Because of the
objectives and characteristics of our study, we use the taxonomy elaborated by this author, which
includes four semantic sub-dimensions of feedback (from the lesser to the major degree of
complexity) that are not exclusive amongst them. So, such sub-dimensions are:
S1. Error identification and correction: the feedback informs about the correction of errors made.
S2. Correct answer: the feedback shows the correct answer or gives the right solution.
S3. Task improvement: the feedback recommends contents and strategies in order to improve the
work, or the test that has been submitted.
S4. In-depth information: the feedback suggests in-depth information in order to continue to move
forward in learning in the future.
Our analysis will focus on the first three sub-dimensions (error identification and correction, correct
answer and task improvement) as they are the most inherent to initial subjects in the framework of
graduate studies. So in our analysis feedback made by the tutor includes the following subdimensions:
1) Guidelines to the answer: once the term to submit the continuous evaluation activities is over, a
correct indicative answer is provided to the whole group of students in the classroom. Thus, students
can carry out a self-evaluation activity, comparing this solution to the contents of the activity they
submitted. This category of feedback fully coincides with sub-dimension 2 of the feedback’s semantic
dimension.
2) Specific feedback: Each tutor has in the virtual classroom a tool (continuous evaluation registry)
that allows the sending of (written) comments to each student. But often, groups are of 70 students or
more and there is the commitment to correct and comment on the test in a term no longer than a
week. So, this makes a personalised input to every student very costly.
The solution we have adopted is twofold. On one hand, there is a feedback according to the mark
obtained. All students that have got the same mark receive in the private space an identical comment
on their tests.. Although, this is not a fully personalized feedback; obviously we can assimilate this
kind of feedback to the third semantic sub-dimension.
On the other hand, it is also common to all subjects to send to the public space of the virtual
classroom a more generic comment on the most common mistakes done in the evaluation test as well
as some advices in order to improve the learning process (feedback S1).
3) Answer to doubts: Moreover, each tutor answers the doubts raised by the students on the activities
and contents they refer to. This could be labelled as a reactive, non-proactive feedback. The aim of
this kind of input is to improve the work to be done by students, coinciding with the third subdimension
previously described.
1.2 Structural dimension
As regards the structural dimension of feedback, it refers to the presentation or form of the feedback
provided in a given context. Inspired on contributions made by Narciss, S. & Huth, K. (2004), Hyland,
F. (2001, 2003) and Espasa, A. (2008) in a higher degree e-learning context as it is the case of the
Universitat Oberta de Catalunya, the main elements that define the feedback's structural dimension
are:
Space of virtual communication where the processes of feedback take place: One of the elements
inherent to feedback are the spaces where its processes take place. This study has been
developed with the use of a technological platform that integrates among others two tools that
466

Maria-Jesús Martínez-Argüelles et al.
serve to provide the student with feedback. The first one is the continuous evaluation registry, a
space in the virtual classroom where the tutor inserts the marks of the continuous evaluation and
facilitates the personalised feedback (which is sent to a private space only shared by tutor and
learner). The second one is the calendar and it can be found in the planning space of the virtual
classroom. There, the student can accede to the solutions of the continuous evaluation tests,
prepared by the tutor and published one day after the students have submitted their solutions.
Publicness or privacy of the feedback’s processes: Another component is the virtual space where
the feedback takes place, be it public or private. In this study we analyse the effects of feedback
in a private space (from now on, personalised feedback). Authors such as Rice, M., et al. (1994)
have proved that personalised feedback (private spaces) has advantages over feedback in public
spaces because it fits better to the needs and characteristics of each student.
Moment of feedback: To provide immediate or differed feedback has been the object of
discussion in several researches on this issue. It is remarkable the proposal made by Dempsey,
J.V. & Wager, S.U. (1988), that define some categories used in this work. The first one is
immediate feedback (it provides the student with information on the correct answer) and in this
context it corresponds to the continuous evaluation tests, that are published just one day after the
students submit their answers. The second one is deferred feedback (where the errors made are
commented on, some contents are recommended as well as strategies to improve the learning
process), that provides value added to the solution of the continuous evaluation test.
Extension and form of the feedback: Another special feature of feedback in a virtual environment
is its extension. It is tightly linked to the semantic dimension of feedback and to the kind of
knowledge evaluated (depending on whether it is more abstract or conceptual or more applied
and procedural). To guarantee an effective feedback, it must give enough information to students,
so that they attain the foreseen learning aims. The feedback can have different shapes: text
and/or technological multimedia (audio and video recordings, screen-shots with audio as well as
the possibility to upload compressed videos). The student receives it in the continuous evaluation
registry as an attached file or in the message itself.
Furthermore, this input, apart from contributing to improve the students' learning process, becomes a
key motivating tool for them not to quit the subjects but to go on with the continuous evaluation
process.
So, to grant UOC's pedagogical model's quality the implementation of personalized inputs becomes a
key factor.
On the contrary, elaborating personalised inputs of four or more continuous evaluation activities by
the tutors, that have classrooms with 70 students or more, is often a non-sustainable task due to the
time it requires particularly taking into account that the estimate and demanded dedication of tutors is
maximum of two hours per day.
In this project we analyse the efficiency, of different forms of feedback alternative to written text. The
analysis have been done both from the the student's viewpoint and the tutor’s point of view, but .this
work is focussed on the assessment from the student’s point of view .
The pilot test consists of exploring new ways of feedback with the aim to make it closer, more efficient
and more motivation-generating to the student. One of the UOC's features is the diversity of tutors
that take part in it. Diversity understood as different origins, professions, and particularly, as refers to
abilities in transmitting knowledge. In this sense, we propose to pay special attention to this diversity,
trying to make each one's best communicating skills arise. Thus, we propose to investigate on the
possibilities to give inputs in different formats that current technologies make available (audio, video,
screen-shots with audio, etc.). We propose to integrate this input in the framework of the continuous
evaluation registry in order to make it even less costly to tutors. Furthermore, we integrate different
tools that facilitate adoption of diverse multimedia responses (video recordings, audio recordings and
the chance to upload compressed videos).
In this context, the pilot test's main objective is to assess the effect of introducing a major
personalization in the feedback received by students through an offer of alternative channels (oral,
video, etc.) to the tutors, in order to facilitate their task.
467

2. Methodology
Maria-Jesús Martínez-Argüelles et al.
With the specific aim already mentioned, a pilot test was designed in order to analyse the results and
the students' assessment on the use of multimedia tools to provide personalised feedback.
The pilot test's design has been double-sided. On the one hand, the technical side that focussed on
making feedback feasible by the tutor, and, on the other, the teaching side that focussed on how to
make it have a major impact on the student.
When implementing the pilot test, these criteria were followed:
Five subjects as different as possible were selected, so the implementation of multimedia
feedback could be evaluated according to the typology of the subject of the Business
Administration Degree.
The pilot test has focussed on one of the subject's classrooms in order for it to check its real
impact, while other monitoring classrooms of the same subject were maintained. Furthermore, a
group of small dimensions was considered.
Previous information from the involved students was searched in order to analyse their profile,
preferences and predisposition to receive a better feedback and the importance they give to its
quantity and quality, among other aspects.
Bearing in mind the diverse technical available options when giving feedback (audio, video, audio
and video or commented screen-shots), we opted for trying different tools in different subjects
taking into account which one suits better to the subject's needs.
Feedback in the pilot test was only implemented in specific evaluation tests, searching to
maximize its usefulness. In the rest of tests the option was for written feedback.
In all cases, feedback is linked to the correction of evaluation tests.
Now we are going to take a look at the technical design of the computer tool that facilitated
incorporation of multimedia feedback.
2.1 Technical part
In order to have available all necessary tools in the continuous evaluation registry, to start the pilot
test, an integration of LANGblog was developed in this application of mark registration. Thus, in a
short space of time the options of audio and video recording have been incorporated to the
continuous evaluation registry.
Despite this agile integration makes the implementation of the pilot test of multimedia feedback
possible, it is not a scalable nor accurate solution in the long term. The objective was to allow a quick
and easy sound and video recording as well as its editing and ulterior visualization by the students.
2.2 Teaching part
Teachers responsible of the subjects that make up the pilot test, jointly with the tutors that carried it
out, defined the structure that the feedback should have during the pilot test. To do it, eight items
adapted in every case to the needs inherent to each subject, were defined:
Is feedback necessary in all the activities?
Who must the feedback be addressed to?
What level of personalization should it have? It’s appropriate to group students by marks in order
to send them the same feedback?
Which of the tools must one opt for? Video? Audio? Screen-shots? Etc.
Is it applicable to all classrooms of the subject? Is it a monitoring classroom?
Is feedback complemented with a generic solution?
Which is the specific purpose of feedback?
Bearing these elements in mind, each pilot test was adapted to the main specific features of each
subject. The common elements to all subjects were the ones that refer to the fact that feedback has
been implemented in all activities of all subjects, with a different level of intensity and that the pilot test
468

Maria-Jesús Martínez-Argüelles et al.
was implemented only in Catalan-teaching campus., For all the subjects taking part in the pilot test
there was a monitoring classroom with the same dimension and managed by the same tutor, so that
the comparison between evolution of the students' performance in similar environments was easy,
except for one of the subjects. As we already explained in point 1.1 Semantic dimension.
Then, there is a range of particularities inherent to each subject. Among them, differences in the
number of persons to which feedback was addressed are remarkable. Whereas in some cases,
everybody would receive multimedia feedback during the first continuous evaluation tests and it would
be personalised, in others the option was to focus only in those persons that passed the activity only
tightly or rather, that directly did not pass it. In this case, the aim is to encourage the student to go
ahead with the subject and to be aware of the errors he/she may have made, as well as to be able to
correct and solve them. In this sense, there are also differences when looking at the level of
personalization of feedback. Whereas some subjects opt for it to be personal and individualized, other
prefer them to be more generic and to include students with similar typologies, whether because they
made the same kind of mistake when solving the activity or because they had a similar final result.
Finally, another of the most significant differences between subjects is tools used to elaborate the
feedback. While some chose audio, others chose video or screen-shots
3. Results
Our starting point is that in asynchronous teaching contexts, the feedback provided by the tutor is one
of the basic elements in the students' learning process. Thus, one of the first aspects to be analysed
was the perception that students had about the importance and need of feedback linked to the
continuous evaluation tests.
For this reason, a questionnaire was designed. It was distributed among the students at the beginning
of the semester in order to know their opinions on feedback. This gives an a priori view and, then,
after finishing the semester, it can be compared with the results of a second questionnaire in order to
know the students’ perception of feedback received.
The first questionnaire was answered by a total of 45 students out of 165 registered in the involved
groups, which makes a 27,4%. In it when asked to compare the importance of feedback in an on-line
course with an in-person course, the answer was that more importance is given in an on-line course,
independently if it’s they are enrolled for the first time or not at UOC
Furthermore, if we analyse their opinion, also in a range from 1 (least importance) to 5 (more
importance), about several aspects of feedback, we can prove that more importance is given to
quality of feedback and to the degree of personalization rather than to quantity or to the moment they
receive it. But, most of all, importance is given to the fact of receiving feedback from the tutor.
Table 1: Assessment of the importance of diverse aspects of feedback
Woman Man Total
Receiving feedback from the tutor 4.6 4.7 4.7
Amount of feedback received 3.5 3.6 3.6
Quality of feedback received 4.1 3.9 4.0
The moment when feedback is received 3.5 3.7 3.6
Degree of personalization of feedback 4.3* 3.6* 4.0
* Scores statistically different with a significance of 1%.
If we compare average scores given by men and women, a statistical contrast can be made in order
to know if these differences are statistically significant, that is to say, if we can conclude that there are
clear differences between men and women's opinion. Well, the only case where a significant
difference is observed is with relation to the “degree of personalization” of feedback. This means that
women give more importance to the degree of personalization than men.
This analysis can be repeated by making a distinction between students for whom this is their first
semester at the UOC and those for whom it is not, and between students that already have a
university degree and those who have not. In any of the cases, there are no significant differences.
469

Maria-Jesús Martínez-Argüelles et al.
Conclusion, thus, is double: The big importance that all groups of students give to the fact of receiving
feedback from the tutor. And, to a lesser extent, the significant importance given to aspects such as
quality and degree of personalization and, a little less, to quantity and moment.
These conclusions confirm the interest of a pilot test like this, which intends to explore and assess,
precisely, the effect of introducing channels and methods of alternative feedback that are different
from a written message.
This initial assessment can be complemented with the posterior opinion of these students, once the
semester where multimedia tools to give personalised feedback were used is finished. Following is
the comment on the most relevant results for the five subjects (that we called subjects A, B, C, D and
E).
In the case of this second questionnaire a total of 39 students out of the 164 registered in the groups
involved answered, which means a 23,8%. Regarding the 5 subjects, 8 students answered to subject
A, 8 to subject B, 8 to subject C and 15 to subject D. In this case, we have changed the names of the
subjects by “fictitious” names as we try to assess the results of the pilot test and to the task carried
out by the tutors in each subject.
One of the questions made to students is if they “consider that feedback received was excessive,
accurate or insufficient”. In this case, the answers received make it clear that students, in no case,
considered excessive the feedback received. On the contrary, some of them (only women)
considered it was insufficient. Such consideration is interesting if we bear in mind that the aim of the
pilot test was to assess ways of attaining a better feedback.
Table 2: Assessment on the feedback received (number of women)
A B C D Total
Excessive 0 0 0 0 0
Accurate 2 1 4 3 10
Insufficient 2 3 0 3 8
Total 4 4 4 6 18
Table 3: Assessment on the feedback received (number of men)
A B C D Total
Excessive 0 0 0 0 0
Accurate 4 2 3 9 18
Insufficient 0 0 0 0 0
Total 4 2 3 9 18
Another of the key questions for us to assess the students' opinion is the one where they are asked if
they “consider that the degree of personalization of the feedback received was excessive, accurate or
insufficient”. In this case, the students' perception is more or less the same as in the previous
question.
The two tables that follow, where men and women's answers have been separated, prove that none
of the students that answered the questionnaire feel that the degree of personalization of feedback
received is excessive.
Table 4: Assessment on the degree of personalization of feedback received (number of women)
Woman A B C D Total
Excessive 0 0 0 0 0
Accurate 2 1 4 2 9
Insufficient 2 3 0 4 9
Total 4 4 4 6 18
470

Maria-Jesús Martínez-Argüelles et al.
Table 5: Assessment on the degree of personalization of feedback (number of men)
Man A B C D Total
Excessive 0 0 0 0 0
Accurate 2 2 3 8 15
Insufficient 2 0 0 1 3
Total 4 2 3 9 18
Finally, as a way to try and capture the students' point of view on methods used to give feedback, the
following listing shows the answers to the questionnaire's open question where they were asked to
give their opinion on “how to improve the feedback received in the subject”. Reading these opinions
suggests some aspects that, we think to be very interesting as for example:
It seems that even though rarely students give critical opinions about the dynamics in the
classroom, this does not mean that they do not have a critical view about teaching practices that
can be more useful to them. And, when asked, they give interesting answers (at least some of
them).
All the given opinions go in the line of claiming for a major degree of personalization.
Students prove to be very practical and pragmatic in the sense that they value, above all, the
usefulness of feedback received, much more than the type of feedback or the channel (audio,
video, etc.) used.
4. Conclusions
As conclusions from the experience we want to outline the three most relevant ideas that students
evidenced on the importance and need of receiving feedback:
In general, students give more importance to feedback in an on-line learning than in an in-person
learning context.
Students give more importance to quality of feedback and the degree of personalization rather
than to the amount or the moment they receive it. But, above all, they give importance to the fact
of receiving the feedback from the tutor.
Comparing average scoring given by men and women, there are some clear differences between
men’s and women’s opinion concerning some of the assessed issues. Particularly, women give
more importance to the degree of personalization than men.
And, on the other hand, when assessing the concrete experience with subjects and groups where
feedback based on several channels (audio, video or screen-shots) were used, their opinions show
the following:
In no case, students consider excessive or inaccurate the feedback received. On the contrary,
some of them (in this case only women) considered it was insufficient.
Students value, above all, obtaining clear messages, in which the explanation to their mistakes
are perfectly explained. That is to say, the feedback that, according to the identified dimensions,
aggregates the semantic dimension 2 and 3.
Students show to be very pragmatic as they value, above all, the feedback received, much more
than the type of feedback or the channel (audio, video, etc.) used.
Independently of the channel used, the fact to be able to consult in different moments the
feedback given to a particular activity and, even, that it can be printed out is positively judged by
students.
The average student is usually one that adapts him/herself to working conditions in each subject.
Thus, despite the fact that conditions and tools are improvable, often he/she does not suggest or
claim other options or resources. But, once he/she discovers (or is shown) that there can be other
tools such as videos, personalised messages, particular corrections, etc., he/she gets used to
them and claim them when he/she does not find them.
In conclusion, it seems that students valuate very positively personalised feedback and this has a
facilitating effect on learning, as well as on motivation. But, according to opinions expressed by
471

Maria-Jesús Martínez-Argüelles et al.
students it seems that they value more the contents of communication than the tool through which the
message is transmitted.
On the other hand, the tutors, in order to implement this feedback, need technical, communication
and pedagogical training. Preparing personalized feedback requires additional dedication and new
ways to correct continuous evaluation tests which make it difficult to extrapolate it to a
group/classroom of 70 students. Particularly, if we bear in mind that the more personalized feedback
is given to students, the more demanding they become, so a door opens to increasing interaction
between tutor and learner.
Acknowledgements
This article is the result of a collective work resulting from the project (2010MQD00145) “TOWARDS
AN IMPROVEMENT OF THE E-FEEDBACK”, funded by AGAUR. We are grateful to the work done
by the rest of members of the project: J.M. Batalla-Busquets, P. Noguera-Guerra, Ernest Pons, Muriel
Garreta Domingo, David Trelles Bertran and Antoni Mangas. We also want to thank the collaboration
of Anna Espasa Roca when improving the methodological work around feedback.
References
Dempsey, J.V. & Wager, S.U. (1988). “A taxonomy for the timing of feedback in computer-based instruction”.
Educational Technology, Vol 28, No. 10. pp 20–25.
Espasa, A. (2008). “El Feedback en el marc de la regulació de l’aprenentatge: caracterització i anàlisi en un
entorn formatiu en línia”. Doctoral Thesis. On line,
http://www.tdx.cat/browse?value=Espasa+Roca%2C+Anna&type=author
Hyland, F. (2001). “Providing Effective Support: Investigating feedback to distance language learners”. Open
Learning, Vol 16, No. 3, pp 233-247.
Hyland, F. (2003). “Focusing on form: Student engagement with teacher feedback”. System, Vol 31, 217-230.
Kluger, A.N. & DeNisi, A. (1996). “The effects of feedback interventions on performance: A historical review, a
meta-analysis, and a preliminary feedback intervention theory”. Psychological Bulletin, Vol Vol 119, No. 2,
pp 254-284.
Kramarski, B. & Zeichner, O. (2001). “Using technology to enchance mathematical reasoning: Effects of feedback
and self-regulation learning”. Educational Media International, Vol 38, No. 2-3, pp 77-82.
Kulhavy, R.W. & Stock, W.A. (1989). “Feedback in written instruction: the place of response certitude”.
Educational Pyschology Review, Vol , No. 4, pp 279-308.
Martínez, M.J.; Juan, A.A. & Castan, J. (2010). “Using the Critical Incident Technique to Identify Factors of
Service Quality in Online Higher Education”. International Journal of Information Systems in the Service
Sector, Vol 2, No. 4, pp 57-72.
Mason, J. & Brunning, R. (2001). Providing feedback in computer-based instruction: what the research tell us.
University of Nebraska-Lincoln: http://dwb.unl.edu/Edit/MB/MasonBruning.html
Narciss, S. (2004). “The impact of informative tutoring feedback and self-efficacy on motivation and achievement
in concept learning”. Experimental Psychology, Vol 5, No. 3, pp 214-228.
Narciss, S. (2008). “Feedback strategies for interactive learning tasks”. A J.M. Spector, M.D. Merrill, J. Van
Merriënboer & M. P. Driscoll (Eds.). Handbook of Research on Educational Communications and
Technology (Aect). Lawrence Erlbaum, New Jersey (EUA).
Narciss, S. & Huth, K. (2004). How to design informative tutoring feedback for multimedia learning. A H. M.
Niegemann, R. Brünken & D. Leutner (Eds.), Instructional Design for Multimedia Learning (pp. 181-195).
Münster: Waxmann.
Narciss, S. & Huth, K. (2006). Fostering achievement and motivation with bug-related tutoring feedback in a
computer-based training for written subtraction. Learning and Instruction, Vol 16, No. 4, pp 310-322.
Rice, M., Mousley, J & Davis, R. (1994). “Improving student feedack in distance education: a research report”. A
T. Evans & D. Murphy (Eds.), Research in distance education (pp. 52-62). Deaking University Press
Geelong, Australia.
472

Cyberbullying: A Workplace Virus
David Mathew
University of Bedfordshire, Luton, UK
David.Mathew@beds.ac.uk
Abstract: This paper explores workplace cyberbullying in an education institution in the south of England, in
which declarations of zero tolerance towards bullying masked the reality that it was silently condoned as a means
of controlling staff. As with face-to-face bullying, cyberbullying is a matter of impact and not necessarily of intent;
and here we contemplate the role of the bully and the role of the victim, while viewing an example through a lens
of control theory. The following questions – Is there a need for bullying in the workplace? Does it serve a
function? Does bullying help contain workplace anxiety as well as create it? – are posed. The case of a man in
his mid-thirties, who was systematically bullied by his manager for eighteen months, is presented. Here I
examine the social structures that the bullying enforced (and destroyed) and examine what the subject learned
from the manager's behaviour. In my commentary on this case study I refer to Wilfred Bion's work on workgroup
anxiety. In Group Psychology and the Analysis of the Ego, Freud explores group formation and the giving up of
individual ideals for the group ideal. I argue that something analogous happens in a workplace environment in
which bullying is rife and in which a scapegoat must be found, even if there is no corresponding misdemeanor for
which he must be punished. I examine the need for homogeneity when it comes to bullying, as well as the issues
of power; transference; the defence against paranoid anxieties; and what happens when a manager is troubled
by others' intelligence. Finally, in the second half of the paper, I extrapolate a future of cyberbullying. The name
of the company in question has been made anonymous and throughout this paper is referred to only as 'the
Institution'. Similarly, the name given to the victim – Rob – has been invented for the sake of anonymity, at his
request.
Keywords: cyberbullying, bullying, anxiety, control
1. The poisoned department
At the end of March 2011, the comedy/satire website The Daily Mash produced a faux-news story
entitled ‘Mob seeks new thing to be angry about’. In this article it emerges that, around the United
Kingdom, enraged people 'may be forced to return to their slightly depressing lives unless they can
find a new cause' to rebel against in a violent manner.'The angry mob,' the story continues, 'who had
been shouting a lot outside a building they believed was the site of a trial but was actually the remains
of a Courts Furniture Superstore, have been enjoying being definitely the goodies for once.'
It is interesting to note this use of for once. Bullies who work in crowds – whether their own tags would
be anarchists, agitators, even protesters – are well aware that they are not 'the goodies' in the eyes of
others: by preying on weaker targets they are clearly in the wrong – again, in the eyes of others –
whatever their original motives. But are they in the wrong in their own eyes? And what implications
does self-awareness, or the lack of it, have for the twenty-first century's development of a bullying
culture, namely the rise of cyberbullying? More specifically still, what implications are there for the
victim and the perpetrator of cyberbullying in the workplace?
It might come as a shock to some people that organisations exist in which cyberbullying is not only
accepted, it is silently encouraged as a tried and tested means of controlling staff. When we think of
industries in which bullying is rife, perhaps we think of industries in which a strict and enforced
hierarchical structure is in place: the Army, for example; the Police Force, the Prison Service, or the
Navy. Whether these perceptions are justified or not, these industries are more likely to be thought of
as places were bullying occurs. However, there are industries in which, for all their proclamations of
fairness for all and zero tolerance towards bullying and harassment, a general air of malice and fear is
not only allowed to exist, but is encouraged. The Institution referred to herein is an example in the
education industry. The workplace operates on a system of strict hierarchy, in which management
and the lower-paid education staff rarely communicate beyond the necessary; in which creative
thinking and inititaitve is firmly suppressed. As John Steiner writes in Seeing and Being Seen (2011):
'A tolerance of difference is necessary for development and for creativity, but difference can also
provoke envy, and it is often when this is attached to injustice that the destructiveness becomes so
magnified' (p.12). It is this triangulation – of envy, injustice and destructiveness – that has raised the
bullying bar to new heights of creative cruelty.
Although cyberbullying is in its infancy, it is growing up fast, and effective techniques are maturing
with it. At places like the Institution, where it is at best ignored and at worst condoned, it pays to keep
473

David Mathew
members of staff frightened. Far from reducing them to unproductive emotional jelly, cyberbullying is
used as a means of lowering staff morale and robbing employees of belief in themselves and in their
skills, which results in the employee working harder to prove his place. Counterintuitively, perhaps,
the damage works in a fashion that increases the propensity for taking on extra tasks: to begin with, at
least. After a while, the pressure builds, and the victim either crumples or tries to achieve against
(deliberately) impossible goals as way of securing his manager's receding favour. The victim's
inevitable submission to psychological impacts such as stress, anxiety, a failure of concentration, are
used as spurs to induce more work. Physiological impacts such as heart rate increases, blood
pressure changes, sweating, shortness of breath, headaches, and so on, are regarded as clear signs
that the victim was not the equal of the challenge in the first place.
Traditionally cyberbullying has taken the form of threats and intimidation, impersonation, stalking,
defamation of character, rejection, and the unauthorised (and unwanted) publication of private
material. The workplace cyberbully will not have such tactics at his or her disposal. Something more
subtle, and arguably more insidious, must be employed. And while an email written in an
exaggeratedly formal style that is completely out of character might not sound like cyberbullying, if the
style of such a message is repeated fifteen times a day, from a manager who sits in the same office
as the victim, and if the messages are copied to most of the senior management team, the cumulative
impact constitutes a case of bullying. For although it is fair to say that bullying in general is a matter of
impact and not intent (i.e. if I think I am being bullied then I am being bullied, irrespective of your
original intentions), the case in question relied on such techniques to such an extent that the victim
believed that nearly everyone had been turned against him through fear of the manager.
Increasing anxiety makes the victim feel isolated among his peers. Inadvertently (perhaps), the
bystanders of cyberbullying become perpetrators themselves, their collective misdemeanours
compounded of ignorance and inaction. Before long it is entirely plausible that these accessories will
enjoy the scapegoating of one individual as much as the originator of the offensive material does. The
fact that scapegoating is generally interpreted as a defence against psychological distress in groups is
therefore pertinent: in a failing team in a ‘poisoned department’, the respite from group anxiety takes
the form of making fun of a figure of fresh hatred, whether or not he has done anything to upset
anybody. His perceived tolerance of the accumulated ‘jokes’ and rebukes via email (for example) are
sufficient provocation to continue. As Wilfred Bion (1961) writes in Experiences in Groups: ‘there is
no way in which the individual can, in a group, “do nothing” – not even by doing nothing’ (p118).
This is one way of paraphrasing part of Sigmund Freud’s Group Psychology and the Analysis of the
Ego (1921), in which Freud explores group formation and the giving up of individual ego ideals for the
group ideal. With reference to the emotional homogeneity of cyberbullying, one might argue that
something analogous is taking place. Referencing Bion himself, in ‘The unconscious at work in groups
and teams’ (1994), Jon Stokes describes such a ‘basic assumption dependency’ team (Bion 1961)
perfectly: ‘there is little capability to bear frustration… and quick solutions are favoured… members
have lost their capacity to stay in touch with reality and its demands. Other external realities are also
ignored or denied… instead of seeking information, the group closes itself off from the outside world
and retreats into paranoia. A questioning attitude is impossible; any who dare to do so are regarded
as either foolish, mad or heretical’ (p22-23). Thus the question might be begged at this point: is there
actually a need for bullying in the workplace? Perhaps the fact that cyberbullying becomes pandemic
says more about the group than it does the bully (or the victim). In this case, does cyberbullying serve
a function? Perhaps it helps contain workplace anxiety as well as create it? Even if these hypotheses
are correct, however, they do not help the victims. And this being said, this is a good point to
introduce Rob, who is the victim in the following case.
2. The wolf who cried boy?
Rob is a thirty-five year old professional man who was subjected to eighteen months of systematic
workplace bullying at an institution for further education. Before the bullying began, he had spent a
contented year at the Institution; the change in the working dynamics between Rob and his older
female manager changed gradually, from positive to negative, and followed no impetus of which Rob
is aware. Not once in the past had Rob experienced problems with a manager. Rob is a hard worker
and prides himself of being thought of favourably in the workplace; and while the manager had a
reputation of being troubled by the greater intelligence of others, Rob had always assumed that
although he was better educated, he and his manager had had a reasonable working relationship.
However, as time progressed, the feedback that Rob received from his manager was increasingly
474

David Mathew
negative, to the point that not a single thing that Rob completed remained unscathed from the
manager’s weak editing skills; and the shift in his manager’s attitudes towards him were confusing.
Although the two parties shared an office of twenty employees, and sat no more than five metres from
one another, the manager stopped talking to Rob directly, even in team meetings, unless she
specifically had to. If Rob asked a question of the manager while in company, the manager either
brushed it away or immediately asked Rob a question in return, sometimes the very same question.
The manager made fun of Rob when the chance arose, for example referring to his childless status
with the comment: ‘He hasn’t even started yet’ when she was discussing children with another
manager… Rob became nervous of approaching the manager’s desk: the manager would sometimes
make him wait a minute while finishing off an email or a piece of work, pretending that it was
inconceivable that she could be interuupted at this crucial moment, even though others had
interrupted her in the previous minute. Rob was the only member of the team not to be invited to the
pub for occasional drinks after work, or to the Christmas dinner; the only member of the team not to
be invited to share an infrequent lunch together in the canteen.
When the manager stopped addressing Rob directly and started using email for every
communication, Rob was baffled and started to feel anxious. At this point it did not occur to him that
this constituted cyberbullying, but he was aware that it was peculiar behaviour (by anyone’s
standards) and that it was behaviour that had been reserved for him alone. Before long, every
message that the manager sent to Rob was also copied to most of the senior management team;
what was worse than the feeling that this action created that he was being ‘watched’ for reasons that
he did not understand was the sensation that he experienced that the contents of the messages were
becoming increasingly (and deliberately) unclear. Rob believes that the emails were written in such a
vague way that he would have no choice but to ask for clarification of the task that he had been given,
or even of the topic in question. The clarification would then be seen in any subsequent emails by the
same people that were observing the proceedings.
After months of this sort of intensive scrutiny, interspersed by entire weeks in which the manager
refused to communicate with Rob in any fashion at all, leaving several of his projects in the air, the
resultant effect on Rob’s nerves and emotional condition was severe. In the meantime, Rob’s
applications for the same training that other members had been accepted for were turned down;
although other team members were allowed flexible working conditions, Rob was told that he must
report in at 9 a.m. and finish at 5 p.m. – even though the department had long since had a guideline of
‘early in, early home’ or ‘late in, late to leave’ for people with external responsibilities. Up to this point,
Rob had always favoured an arrival at work an hour earlier than his colleagues as he had a long
commute home. With this new ‘rule’ that was imposed only on him, his working day stretched by an
extra three hours as he was no longer able to catch his usual train home and had to wait some time
for the next one. After ten months of bullying, Rob went off sick.
The treatment drove him to a recourse to medication and a prolonged spell away from work (during
which the subject saw more than one message that confirmed that he would not be returning to work,
even though he had stated nothing for the kind). He admits to feeling anxious that the department was
discussing him; unbeknownst to Rob, his colleagues had been told by his manager not to ‘bother’ him
with any messages of goodwill or any Get Well Soon cards. A regular appointment with a counselling
service was swiftly arranged via Rob’s G.P. The counsellor, who had worked for this particular NHS
Trust, later stated that Rob’s was one of the worst examples of bullying that she had ever seen. And
yet, throughout all of this, Rob clung to the vanishing hope that things would eventually get better
without any form of legal intervention.
It was around this time that Rob started getting texts on his mobile phone from a number that he did
not recognise. The messages were unambiguous but nevertheless perplexing: ‘Failure’, ‘Lonely?’,
‘What did you want NOT to talk to me about?’ At first he deleted them, still attempting to convince
himself that he was looking at the whole matter incorrectly, and that things would get better soon. He
put the texts down to a simple coincidence, and his manager’s former and current silence down to her
own pressures in the workplace. (Not once did the manager call Rob to ask after his welfare. He was
off sick for nearly a third of a year.) As the texts kept coming, sometimes after midnight, Rob began to
suspect that not all was well psychologically and mentally with his manager, and though he knew that
steps can be taken to trace a mobile owner’s details, he also knew that the mobile operator can only
475

David Mathew
disclose this information to the police, and Rob was reluctant to involve the police. He now wishes
that he had involved the police, and much earlier on.
It is somewhat ironic that a piece of work that Rob had authored earlier, during the year in which
everyone seemed happy, had been on the subject of bullying. Thus it was that Rob had some
recourse to information that he never thought that he would use practically; now that his health had
been affected (headaches, anxiety attacks, vomiting) he looked back on this work and at some of the
references therein. He was alarmed to note how little of what he’d written could be used in his own
case. For example, victims of cyberabuse are advised to block emails or remove names from
contacts; but how is this possible if one’s bully is also one’s manager? Similarly, when a YouTube
‘joke’ had been sent to everyone in the department as a link, with the single line in the body of the
email that read ‘One for Rob…’ and the film contained content that addressed something personal to
Rob, the victim was unsure of how to proceed. After all, everyone had been sent the joke film. How
was he meant to say that he alone had found it offensive without appearing as a crybaby?
Eventually Rob screwed up the courage to approach a Union representative. Next, on his eventual
return to work, he reported the cyberbullying to his manager’s manager, even though he suspected
that the latter would be futile. So it proved: the manager’s manager merely told Rob that his
perception was wrong and that what he thought was happening was not happening at all. It was all in
his mind: his tormentor didn't mean him any harm, and so on. Rob was flatly informed that
cyberbullying did not exist in the Institution for the simple reason that it was not acceptable. This was
the extent of the mitigation: it does not exist because we say that it not allowed to exist. (Surely this is
the workplace equivalent of a child holding his hands over his own eyes and assuming that no one
can see him because he is now in darkness. It is workplace equivalent of putting fingers in one's ears
and chanting: Can't here you! Can't hear you!)
There followed months of meetings (while the counselling continued) and Rob was eventually forced
out of the Institution with a payment of several thousands of pounds and a copy of a contract that he
was obliged to sign that said that he accepted this payment as a redundancy package. Needless to
say, the contract said nothing about bullying: Rob, or rather the man whose real name I must not
disclose, had effectively been gagged on the subject, legally, for ever. If the title of one of Slavoj
Žižek’s many papers – namely, ‘Is this digital democracy, or a new tyranny of cyberspace?' – was not
applicable enough in this instance, then let us take a glance at what the author has to say. ‘Our social
identity, the person we assume to be in our social intercourse, is already a "mask" that involves the
repression of our inadmissible impulses,’ Žižek’ writes. ‘The fact that I perceive my virtual self-image
as mere play thus allows me to suspend the usual hindrances which prevent me from realising my
"dark half" in real life. My electronic id is given wing’ (web article).
3. Unpatrolled boundaries
When cases of cyberbullying are of a severity to make the news (ref the Houghton case,
NetFamilyNews 2009) there is often a reference to the anonymity (or attempted anonymity) or the
perpetrator. The idea is for the audience to recognise the sly and insidious fashion with which the
bully reached his/her victim. Arguably, however, the case where the bully is completely upfront and
'open' is worse. In cases such as this, it becomes a matter of misdirection and stealth. The bully
knows enough about the victim to be able to hurt him by using languages/images that might be
regarded (by others outside the dualogue) as innocuous enough. This serves, of course, to make
matters worse. If the bully is hurt by words/images that might be innocent in an uncontextualised
situation, the bully is thought to be 'too sensitive' or 'not tough enough'. It represents another slur on
the bully's character, and now a wider audience is allowed to join in.
The workplace bully is an abuser of power, whether this power is actual (where the bully might have
influence over someone’s career, for example) or supposed (where the bullying has been left alone
because it is easier to accept than the challenge to its existence would be). Either way, relationships
in the workplace should be governed by the same sort of boundaries that survive in any other
relationship; and where cyberbullying has been ignored we see a clear example of unpatrolled
boundaries. No one is checking. According to Wenger (2000), members of a community are bound
together by a collective understanding of what their community is about; they hold each other
accountable to his joint enterprise. In the absence of a possibility to maim or kill physically, the
workplace bully’s only real ‘weapon’ is the use of technology, and the only real target the victim's
476

David Mathew
emotional world: a world that has opened up for everybody who is interested in exploring it, since the
mass market evolution of the Internet and of social networking in cyberspace.
Social control theory would suggest that instead of looking forfactors that make people reveal criminal
tendencies, we should try to explain why people do not become criminal. In Causes of Delinquency
(1969),Travis Hirschi revealed four personal characteristics – attachment to others, a belief in the
moral validity of rules, the commitment to achievement, and an involvement in conventional activities
– the possession of which might keep a person away from criminal intent. But what if the attachment
to others is the attachment to likeminded individuals in a bullying group? What if the ‘rules’ are not
understood by everybody (in the way that the victim will fail to understand them)? What if
‘achievement’ is only scored via the downfall of others (or one in particular)? And so on. There is no
doubt that Hirschi’s research has been studied for decades, but there is equally no doubt that he was
writing before the advent of the Internet, and that times change. Violence has changed for ever.
4. Future dissonance
Neimeyer (1993) and Steenbarger (1991 ) suggest that anyone is capable of creating a variable future
based on one’s own history and experience, one’s perceptions of the moment, and personal values.
So a pertinent question, perhaps, is not 'Whence Cyberbullying?' or even 'Why Cyberbullying?':
perhaps we should be asking: 'Where Next, Cyberbullying?' When the achievements have already run
to financial disaster due to staff dissatisfaction, and to employee suicide – even murder – what is left
for cyberbullying to work towards, and how will it set about this challenge?
A cyberbully who exhibits the efficacy of the one highlighted in this paper does not arrive fully formed:
effective cyberbullying is an iterative approach. The treading of a fine line between cruelty-and-notgetting-caught
and misfired-slander-and-getting-caught takes practice and resolve. The individual
concerned has almost certainly executed similar hate crimes in the past; it is likely that he or she will
execute more in the future, unless the organisation takes a robust stand against the habit. Quite
possibly the bully was a victim of bullying in the past, or began the practice as early as during the
school years. The bully uses the psychoanalytic device of transference – the replacement of early
feelings for someone onto someone new, years later, and cyberbullying thereby becomes a bully’s
defence against paranoid anxieties. Or to put it another way, in the findings of Loeber and Hay
(1997), there is much to learn about contemporary aggression in studies of the aggressor’s
adolescence. As Steiner (1993, p4) would have it: ‘Traumatic experiences with violence or neglect in
the environment leads to the internalisation of violent disturbed objects which at the same time serve
as suitable receptacles for the projection of the individual's own destructiveness.’
While we may hypothesise that cyberbullying constitutes a resolution of issues of physical space, or a
delineation of one's struggle with personal boundaries, we must not forget (or ignore) the painful truth
that some people simply enjoy being unpleasant. It might not have been a uniform that was worn (in
this case), and the 'weapon' might not have fired shells, but cyberbullying is every bit as distressing as
a case of (say) a sergeant showing up a constable's faults in front of others. The context has
changed; the consequences and the motives have not. What is certain is that the bully will continue
to derive personal satisfaction (and arguably ersatz sexual gratificationo) from the attempts to wound
and vilify unless the emotional imbalances and lacks that he or she endures are directly addressed.
The bully fears that someone else can function better or is more capable of carrying out the job at
hand; the quest for power is the by-product of individual impotence, and the incompetence that the
bully shows in the workplace is often camouflaged by the actions that are aimed at the victim. Without
a change in a failing department that addresses these shortcomings head on, the bully thrives and
continues with identical behaviours, unable to change. The analogy in the following should be simple
enough to grasp, from Ricky Emanuel’s seminal paper on psychological retreats, 'A Void':
‘The baboon smells the watermelon seeds and puts its hand through the hole. By
grabbing the seeds in its fist, it cannot get its fist out of the hole and remains trapped
there until the Bushmen come and capture it. It cannot think that releasing the
attachment to the seeds will result in its freedom. It is something like this that I am
discussing, in relation to becoming fixed or attached to one’s sense of identity or even
true self’ (pX).
However, we must not disregard the human being's simple reluctance to let go of a situation – and an
identity – once it has been established, regardless of how unpleasant this situation and identity might
be. In his seminal paper on the psychoanalytic 'Void', Ricky Emanuel ( explains as follows: 'The
477

David Mathew
baboon smells the watermelon seeds and puts its hand through the hole. By grabbing the seeds in its
fist, it cannot get its fist out of the hole and remains trapped there until the Bushmen come and
capture it. It cannot think that releasing the attachment to the seeds will result in its freedom. It is
something like this that I am discussing, in relation to becoming fixed or attached to one’s sense of
identity or even true self.’
At the start of this paper I wrote that it 'might come as a shock' to learn that there were institutions in
which cyberbullying was tacitly endorsed. Please note my use of the word might for I used it
deliberately. What I could have said was 'It will come as a shock' but when I typed this I had a failure
of confidence in my sentence. Would it come as a shock, I wondered, to consider cyberbullying in the
workplace? I wasn't sure so I opted for might. I am still not sure, the truth be told; but if what has been
said has not come as any sort of shock (or shall we compromise with 'unpleasant surprise'?) then
questions are inevitably begged. The logical sequiturs are at least twofold. If information such as this
does not come as a shock, first why does it not come as a shock; and second, where have you seen
such behaviour yourself? A third question, on which I would like to conclude, is this:
What did you do about it?
References
Bion, W.R. (1961) Experiences in Groups. East Sussex/New York: Routledge.
Daily Mash (2011) Mob seeks new thing to be angry about. http://www.thedailymash.co.uk/news/society/mobseeks-new-thing-to-be-angry-about-201103303673/.
Accessed on 30 March 2011.
Emanuel, R. (2001) ‘A Void – An Exploration of Defences Against Sensing Nothingness’ in Int. J. Psychoanal. 82,
1069.
Freud, S. (1921) Group Psychology and the Analysis of the Ego. SE 18. The Standard Edition of the Complete
Psychological Works of Sigmund Freud. Trans. James Strachey. London: Hogarth, 1953-74.
Hirschi, T. (1969) Causes of delinquency.Berkeley: Unversity of California Press.
Loeber, R., & Hay, D. (1997). Key issues in the development of aggression and violence from childhood to early
adulthood. Annual Review of Psychology, 48, 371-410.
Neimeyer, R.A. (1993) , An appraisal of constructivist psychotherapies. Journal of Consulting and Clinical
Psychology , 61(2), 221-234.
NetFamilyNews. (2009) First UK teen to be jailed for cyberbullying. http://www.netfamilynews.org/?cat=5526.
Accessed 31 May 2011.
Olweus, D. (1993) Bully/victim problems among schoolchildren: Long-term consequences and an effective
intervention program. In S. Hodgins, Mental disorder and crime (pp. 317-349). Thousand Oaks, CA: Sage.
Parkes, K. R. (1991) Locus of control as moderator explanation for additive versus interactive findings in the
demand-discretion model of work stress? British Journal of Psychology, 82, 291 - 312.
Salmivalli, C., Lagerspetz, K., Bjorkqvist, K., Osterman, K., & Kaukiainen, A. (1996). Bullying as a group process:
Participant roles and their relations to social status within the group. Aggressive Behavior, 22, 1–15.
Steenbarger, B.N. (1991) . All the world is not a stage. Journal of Counselling , 70(Nov-Dec), 228-296.
Steiner, J. (1993). Psychic Retreats. London: Routledge.
Steiner, J. (2011) Seeing and Being Seen. East Sussex/New York: Routledge.
Stokes, J. (1994) The unconscious at work in groups and teams. In Obholzer, A. Roberts, V.Z. The Unconscious
at Work. East Sussex/New York: Routledge, 1994.
Symington, N. (1994) Emotion and Spirit. London: Karnac.
Wenger, E. (2000) Communities of Practice and Social Learning Systems. Organization 7(2): 225-246
Žižek, S. (2006) Is this digital democracy, or a new tyranny of cyberspace? In: Guardian. December 30, 2006.
478

Learning in Smart Environments – From Here to There
Peter Mikulecky
University of Hradec Kralove, Hradec Kralove, Czech Republic
peter.mikulecky@uhk.cz
Abstract: Recently, ten years after the Information Society Technologies Advisory Group (ISTAG) Report
(Ducatel et al 2001) was issued with several visions of smart environments for various purposes, we could
already evaluate the results achieved. One of the original scenarios by ISTAG was the Scenario 4: Annette and
Solomon in the Ambient for Social Learning. The vision presented in the scenario started an intensive research
resulting in a number of projects aiming at design and utilization of various smart learning environments. In the
paper, which is based on our ongoing research, we wish to remind several recent important attempts in the area
of smart environments designed for learning.
Keywords: smart environments, ambient intelligence, ubiquitous learning, scenarios
1. Introduction
The famous ISTAG Report started from 2001 a decade of various research initiatives covered by the
recently very successful area of Ambient Intelligence (AmI). The AmI vision can be applied to a
number of diverse application environments, varying from homes, offices, or cars to homes for the
elderly and hospitals. It refers to a wide range of human emotional and intellectual needs, from
comfort, pleasure, and entertainment to safety, security, and health. AmI scenarios provide examples
of such environments. They are included in the ISTAG report (Ducatel et al 2001) that introduced four
scenarios of the future development in information society.
One of the ISTAG scenarios – Scenario 4: Annette and Solomon in the Ambient for Social Learning –
brought a vision of a smart learning environment, based on a position that learning is a social
process. According to the original description in the ISTAG report, the Ambient for Social Learning
(ASL) is an environment that supports and upgrades the roles of all the actors in the learning process,
starting with the roles of the mentor and the students as most concerned parties. The systems that
make up the ASL are capable of creating challenging and interacting learning situations that are
codesigned by the mentor and students in real-time. Students are important producers of learning
material and create input for the learning ‘situations’ of others. In other words, the ASL is both an
environment for generating new knowledge for learning and a ‘place’ for learning about learning
(Ducatel et al 2001, p. 7).
Recently, ten years after the ISTAG Report was issued with several similar visions of smart
environments for various purposes, we could already evaluate the results achieved. An intensive
research has been provided since that time with a number of interesting results. However, recent
experience with new achievements in the area of smart classrooms and other similar smart
environments seems not to be always satisfactory.
Smart environments for learning, as a result of intensive research in the area of Ambient Intelligence,
deserve also attention of the large community oriented on e-learning and technology enhanced
learning. Smart environments could be naturally considered a new degree of computer enhanced
learning, with a number of new facilities. The area of Ambient Intelligence can be studied from several
perspectives. As (Bureš, Čech, and Mls 2009) pointed out, besides its technological perspective,
social perspective, or ethical perspective, we can also identify an educational perspective. The
educational perspective deals with problems and challenges related to proper education in relevant
AmI areas.
In the paper we intend to map the recent state of the art in the area of smart environments designed
for learning. On the basis of our ongoing research we wish to present a number of open problems and
possible ways to their solutions, as well as a critical view on the research of smart environments for
learning.
2. Context-aware ubiquitous learning
The context aware and ubiquitous learning as being naturally close to the educational perspective of
AmI as well as to the idea of smart learning environments, was defined and studied by several
authors. (Winters et al. 2005) pointed out that ubiquitous computing has tremendous potential for
479

Peter Mikulecky
framing learning, particularly in informal and socially constructed contexts. To reach this potential
according to (Winters et al. 2005) it is necessary for the current desktop-focus development of
technology in education to be challenged through the design, development and testing of new
ubiquitous prototypes for learning.
For instance, according to (Yang et al 2008), context-aware and ubiquitous learning is a computer
supported learning paradigm for identifying learners' surrounding context and social situation to
provide integrated, interoperable, pervasive, and seamless learning experiences. The objective of
context-aware and ubiquitous learning is to enhance Web-based learning a step further from learning
at anytime and anywhere to learning enabled at the right time and the right place using right
resources and right collaborators. Alternatively, according to (Hwang et al 2009) context-aware
ubiquitous learning is an innovative approach that integrates wireless, mobile, and context-awareness
technologies to detect the situation of learners in the real world and provide adaptive support or
guidance accordingly.
Already mentioned authors (Yang et al 2008) summarize the characteristics of context-aware and
ubiquitous learning in the following eight aspects: mobility, location awareness, interoperability,
seamlessness, situation awareness, social awareness, adaptability, and pervasiveness. More detailed
descriptions of these aspects are as follows:
Mobility: The continuousness of computing while learners move from one position to another.
Location awareness: The identification of learners’ locations.
Interoperability: The interoperable operation between different standards of learning resources,
services, and platforms.
Seamlessness: The provision of everlasting service sessions under any connection with any
device.
Situation awareness: The detection of learners’ various situated scenarios, and the knowledge of
what learners are doing with whom at what time and where.
Social awareness: The awareness of learners’ social relationship, including what do they know?
What are they doing at a moment? What are their knowledge competence and social familiarity?
Adaptability: The adjustability of learning materials and services depending on learners’
accessibility, preferences, and need at a moment.
Pervasiveness: The provision of intuitive and transparent way of accessing learning materials and
services, predicting what learners need before their explicit expressions.
On the other hand, (Bomsdorf 2005) considered ubiquitous learning as the next step in performing elearning
and by some authors it was expected to lead to an educational paradigm shift, or to new
ways of learning. The potential of ubiquitous learning results from the enhanced possibilities of
accessing learning content and computer-supported collaborative learning environments at the right
time, at the right place, and in the right form. Furthermore, and this is close to the ideas of AmI
presented by (Ducalet et al. 2001), it enables seamless combination of virtual environments and
physical spaces. Ubiquitous computing leads to ubiquitous learning that allows embedding of
individual learning activities into everyday life. As Bomsdorf (2005) mentioned, early applications of
ubiquitous learning were tourist and museum guides, by means of which a visitor gets information
based on his current position, e.g., facts about a painting he is standing in front of. In most ubiquitous
learning approaches the physical environment is directly (semantically) related to learning objectives
and activities (e.g., the museum visitor gets information each time based on his current location). As it
was already stressed, the fundamental issue in a ubiquitous learning environment is how to provide
learners with the right material at the right time in the right way. Context aware adaptation is therefore
indispensable to all kinds of learning activities in ubiquitous learning environments.
In addition to the context-aware ubiquitous learning characteristics by (Yang et al 2008) mentioned
earlier in this chapter, (Hwang et al 2008) formulated the potential criteria of a context-aware
ubiquitous learning environment as follows:
It is context-aware; that is, the learner’s situation or the situation of the real-world environment in
which the learner is located can be sensed, implying that the system is able to conduct the
learning activities in the real world.
480

Peter Mikulecky
It is able to offer more adaptive supports to the learners by taking into account their learning
behaviours and contexts in both the cyber world and the real world.
It can actively provide personalized supports or hints to the learners in the right way, in the right
place, and at the right time, based on the personal and environmental contexts in the real world,
as well as the profile and learning portfolio of the learner.
It enables seamless learning from place to place within the predefined area.
It is able to adapt the subject content to meet the functions of various mobile devices.
As (Hwang et al 2008) pointed out, researchers have different views of the term “ubiquitous learning”
till now. A popular view is “anywhere and anytime learning”, which is a very broad-sense definition of
ubiquitous learning. With this definition, any learning environment that allows students to access
learning content in any location at any time can be called a ubiquitous learning environment, no
matter whether wireless communications or mobile devices are employed or not. From this viewpoint,
the mobile learning environment which allows students to access learning content via mobile devices
with wireless communications is a special case of the broad-sense definition of ubiquitous learning.
However, if we took into account the ISTAG scenario Annette and Solomon in the Ambient for Social
Learning that could serve as an ideal case for a smart learning environment, which was undoubtedly
context-aware as well as ubiquitous at the same time, the popular view of “anywhere and anytime
learning” should be considered as impractically broad. As (ElBishouty et al 2010) pointed out, the
challenge in the information-rich world is not to provide information at anytime and at anywhere but to
say the right thing at the right time in the right way to the right person.
This approach is supported also by (Yang 2006, p. 188), stating that a ubiquitous learning
environment provides an interoperable, pervasive, and seamless learning architecture to connect,
integrate, and share three major dimensions of learning resources: learning collaborators, learning
contents, and learning services. Therefore ubiquitous learning is characterized by providing intuitive
ways for identifying right learning collaborators, right learning contents and right learning services in
the right place at the right time.
The main characteristics of ubiquitous learning are permanency, accessibility, immediacy,
interactivity, and situating of instructional activities. (ElBishouty et al 2010) mentioned that the
ubiquitous environment should be personalized according to the learner’s situation. They defined
personalization as the way in which information and services can be tailored in a specific way to
match the unique and specific needs of an individual user. While a learner is doing learning task or
activity, she usually looks for some knowledge. In a ubiquitous learning environment, as (ElBishouty
et al. 2010) stressed, it is very difficult for a learner to know who has this knowledge even though they
are at the same place. In this case, the learner needs to be aware of the other learners’ interests that
match his request.
There are only a few studies that have attempted to induce the educational affordances of contextaware
ubiquitous learning environment. (Liu and Chu 2009) devoted attention to the problem of what
educational affordances can be provided by a context-aware ubiquitous learning environment. They
proposed a system named EULER that can provide eight educational affordances: knowledge
construction, apply, synthesis, evaluation, interactivity, collaborative learning, game-based learning,
and context-aware learning. Moreover, they stressed that ubiquitous learning provides context-aware
information and self-learning opportunities for learners. Therefore, it not only enables students to
achieve learning goals anytime and anywhere, it is also cultivating their ability to explore new
knowledge and solve problems. This should be considered to be one of most important characteristics
of ubiquitous learning.
Interesting ideas about learning in smart environments can be found in (Winters et al. 2005).
According to them, learning is no longer viewed only as a form of delivered instruction, undertaken
within the confines of traditional educational environments. It is now understood as a social process
that happens at a time and place of the learner's choosing, continuing throughout one’s life. It is
collaborative, evolving and informed by a process of self-paced development.
(Winters et al. 2005) define a smart environment as any space where ubiquitous technology informs
the learning process in an unobtrusive, social or collaborative manner. Thus a smart environment can
be an ‘aware’ room or building, capable of understanding something about the context of its
481

Peter Mikulecky
inhabitants or workers; it can be a digitally enhanced outdoor space – park, cityscape or rural
environment; or it can be the environment created when peoples’ meetings or interactions are
augmented by wearable devices. These ideas are very close to that of original Scenario 4: Annette
and Solomon in the Ambient for Social Learning from the ISTAG Report (Ducatel et al 2001).
3. Examples of ubiquitous learning environments
There is a number of interesting attempts to propose ubiquitous learning environments of the type
discussed in previous chapters. For instance (Chen, Kinshuk, Wei, and Yang 2008) proposed a
wireless communication based network called GroupNet. It is a Group Area Network that is proposed
on the basis of P2P wireless network connection to fit with this type of mobile scenario. GroupNet
consists of a set of interconnecting handheld devices with wireless access, carried by a group of
people within proximity.
GroupNet works with wireless modules of the handheld devices to achieve the best of ubiquitous
networking. Ubiquitous networks enable secure access to data from everywhere on multiple devices
to achieve the ubiquitous learning environment. The ubiquitous learning environment can connect,
integrate and share learning resources in the right place at the right time by an interoperable,
pervasive and seamless learning architecture. P2P networking used in GroupNet is one approach of
creating ubiquitous networks for supporting ubiquitous learning.
Another interesting proposal of intelligent learning environments published (Mhiri and Ratté 2009).
They proposed an intelligent environment for human learning (the AARTIC project) that assists
software engineering students in their assignments. The system resolve real problems: for the
students, too much time to complete each assignment, for the teacher, too many students to offer any
personalized help. Moreover, because students find themselves in a precarious situation (the
concepts are new and complex), they rely on old primary reflexes: zero collaboration or planification.
The proposed system aims to help the student in the understanding of concepts by suggesting
examples. Two pedagogical agents compose the adaptive aspect of the system. The first monitors
students’ activities in the environment. The second allows the teacher to observe the performance of
each student and of the class as a whole. The environment also emphasizes collaboration.
(ElBishouty et al 2010) present a model of personalized collaborative ubiquitous learning environment
in order to support learners doing learning tasks or activities. It utilizes RFID tags to detect the
surrounding physical objects and provides personalized recommendations based on the detected
objects. It provides the learner with social knowledge awareness map for the peer helpers. The map
visualizes the learners’ surrounding environmental objects, peer helpers and the strength of the
relation in the social network perspective. The learner can contact, interact, and collaborate with the
peer helpers to address the learning goal.
Another example of a different approach towards intelligent learning environments presented (Winters
et al. 2005). They defined an intelligent environment as any space where ubiquitous technology
informs the learning process in an unobtrusive, social or collaborative manner. In their paper, two
ubiquitous devices for use in such an environment were presented: the Experience Recorder and the
iBand. The Experience Recorder is an embedded system that records the paths travelled by users –
i.e. trails – in a particular place, for example at a museum or trade fair. It then recreates this visit in
digital form, for example as a personalised website, enhanced for learning. The iBand is a wearable
bracelet-like device that exchanges information about its users and their relationships during a
handshake. (Winters et al 2005) stressed that the challenge of ubiquitous computing was to design
and build systems for augmenting human capabilities rather than to replace them. In the context of
learning, any ubiquitous computing tool cannot be viewed as deskilling the user. It must encourage
skills development in a manner in which the learner is comfortable and engaged with. We cannot
agree more.
The last example of a smart environment is the system ISABEL described in (Garruzzo et al., 2007).
The ISABEL is a new sophisticated multi-agent e-learning system, where the basic idea is in
partitioning the students in clusters of students that have similar profiles, where each cluster is
managed by a tutor agent. When a student visits an e-learning site using a given device (say, a
notebook, or a smart phone), a teacher agent associated with the site collaborates with some tutor
agents associated with the student, in order to provide him with useful recommendations. Generally,
these systems use a profile of the student to represents his interests and preferences, and often
482

Peter Mikulecky
exploit software agents in order to construct such a profile. More in particular, each student is
associated to a software agent which monitors his Web activities, and when the student accesses an
e-learning site, his agent exploits the student’s profile interacting with the site. In this interaction, the
site can use both content-based and collaborative filtering techniques to provide recommendations to
the student’s agent by adapting the site presentation.
4. On the way to the Annette and Solomon scenario
Quoting (Weiser 1991), the most profound technologies are those that disappear. They weave
themselves into the fabric of everyday life until they are indistinguishable from it. As a matter of fact,
ubiquitous learning could be a good example of such a technology.
Without any doubts any AmI application bringing new ideas and approaches into educational process
at every level of education deserves a special attention. One of educational applications is the Smart
Classroom project (Shi et al. 2010). It aims to build a real-time interactive classroom with teleeducation
experience by bringing pervasive computing technologies into traditional distance learning.
The goal of Smart Classroom project is to narrow the gap between the teacher’s experience in teleeducation
and that in the traditional classroom education, by means of integrating these two currently
separated education environments together. The used approach was to move the user interface of a
real-time tele-education system from the desktop into the 3D space of an augmented classroom
(called Smart Classroom) so that in this classroom the teacher could interact with the remote students
with multiple natural modalities just like interacting with the local students.
A more general overview of the AmI possibilities in education brings our recently published book
chapter (Mikulecky et al. 2011). The objective of the paper is to identify and analyze key aspects and
possibilities of AmI applications in educational processes and institutions (universities), as well as to
present a couple of possible visions for these applications. A number of related problems are
discussed there as well, namely agent-based AmI application architectures. Results of a brief survey
among optional users of these applications are presented as well. The conclusion of this research
was that introduction of Ambient Intelligence in educational institutions is possible and can bring us
new experiences utilizable in further development of AmI applications.
The scenario Annette and Solomon was considered in the time of its origin as a long term future.
However, we presented a lot of examples and arguments in favour of the idea, that the scenario can
be nowadays implemented, as the relevant technology has matured enough.
5. Conclusions
Taking advantage of AmI technologies educational institutions can become real “knowledge centres”,
which are formed by intelligent applications, devices, or technologies. Such a technologically intensive
system, whose significant attribute is an intelligent interaction with its users, can support following
activities:
Plan classroom instructions in a way that increases individual attention in critical areas,
Identifying training needs for teachers,
Aim to develop competence (opposed to awarding certificates, which cannot be an end in itself),
Enable students to learn in a way that promotes interest in learning & continue learning in
environments other than the institute which imparts formal education etc.
The idea of enhancing educational environment by suitably chosen AmI solutions has been already
sketched e.g. in (Mikulecky 2009) and for the case of e-learning education more elaborated in a
number of sources we mentioned (see e.g. (Bomsdorf 2005), (Bureš, Čech, and Mls 2009), (Mhiri and
Ratté 2009) and others). In this paper, we presented a whole range (from here to there) examples of
smart environments, which certainly can be implemented and used by any modern university which
uses nowadays information and communication technology. The basic principle of any further
development lies, apart of its technological basis, also in understanding and using new approaches,
new ways how these technologies can be utilized. We are deeply convinced, that AmI solutions are,
among other recent approaches, the right way of achieving a new quality of educational process,
which will be both motivating and effective for anyone.
483

Acknowledgements
Peter Mikulecky
The research has been partially supported by the Czech Grant Foundation, grant No. P403/10/1310.
References
Bomsdorf, B. (2005) “Adaptation of Learning Spaces: Supporting Ubiquitous Learning in Higher Distance
Education.”, Mobile Computing and Ambient Intelligence: The Challenge of Multimedia, Dagstuhl Seminar
Proc. 05181, available on http://drops.dagstuhl.de/opus/volltexte/2005/371.
Bureš, V., Čech, P., Mls, K. (2009) “Educational Possibilities in the Development of the Ambient Intelligence
Concept”, Problems of Education in the 21 st Century, Vol. 13, No. 13, pp 25-31.
Chen, N.-S., Kinshuk, Wei, C.-W., & Yang, S. J. H. (2008). „Designing a Self-contained Group Area Network for
Ubiquitous Learning”. Educational Technology & Society, Vol. 11, No. 2, pp 16-26.
Costello, R., Mundy, D.P. (2009) “The Adaptive Intelligent Personalised Learning Environment”, IEEE ICALT
2009, IEEE Press, pp 606-610.
Ducatel, K. et al. (2001) Scenarios for Ambient Intelligence in 2010: Final report. Seville, Spain, Information
Society Technologies Advisory Group. Retrieved June 7, 2011, from
ftp://ftp.cordis.lu/pub/ist/docs/istagscenarios2010.pdf.
El-Bishouty, M. M., Ogata, H., Rahman, S., & Yano, Y. (2010). “Social Knowledge Awareness Map for Computer
Supported Ubiquitous Learning Environment”. Educational Technology & Society, Vol. 13, No. 4, pp 27–37.
Garruzzo, S., Rosaci, D., Sarné, G.M.L. (2007) ISABEL: A Multi Agent e-Learning System That Supports Multiple
Devices. In: IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IEEE, New York, pp.
485-488.
Hwang, G.-J., Tsai, C.-C., & Yang, S. J. H. (2008) “Criteria, Strategies and Research Issues of Context-Aware
Ubiquitous Learning”. Educational Technology & Society, Vol. 11, No. 2, pp 81-91.
Hwang, G.-J., Yang, T.-C., Tsai, C.-C., Yang, S.J.H. (2009) “A context-aware ubiquitous learning environment for
conducting complex science experiments”. Computers & Education, Vol. 53, pp 402-413.
Liu, T.-Y., Chu, Y.-L. (2009) “Educational Affordances of Ubiquitous Learning Environment for Natural Science
Learning”. IEEE ICALT 2009, IEEE Press, pp 290-292.
Mhiri, F., Ratté, S. (2009) “AARTIC: Development of an Intelligent Environment of Human Learning”. ITiCSE 09,
Paris, ACM Press, p 359.
Mikulecký, P. (2009) “Remarks on Ubiquitous Intelligent Supportive Spaces”. Recent Advances in Applied
Mathematics and Computational and Information Sciences, Proc. of the 15th American Conference on
Applied Mathematics and Proc. of the International Conference on Comp. and Information Sciences (Eds.
Jegdic, K., Simeonov, P., Zafiris, V.), Vols. I and II, pp 523-528, WSEAS Press, Athens.
Mikulecký, P., Olševičová, K., Bureš, V., Mls, K. (2011) “Possibilities of Ambient Intelligence and Smart
Environments in Educational Institutions.” In: Handbook of Research on Ambient Intelligence and Smart
Environments: Trends and Perspectives (Mastrogiovanni, F., Nak-Young Chong, eds.), IGI Global, Chap.
29, Information Science Reference (IGI Global), pp 620-639.
Shi, Y., Qin, W., Suo, Y., Xiao, X. (2010) “Smart Classroom: Bringing Pervasive Computing into Distance
Learning”, In: Handbook of Ambient Intelligence and Smart Environments, Springer Science+Business
Media, pp. 881-910.
Weiser, M. (1991). “The computer for the twenty-first century”. Scientific American, September, 94-104.
Winters, N., Walker, K., Rousos, D. (2005) “Facilitating Learning in an Intelligent Environment”. The IEE
International Workshop on Intelligent Environments. Institute of Electrical Engineers, London, pp. 74-79.
Yang, S.J.H. (2006) “Context Aware Ubiquitous Learning Environments for Peer-to-Peer Collaborative Learning”.
Educational Technology & Society, Vol. 9, No. 2, pp 188-201.
Yang, S. J. H., Okamoto, T., & Tseng, S.-S. (2008). „Context-Aware and Ubiquitous Learning (Guest Editorial)”.
Educational Technology & Society, Vol. 11, No. 2, pp 1-2.
484

Using Courseware for More Than Courses: You May
Already Hold the Lease on a Versatile Virtual Meeting
Space
Karen Hughes Miller 1 and Linda Leake 2
1
University of Louisville School of Medicine, USA
2
University of Louisville Delphi Center for Teaching and Learning
Louisville Kentucky, USA
Karen.miller@louisville.edu
Laleak01@louisville.edu
Abstract: The University of Louisville (Kentucky, USA) is a large, urban institution on three separate campuses
that includes undergraduate, graduate, and professional schools such as education, business, medicine, law, and
engineering. The institution also offers staff/faculty development opportunities, corporate learning, and life-long
learning for the community as a whole. Blackboard is our online course platform and every section of every
course (face-to-face, online, and blended) is automatically issued a course site to be used at the discretion of the
instructor. A few years ago, those of us who teach online realized that the “organizations” option in Blackboard
had far more potential for student, faculty, and staff support than we were using. The “organizations” option
provides a virtual meeting space that can be used to facilitate discussions, share documents, administer events,
conduct quizzes and surveys, broadcast e-mails, link to media, wikis, and blogs, and track the type and
frequency of activities by each participant. While none of these capabilities are unique to BlackBoard, what is
unique is that in BlackBoard these capabilities are bundled so users have access to all capabilities in a
controlled (password protected) environment. The two critical differences between “organization” sites and
traditional “course” sites are (1) once an organization leader has been designated, the leader controls who may
log into the site and the role they play; and (2) organization sites are not time-bound by the academic calendar.
Admission to traditional course sites are limited by administrators to include only faculty assigned to the course
and students enrolled in the course and the site usually expires at the end of the term, semester, or quarter. In
Blackboard, organization sites have all of the functionality of course sites without the two major limitations;
therefore we could use our imaginations, and either apply the functions as they were intended or repurpose them
to meet other needs. The purpose of this paper is to share five brief case studies that illustrate the range of uses
to which we have applied the BlackBoard organizations option to support students, faculty, and staff in hopes
that these ideas may be transferable to other institutions. Each case shows at least one unique application.
Although there are a few technical limitations to Blackboard organization sites, in general the benefits outweigh
the limitations. We argue that when an institution uses fewer types of platforms (or other software packages);
they reduce training time for developers and users and maximize the value delivered by the learning platform
provider. Using a platform provided thru central administration also reduces costs to individual schools,
departments, and programs as they need not purchase additional software to support their unique needs.
Organizational learning also increases as departments learn from each other’s examples of how the
organizations option can be applied.
Keywords: non-traditional applications of learning platforms; student, faculty, staff support
1. Case Study #1: Supporting a large interdisciplinary student learning event
on a health sciences campus
As the population of the USA becomes more racially and ethnically diverse, students in the health
professions need more instruction to increase their cultural competency. Whether they plan to practice
medicine or dentistry, they will not be able to deliver successful healthcare if they do not understand
their patient’s cultural and/or religious frame-of-reference relating to health and wellness (Bonder,
Martin, and Miracle, 2001; Mott, 2003). For the past five years, a half-day workshop has been offered
on the University of Louisville Health Sciences Campus (U of L HSC) on Increasing Cultural
Competence. The workshop is designed much like a professional conference with plenary sessions
and breakout sessions that learners may select according to their interests. First year students from
medicine, dentistry, and dental hygiene along with first year dental residents and first year audiology
students are required to attend. To justify the instructional time and the budget dedicated to this
annual event, it has been important to measure learning outcomes. For the past several years,
students have completed pre- and post- attitude and knowledge surveys to ensure that they are
benefiting from the instruction.
A Blackboard organization is created each year to manage enrollment, session selection, learner
satisfaction, and share background information and bios of the various speakers. The process begins
485

Karen Hughes Miller and Linda Leake
with the Schools of Medicine and Dentistry providing the list of eligible students and their User IDs to
the event organizers. Students are then entered as “members” of the organization. (For some years,
this has been as many as 400 learners, but still seems to be within the capacity for a Blackboard
organization site to function well.) Using the site’s e-mail function, learners are contacted with an
invitation to log in and select their preferences of presentations and discussions. Reminder e-mails
are sent to ensure the presentation selections are made, and learners are assigned groups according
to the sessions they select. The conference agenda is posted as a printable document, as are print-ityourself
name badges, meal tickets, bus passes (when sessions are held off campus), and other
conference essentials. Some speakers also enjoy sharing their PowerPoints or links to their web sites
when the event has concluded so students can use it as an ongoing reference.
Originally, students who came prepared with all their materials pre-printed were invited to the “fast”
registration lines and students who came without were in the “slow” lines, but preparation seems to
have improved over the years. Currently students’ ID badges are scanned electronically as they enter
and this list can be compared to the “roster” from the Grade Center to ensure each student has
participated.
Several assessment strategies have been applied over the past five years including learner
satisfaction surveys and content quizzes for specific sessions. Because we knew which sessions
specific learners planned to attend, we could direct content quizzes to specific groups. For program
evaluation purposes, the Grade Center provides a good record of which sessions were most attended
and which session’s content quizzes scored the highest.
Over the past five years there have been several staff members assigned to manage the site;
therefore, several variations of Blackboard’s technical capabilities have been applied. But for each
event, the use of Blackboard to administer this large face-to-face event for students from different
schools and departments has been uniformly successful. Because there are so many Blackboard
users on campus, there are many individuals who can offer advice and support to a new staff member
assigned to this task. This would not be the case if we had elected to create an event website using
software that required unique training. Students, because they are familiar with using Blackboard
for their coursework, have no hesitation logging on to find the documents they need or to reply to
quizzes and surveys. The course “banner” for the organization site is the same design as the cover for
campus flyers and other publicity in order to help learners connect the site and the event, and all
language in the site is designed to be as inclusive as possible to reflect the overall theme.
The major limitation to using a Blackboard organization for this event (or for any other purpose) is
that while quiz results are linked to specific learners and reported to the leader (instructor) as discreet
data; survey results are anonymous but only available as aggregate data thus limiting the statistical
analysis that can be applied. So, if you prefer having survey data as discrete data for in-depth
analysis, you must create the survey as a quiz and sacrifice anonymity. You can certainly assure
learners that their identities will be kept confidential, but that is somewhat different from true
anonymity.
2. Case #2: Preparing advanced students to be effective tutors
Peer tutoring has become an important component of student support on many college campuses,
but those who study this trend have realized that simply being knowledgeable with the content of a
course does not equate to being able to tutor others successfully in that content. Teaching/training
skills such as explaining content and responding to questions (Roscoe and Chi 2007, 2008) are skills
that can be taught to tutors resulting in measurable improvements in their success.
U of L’s undergraduate tutoring program REACH (Resources for Academic Achievement) has been
internationally recognized. The program’s web site was awarded Third Place in the annual 2010
Learning Support Centers in Higher Education (LSCHE) Website Excellence Award; and in 2009,
REACH received both a Program of Excellence and the Tutor of Excellence by the Association for the
Tutoring Professions (ATP). REACH recently received its three-year certification renewal from the
College Reading and Learning Association (CRLA), and is one of the only about 260 Tutor Training
programs worldwide to gain certification from the CRLA.
At the beginning of each academic year, REACH holds two days of tutor training. The first day is
spent orienting the tutors to the mission and goals of REACH, reviewing learning styles, and providing
486

Karen Hughes Miller and Linda Leake
general tutoring basics. The second day provides campus speakers and breakout sessions, where
each center meets as a group and discusses their specific area and how to best meet the needs of
the students who take advantage of that particular service. These two days not only provide
information about tutoring with REACH, but also allow an opportunity for the tutors to begin to develop
relationships with one another. An additional day of tutor training is held at the beginning of each
spring semester. This one-day event continues the information from the two-day training. During the
first part of the day, campus updates are discussed and tutoring basics are covered. Then, tutors
break out into small groups for center-specific information for the second part of the day.
In addition to attending the two-day training, all tutors are required to use the REACH Blackboard
Tutor Training course (developed as an “organization”) to complete modules on Diversity Training and
on Sexual Harassment. These modules are completed by each new tutor and then graded by REACH
staff. When the modules are completed successfully, the tutor becomes eligible for the first level of
certification and a slight pay increase. The complete Blackboard Tutor Training course is required
for tutors who were unable to attend the face-to-face sessions or who were hired after this training
took place. Modules on the Blackboard course cover the topics and basic information from training
so that tutors will be given the same information as those who attended the new tutor training.
Online tutor training has several advantages. As in all online learning, the Blackboard site provides
a place for learners to return to specific content if they feel they need a “refresher” on certain topics.
Having the modules online also standardizes the training being given to tutors from semester to
semester, and provides a way to document the completion of specific modules that can earn the
various levels of certification and the differential pay rates. The combination of face-to-face and online
training creates a blended learning model for tutor training, and because the U of L access to
Blackboard is thru the central University web portal, they are already situated to access the Virtual
Math Center and the Virtual Writing Center when the students they are tutoring need specialized
assistance. When necessary, tutors may be given guest privileges into a specific course site in
Blackboard to assist with linked Supplemental Instruction (SI) study sessions.
3. Case #3: Keeping medical faculty, students, and staff up to date with
patient privacy training
A very simple but effective application of a Blackboard organization site is keeping medical and
nursing faculty, students, and staff up-to-date with their training and certification on patient privacy.
The US statute, The Health Insurance Portability and Accountability Act of 1996 (HIPAA), includes
several objectives relating to health insurance availability and medical record keeping. The aspects
most relevant in a healthcare teaching and research environment are: (1) the basics of patient
privacy, (2) HIPAA information security fundamentals, and (3) privacy when patients are the subjects
of research. The U of L Privacy Office has developed Blackboard training courses and quizzes for
each of these three essential areas, and, over the past three years, there have been more than 2,000
individuals enrolled to take one or more courses. Because not every employee or student is required
to take every course, but passing the required courses is their legal obligation, individual record
keeping is very important. Blackboard uses the University’s User ID system; therefore it has
reduced the complexity for the Privacy Office to track users by name. In Blackboard, users can not
have more than one “record” of performance as can happen with some other systems.
When someone’s certification in a specific topic is due for renewal, the course name will appear on
their own personal Blackboard menu under Organizations. It is then their responsibility to take the
course and pass the online quiz. Modules and quizzes are asynchronous and untimed so participants
can work at their own pace. As with all Blackboard courses and organizations, grades are recorded
in the central Gradebook, but participants are also encouraged to print out their own record of
performance. One “cross check” to this system is that HIPAA training is verified when a faculty
member or student is named as a researcher in any clinical study presented for approval to the
Institutional Review Board (IRB).
Regulatory training modules like this are usually developed by committees of content experts and in
this case, the Privacy Office was able to develop its own pool of questions to meet their specific
needs. The “question pool” feature in Blackboard allows the order of questions presented to any
given learner to be randomized thus increasing the reliability of the score as a measure of their actual
knowledge. While managing this training and testing process this is still a labor intensive activity,
using a Blackboard organization site to deliver and monitor training gives the Privacy Office two key
487

Karen Hughes Miller and Linda Leake
advantages: the first is control over content and presentation since they did not need to rely on an
external vendor/developer; and the second is control of learner identification since University User IDs
are unique and learners are not likely to be confused or to have more than one record - an issue that
had occurred when using other software.
4. Case #4: Presenting online staff development with complex content
One of the newest and most technologically advanced applications of a Blackboard organization
site (with several course modules still under development) is an employee certification program on
university business practices. As budgets for higher education become more stringent and regulation
of incoming funds from research and other projects becomes more complex, there is increased
demand on the skills of individual clerks and business managers within college and university
departments. At U of L, the role of Unit Business Manager (UBM) has become highly specialized and
complex so even those employees with degrees in accounting may not have the specific knowledge
necessary to navigate the system. The Unit Business Training (UBT) curriculum was developed in
consultation with senior unit business managers and university leaders. The course content is created
with input from university subject matter experts and designed by the Delphi Center for Teaching and
Learning. Employees with certain job titles and technical permissions within the financial system are
required to complete the certification program.
To become a certified UBM, an employee must successfully complete 18 hours of general electives
focusing on soft-skills and technical training. Almost all content and testing is delivered online using a
Blackboard organization with face-to-face sessions to enhance learning. Employees enrolled in a
course must pass a quiz on each module with a score of 85% or higher with no more than three
attempts permitted. (Participants who are not able to pass the test after 2 attempts receive additional
coaching from the subject matter experts before they attempt the quiz again.) There is also a final quiz
where a score of 85% or higher is required. Experienced employees may “test out” of a course by
obtaining a score of 85% or higher on a comprehensive test in the course subject area. Both final
course tests and the “test-out” examinations are administered outside of Blackboard. A single
Blackboard organization site is used to house all courses (designed as modules) (see Figure 1) so
all employee grades are recorded in one central Gradebook.
Figure 1: The announcements screen of unit business training
After completing a prerequisite introductory course called University 101, the required core courses
focus on two general areas: “financial” systems and “people” systems. Examples of financial systems
488

Karen Hughes Miller and Linda Leake
courses include Financial Basics, Procuring and Paying for Goods and Services, and Sponsored
Program Administration Core (for managing extramural funded research). Examples of people
systems courses (HR) include Hiring and Compensation, Benefits, and Personnel Actions. Specialty
electives, while once envisioned as part of the core curriculum, have recently been re-defined as a
continuing education requirement to maintain certification. Examples of specialty electives include
Advanced Sponsored Program Management, Revenue-Based Budgets, and Student Administration.
The most challenging component of developing online courses in business or finance is the same
regardless of the course being for academic credit or for staff development; in general, it is more time
consuming to develop numbers-based content than it is to develop language based content for online
learning (Chao and Miller 2002). However, some of the newer development tools located in
Blackboard’s text boxes including MathML, WebEQ, and adding QuickTime, audio content,
Flash/shockwave, and Mashups. All of these multi-media capabilities expand the range of options
for sharing content. With these multi-media options, materials can be created that allow the
participants to actually “walk through” or rehearse the process to fill out a needed form or process in
real-time. Many software programs can be packaged in a format that can be uploaded into
Blackboard to allow for more capabilities for interactivity to help the participants of the UBT program
retain more knowledge of the related information. The information can then be available as a
reference or resource for all the participants.
5. Case #5: Blended orientation and graduate advising
U of L’s Raymond A. Kent School of Social Work found a way to provide a blended approach to
orientation for incoming students to their Master of Science in Social Work (MSSW) program as well
as a way to communicate important advising information for all MSSW students. For ease of use, the
Kent School organization site is divided into separate sections; one for Orientation and one for
Advising. When administrators realized that using a Blackboard organization to support Orientation
was actually saving approximately four hours in the required face-to-face sessions, they were able to
reduce the sessions by half a day. As far as the Advising section of the site, it has saved countless
advising staff hours answering basic advising questions via e-mail.
The Orientation section of the site includes required readings and information on which students are
quizzed using Blackboard’s testing feature. There are also optional readings and information as
well as the information about the half-day face-to-face orientation session. The discussion board is
used to create a dialogue among the new students on practical issues such as car pooling, looking for
someone to share an apartment, as well as areas (or “forums”) to generate ideas for the
organization’s upcoming priorities. The information on which students are quizzed is taken from the
Kent School Catalog, and 8 out of 10 correct replies are required for successful completion.
Since each student in the MSSW program has a specific course plan and specializations, the
Orientation and Advising Staff have developed a truly unique application of the Blackboard Grade
Center. To help students stay organized, each student’s My Grades page reminds them not only of
which items they need to complete but also (after their first advising appointment) the name of their
advisor, the program plan that the student has selected, and their specialization. If a student has been
admitted with conditional status, the Grade Center also contains columns that list any pre-requisite
courses the student may need to complete as well as if any documentation is needed such as
transcripts for the student’s files. This innovative use of the Grade Center is possible because
columns in the Grade Center accept text as well as numerals. Column content does not need to be
included in a calculated “score”, so any type of relevant information can be included in column format.
While the “master” Grade Center includes a complete spread sheet of student data, individual
student’s My Grades page (see Figure 2) displays only their personal records.
The Advising section of the site includes a list of curriculum plans for the current year, information on
what to expect if pre-requisites are needed, a list of important dates, and the Kent School Phone
Directory. Having all of this information easily at hand helps students get their questions answered
without having to e-mail the advising staff. The Catalog, the Academic Calendar for the current three
semesters, and the MSSW Student Handbook are also included under Advising. Once a student is
assigned to an advisor, they are enrolled in a “Group” for that advisor within the organization. Groups
include between 10 and 15 students. The Groups function in Blackboard allows each advisor to use
tools such as Group E-mail, Group Discussion Board, and File Exchange to communicate with each
of the students assigned to the advisor.
489

Karen Hughes Miller and Linda Leake
Figure 2: A “My Grades” page used as a personal advising record
In addition to these innovative applications, the site contains information more typical of an
“organization” site such as information about The Kent School Student Association (KSSA). This
provides students the opportunity to network with peers and participate in activities each year that reenforce
building a learning community within the school. Other important information at the site
includes master syllabi for each class in a given semester, graduation information, and links to other
campus sites including “Parking” for parking passes and “Bookstore” so that students can order books
in advance.
This site has become a single resource for all of the MSSW students to find the majority of information
with regard to the school, program, and networking opportunities available. It has made the sharing of
information much easier and has saved time and resources that Kent School can now use for other
endeavors.
6. Conclusion
Blackboard organization sites have far more potential than simply supporting extracurricular student
activities. If we consider each of Blackboard’s technical capabilities as platform (or software)
capabilities rather than being limited by the current “label” or how they are typically used, we can see
there are multiple opportunities to use an online organization to support faculty, student, and staff
training and communications needs. As noted by Gautreau and Ahmed (2008), Blackboard is a far
more robust platform than just a traditional learning management system. At U of L, we currently
support 663 Blackboard organization sites and receive an average 20 requests per month to create
new sites. The range of applications is still growing.
The Delphi Center for Teaching and Learning under the auspices of the Office of the Provost supports
learning technology needs on all three University campuses. They have simplified the process for
faculty members and departments to be granted Blackboard organizational sites so it is done online
and at no cost to the department. Student groups may also be given sites under the guidance of a
faculty advisor. When a site has served its purpose, it may be archived, or portions of the site copied
490

Karen Hughes Miller and Linda Leake
into another site that is now filling that purpose. As with all learning platforms or other interactive
software, there are some limitations, but we believe that overall the advantages make this a
worthwhile endeavor (see Table1).
Table 1: Positive and negative aspects of using the Blackboard organizations function for nontraditional
applications
Positives:
The Blackboard platform includes a rich variety of
functions bundled into one proprietary (password
protected) venue.
Most institutions’ contracts already include some form
of “organization” application.
Faculty, students, and staff are usually familiar with
using the online platform and these skills can be
applied to “organization” applications.
Training and technical support can be the same for
regular course application and organization
applications.
Success breeds success and other departments learn
from example and develop new applications.
Negatives:
Users’ browser compatibility: occasional issues with
newer browsers (Firefox 4 and Internet Explorer 9).
Needing to obtain user IDs (passwords) for non
college or university personnel.
Not being able to access “discreet” data on
participant replies when using the survey function
(this is viable only in the “quiz” function and
anonymity is sacrificed).
Although all online learning platforms have become
more user-friendly over the years, faculty and staff
time must still be dedicated to site development and
site maintenance.
The term “thinking outside the box” is certainly overused, but in this case, if we can think outside the
idea of a “course site” and imagine to what other purposes we can put all of that online functionality,
we may find we have a wonderful new set of tools already at our fingertips.
Acknowledgments
Sincere thanks to Ronald M. Welch, Kentucky Area Health Education Centers (AHEC) for review of
Case Study #1; to Cathy Liest, Executive Director of REACH, for review of Case Study #2; to Gwen
Holland, Senior Compliance Analyst, U of L Privacy Office, for review of case #3; to Dr. Virginia
Weber Denny, Assistant Director for Professional Development, and John Spence, Program Manager
for Professional Development, Delphi Center for Teaching and Learning, for review of Case #4; and to
Angela Napier, MSSW, ABD, Coordinator of Academic Affairs Graduate Programs, and Dr. Pamela
Yankeelov, Associate Dean, Student Services, Raymond A. Kent School of Social Work, for review of
case #5.
References
Bonder, B., Martin, L. and Miracle, A. (2001) “Achieving Cultural Competence: the Challenge for Clients and
Healthcare Workers in a Multicultural Society. Generations, Vol 25, No. 1, pp 35-42.
Chao, Y-C. and Miller, G. (2002) “Effective Math Online Learning (EMOL), Master Project Proposal.” Learning
Design & Technology Program. Stanford University, 18 January, 2002. Advisor: Professor Jim Greeno.
[online] http://ldt.stanford.edu/~chaoyc/project/emolproposal.pdf
Gautreau, C. and Ahmed, S. (2008). “Blackboard Management and Professional Development Strategies to
Augment Teaching and Learning”, MERLOT Journal of Online Learning and Teaching. Vol 4, No. 3. [online]
http://jolt.merlot.org/vol4no3/gautreau_0908.pdf
Mott, W.J. (2003) “Developing a Culturally Competent Workforce: A Diversity Program in progress.” Journal of
Healthcare Management, Vol 48, No. 5 (Sep/Oct) pp 337-342.
Roscoe, R.D. and Chi, M.T.H. (2007) “Understanding Tutor Learning: Knowledge-Building and Knowledge-Telling
in Peer Tutor’s Explanations and Questions”, Review of Educational Research, Vol 77, No. 4, pp 543-574.
Roscoe, R.D. and Chi, M.T.H. (2008) “Tutor learning: the Role of Explaining and Responding to Questions”,
Instructional Science, Vol 36, pp 321-350.
US Department of Health and Human Services (2003). Health Information Privacy: Research. [online]
http://www.hhs.gov/ocr/privacy/hipaa/understanding/special/research/index.html
491

An Analysis of Collaborative Learning as a Prevalent Instructional
Strategy of South Africa Government eLearning
Practices
Peter Mkhize, Magda Huisman and Sam Lubbe
University of South Africa, South Africa
mkhizpl@unisa.ac.za
madga.huisman@nwu.ac.za
Sam.Lubbe@nwu.ac.za
Abstract: A skills shortage seem to be engulfing the South African workplace in all economic sectors (Statistics
South Africa 2010). In turn, increasing unemployment rates feed off such lack of critical skills, that cripples the
productive capability of those who are suppose to be economically active. Subsequent to the 2001 parliamentary
decision to migrate the government’s information system (IS) to an open source platform, the State Information
Technology Agency has been faced with the task, among others, to facilitate training during and after the migration
process. According to Department of Public Service and Administration (2006), different instructional delivery
methods will be used – including eLearning, which allows for a lifelong learning, collaborative learning, flexible,
just-in-time, just-in-context learning experience for users who are learning the new platform. In this paper, the
researcher will determine the prevalent instructional strategy in relation to current instructional technology available
to support eLearning practices in SITA. In doing so, the authors will employ grounded theory analysis techniques
within a case study conducted in SITA – the custodian of the open source migration project (Charmaz
2006). The grounded theory analysis technique allows the authors to create a relational networks of codes, as
they emerge, for interview transcripts that will be interpreted to determine the instructional value of the instructional
tools used in learning practices. In this study, an instructional strategy framework is developed based on
code networks as emerged from grounded theory analysis. The result proves to favour collaborative learning as
an efficient and effective instructional strategy to support eLearning practices in the public sector, given multiple
constraints, towards realising a fully-fledged eLearning programme.
Keywords: instructional design, instructional technology, instructional strategy, collaborative learning, eLearning
and eLearning delivery modes
1. Introduction
After a parliamentary decision to migrate Government’s Information System (IS) to an open source
platform, the project team decided on user training methods. One of these was eLearning. Meanwhile,
the South African labour market has been engulfed by a skills shortage. It would thus be beneficial for
those interested or struggling with facilitating skills training, to learn from eLearning practices as applied
during the open source migration project.
There are no reports or publications covering successful eLearning experiences in South Africa as
there are reported successes in other countries (Bell 2007). In this paper, the authors investigate
eLearning practices in the public sector, as used to impart skills to employees who are going to use
the new open source systems. This is done by discussing relevant concepts as found in literature,
and conducting an empirical investigation using the research method outlined in the research methodology
section below. After that, a discussion of the results follows, incorporating a proposition emanating
from the results discussion, and the conclusion of the study.
2. Literature review
In this paper, the authors decided to conduct a conceptual literature review, instead of the chronological
review giving a short history of the eLearning experience, and studies that investigate eLearning
practices in the South African context. The following sections unpack the conceptual underpinning of
eLearning practices.
2.1 eLearning in a dynamic socioeconomic environment
Cross and Hamilton (2002) define eLearning as a learning process that takes place anywhere, any
time, at the learner’s own pace. Acton and Golden (2003) add that eLearning is just-in-time education
integrated into a high-velocity value chain. However, Asgharkani (2004) asserts that eLearning is
aimed at replacing the old, traditional time/place/content predetermined learning, with a just-intime/artwork
place/customised/on demand process of learning. This is learning made easy. According
to Alsultanny (2006), eLearning programmes should be supported by management, in order to define
492

Peter Mkhize et al.
and align the vision and plan for learning in the organisational strategy, so that learning can be integrated
into the organisation’s daily activities. The author further suggests that eLearning should be
used to cultivate a culture of learning in the morning, and “do” in the afternoon.
Basically, organisational training has been at the heart of organisational development and performance
improvement, used to support skills acquisition and development for employees, and thereby
facilitating the shift from traditional (or old) methods to new methods of doing business (Taran 2006).
According to Alsultanny (2006), such training should be flexible in terms of place, time and pace, in
order to accommodate the different learning styles preferred by learners/employees from differing
educational backgrounds. This makes it necessary for an organisation to respond more quickly to the
demands of the knowledge-based global economy with an ever-growing demand for learning new
skills (Alsultanny 2006). eLearning could therefore contribute towards a solution to training problems –
which are related to flexibility and access to developmental training programmes, in supporting organisational
growth while sustaining productivity during training.
The learning process should therefore no longer be confined to lecture rooms or training rooms, as
long as the student or trainee has access to internet connectivity. This is affirmed by Hunter and Carr
(2002), who also argue that eLearning is easily accessible to those who have time constraints and are
thus unable to attend full-time, class-bound learning programmes.
Even though eLearning is gaining in popularity, Li et al. (2003) argue that it dehumanises the learning
process, because students or trainees do not come into contact with the facilitator, thereby losing a
sense of “belonging”. Meanwhile, Alsultanny (2006) points out the relevance and benefits of using
eLearning in the training programme, where eLearning can be reactionary in dealing with the skills
problems as they occur in a non-linear manner. Training programmes can then be accessible to employees
in times of need, or as the need arises, at the trainee’s own pace. This is in line with the underpinning
principles of eLearning as lifelong learning, flexibility of the learning process, collaborative
learning, and easy access to learning. These principles are discussed in the following section.
2.2 Lifelong learning
El-Khatib et al. (2003) reiterate the fact that the sustenance of the information economy is highly dependent
on lifelong learning. Organisations and employees have to keep on learning to survive the
revolutionary global competition and the explosion of information technology that incite the need to
acquire new skills, knowledge and training. The prevalence of technology in the dynamic information
economic conditions necessitates change in the workplace; likewise, skills have to be up to date (Bell
2007; Hunter and Carr 2002). This could means that employees have to improve their skills levels
with the latest skills, which are necessary to face the challenges of the new economic conditions and
demands.
Therefore, employees who want to be productive in their area of expertise must have access to training
or learning programmes that would allow for just-in-time and just-in-context learning, to address
the specific skills shortage. However, employers have to provide such an environment by investing in
eLearning-based training systems that will allow employees to learn as long as they still need to improve
their skills, thereby contributing to corporate intelligence (Cross 2004).
Lifelong learning has been a maturing topic over the years in Europe (Bell 2007; Asgharkani 2004), as
many organisations recognise the need to engage in ongoing learning. In order to realise the benefits
of lifelong learning, organisations utilise eLearning-based training programmes that are flexible
enough to accommodate personalised learning styles – unlike traditional training models (Cross and
Hamilton 2002). Fry (2001) affirms that lifelong learning is becoming the only way to survive the everchanging
global economy for organisations that want to sustain competitiveness and ascertain employees’
productivity. This is even more relevant where the knowledge and skills lifespan is rapidly
diminishing in the advent of new technology, which is then compelling organisations to change their
production methods (Abdul-Salam et al. 2006; Acton and Golden 2003). Employers have to provide a
lifelong learning environment that is timely, accessible and affordable for employees who want to upgrade
their skills and knowledge in order to successfully fulfill their job responsibilities (Wiegel 2002).
The eLearning model makes lifelong learning possible, as it is one of the intertwined underpinning
principles of eLearning. Again, lifelong learning is more effective in a collaborative learning environment
– which is discussed in the following section.
493

2.3 Flexible and accessible learning
Peter Mkhize et al.
Traditional classrooms have been, and still are, convenient and suitable for learning in the residential
institutions. However, traditional learning is said to be unresponsive to urgent training needs that
might require just-in-time and just-in-context learning aimed at solving specific work problems. In a
flexible learning environment, students should be able to set out their own learning goals and pursue
their learning objectives independently (Acton and Golden 2003). Students should also be able to
learn to construct their own meanings by interpreting and making knowledge claims based on the
learning material provided to them – thereby contributing to generation of knowledge.
Asgharkani (2004) emphasises the importance of technology integration into learning to support the
flexible learning strategy, so that learning material can be distributed to students using technology,
and, specifically, web-based technology. The flexible learning strategy is technology intensive, because
technology plays an integral role in facilitating asynchronous and synchronous modes of delivering
learning material. This makes learning material accessible to students who would otherwise not
be able to attend classes in real time because of other daily commitments (Hunter and Carr 2002).
According to Asgharkani (2004), many companies are focusing training investments more towards
online-based learning systems than on traditional classroom learning – that way, boosting turnaround
time from learning to performance of tasks. Flexible learning could be facilitated well where students
can collaborate in the development of new meaning, given appropriate instructional technology.
2.4 Collaborative learning
The advent of internet transfer of information between individuals has been made easy through a multitude
of ICT tools that have enabled interconnectivity between students in distant places, both asynchronously
and synchronously (Hunter and Carr 2002; Abdul-Salam and Mansur 2006). The internet
also enables global communication between students/trainees and experts in their field of specialisation
(Matula 2004). As a result, students learn from each other by exchanging ideas, information and
knowledge in online interactive platforms, both asynchronously and synchronously. The sharing of
information and knowledge between students is done within communities of practice, where specialists
and experts in the field share their experiences and learning from each other (Wenger and Snyder
2000).
Collaborative learning provides opportunities to critically engage with the learning material that is being
discussed within collaborative groups (Drinkwater et al. 2004). Even though some of the eLearning
programmes are not designed to enable a collaborative learning strategy, students nevertheless
still enjoy some level of online interaction with the facilitator, experts in the industry, and other students.
Cross and Hamilton (2002) assert that learning in the collaborative learning environment is
student centred, rather than lecturer centred (which is prevalent in the traditional learning model). In
addition to critical engagement, students learn by discovery – where they interrogate learning material
to discover implied meaning and interpret it to formulate new valid conclusions.
Leacock (2005) suggests that organisations should create an environment for skills development and
training, that is continuously improved through open sharing of ideas, frequently, and regular evaluation
of processes across different areas of the organisation, by collaborators. The facilitators and
learners/employees engaging in the evaluation of the learning process thereby encourage development
and/or subscription to a community of practice where employees can share relevant information
about a technology-driven development in their industry.
The collaborative learning environment strengthens situational learning that grants students access to
the relevant training programmes addressing particular skills deficiencies directly linked to their work
(Knowles 1984). Williams (2003) confirms that situational learning inculcates the “knowing how to act
in practice” rather than “knowing about practice”. This is imperative for a learning organisation in the
midst of technological changes in the global marketplace (Senger 1990).
2.5 Student centeredness
Cross (2004) maintains that eLearning systems provide continuous flexible access to learning programmes
that are accessible at any time and anywhere, meeting diverse student requirements, as
they have varying preferences as to time and location of learning delivery. This requires a student-
494

Peter Mkhize et al.
driven learning mechanism that would facilitate learning in ways that allows students to drive the pace
and learning process (Knowles 1984).
In a fast-pace, changing work environment it important for an organisation to develop training programmes
that are sensitive to employees’ skills needs. Failure to align learning programmes with student/employee
needs could lead to rejection of the learning programmes (Matlhape and Lessing,
2002). eLearning provides a flexible and student-centred learning environment, enabling a responsive
learning process in an attempt to close skills gaps that could be caused by changes in the work
environment (Hunter and Carr 2002). In a student-centred learning environment, a student would be
able to access learning programmes at any time of the year, directly, anywhere that they are situated,
based on the current skills deficiencies (Cross 2004). Cross and Hamilton (2002) assure that studentcentric
learning programmes are developmental and enriching, as students gain a competitive edge in
the labour market over those who do not update their skills.
Instructional designers of eLearning experiences need to assess student profiles in order to customise
their offering. Customisation of the eLearning solution would enable students to learn in their own
learning style (Hunter and Carr 2002). The student becomes the centre of the learning practice, as the
facilitator should be there to facilitate the learning process. All the learning material is designed
around the student, and they are encouraged to contribute towards the design of learning programmes.
Williams (2003) assures that students will be interested in engaging learning material to
learn even more abstract concepts on their own, where they have a sense of ownership of the learning
experience.
Students are encouraged to engage learning material and deduct new meaning from learning experiences,
which would enable them to learn through discovery (Li et al. 2001). Williams (2003) asserts
that learning by discovery increases the sense of ownership of the learning process. The facilitator
would not have to push students to learn new concepts, and students will pull all concepts that would
add value to their career and life.
Student centeredness complements collaborative learning as both a principal advocate for student
ownership of the learning process, and an encouragement to facilitators to create a conducive learning
environment for discovery of new knowledge. Levinson (2004) suggests that the learning programmes
developer should be familiar with the constructivist theory in order to promote student ownership
of the learning process, as they could be assisted by the facilitator in discovering new knowledge.
The constructivists’ epistemological perspective promotes co-creation of meaning and knowledge
between the learner and the facilitator of the learning experience (Duffy and Cunningham 1996).
In designing the eLearning platform, Li et al. (2003) argue that instructional designers should first
consider or give priority to educational principles over technology infusion, in order to keep the content
in context and relevant, instead of focusing attention on the technological components of eLearning.
3. Research methodology
This is in line with the objective of the study, which is to establish prevalent instructional strategy in
the public sector, and to formulate propositions that can be used as guidelines for instructional design
in the public sector. In doing that, the authors will try to gain an in-depth understanding of eLearning
practice in the public sector, by employing a qualitative method of inquiry (Creswell 2007). The unit of
analysis will be Government officials interested or working with eLearning in different departments. A
grounded theory analysis method will be used to get a dense description of eLearning practice and to
find the most prevalent/appropriate instructional strategy in the public sector (Charmaz 2006). Once
the authors understand the eLearning practices in the public sector, they will be able explain how an
eLearning designer could align different aspects and concepts into designing an effective eLearning
environment.
It is important to note that grounded theory is not applied as the research design in this paper, but as
the method of analysis, because it sets out clearly-defined steps in analysing qualitative data. The
analysis of data entails open coding that allows the researcher to let codes emerge from interview
transcripts, by applying bracketing.
495

3.1 Definition of case study
Peter Mkhize et al.
Case study is one of the government departments that is in charge of open-source migration from
proprietary software. Among other training delivery mechanism is eLearning. This change happens in
the midst of high unemployment rates due to skills shortages in South Africa. This department is also
in charge of end-user training. This is disconcerting, because training initiatives have failed to serve
training needs that existed before the latest requirement of open-source training.
The authors interviewed six members of migration training programmes. They all work for the same
departments, but consult in different departments that have already started to migrate. They are faced
by different challenges, which they shared during interview sessions. The average interview duration
recorded was 43 minutes. Each interview was transcribed and open coded before the subsequent
interview, in order to grasp the exact situation, thereby adapting the interview schedule where necessary.
After open coding comes axial coding – which allows the researcher to categorise open codes
into families. This is a term used in AtlasTI, the qualitative data analysis software used to analyse
data in this study. Subsequent to that is selective coding, where relationships between families are
established; in this study, a network of codes is used to represent such relationships. Code networks
are discussed in the discussion of findings.
4. Discussion of findings
Findings are discussed in relation to categories deduced from codes as they emerged from interview
transcripts. Two major categories emerged during the analysis of data. These categories were formed
by grouping related codes together into families, as is the term used in AtlasTI to describe groups of
codes. Within these categories, codes were interconnected according their relationships with each
other in the family, in order to create a story based on the interconnection (Creswell 2007; Charmaz
2006). Two categories emerged with networked codes as discussed below.
4.1 Management and administration of collaboration
Collaborative learning emerged as the prevalent instructional strategy in the public sector. This category
illustrates the setting up of collaborative forums, allows for flexible learning between collaborators
as they define the learning process and what is to be learnt (Acton and Golden 2003). However,
membership of collaborative forums is controlled, to ensure that everybody in the forum makes a
meaningful contribution, and to facilitate the sharing of ideas around development and adoption of
open sources software in the public sector. Since open sources adoption takes place in the initial
stages of the migration process, most of the communication among participants in the collaboration
forums is kept confidential; therefore, requests to join should be trusted.
I think it’s just about the confidentiality of the information, say you are in government you
typically want to control whoever joins cause you might discuss strategies, communication
strategies, ...in that case you will firstly have the private network ... you might be able
to invite external people but then you would qualify them ...
Even though membership of collaborative forums is closed, there is a need to attract or invite expert
people to fill knowledge and skills gaps in the group of collaborators. Sometimes, social networks are
used to attract prospective collaborators who have something to contribute, as the administrator
would scan the social space to find bloggers who seem to have some level of expertise in open
source – or any other skills that is deemed useful to the group. This makes collaborative learning
even more accessible to those who are not employees in same department. Knowledge sharing is
then extended to specialised experts.
I would maybe do some research and see...the thinkers in that area and I would extend
an invite to that somebody that writing a blog that’s...or start talking to that network and
say who are the real experts in open source ... Stakeholders onto that network the multiplier
effect, wow ...
Proposition 1: Profile the audience of the intended learning programme, then establish instructional
technology that is accessible to all participants in the learning or training initiative.
496

Peter Mkhize et al.
Figure 1: Management and administration of collaboration
497

Peter Mkhize et al.
Basically, social networks become an integral part of collaborative learning, even for a closed community
of collaborators. Social networks are normally used because of their relaxed and informal learning
characteristics, where collaborators could learn from each other and advance their current knowledge
and skills (Drinkwater et al. 2004). One of the participants suggested that social networks should be
strategically integrated into a credible educational model instead of loosely infusing technology where
it seems convenient, and then hoping that learning will take place.
Social network is the good model to look for, I mean it’s more of a forum environment; it’s
more, what is this? One of these educational models, constructivism...
In formulating an instructional strategy, the instructional designer should avoid the temptation to get
carried away by the potential of instructional technology, and then just transpose traditional instructional
material into technology media. This could deprive learner/trainees of efficient learning, and result
in chaos (Buhrmann 2003). One of the participants went on to suggest that there should at least
be one facilitator to direct the learning process in the collaborative forum (Abdul-Salam and Mansur
2006).
have a central person that will be able to guide ... it is informal but also learning because
the tool allows us to do that in a unified way then we’ll be able to collaborate at different
levels. One way or the other would need to collaborate with other people in the specific
area ...
Proposition 2: Once instructional designers have profiled participants, they should evaluate different
instructional strategies that could be applied in conjunction with the most relevant instructional technology
accessible to most participants.
Even though collaborative learning is student centred, there is still a need to monitor collaborative forums,
to avoid chaos (Abdul-Salam and Mansur 2006). Participants suggested that there should be a
facilitator who understands the learning objectives that have to be achieved, in order to declare the
learning initiative a success. The following section discusses learning processes in a collaborative
environment as experienced by participants in this study.
4.2 Learning through discovery
Collaborative learning in the public sector allows learners to be grouped, or to group themselves, into
communities of practice where they take charge of their learning process (Wenger and Snyder 2000).
This is in line with the principle of student centredness advocated by many authors in literature (Asgarskhani,
2004; Acton and Golden, 2003; Alsultanny, 2006). Figure 2 shows a network of relationships
between codes, that could be interpreted as elements of collaborative learning.
Some of the participants in this study argued that instructors and learners should become co-creators
of knowledge in the workplace, as they engage learning material according to the skills requirements,
with the assistance of instructional designers (Knowles 1984). Instructional designers should be familiar
with educational principles, such as the constructivist theory, in order to facilitate learning in the
workplace and matching instructional technology (Hunter and Carr 2002). In the case of public sector
training, a constructivist approach is used to design a learning experience that allows for co-creation
of knowledge and the sharing of ideas within a community of practice (Duffy and Cunningham 1996).
you look at the philosophy behind Moodle eeh it is social constructivism where you cocreate
knowledge, so in one stage you need the instructional designer and knowledge of
teaching presenting online lessons and exams and even on facilitators ...
In communities of practice, the role of a facilitator is to ensure that progression of activities leads to
achievement of learning objectives (Leacock 2005). Even though communities of practice are not
formalised, there are groups of people sharing the same interests and expertise, interacting on a
regular basis (Wenger and Snyder 2000). Their interactions are based on experiences and open
source problem solving strategies, as some members of the community have faced some of the open
source migration challenges.
In community of practice would see people who share common interest in a particular
field or something, they will be able to seat and collaborate, discuss, move forward and
look at the development, ... management of knowledge and knowledge gathering that
can be preserved ...
498

Figure 2: Learning through discovery
Peter Mkhize et al.
499

Peter Mkhize et al.
you’ve got your structured learning and unstructured learning so also contribute to that
unstructured learning ...
collaboration in terms of the different roles, to …, and that what’s exciting us, getting together
to work out a solution for the government we’ve had business analysts, we’ve
had the now the content developers ...
Proposition 3: The collaborative learning strategy should be encapsulated within the community of
practice where collaborators could share common interests, experiences and/or expertise, thereby
learning from each other by using appropriate instructional technology.
The primary objective of the communities of practice is to share experiences and knowledge, thereby
empowering colleagues in business – especially in the specialised business environment (Wenger
and Snyder 2000), which requires collaboration with experienced colleagues (Drinkwater et al. 2004).
Knowledge and skills sharing could be cross functional within the public sector, where interaction is
elevated through the levels of information sharing, to high-level competencies.
we need to engage with a department, so department x whether it’s Arts and Culture or
Home Affairs whatever, the SITA training department can engage with the department
and then setup a development plan that takes people from the basic skills required, basic
skills and basic knowledge through to the competence that you want in the end, ... to
the point where they are fully competent for their job and beyond, moving to the next
level ...
Within communities of practice, learning takes place through discovery of the facts and behaviours
that are useful to the improvement of current methods – to the extent that discovery incites changes in
the behaviour of the community (Li et al. 2001). One of the participants went on to say that collaboration
is a culture that has changed ways of doing business in the public sector. It has removed the silo
effect in the public sector, to enable sharing of best practices, so that everyone can learn from one
another.
think the main thing about collaboration in government is the culture is pretty much …so
everybody is busy with their own stuff, fighting their own battles and not knowing that another
department..., so collaboration for us is about creating the tool that can connect
these people so that they can form groups, networks of interests ...
Communities of practice could be interconnected, to allow the transfer of specialised skills from one
group to another, without formal coordinated learning programmes. However, this process is conducive
to a self-directed learning environment where individual learners know what they want to learn.
Such learners initiate the learning process that is directed at the achievement of situation-specific
learning.
so if you are in a social network ... there’s more than I think 14 groups already so they
are communities of practice because one the e-government group, ... I can be a member
of the government wide collaboration group as well as the e-government I’m actually a
part of 3 groups one I’m administrators the other one are I’m members ...
Proposition 4: Evaluate different social media to establish appropriateness for specific learning objectives,
then adapt one found to be appropriate to instructional design.
Self-directed learners would use any vehicle of knowledge transfer if it would help in achieving a specific
learning objective as required by the learner to fulfil their job responsibilities or to improve performance.
Communities of practice would then be useful for employees who want to gain an insight
into a specific knowledge area, and then probe beyond what is currently available in the forum. Learning
would take place without anxiety related to being in a classroom and getting ready to be tested.
Employees could navigate their way into different communities of practice, and then learn collaboratively
from peers.
5. Conclusion
Through grounded theory analysis, two themes emerged – namely: (1) Management and administration
of collaboration, and (2) learning through discovery. Collaboration proved to be the prevalent instructional
strategy in the public sector, as all participants agreed that collaborative learning would
help in paving the way for massive skills development, both in the public and private sectors. In addition
to that, participant agreed that collaboration would enable industry practitioners to develop the
500

Peter Mkhize et al.
industry by sharing information and knowledge with colleagues, thereby negotiating new meanings.
The authors went on to make a proposition that will serve as a guideline for instructional design in the
public sector.
References
Acton T. and Golden W. (2003). Training the knowledge worker: a descriptive study of training practices in Irish
software companies. Journal of European Industrial Training, 27(3), pp. 137-146.
Alsultanny Y. (2006). eLearning System Overview based on semantic Web. Electronic Journal of eLearning, 4(2),
pp.111-118.
Abdul-Salam Z. A. and Mansur A. (2006). MyGfL: A Lifelong Learning Platform for Malaysian Society. The Electronic
Journal of eLearning, 4(1), pp. 7-14.
Asgharkani M. (2004). The Need for Strategic Foundation for Digital Learning and Knowledge Management Solutions.
Electronic Journal on eLearning pp. 31-42.
Bell J. (2007). ELearning: your flexible development friend? Development and Learning in Organisations, 21(6),
pp. 7-9.
Charmaz J. (2006). Constructing Grounded Theory: A practical guide through qualitative analysis. Thousand
Oaks, CA: Sage.
Creswell J.W. (2007). Qualitative inquiry and research design: Choosing Among Five Approaches. Thousand
Oaks, CA: Sage.
Cross J. (2004). An informal history of eLearning. On the Horizon, 12(3), pp.103-110.
Cross J. and Hamilton I. (2002). The DNA of eLearning, excerpt from Beyond eLearning. Accessed: March 26,
2003 from www.internettime.com.
Department of Public Service and Administration. (2006). Policy on Free and Open Source Software Use for
South African Government. Accessed: 13 May 2008 from
http://unpan1.un.org/intradoc/groups/public/documents/CPSI/UNPAN025432.pdf.
Duffy T. M. and Cunningham D.J. (1996). Constructivism: Implications for the design and delivery of instruction.
In: Jonassen D.H. (ed.) Handbook of Research for Educational Communications and Technology. New
York: Macmillan Library Reference
Drinkwater P.M., Adeline C.M., French S., Papamichail K.N. and Richards T. (2004). Adopting a Web-based Collaborative
Tool to Support The Manchester Method Approach to Learning. Electronic Journal of eLearning,
2(1), pp. 61-68.
Fry K. (2001). ELearning market and providers: some issues and prospects. Education and Training, 43(5), pp.
233-239.
Glaser B. and Strauss A. (1967). The discovery of grounded theory. Chicago: Aldine.
Hoffman E.C. (2002). Information and communications technology, virtual offices and telework. South African
Journal of Information Management, 4(2) pp. 1-27.
Horton W. and Horton K. (2003). ELearning tools and technologies: A consumer's guide for trainers, teachers,
educators and instructional designers. London: Wiley.
Hunter M.G. and Carr P. (2002). Technology in Distance Education: A Global Perspective To Alternative Delivery
Mechanisms. Journal of Global Information Management, 10(2), pp. 52-56.
Knowles M.S. (1984). Andragogy in action. San Francisco: Jossey-Bass.
Leacock T. (2005). Building a sustainable eLearning development culture. The Learning Organization, 12(4), pp.
355-367.
Li R., Cheng S.T. and Willis R.J. (2001). A framework for the implementation of a collaborative flexible learning
environment for academic institutions”. In: Khosrow-Pour M. (ed.) Managing information technology in a
global economy. Hershey: Idea Group Publishing, pp. 89-93.
McCallin A.M. (2003). Designing a grounded theory study: some practicalities. Nursing in Critical Care, 8(5), pp.
203-208.
Matula S.M. (2004). Peculiarities of the digital divide in sub-Saharan Africa. Electronic library and information
systems, 39(2), pp. 122-138.
Matlhape M.G. and Lessing N. (2002). Employees in Total Quality Management. Acta Commercii, 2, pp. 21-34.
Sambrook S. (2003). ELearning for small organisations. Education and Training, 45(9), pp. 506-516.
Senge P.M. (1990). The Fifth Discipline: The Art and Practice of the Learning organisation. New York: Doubleday.
Taran C. (2006). Enabling SME to deliver synchronous online training – practical guidelines. Campus-Wide Information
Systems, 23(3), pp. 182-195.
Wenger E.C. and Snyder W.M. (2000). Communities of Practice: The Organizational Frontier. Harvard Business
Review, pp. 139-145.
Williams R. (2003). Integrated Distributed Learning with just-in-context Knowledge Management. Electronic Journal
of eLearning, 1(1), pp. 45- 50.
501

Ideas for Using Critical Incidents in Oral Debriefing From a
Business Strategy Simulation Game
Jonathan Moizer and Jonathan Lean
University of Plymouth, UK
jmoizer@plymouth.ac.uk
jlean@plymouth.ac.uk
Abstract: Reflecting on events or experiences can be an important means by which learning is achieved.
Specifically, a critical incident is some kind of significant occurrence that can stimulate learning through
questioning what has happened and why. Essentially, critical incident analysis is concerned with reflecting on
causes and impacts. An incident can be regarded as critical when the action taken has contributed to an
outcome, either positive or negative. A series of incidents can be reflected upon in a debriefing setting using a
structured approach to help to challenge people’s assumptions and promote personal and professional learning.
Critical incidents are a good medium for achieving reflective learning on the nature of situational events.
However, there are a number of reasons why it can be difficult to learn from incidents in the real world, and why
therefore, simulation games may facilitate effective learning through the application of a critical incident method.
This working paper seeks to present ideas on how critical incidents can be used by instructors in oral debriefing
to stimulate learning in a business simulation gaming setting. This is based on a case study of using a total
enterprise game named the Business Strategy Game to teach part-time, post-graduate / post-experience
students. The students, in teams, took on the roles of Directors of a global corporation and played out strategies
and decisions over a simulated period in the life of the company. Following this, students were orally debriefed
and assessed on what they had learnt about strategy making from the critical incidents that surfaced in course of
playing the game. Three sources of critical incidents were introduced during the debriefing, upon which the
students could comment, namely: macro-environment, competitive environment and internal environment of each
simulated company. Preliminary findings from the debriefings has indicated that using a critical incident method
did provide significant opportunity for students to think causally and better reflect on the connections between
actions and consequences under circumstances of gaming complexity. An agenda for a fuller qualitative analysis
of the student debriefings is proposed.
Keywords: critical incident, event, reflection, learning simulation game, business strategy game, oral debriefing
1. Introduction
Reflecting on events or experiences can promote learning to be achieved. A critical incident is a
significant occurrence that can stimulate learning through questioning what has happened and why.
Critical incident analysis is concerned with reflecting on causes of events and their impacts. A series
of incidents can be reflected upon in a debriefing setting using a structured approach to help to
challenge people’s assumptions and encourage learning. Although critical incidents can provide
understanding about the nature and repercussions of discontinuities and events, there are a number
of reasons why it can be difficult to learn from incidents in the real world, and why therefore,
simulation games may facilitate effective learning through the application of a critical incident method.
The aim of this working paper is to describe how critical incidents can be used within a business
simulation gaming exercise to benefit student learning. The objectives are to:
Explain how critical incidents can be harnessed to generate higher order learning.
Describe a learning and assessment approach using a critical incident method.
2. Background to study
The critical incident technique (method) was originated by Flanagan (1954). It was developed as a set
of procedures for collecting direct observations of human behaviour to assess an individuals’
performance in problem solving situations. It was developed to help assess and classify US air force
flying personnel’s behavioural competence. Since that time, the approach has evolved, and has been
applied to a range of different situations to help develop learning and understanding. These include,
amongst other applications, customer service encounters (Kemppainen, 2000; Edvardsson and Roos,
2001; Roos, 2002), strategic innovation decisions (Moenaert et al, 2010), and entrepreneurial learning
processes (Cope and Watts, 2000; Cope, 2003; Taylor and Thorpe, 2004). Critical incidents have also
been used for teaching and learning in higher education, often to help students to reflect on real
workplace incident vignettes (Macfarlane, 2001; 2003) or business case studies (Gulezian, 1981).
502

Jonathan Moizer and Jonathan Lean
The critical incident method has long been used to help people to reflect on their practice and develop
new understandings of the processes they are engaged with (Finch, 2010). Critical incidents are often
unplanned and unanticipated and perceived to have the potential to influence major change. The
learning generated from more unusual events can often be more transformational, in that individuals
when faced with critical incidents need to rethink their behaviours, as their habitual ways can prove
ineffective in the face of new conditions (Cope, 2003). Critical incidents provide a means for
individuals to transform their learning from a lower-order and more routinised level, to a higher-order
generative level.
A number of structured and unstructured methods are available for collecting data on critical incidents
(Kemppainen, 2000). Flanaghan (1954) originally advocated using interview or observation. More
latterly, others have used more structured survey instruments (e.g. Johnston, 1995; Mallak et al,
2003; Voss, 2009) to unpack critical incidents with individuals. The predominant methods for critical
incident analysis are via interview or focus group. Edvardsson and Roos, (2001) suggest developing
an critical incident interview guide around cause, course and result.
Through applying the critical incident method, data can be gathered through a variety of structured or
unstructured methods and researchers are given considerable latitude in study design. Critical
incident studies are more highly focused on providing solutions to more practical problems than other
qualitative methods that simply seek to describe phenomena. The format devised by Flanagan was
aimed at pinpointing facts and reducing personal opinions, judgements and generalizations
(Kemppainen, 2000).
To understand how a critical incident method can be employed within a simulated gaming
environment to promote reflective learning, it is important to first understand the nature of simulation
games and their role in providing experiential learning to participants (students).
There is quite an extensive literature dealing with how simulation games can be used as instructional
tools to engage students with active learning (see Pasin and Giroux, 2011; and Sitzmann, 2011 for
recent expositions of the simulation and gaming literature). Hainey et al (2011) illustrate the breadth
and depth of disciplinary application of simulation gaming for learning from business through to
medicine. Simulation games (SGs) are increasingly computer based. They are highly complex
synthetic artefacts where students are provided with the opportunity to ‘learn by doing’ through
immersion in simulated experiences of the real world (Ben-Zvi, 2010). With regard to business
education, many SGs are characterised by a simulated competitive environment in which rival
companies make periodic decisions, this provides inputs into the software which in turn generates
information which then forms the basis for the next round of decision making (Vos and Brennan,
2011).
The benefits of employing business simulation games to help develop decision-making skills are well
established (see Moizer et al, 2006; Edelheim and Ueda, 2007; Gilgeous and D’Cruz, 1996). SGs can
bring experiential learning into the classroom through providing structured environments for learning
about complex problems (Doyle and Brown, 2000). Such learning is often deeper than that achieved
when conventional teaching approaches are adopted, with the opportunity for learning to occur at
different levels in Bloom’s taxonomy (Anderson and Lawton, 1988; Ben-Zvi, 2010) as well as allowing
students to pass through all four phases of Kolb’s ‘experiential learning cycle’ (Moizer et al, 2004). As
well as being perceived by many students as a more enjoyable way to learn, SGs do provide a means
of integrating subject matter into the mainstream of students’ interests. A SG can be viewed as a
‘learning laboratory’ where students are able to practice analytical and decision-making techniques in
preparation for the uncertainties of life (Gilgeous and D’Cruz, 1996). Experimentation can be
undertaken at low cost, where understandings of the nature of cause-and-effect can be promoted and
developed, both under conditions of complexity and within compressed time periods. SGs can
correspond with reality; but at the same time can provide safe environments for students to play out
risks and live with ‘synthetic’ rather than real consequences, i.e. they easily allow learning through
mistakes to occur. In the course of playing a SG, students begin to attain a level of ownership which
moves them from passive to active learners. Within the simulation setting, they are presented with
opportunities to put the espoused theories that they have learnt into action, and actually try out new
behaviours that they would not readily attempt in a workplace, often for fear of failure.
503

Jonathan Moizer and Jonathan Lean
Interactive role-play is inherent to the simulation gaming experience. In business SGs, participants
invariably assume prescribed roles of characters from a company; which allows students to move
towards developing an understanding of circumstances and roles that are beyond their immediate
experience (Sutcliffe, 2002). They are given the opportunity to feel what is at stake, albeit in a
synthetic setting. It is hoped that better understanding of roles and relationships, and a greater
awareness of their own activities emerges from a SG role-play experience.
3. Critical incidents to promote learning in a simulation game
A review of the critical incident literature reveals various application of the critical incident methods,
particularly as a means to allow individuals’ to reflect on their experiences and change their thinking.
Therefore, it is perhaps surprising that there appears to have been very little research published in the
domain of simulation gaming other than work by Wolfe (1975) who used Flanagan’s critical incident
technique to evaluate the behavioural effectiveness of students participating in a management SG.
Simulation gaming would appear to present as a useful method to enable students to reflect on a
series of simulated events and actions, and therefore, to deepen their understanding of judgement
and decision making.
Critical incidents are a good medium for achieving reflective learning on the nature of situational
events. However, there are a number of reasons why it can be difficult to learn from incidents in the
real world, and why therefore, simulation games may facilitate effective learning through the
application of a critical incident method.
Firstly, in real environments critical incidents can be infrequent, whereas in a simulation game, a
longer period of simulated time can be compressed into a much shorter period of real playing time, for
example, .a number of simulated years can unfold in a matter of real hours. Hence, it is possible for a
number of incidents to be introduced into the game by the instructor in a planned way, or to allow
them to be generated in an unplanned way through the interactions and complex dynamics of the SG.
Furthermore, the consequences of an event can be seen to emerge and develop in full over an
extended simulation period. This allows the potential for rich learning opportunities to present
themselves through critical incidents within a manageable period of play.
Secondly, not only do SGs offer the opportunity to compress real time into simulated time, they also
offer the opportunity for instructors to stop time, and therefore ring-fence space for reflection, both
forensically and in real time within a simulation gaming exercise. This is in contrast to the real world
where day to day operational pressures invariably mean that managers have limited time to think
about and learn from events that have unfolded. As a result, debriefing lends itself well to post-play
critical reflection in a simulation gaming environment. In turn, such debriefing allows the learning
benefits arising from in-play critical incidents to be harnessed more effectively. Assessment criteria
based upon critical incidents can be introduced to require students to reflect on how an outcome was
governed by a course of action, and what the causal influences were behind such an action or
actions.
Thirdly, through technological advances, computer-based simulation games have become more
authentic. These new higher levels of fidelity and verisimilitude have allowed learners to become
more immersed in the simulated ‘microworlds’ of the games; hence, critical incidents that occur within
the gaming period are felt in an impactful way that is closer to a real world experience. They can be
viewed as more concrete than abstract. Although critical incidents may feel real and can generate
emotional reactions, they do not have negative consequences for the players themselves or any real
organisations; in other words, simulations provide a safe environment for learning through critical
incidents.
Although SGs have a higher level of authenticity, they are by definition ‘simplifications of reality’ and
do still work at a level of abstraction relative to the real world. In a real world situation, marked by
problems that are ill-defined, amorphous, messy and complex, it is often difficult to identify a critical
incident, never mind understand its nature, in terms of cause, course and results. In contrast, because
computer based SGs are more rigid (i.e. they comprise of structure, rules and behaviour) and
simplified than real world circumstances, it is easier to identify and separate out different impacting
variables and events within the boundaries of the game. With fewer variables than in the real world for
the students’ to consider, it is easier for the them to actually trace through and establish the causal
influences that are contributing to a course of action and its resultant outcomes; thus making critical
504

Jonathan Moizer and Jonathan Lean
incidents more explicit. This is assisted greatly within simulations by the outputs or results generated
automatically after each playing period of the game. Within some SGs, critical incidents are either predetermined
(pre-programmed) or the instructor has the opportunity, pre-play, to introduce them
(unknown to the player) at key points during game play. In a number of games, there is a facility for
instructors to flag these interventions to aide the learners’ recognition of critical incidents, for example
Airline: A Strategic Management Simulation (Smith and Golden, 2008), Human Resources
Management Simulation (Smith and Golden, 2005) and Corporation: A Global Business Simulation
(2003). This explicit flagging and signposting of critical incidents can aid learning that may not be
readily apparent in the real world.
Given the benefits that SGs offer (in terms of authenticity and an ability to manipulate time), they can
provide a potentially powerful mechanism for students to discover and learn about the nature of
decision making through experiencing and reflecting upon critical incidents in a safe, risk-free and
experimental learning environment.
4. A learning and assessment approach to using a critical incident method in
a simulation game
Within The University of Plymouth Business School, part-time, post-graduate / post-experience
students are taught strategic management as a capstone module of study, dovetailing the various
strands of business and management. As part of an effort to make the subject more engaging and
relevant to their current and future work organisations, simulation gaming was adopted as an
experiential learning approach. The Business Strategy Game (Thompson et al, 2010) was selected as
an off-the-shelf total enterprise simulation game that could help address the teaching, learning and
assessment imperatives of the subject module.
The Business Strategy Game (BSG) is web-based with 24-7 student and instructor access which
simulates the strategies and decisions of companies serving the global marketplace for athletic
footwear. Student teams (companies) adopt the roles of Directors and compete against other teams
for a predetermined market demand. An instructor is responsible for overseeing the running of the
game and can shape many of the externalities that will govern company decision-making (such as
exchange rates, material prices and shipping costs). The game requires the input of yearly business
decisions, such as volumes of goods to manufacture, shipment volumes, pricing levels and
advertising spend. These decisions are collectively processed on an on-line server program, and the
game then rolls on to another year’s play. Scores based on a number of performance metrics (for
example, profit, market share, capitalisation, sales volume) are determined, resulting in the teams
moving up or down a league table of companies.
Given the wide managerial experience of this particular student cohort, the totality of the BSG
environment does provide resonance and correspondence with their real workplace setting, allowing
them to not only draw on theory, but their own management practice to inform how they strategise for
and operate their simulated companies. This can result in the levels of engagement and
understanding of the game dynamics being higher than that of learners with little or no managerial
experience. Consequently, this does provide good opportunities to extend the real world feel of the
learning through to using more innovative methods of assessment of the knowledge and
understanding developed through playing the BSG.
Incidents or events that unfold during the course of playing the BSG are presented as opportunities
for critical reflection. Throughout the course of play, the student teams are confronted by significant
events which provide learning opportunities. Such events are not routine, rather critical.
Consequently, they help students, through reflection, to develop high-order skills. In designing the
summative assessment of the simulation gaming exercise, what was required was a method that was
based on reflection and would enable students to identify and discuss, post-game, the critical
incidents that occurred during play. It was decided that a form of debriefing activity would allow critical
reflection to occur.
Reflection can play an important part in constructing knowledge and improving learning (Chen et al,
2011). A number of author emphasise using critical reflection or debriefing as an important element of
learning with SGs (Garris, Ahlers, and Driskell, 2002; Gredler, 2002). In-game, students may play
without actually reflecting on decision and events, hence the importance for them to be aware of
reflection and how it can be facilitated (Lee and Chen, 2008). Post-game, getting students to reflect
505

Jonathan Moizer and Jonathan Lean
on the knowledge gained holistically from playing a SG, and how the various strands of information fit
together, post-play, can help promote deeper learning. Knotts and Keys (1997) recognise that
students learn at various points along the gaming period. They specifically argue that learning occurs
when students are forced to reflect on their experiences, as the experience itself will not provide
learning alone. Students need to learn how they can to learn from experiences tested against
espoused theories. This process of reflection can be facilitated through debriefing, where learning
closure can take place. End of game debriefing can crystallise or integrate the learning events that
occurred during play (Léger et al, 2011). Debriefing is a form of instructional support that acts as
scaffolding to help provide for an effective learning environment (Garris et al, 2002). It provides the
means by which the game itself can be linked to the achievement of learning outcomes. It allows the
opportunity to review and analyse the events that occurred during game play, allowing argumentation
to be developed and assessed (Sutcliffe, 2002).
For this student cohort and module, it was concluded that oral debriefing offered particular learning
benefits. Oral debriefing conducted in-role can be extremely valuable, as it allows the realism of the
experience to be extended. Arguably, in this situation, the gaming participants are still thinking and
acting as decision-makers, and thus, they are able to reflect more easily on the past, present and
future direction of their company. Oral debriefings are commonly used, post-game, to guide students
through a reflective process of their learning, where students and instructors engage in a question
and answer type session (Petranek et al, 1992).
Post-game, oral debriefing was particularly suited to this cohort of students given their managerial
backgrounds, work experience and confidence in engaging in verbal discourse. The debriefing was
used to measure the extent to which student participants learnt about managing their company’s
strategy and operations within the BSG. The debriefing took the form of a role play discussion, with
each student team (Company Directors) over a period of 20 to 30 minutes. This discussion was
summatively assessed and contributed to 30% of the overall module grade. Prior to the debriefing, the
teams prepared by producing a short formative report, incorporating relevant information, models and
graphics that would clarify their understanding of key events and performance as well as provide the
instructors with the necessary evidence to shape a full discussion.
Within the debriefing discussions, the teams remained in-role as company directors and two
instructors took on the parts of corporate investors seeking potential investment opportunities with
each company. A scripted role-play was employed to structure the verbal interactions to ensure that
the discussion addressed the assessment criteria and intended learning outcomes of the simulation
gaming exercise. A discussion of critical incidents experienced during play featured as a key
component of the debriefing activity.
Flanagan’s technique was adapted and simplified in order to provide reflective insight at the postgame
debriefing stage into a series of events or incidents that the students faced at certain points
along the gaming timeframe. The students were advised that a critical incident in the context of the
game play was an event which made them think differently than before about aspects of their
simulated business and its operating environment.
As part of the debriefing process, students were prompted to look back and reflect on the game play
and to explore critical incidents (both positive and negative) which may have impacted on their
understanding of business strategy and the running of their simulated company. They shared with the
instructors up to three critical incidents (occurring at the company, industry and macro-environmental
levels). They were asked to frame incidents in terms of cause, course and result (as per Edvardsson
and Roos, (2001) interview guide method for critical incident capture).
The surfacing of critical incidents centred on three questions:
What happened in the critical incident? - CAUSE
How did your strategic thinking change as a result of this event? - COURSE
What strategic actions followed on from this event (behaviour)? - RESULT
Evaluation of the critical incident debriefing activity is currently on-going; however, initial feedback
from instructors and students involved indicates that students were able to frame and reflect upon
incidents. In other words, the student teams were able, post-game, to identify critical incidents and
506

Jonathan Moizer and Jonathan Lean
make meaning of them from strategic perspective enabling them to evaluate the decision made and
actions taken during the course of the game.
5. Summary and conclusion
Anecdotally, students were highly engaged with the BSG, and an evaluation of grade performance
has indicated that an emphasis on critical incidents within the assessment of the gaming experience
has brought about an enhanced level of reflection, allowing more comprehensive and insightful oral
debriefings to take place. However, the researchers have been keen to conduct a fuller evaluation of
the reasons why student understanding of decision strategic management has improved even when
using the same simulation game to teach with; in particular what role has been played by the focus on
understanding critical incidents within the game playing period and how it teases out a better dialogue
bout the connections between causes, courses and results. To this end, over the last four years, data
has been collected from the oral debriefings to establish how the learning and assessment design has
fed through into the business understanding of students, i.e. how and to what extent has the
debriefing approach had a positive impact on reflective learning. This will provide a fuller
understanding of the contribution that critical incidents make to students’ understanding of and
engagement with the strategic issues within the simulated business, to allow meaning to be made of
decisions and outcomes with complex causes and consequences.
References
Ben-Zvi, T. (2010) “The Efficacy of Business Simulation Games in Creating Decision Support Systems: An
Experimental Investigation”, Decision Support Systems, Vol 49, pp 61-69.
Chen, N.S., Kinshuk, C.W. and Liu, C.C. (2011) “Effects of Matching Teaching Strategy to Thinking Style on
Learner’s Quality of Reflection in an Online Learning Environment”, Computers & Education, Vol 56, pp 53-
64.
Cope, J. (2003) “Entrepreneurship Learning and Critical Reflection: Discontinuous Events as Triggers for ‘Higherlevel’
Learning”, Management Learning, Vol 34, No. 4, pp 429-450.
Cope, J. and Watts, G. (2000) “Learning by Doing: An Exploration of Experience, Critical Incidents and Reflection
in Entrepreneurial Learning”, International Journal of Entrepreneurial Behaviour & Research, Vol 6, No. 3,
pp 104-124.
Doyle, D. and Brown, F.W. (2000) “Using a Business Simulation to Teach Applied Skills: The Benefits and the
Challenges of Using Student Teams from Multiple Countries”, Journal of European Industrial Training, Vol
24, No.6, pp 330-336.
Edelheim, J. & Ueda, D. (2007) “Effective Use of Simulations in Hospitality Management Education – a Case
Study”, Journal of Hospitality, Leisure, Sport & Tourism Education, Vol 6, No. 1, pp 18-28.
Edvardsson B. & Roos, I. (2001) “Critical Incident Techniques: Towards a Framework for Analysing the Criticality
of Critical Incidents”. International Journal of Service Industry Management, Vol 12, No. 3, pp 251-268.
Finch, A. (2010) “Critical Incidents and Language Learning: Sensitivity to Initial Conditions”, System, Vol 38, pp
422-431.
Flanagan, J.C. (1954) “The Critical Incident Technique”, Psychological Bulletin, Vol 51, No.4, pp 327-358.
Garris, R., Ahlers, R., and Driskell, J. E. (2002) “Games, Motivation, and Learning: A Research and Practice
Model”, Simulation & Gaming, Vol 33, No. 4, pp 441–467.
Gilgeous V. and D’Cruz, M. (1996) “A Study of Business and Management Games”, Management Development
Review, Vol. 9, No. 1, pp 32-39.
Gredler, M. E. (1996) “Games and Simulations and their Relationships to Learning”, in D. H. Jonassen (Ed.),
Handbook of Research for Educational Communications and Technology. pp. 571–603, Macmillan Library
Reference, New York.
Gulezian, A.K. (1981) “Teaching Business Policy and Strategy Using the Incident Process”, Developments in
Business Simulation & Experiential Exercises, Vol 8, pp 202-204.
Hainey, T. Connolly, T.M., Stansfield, M. and Boyle, E.A. (2011) “Evaluation of a Game to Teach Requirements
Collection and Analysis in Software Engineering at Tertiary Education Level”, Computers & Education, Vol
56, pp 21-35.
Johnston, B. (1995) “The Determinants of Service Quality: Satisfiers and Dissatisfiers”, International Journal of
Service Industry Management, Vol 6 (5), pp 53-71.
Kemppainen, J.K. (2000) “The Critical Incident Technique and Nursing Care Quality Research”, Journal of
Advanced Nursing, Vol 32, No. 5, pp 1264-71.
Knotts, U.S., and Keys, J.B. (1997) “Teaching Strategic Management with a Business Game”, Simulation
Gaming, Vol 28, No.4, pp 377-394.
Lee, C.Y. and Chen, M.P., Kinshuk, C.W. and Liu, C. (2008) “A Computer Game as a Context for Non-routine
Mathematical Problem Solving”, Computers & Education, Vol 52, pp 530-542.
Léger P.M., Charland, P., Feldstein, H.D. and Robert, J. (2011) “Business Simulation Training in Information
Technology Education: Guidelines for New Approaches in IT Training”, Journal of Information Technology
Education, Vol 10, pp 39-53.
507

Jonathan Moizer and Jonathan Lean
MacFarlane, B. (2001) “Developing Reflective Students: Evaluating the Benefits of Learning Logs within a
Business Ethics Programme”, Teaching Business Ethics, Vol 5, No. 4, pp 375-387.
MacFarlane, B. (2003) “Tales from the Front-line: Examining the Potential of Critical Incident Vignettes”,
Teaching Business Ethics, Vol 7, No. 1, pp 55-67.
Mallak, L.A., Lyth, D.M., Olson, S.D., Ulshafer, S.M. and Sardone, F.J. (2003) “Diagnosing Culture in Health-care
Organizations using Critical Incidents”, International Journal of Health Care Quality Assurance, Vol 4, pp
180–190.
Moizer, J.D., Lean, J., Smith, G., and Towler, M.J. (2004) “Experiences of Using a Business Strategy Simulation:
Lessons for Promoting Effective Learning”, International Journal of Management Education, Vol 4, No. 1, pp
19-28.
Moizer, J., Lean, J. Towler, M. and Smith, G. (2006) “Modes of Learning in the Use of a Computer-based
Business Simulation Game”, International Journal of Learning Technology, Vol 2, No. 1, pp 49-60.
Moenaert, R.K., Robben H., Antioco, M., De Schamphelaere, V. and Roks, E. (2010) “Strategic Innovation
Decisions: What You Foresee is Not What You Get”, Journal of Product Innovation Management, Vol 27, pp
840-855.
Pasin, F. and Giroux, H. (2011) “The Impact of a Simulation Game on Operations Management Education”.
Computer & Education, Vol 57, pp 1240-1254.
Petranek, C.F., Corey, S. and Black, R. (1992) “Three Levels of Learning in Simulations: Participating, Debriefing
and Journal Writing”, Simulation & Gaming, Vol 23, No. 2, pp 174-185.
Roos, I. (2002) “Methods of Investigating Critical Incidents: A Comparative Review. Journal of Service Research”,
Vol 4, No. 3, pp 193-204.
Sitzmann, T. (2011) “A Meta-Analytic Examination of the Instructional Effectiveness of Computer-Based
Simulation Games”, Personnel Psychology, Vol 64, pp 489-528.
Smith, J.R. and Golden, P.A. (2003) Corporation: A Global Business Simulation (4 th ed.), Prentice Hall, NJ.
Smith, J.R. and Golden, P.A. (2005) Human Resources Management Simulation (2 nd ed.), Prentice Hall, NJ.
Smith, J.R. and Golden, P.A. (2008) Airline: A Strategic Management Simulation (4 th ed.), Prentice Hall, NJ.
Sutcliffe, M. (2002) Games, Simulations and Role-playing. The Handbook for Economics Lecturers, LTSN,
Bristol.
Taylor, D.W. & Thorpe, R. (2004) “Entrepreneurial Learning: A Process of Co-participation”, Journal of Small
Business and Enterprise Development, Vol 11, No. 2, pp 203-211.
Thompson, A.A., Stappenbeck, G.J. and Reidenbach, M.A. (2010) The Business Strategy Game: Competing in a
Global Marketplace, (2010 ed.), McGraw-Hill/Irwin, IL.
Voss, R. (2009) “Studying Critical Classroom Encounters: The Experiences of Students in German College
Education”, Quality Assurance in Education, Vol 17, No. 2, pp 156-173.
Wolfe, J. (1975) “Effective Performance Behaviors in a Simulated Policy and Decision Making Environment”,
Management Science, Vol 21, No. 8, pp 872-882.
508